Pages:
Author

Topic: Request: Merit history downloadable as raw data (Read 721 times)

copper member
Activity: 168
Merit: 34
there is no base in this form
Today at 04:46:08 AM: 1 from NLNico for   need full name post >>>>  Re: BetKing.io - Poker, Dice, 50 BTC Jackpot, 20 currencies Huh?
legendary
Activity: 1862
Merit: 1469
https://Ecua.Mobi
There's also this from LoyceV
New: theymos' data in human readable format (14 MB).
I'll update it with post titles if Cloudflare once turned off again. At the moment, I can't download from the forum using command line:
Code:
ERROR 503: Service Temporarily Unavailable.
Have you tried using this?
https://github.com/Anorov/cloudflare-scrape

It works fine for me (but I didn't try today, I'm not sure if you mean an additional issue)
copper member
Activity: 168
Merit: 34
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]

(I may retract this offer if I get too many requests)

Sending an email, thank

There's also this from LoyceV

New: theymos' data in human readable format (14 MB).

Not suitable,  Today at 04:46:08 AM: 1 from NLNico for   need full name post    Re: BetKing.io - Poker, Dice, 50 BTC Jackpot, 20 currencies
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
There's also this from LoyceV
New: theymos' data in human readable format (14 MB).
I'll update it with post titles if Cloudflare once turned off again. At the moment, I can't download from the forum using command line:
Code:
ERROR 503: Service Temporarily Unavailable.
hero member
Activity: 2576
Merit: 882
Freebitco.in Support https://bit.ly/2I9BVS2
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]

(I may retract this offer if I get too many requests)

Sending an email, thank

There's also this from LoyceV

New: theymos' data in human readable format (14 MB).

copper member
Activity: 168
Merit: 34
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]

(I may retract this offer if I get too many requests)

Sending an email, thank
Vod
legendary
Activity: 3668
Merit: 3010
Licking my boob since 1970
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]

(I may retract this offer if I get too many requests)
copper member
Activity: 168
Merit: 34
Hi, have anyone in this form of a base or in a similar



or html ... .. in any format the main thing that was name post
jr. member
Activity: 308
Merit: 5
My progress is very slow at the moment (still recovering from being ill).

I have one request for theymos though: would it be possible to get a one-time complete overview over all initial Merit at the introduction at January 25th? Having such a baseline would make it much easier to analyze who ranked up after the Merit system was introduced.

Be better soon LoyceV.

I agree. It looks like exciting to see what the progress of merit to entire members?
It could clear up the doubt of merit effectivity nor it could clear that merit is not effective so far?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
My progress is very slow at the moment (still recovering from being ill).

I have one request for theymos though: would it be possible to get a one-time complete overview over all initial Merit at the introduction on January 25th? Having such a baseline would make it much easier to analyze who ranked up after the Merit system was introduced.
hero member
Activity: 2254
Merit: 831
That is the link in the post you quoted. You have to manipulate the data yourself and crawl the forum for the data as to ranks and boards.

LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328


Thanks, TheQuin. I am not developers, so I don't have skills to get raw data from website/ forum like this one. I can do good statistical analysis and nice, informative plots for the forum users to get general trend of merit system operation over the last two months. But I can only do it if I get data.
It is likely that I can't get it. Anyway, thanks again for your thread.
hero member
Activity: 2576
Merit: 882
Freebitco.in Support https://bit.ly/2I9BVS2
Here you go: https://bitcointalk.org/merit.txt.xz

Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.

I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.

Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Hi Theymos,

Would you mind giving me link to download full data of the forum related to merit (receivers, senders and their ranks, boards in the forum, etc.).
Thank you very much for your help.

That is the link in the post you quoted. You have to manipulate the data yourself and crawl the forum for the data as to ranks and boards.

LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328

hero member
Activity: 2254
Merit: 831
Here you go: https://bitcointalk.org/merit.txt.xz

Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.

I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.

Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Hi Theymos,

Would you mind giving me link to download full data of the forum related to merit (receivers, senders and their ranks, boards in the forum, etc.). The current ones lack those essential variables, and I don't have skills to get them from the forum raw dataset you gave.
Thank you very much for your help.
legendary
Activity: 2604
Merit: 2353
Another good thread made with these datas

"Merit Stat from theymos data"
https://bitcointalksearch.org/topic/merit-stat-from-theymos-data-3082289
hero member
Activity: 536
Merit: 513
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
Great, thanks. I'll start downloading it to my server and writing some scripts

I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
I'll try to do something like that. However it seems knowing what board/sub-community every msg belongs to would require crawling the forum a lot.


For the moment I'll aggregate total merits sent from one user to another and direct exchange of merits between 2 users; and I'll do my best about finding small groups of users sending merits to each other.

Nice, I posted my quick research here:

Merit stats & all transactions more than 40 Merits
https://bitcointalksearch.org/topic/merit-stat-all-transactions-more-than-40-merits-3046077

where I created a histogram to try to see tendencies and a clickable table for all transactions more than 40 Merits.
From the link you can directly check user profile, merit history, and the thread where the large transaction performed.

In total 85524 sMerits have been sent, which is much smaller than 11975 sMerits created by 57 merit sources per month.
It seems that we need more merit sources.
legendary
Activity: 1862
Merit: 1469
https://Ecua.Mobi
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
Great, thanks. I'll start downloading it to my server and writing some scripts

I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
I'll try to do something like that. However it seems knowing what board/sub-community every msg belongs to would require crawling the forum a lot.


For the moment I'll aggregate total merits sent from one user to another and direct exchange of merits between 2 users; and I'll do my best about finding small groups of users sending merits to each other.
administrator
Activity: 5166
Merit: 12850
Here you go: https://bitcointalk.org/merit.txt.xz

Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.

I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.

Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
legendary
Activity: 1862
Merit: 1469
https://Ecua.Mobi
The merit system is still very new, users are still leaning and experimenting with it, and there is the strong potential it will be tweaked in the coming months to address issues discovered.

I believe any effort into tracking stats about the merit economy this early into its life will probably be of little use, and will contain mostly outlier data. After a few months, especially after most of the merit initially given to various users is spent, stats about merit will be more useful.
Yes, the system is new and it could change. More sources may be added, people could learn to use it better.
But I don't think the basics will change. It will still be "User A merits users B's post X" and "User A has granted a total of Y merit points to user B", so most stats would still work.

Stats will definitely be more useful in the future, but that doesn't mean it won't be useful at the moment, or that developing a script now will be useless.
Maybe the analysis with the current little available data would be of little help, but it is required to start grabbing the data now so it's useful soon.
copper member
Activity: 2870
Merit: 2298
The merit system is still very new, users are still leaning and experimenting with it, and there is the strong potential it will be tweaked in the coming months to address issues discovered.

I believe any effort into tracking stats about the merit economy this early into its life will probably be of little use, and will contain mostly outlier data. After a few months, especially after most of the merit initially given to various users is spent, stats about merit will be more useful.
legendary
Activity: 1862
Merit: 1469
https://Ecua.Mobi
The only (possible) downside to this suggestion is that it will probably make unmasking of the merit sources easier.
Theymos didn't want to publish the list, because it could result in them being pestered for merits...

Theymos also said:
If you pay attention, you'll get a pretty clear idea of who the active sources are
and that is true with or without third-party scripts. Mainly, you'll just have to check this in a few months and you'll have a good idea of the list.

So I don't think this suggestion will make a difference on that matter. It will help identifying other things much harder to spot only with the currently available tools, such as abuse and advanced stats.
Pages:
Jump to: