Today at 04:46:08 AM: 1 from NLNico for need full name post >>>> Re: BetKing.io - Poker, Dice, 50 BTC Jackpot, 20 currencies
? 
Topic: Request: Merit history downloadable as raw data (Read 754 times)
there is no base in this form
Today at 04:46:08 AM: 1 from NLNico for need full name post >>>> Re: BetKing.io - Poker, Dice, 50 BTC Jackpot, 20 currencies?
Today at 04:46:08 AM: 1 from NLNico for need full name post >>>> Re: BetKing.io - Poker, Dice, 50 BTC Jackpot, 20 currencies
? 
There's also this from LoyceV
I'll update it with post titles if Cloudflare once turned off again. At the moment, I can't download from the forum using command line:Code:
ERROR 503: Service Temporarily Unavailable.
https://github.com/Anorov/cloudflare-scrape
It works fine for me (but I didn't try today, I'm not sure if you mean an additional issue)
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]
(I may retract this offer if I get too many requests)
(I may retract this offer if I get too many requests)
Sending an email, thank
There's also this from LoyceV
Not suitable, Today at 04:46:08 AM: 1 from NLNico for need full name post Re: BetKing.io - Poker, Dice, 50 BTC Jackpot, 20 currencies
There's also this from LoyceV
I'll update it with post titles if Cloudflare once turned off again. At the moment, I can't download from the forum using command line:Code:
ERROR 503: Service Temporarily Unavailable.
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]
(I may retract this offer if I get too many requests)
(I may retract this offer if I get too many requests)
Sending an email, thank
There's also this from LoyceV
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]
(I may retract this offer if I get too many requests)
(I may retract this offer if I get too many requests)
Sending an email, thank
I (BPIP) have collected all merit data in a database, and I can send you a copy in the format you'd like if you send an email to [email protected]
(I may retract this offer if I get too many requests)
(I may retract this offer if I get too many requests)
Hi, have anyone in this form of a base or in a similar
or html ... .. in any format the main thing that was name post
or html ... .. in any format the main thing that was name post
LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328
My progress is very slow at the moment (still recovering from being ill).I have one request for theymos though: would it be possible to get a one-time complete overview over all initial Merit at the introduction at January 25th? Having such a baseline would make it much easier to analyze who ranked up after the Merit system was introduced.
Be better soon LoyceV.
I agree. It looks like exciting to see what the progress of merit to entire members?
It could clear up the doubt of merit effectivity nor it could clear that merit is not effective so far?
LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328
My progress is very slow at the moment (still recovering from being ill).I have one request for theymos though: would it be possible to get a one-time complete overview over all initial Merit at the introduction on January 25th? Having such a baseline would make it much easier to analyze who ranked up after the Merit system was introduced.
That is the link in the post you quoted. You have to manipulate the data yourself and crawl the forum for the data as to ranks and boards.
LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328
Thanks, TheQuin. I am not developers, so I don't have skills to get raw data from website/ forum like this one. I can do good statistical analysis and nice, informative plots for the forum users to get general trend of merit system operation over the last two months. But I can only do it if I get data.LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328
It is likely that I can't get it. Anyway, thanks again for your thread.
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Hi Theymos,Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Would you mind giving me link to download full data of the forum related to merit (receivers, senders and their ranks, boards in the forum, etc.).
Thank you very much for your help.
That is the link in the post you quoted. You have to manipulate the data yourself and crawl the forum for the data as to ranks and boards.
LoyceV has already started to do some work on that. https://bitcointalksearch.org/topic/loycevs-merit-data-analysis-full-data-since-jan-24-2018-not-just-120-days-3078328
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Hi Theymos,Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Would you mind giving me link to download full data of the forum related to merit (receivers, senders and their ranks, boards in the forum, etc.). The current ones lack those essential variables, and I don't have skills to get them from the forum raw dataset you gave.
Thank you very much for your help.
Another good thread made with these datas
"Merit Stat from theymos data"
https://bitcointalksearch.org/topic/merit-stat-from-theymos-data-3082289
"Merit Stat from theymos data"
https://bitcointalksearch.org/topic/merit-stat-from-theymos-data-3082289
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
Great, thanks. I'll start downloading it to my server and writing some scriptsSimilar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
I'll try to do something like that. However it seems knowing what board/sub-community every msg belongs to would require crawling the forum a lot.For the moment I'll aggregate total merits sent from one user to another and direct exchange of merits between 2 users; and I'll do my best about finding small groups of users sending merits to each other.
Nice, I posted my quick research here:
Merit stats & all transactions more than 40 Merits
https://bitcointalksearch.org/topic/merit-stat-all-transactions-more-than-40-merits-3046077
where I created a histogram to try to see tendencies and a clickable table for all transactions more than 40 Merits.
From the link you can directly check user profile, merit history, and the thread where the large transaction performed.
In total 85524 sMerits have been sent, which is much smaller than 11975 sMerits created by 57 merit sources per month.
It seems that we need more merit sources.
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
Great, thanks. I'll start downloading it to my server and writing some scriptsSimilar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
I'll try to do something like that. However it seems knowing what board/sub-community every msg belongs to would require crawling the forum a lot.For the moment I'll aggregate total merits sent from one user to another and direct exchange of merits between 2 users; and I'll do my best about finding small groups of users sending merits to each other.
Here you go: https://bitcointalk.org/merit.txt.xz
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
Similar to trust.txt.xz, it'll be updated weekly. It will show only the last 120 days of data; someone else should archive the old ones if you want them.
I am especially interested in analyses of this data which could point to sub-communities where the initial sMerit is exhausted and new sources are necessary, and people who might be good merit sources.
Edit: Note that for a little while I had user_to and user_from as names, but I decided to change it to IDs.
The merit system is still very new, users are still leaning and experimenting with it, and there is the strong potential it will be tweaked in the coming months to address issues discovered.
I believe any effort into tracking stats about the merit economy this early into its life will probably be of little use, and will contain mostly outlier data. After a few months, especially after most of the merit initially given to various users is spent, stats about merit will be more useful.
Yes, the system is new and it could change. More sources may be added, people could learn to use it better.I believe any effort into tracking stats about the merit economy this early into its life will probably be of little use, and will contain mostly outlier data. After a few months, especially after most of the merit initially given to various users is spent, stats about merit will be more useful.
But I don't think the basics will change. It will still be "User A merits users B's post X" and "User A has granted a total of Y merit points to user B", so most stats would still work.
Stats will definitely be more useful in the future, but that doesn't mean it won't be useful at the moment, or that developing a script now will be useless.
Maybe the analysis with the current little available data would be of little help, but it is required to start grabbing the data now so it's useful soon.
The merit system is still very new, users are still leaning and experimenting with it, and there is the strong potential it will be tweaked in the coming months to address issues discovered.
I believe any effort into tracking stats about the merit economy this early into its life will probably be of little use, and will contain mostly outlier data. After a few months, especially after most of the merit initially given to various users is spent, stats about merit will be more useful.
I believe any effort into tracking stats about the merit economy this early into its life will probably be of little use, and will contain mostly outlier data. After a few months, especially after most of the merit initially given to various users is spent, stats about merit will be more useful.
The only (possible) downside to this suggestion is that it will probably make unmasking of the merit sources easier.
Theymos didn't want to publish the list, because it could result in them being pestered for merits...
Theymos didn't want to publish the list, because it could result in them being pestered for merits...
Theymos also said:
If you pay attention, you'll get a pretty clear idea of who the active sources are
and that is true with or without third-party scripts. Mainly, you'll just have to check this in a few months and you'll have a good idea of the list.So I don't think this suggestion will make a difference on that matter. It will help identifying other things much harder to spot only with the currently available tools, such as abuse and advanced stats.
The only (possible) downside to this suggestion is that it will probably make unmasking of the merit sources easier.
Theymos didn't want to publish the list, because it could result in them being pestered for merits...
Theymos didn't want to publish the list, because it could result in them being pestered for merits...
I think this is a good suggestion.
Personally I don't like the new Merit system but... If it's supposed to continue, we should be sure no one will make shady things
Personally I don't like the new Merit system but... If it's supposed to continue, we should be sure no one will make shady things
- The script must override the logging in and parse the HTML. This can change at any time and the script would stop working
What is this supposed to be? I just see this:
I just converted the page to a file like thisBut it's unreliable as I explained in my previous post.
(And you're not really being helpful by just posting the result of the conversion. It could have been done manually for all I know)
- The script must override the logging in and parse the HTML. This can change at any time and the script would stop working
What is this supposed to be? I just see this:
I just converted the page to a file like thisyou want Real-time data or just txt file "manually updated"
I want raw data updated every specif amount of time, every day, week, whatever makes sense, so that third party scripts can download it regularly without overloading bitcointalk's server and knowing no data is lost.it’s easy to make .txt file from pages like https://bitcointalk.org/index.php?action=merit;stats=topusersat.
I'm more interested in this, actually: https://bitcointalk.org/index.php?action=merit;stats=recentI wouldn't say easy, but yes it's possible. There are a few problems about doing that though:
- The script must override the logging in and parse the HTML. This can change at any time and the script would stop working
- Because there's no specif amount of time after which the data is updated, the script must download the data regularly, very often. This can cause problems with the anti-DDos system
- If the script downloads data from here, it doesn't know immediately who received the Merit. It would need to open every post to see who the author is, causing a lot of traffic and, again, having problems with the DDoS system
Providing raw data (like this) is by far the best solution.
I'm willing to write a Merit script and generate some stats, but I'm really not willing to do so when it represents so much work and knowing it will stop working at any time.
you download it from here https://file.io/LOKibO
What is this supposed to be? I just see this:Quote
{"success":false,"error":404,"message":"Not Found"}
I know that admins don't have time to analyse this and would appreciate help from the community. We wouldn't have to rely on people spotting and reporting possible cases of merit trading and could do it ourselves. I've seen some posts where people were getting 50 merit from a single account all at once which is at the very least suspicious. A list of merit activity would allow us to clearly see what's going on all around the forum. I'd expect most merit trading to be happening in the local boards and if you delete the merited post all visible trace is lost.
This would be especially useful in early days of implementation of the system, to detect cases of blatant abuse.
Yes, definitely! And also in the long-term to have better stats, to both keep us informed and to help improving it.
I don't think it would be too difficult to provide the data, and it will allow all sorts of development.
This is a very good idea. People can use the raw data to generate statistics they need.
This would be especially useful in early days of implementation of the system, to detect cases of blatant abuse.
This would be especially useful in early days of implementation of the system, to detect cases of blatant abuse.
Bumping this as I think it's a good idea. I'd also like to see more stats rather than just the top 50 for each category.
I already posted this on the official Merit thread. However it just got a couple of comments and then got buried.
So I'm re-posting it here. I really think this would be useful:
I'd like to see a link to download raw data regarding the latest merit activity.
Basically this https://bitcointalk.org/index.php?action=merit;stats=recent but as txt, json, csv or similar format including name/id of sender, name/id of receiver and post ID.
When the DT was provided as text some interesting processing was possible.
With merit raw data we could easily write scripts to find suspicious activity, to show it graphically, advanced stats, among other things.
So I'm re-posting it here. I really think this would be useful:
I'd like to see a link to download raw data regarding the latest merit activity.
Basically this https://bitcointalk.org/index.php?action=merit;stats=recent but as txt, json, csv or similar format including name/id of sender, name/id of receiver and post ID.
When the DT was provided as text some interesting processing was possible.
With merit raw data we could easily write scripts to find suspicious activity, to show it graphically, advanced stats, among other things.
Jump to: