Here’s the data, up to last Friday.
Thanks DdmrDdmr. The data worked perfectly.
I have updated the OP with lots of new local animations, as requested by some people.
Topic: [UPDATED] bitcoin/bitcointalk.org Stats Animation - all in one thread (Read 1360 times)
November 24, 2021, 06:10:28 AM
Thanks DdmrDdmr. The data worked perfectly.
I have updated the OP with lots of new local animations, as requested by some people.
November 14, 2021, 10:42:38 AM
<…>
I’m still using the forum’s search function to detect mentions, but somehow skipped this one, and wasn’t really aware of the petition until @fillippone posted today …You know, @ddmrddmr, resistance is futile.
Time has come:
#meritislife: how to be notified on a smartband of merits and mentions
Join the army of the hairy wrist band so you will never skip a mention ever!
November 14, 2021, 09:57:49 AM
<…>
I’m still using the forum’s search function to detect mentions, but somehow skipped this one, and wasn’t really aware of the petition until @fillippone posted today …Here’s the data, up to last Friday. I’ve exported it using a different method, but the format should be the same as on prior occasions:
https://drive.google.com/file/d/1wdUBx9p0KTtbxlucBm_S6CIR9UqOgRxs/view?usp=sharing
Let me know if anything is off in the file.
<…>
I’m must be old fashioned (which doesn’t mean I won’t cave-in eventually): I want to go and get my notifications at my convenience and pace … November 13, 2021, 06:09:41 PM
Hey,
I know this thread hasn’t been updated for a long time.
Is there the possibility to have a merit race for the Italian board?
Or any new idea you came up with in those few months?
Thanks!
I know this thread hasn’t been updated for a long time.
Is there the possibility to have a merit race for the Italian board?
Or any new idea you came up with in those few months?
Thanks!
Of course!
For local boards I need some extra data from @DdmrDdmr. Can you please send me an updated file of including Section/subsection column? no need to have too many details, I will consider any local child board as local only.
Thank you DdmrDdmr
Any update on this?
@ddmddmr can you do that for us?
I have just updated my meta post, and I would like to include that video in the post!
October 27, 2021, 11:14:02 AM
Hey,
I know this thread hasn’t been updated for a long time.
Is there the possibility to have a merit race for the Italian board?
Or any new idea you came up with in those few months?
Thanks!
I know this thread hasn’t been updated for a long time.
Is there the possibility to have a merit race for the Italian board?
Or any new idea you came up with in those few months?
Thanks!
Of course!
For local boards I need some extra data from @DdmrDdmr. Can you please send me an updated file of including Section/subsection column? no need to have too many details, I will consider any local child board as local only.
Thank you DdmrDdmr
October 26, 2021, 06:21:33 PM
Hey,
I know this thread hasn’t been updated for a long time.
Is there the possibility to have a merit race for the Italian board?
Or any new idea you came up with in those few months?
Thanks!
I know this thread hasn’t been updated for a long time.
Is there the possibility to have a merit race for the Italian board?
Or any new idea you came up with in those few months?
Thanks!
February 28, 2021, 02:55:28 AM
<...>
I’ve uploaded a file with the same structure as last time to the following link:https://drive.google.com/file/d/1dANGltunbyBo1JwNaVLUhdGfQVnTdR4O/view?usp=sharing
The file is a compressed file, that contains the flat file "TXs_Bitmover_export_20210228.txt", with data as of last Friday.
Bear in mind that I haven’t had my morning coffee yet, so please check that the content is as expected…
February 27, 2021, 09:32:34 PM
You can use the following file: https://drive.google.com/file/d/1rOvFPbd9T6q5qrtLtAD1gy4fBMcUrg3Z/view?usp=sharing
It contains all the historical TXs, including the board description (section/subsection) It occupies some 75 MBs. You’ll have to load them to a DB or whatever repository you are going to use, and filter by subsection = Turkish (last column).
The data is as of last Friday 19/06/2020. How you create a the race itself is up to you (python, Tableau, etc.).
It contains all the historical TXs, including the board description (section/subsection) It occupies some 75 MBs. You’ll have to load them to a DB or whatever repository you are going to use, and filter by subsection = Turkish (last column).
The data is as of last Friday 19/06/2020. How you create a the race itself is up to you (python, Tableau, etc.).
Hello @DdmrDdmr,
Can you update this file again, please?
I have some ideas about new charts, a network chart for some local boards.
Thank you.
September 25, 2020, 10:17:11 AM
<…>
Since tomorrow we receive the new Merit.txt file, it’s probably better to wait for that update, in order to have the most recent version. It should therefore be ready tomorrow (max. Saturday). I’ll let you know when it’s done, and I’ll trim the information on child boards from it (leaving the granularity at subsection level again).Edit: File-> https://drive.google.com/file/d/1meWPgG_bIyChkUIcxHhTQnB4WPwp1oAV/view?usp=sharing
Thank you DdmrDdmr.
The page was just updated with the latest data
link
September 24, 2020, 08:27:21 AM
<…>
Since tomorrow we receive the new Merit.txt file, it’s probably better to wait for that update, in order to have the most recent version. It should therefore be ready tomorrow (max. Saturday). I’ll let you know when it’s done, and I’ll trim the information on child boards from it (leaving the granularity at subsection level again).Edit: File-> https://drive.google.com/file/d/1meWPgG_bIyChkUIcxHhTQnB4WPwp1oAV/view?usp=sharing
September 24, 2020, 07:17:05 AM
<...>
I've uploaded all the TXs to the following file, including the TXs from today’s merit.txt file: https://drive.google.com/file/d/1MnRa6e2O0RH7K2gw7AUoT7FwK8rHbX1-/view?usp=sharing
Hello DdmrDdmr.
Do you have an updated file for merit transactions? So i can update the merit table.
It doesn't need to be so big like the old one. I decided not to include those child boards, and focus more in other projects. Please share the link with me when you can.
Thank you
August 17, 2020, 09:04:57 AM
Wow, well done, glad to see this 
I have a suggestion for you. Maybe you can disable report from zero value column. it only causes additional scrolling.
I have a suggestion for you. Maybe you can disable report from zero value column. it only causes additional scrolling.
Thanks examplens
I liked your suggestion, however I am not able to do it for now.
As I explained here, I tried to generate the table myself using bootstrap, which would allow a lot of customization options. But the loading time was terrible. I think I need to learn more about SQL or AJAX or soemthing like that to make a server side processing.
I am not able to do it.
I tried to use bootstrap + datatable.net to create the table, so I would have more customization options. However, I do not know how to efficiently parse the data, and my HTML file had 30mb and consumed a lot of processing power.
Flourish is very efficient in parsing the data, so I will keep the table in this format and move on to other projects. At least for now. Maybe in the future as I learn more about JS and SQL...
I tried to use bootstrap + datatable.net to create the table, so I would have more customization options. However, I do not know how to efficiently parse the data, and my HTML file had 30mb and consumed a lot of processing power.
Flourish is very efficient in parsing the data, so I will keep the table in this format and move on to other projects. At least for now. Maybe in the future as I learn more about JS and SQL...
Quote
Also when we talking about the scroll, if it's possible to change horizontal to vertical scroll, it is easier to use on desktop pc. That will be fine.
Tried on a mobile device, and there is okay. Table fields are placed one below the other.
Tried on a mobile device, and there is okay. Table fields are placed one below the other.
This I can do. This is a dataframe transpose. And it is really a nice suggestion. I will make some tests
August 17, 2020, 07:08:23 AM
The process was so painful that I intentionally edited the file in notepad to change it, intentionally breaking unwanted characters.
I was able to update my table using your data, you can check it on https://www.bitcoindata.science/supertable.html
I will try to work in the new categories (path l4 first) on Monday.
I was able to update my table using your data, you can check it on https://www.bitcoindata.science/supertable.html
I will try to work in the new categories (path l4 first) on Monday.
Wow, well done, glad to see this
I have a suggestion for you. Maybe you can disable report from zero value column. it only causes additional scrolling. Also when we talking about the scroll, if it's possible to change horizontal to vertical scroll, it is easier to use on desktop pc. That will be fine.
Tried on a mobile device, and there is okay. Table fields are placed one below the other.
August 14, 2020, 03:54:56 PM
<…> I was able to update my table using your data, you can check it on https://www.bitcoindata.science/supertable.html <...>
Looks good. We differ a bit on the last decimal of the percentages, but minimal, likely due to using different rounding precision.<…> I just changed encoding to UTF-16 and it worked without too much a hashle (just did a few adjustments). My line 20 is intact now and I can see Aльтepнaтивныe кpиптoвaлюты correctly <…>
Great! slayed the Unicode dragon pretty swiftly there. August 14, 2020, 03:32:08 PM
Ideally, without needing to edit the file in between, since the editor may change the format and be part of the cause. This is normally a small painful process of getting the environment to interpret this kind of file properly.
The process was so painful that I intentionally edited the file in notepad to change it, intentionally breaking unwanted characters.
I was able to update my table using your data, you can check it on https://www.bitcoindata.science/supertable.html
I will try to work in the new categories (path l4 first) on Monday.
Edit:
The resulting file opens with Excel seamlessly, as does notepad (notepad indicates that the format is actually UTF 16).
I just changed encoding to UTF-16 and it worked without too much a hashle (just did a few adjustments). My line 20 is intact now and I can see Aльтepнaтивныe кpиптoвaлюты correctly
August 14, 2020, 02:23:50 PM
<...>
Actually, the data I scrape and store goes to a table where the fields are defined as nVarchar (as opposed to varchar). nVarchar supports Unicode characters, which is what I need to store the scraped information whilst preserving the content’s format.Due to the above, the export I created requires me to mark Unicode in the export parametrization (forget the UTF8 -> Unicode overrides it even if I select it).
The resulting file opens with Excel seamlessly, as does notepad (notepad indicates that the format is actually UTF 16). You probable need to see how to treat Unicode files in your environment without all those prior conversions. Ideally, without needing to edit the file in between, since the editor may change the format and be part of the cause. This is normally a small painful process of getting the environment to interpret this kind of file properly.
August 14, 2020, 12:26:55 PM
The "path" field (last field) does indeed look weird (*) now on your quoted example, but the rest seem ok.
Check if row number 20 shows as this:
Check if row number 20 shows as this:
Quote
"20","2018-01-24 22:33:27","1516833207","1189487","Prometheu$","Full Member","1028592","nikola22","Full Member","1","https://bitcointalksearch.org/topic/m.28837085","Local","Russian","Aльтepнaтивныe кpиптoвaлюты ","","","Bitcoin Forum > Local > Pyccкий (Russian) > Aльтepнaтивныe кpиптoвaлюты (Moderators: xandry, Xal0lex) > [ANN][ICO] TOKENGO BUSINESS TOKENIZATION PLATFORM - Bonus up to 50%"
If the fields are displayed as above (and not as a bunch of question marks), you’re good to go.This is my line 20
Code:
"20","2018-01-24 22:33:27","1516833207","1189487","Prometheu$","Full Member","1028592","nikola22","Full Member","1","https://bitcointalksearch.org/topic/m.28837085","Local","Russian","Ðльтернативные криптовалюты ","","","Bitcoin Forum > Local > РуÑÑкий (Russian) > Ðльтернативные криптовалюты (Moderators: xandry, Xal0lex) > [ANN][ICO] í ¼í¼€TOKENGOí ¼í¼€ BUSINESS TOKENIZATION PLATFORM - Bonus up to 50%"
I can clean this with regex. I will just lose the pathl4 column in some rows (specially locals), which is the one with this crazy " Ðльтернативные криптоваÐ... "...
But, overall I will can use pathl4 column, unless it has some crazy encoding.
How did you manage to work with this kind of characters? льтepнaтивныe кpип and . Are you using UTF-8?
August 14, 2020, 10:33:01 AM
<...>
<…>
Yep, I sometimes encounter issues with encodings I receive on files. Takes some time fighting to overcome it. If you’ve now got a method to load the file, that is not too much of a hassle, we can leave the file as is (or go over it if you want me to change the format). The "path" field (last field) does indeed look weird (*) now on your quoted example, but the rest seem ok.
Check if row number 20 shows as this:
The problem is being caused by UTF-8 bytes being interpreted as Windows-1252 (or ISO 8859-1) bytes.
https://www.i18nqa.com/debug/utf8-debug.html August 14, 2020, 10:08:27 AM
Let's see if you can load this file properly. I has to make it Unicode because of the non Latin characters - i.e. russian charactrs on path_l4, which is the first childboard level. RowId number 20 is the first case with such characters.
Thanks for that.
This file was not easily parsed... I had to change a lot of configurations in python notebook, even data rate limit... I learned a lot parsing your data.
I also had to open it in Notepad and I changed it encoding manually to UTF-8, breaking all those special characters (as I won't use them anyway)
My final csv was like this:
Code:
"8","2018-01-24 22:29:38","1516832978","1344962","dspencer","Member","1101839","Scout Georgenson","Jr. Member","1","https://bitcointalk.org/index.php?topic=2384335.msg28854772#msg28854772","Alternate cryptocurrencies","Announcements (Altcoins)","Tokens (Altcoins) (Moderators: mprep, Welsh)","","","Bitcoin Forum > Alternate cryptocurrencies > Announcements (Altcoins) > Tokens (Altcoins) (Moderators: mprep, Welsh) > [ANN] [ICO] Xwin – CryptoBet betting platform based on blockchain"
then I was finally able to open your dataset
Thank you again. I will work in that data to update my Super Merit table soon.
August 14, 2020, 05:01:03 AM
<...>
I've uploaded all the TXs to the following file, including the TXs from today’s merit.txt file: https://drive.google.com/file/d/1MnRa6e2O0RH7K2gw7AUoT7FwK8rHbX1-/view?usp=sharing
Observations:
- I’ve only uploaded the .rar file, which is 16 Mb. If you’d rather have I upload the csv file without compression, tell me (264 Mb).
- The files is csv, with quote qualifiers.
- Code page is 1252 (ANSI – Latin 1) and Unicode.
- The file contains the additional fields described here: https://bitcointalksearch.org/topic/m.54895308
Let's see if you can load this file properly. I has to make it Unicode because of the non Latin characters - i.e. russian charactrs on path_l4, which is the first childboard level. RowId number 20 is the first case with such characters.
Jump to: