December data is available here:
https://drive.google.com/open?id=1mGEk6V3c_D-IhSYbuJvPrGGVEWLb0V8LThere is both raw data and the SQL Lite DB. There are now
2.481.270 users in it and about
44.206 new users since the last run.
Since from time to time people get interested in having all user data available for different purposes, but they don’t know how to scrape it or they scrape each time their own (and can take a really long time to do it), i thought i would do it and share the data with everybody.
We had already some great thread showing different kind of stories and aspects of users over time, i’m sure somebody will come up with something new we have't seen so far (this contain also a fresh snapshot of the trust details of all users) or can simply be useful to update own user data.
The data was taken between the 3th and 8th of November, there are
2.437.064 users.
This is the table and data available for each user in the database, which you can use to run your query as per example below:
TABLE UserData(
UserId Integer PRIMARY KEY,
UserName TEXT,
Trust TEXT,
Merit Integer,
Posts Integer,
Activity Integer,
Rank TEXT,
DateRegistered Integer,
LastActive Integer,
ICQ TEXT,
AIM TEXT,
MSN TEXT,
YIM TEXT,
Email TEXT,
Website TEXT,
Bitcoinaddress TEXT,
Gender TEXT,
Age TEXT,
Location TEXT,
LastUpdatedData Integer
);
You can download the data here, both db and raw(db is about
366MB):
https://drive.google.com/open?id=1l0xz1OC4mc3FvXzX18scSbSTzsnUkD4fHere is a brief description of the necessary step to use the database:Download the Precompiled Binaries for your favourite platform here (we just need the sqlite3):
https://www.sqlite.org/download.html"A bundle of command-line tools for managing SQLite database files, including the command-line shell program, the sqldiff program, and the sqlite3_analyzer program."
If you want to know more in detail how to use Sqlite3 directly:
https://www.sqlite.org/cli.htmlExample for usage under Windows platform, from command line:
Put the database, scripts and the sqlite3.exe in the same folder and you have everything you need to start:
Query.bat file:
sqlite3 btctalk_full.db < CommandsQuery.txt
CommandsQuery.txt:
.mode csv
.separator "\t"
.output ResultQuery.csv
SELECT * FROM UserData WHERE Activity > 100 AND LastActive <= '2015-01-01';
.output stdout
.quit
Now from command line call Query.bat and after a bit of crunching you will get the data in the ResultQuery.csv tab separated.
