Pages:
Author

Topic: Bitcoin block data (728 GB): inputs, outputs and transactions - page 7. (Read 3356 times)

copper member
Activity: 1554
Merit: 489
Stop the war!
I never saw the blockchain in CSV format, which is a easy to use format for data analysis.
Will you analize 500 GB data by-hand?  Shocked
I think the best format is a relational database. SQLite for example. But it is not exactly  Smiley
legendary
Activity: 2352
Merit: 6089
bitcoindata.science
The data
See blockdata.loyce.club
blockdata.loyce.club/inputs/ (159 GB - currently incomplete, growing 8 GB per day)
blockdata.loyce.club/outputs/ (117 GB)
blockdata.loyce.club/transactions/ (43 GB)

Hello LoyceV.

I am curious this file.

Can you create a sample of each file, with just a few MB, so people can see the that and how it is organized before downloading 500GB?

What is the problem in to make your own blockchain parser?
This is easier Smiley

I never saw the blockchain in CSV format, which is a easy to use format for data analysis. Is there anywhere else where you can download the blockchain in csv format other than in loyce.club and blockchair?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
What is the problem in to make your own blockchain parser?
This is easier Smiley
And I don't mind the fact that each file can be verified at an independent third party.
copper member
Activity: 1554
Merit: 489
Stop the war!
Quote
I've used Blockchair Database Dumps for several projects, but when I need it I don't want to wait weeks to download all files. So I created a mirror for Bitcoin data
What is the problem in to make your own blockchain parser?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
October 7, 2022: I've consolidated a few posts here, so that I can remove Vod's post (sorry Vod) and have the second post in this topic for myself.

Credits
Blockchair Database Dumps has a staggering amount of data, easily accessible (at 10 100 kB/s) with daily updates. All data in this topic comes from Blockchair.
Spam discussion/question.    The page mentions 10 kb/s - is there any cost or requirement for the 100?

The page mentions 10 kb/s - is there any cost or requirement for the 100?
At some point they increased the speed limit, but didn't update their Readme.
The fool speed is a tad above my budget for this:
I just contacted Blockchair for an API key, which enables people to download at the fast rate, and a support rep told me they cost $500/month.

Bump:
blockdata.loyce.club/inputs/ (159 GB - currently incomplete, growing 8 GB per day)
This directory is now 349 GB in size. It's almost complete Smiley
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Ethereum data

I've added ethereumdata.loyce.club Abandoned! I'm out of disk space. I have a local backup, if you really need this: ask!
ethereumdata.loyce.club/calls/ (345 GB)
ethereumdata.loyce.club/erc-20_transactions/ (378 GB)
ethereumdata.loyce.club/transactions/ (71 GB)

All funded Ethereum addresses: ethereumdata.loyce.club/blockchair_ethereum_addresses_latest.tsv.gz (5.1 GB).  Blockchair doesn't updated this file regularly, so I don't do automatic updates.

Disclaimer
I don't like Ethereum, which is a centralized shitcoin that abandoned it's one unique selling point ("code is law") the moment it was convenient for the creator. Don't waste your money on it!
And since it's a shitcoin, I don't think it deserves it's own topic.

Missing files
Those 2 files are missing from my mirror because they're corrupted:
https://gz.blockchair.com/ethereum/calls/blockchair_ethereum_calls_20200113.tsv.gz
https://gz.blockchair.com/ethereum/calls/blockchair_ethereum_calls_20211110.tsv.gz
If anyone has the tools to rebuild those 2 files, I'd love to add them!
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Background
I've used Blockchair Database Dumps for several projects, but when I need it I don't want to wait weeks months to download all files. So I created a mirror for Bitcoin data Smiley

The data
See blockdata.loyce.club
blockdata.loyce.club/inputs/ (514 GB)
blockdata.loyce.club/outputs/ (159 GB)
blockdata.loyce.club/transactions/ (56 GB)
NEW! block_hash_version_versionHex_merkleroot_time_mediantime_nonce_bits_difficulty_ chainwork_nTx_strippedsize_size_weight.tsv.gz (80 MB)

Updates
New files are added daily.

Samples
inputs:
Code:
block_id transaction_hash index time value value_usd recipient type script_hex is_from_coinbase is_spendable spending_block_id spending_transaction_hash spending_index spending_time spending_value_usd spending_sequence spending_signature_hex spending_witness lifespan cdd
141719 3c86cc88726f38d2f1e861850b143fe8d710e171a91291628ffe6418d8b60190 1 2011-08-20 06:12:23 990322200 112.8967 1L2wqJ4z4pjXpqP4EFudnn7TKE4sjksQQj pubkeyhash 76a914d0c893ee2fa4278a2542691e2f569247d833805988ac 0 -1 152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 0 2011-11-11 00:05:55 28.4222 4294967295 4830450220015c796fc6ddf0040c70115539d476426f5c69e1537bb8b7da90d4d671c564d302210094fc145108b617352db33880118a69c059055538af7472ae60a5f4b37d558a060141048396aebc72f8a7b26ea0f676c7c8b9379b0a93b487058787f78b2d256d2a4a6e37d7df1ac8314ee09c1f9a57d48f8c07fd1e4e3056bb17c0075ec703599cada9 7149212 819.4471476975
152763 5733d4a453dcea2690bdc8e8866f5788adadca82ba4639a3dada3d26f738ae9a 0 2011-11-10 23:53:08 1167280 0.035 1JFVo6cGL3cpyXZctC5Tw3HHvmAzuwkjSj pubkeyhash 76a914bd37ff9aa142d5ed87b1f9385c66b856b6c60e6088ac 0 -1 152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 1 2011-11-11 00:05:55 0.0335 4294967295 493046022100ba9fa562476163f5eaac9a6bbee123a0c3f1421377c35208af4b9606816603d1022100c1a797af43d8297f08011248c16f01ee9e95d03ff5d527c2ace2adb545eb88de0141047d949c30d6ffffe8e5ecd2ee774745bda2563e4842b7a9836bd0be8461786a2cceb1d844b0789afde927eace479955f9c27ff98e0e5afe71d4dde54e621aaa1f 767 0.0001036231
141326 5c51c88388d3452c0da455ad03ee50ca121ab4812d8790402527df74cf327db1 0 2011-08-17 11:03:42 700000000 77.35 17AqNueHmeFbk6rxnpvQM3QGFJM9oAs1XY pubkeyhash 76a91443ace1cda1b0aede06ca68bc0f60f4c11ecbfdc288ac 0 -1 152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 0 2011-11-11 00:05:55 20.09 4294967295 493046022100b848925df3d18baa10f26b169bd917927c0787aa5719d3e2811176ec66e8297f022100abf4688ed1b744fd3d64e2263137feb2b057ff93d35d8e8e8ddb13fee49c40780141049042d124b779c07a9c825fd572a5002c7a50efc1fea451ad390308196f75ea7b0747c86dbfa369400b0b8d7c1965999d3fe0ad2a4f87afd96c90a5d3ed2c5ecb 7390933 598.8024421296
152754 6b5cd836c52d17a1390e1a9533303941ad23e497f0b508922f1dc55c00d293f0 0 2011-11-10 21:46:08 59000000 1.77 1KGBGWUVbVpvW6Sw7GBVsjS2NS8uWbMsgM pubkeyhash 76a914c85113063d2c642f0566d605572581075dd7bd7188ac 0 -1 152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 1 2011-11-11 00:05:55 1.6933 4294967295 483045022100a4a8ce69ccbee08899a0c5693c1c43ddac5e4c187093fd7bd1efdcea70a92c3502200fe64020f94229976febf8427d4b1510187f500f95a004a4a7be498b0439c565014104175747a43f4f51bcb9f1753398157bf5b02cd9b239ea11b544d95b3f7c81881c46c0a86b57e445f0e803c630dc1f51e7b6f7d59f36b408509cd1b3cdda918625 8387 0.057272338
145520 f49804f14a44a7a478a870635662df8b6b089317c2e878b701c6ab4c1955cf2d 1 2011-09-15 22:35:54 600000000 33.3 1HxmXQjcU51CGuQuugjFBL4Aa2VvroqpsF pubkeyhash 76a914ba0e0eb50dce94d5979626f15745ab12e89369eb88ac 0 -1 152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 0 2011-11-11 00:05:55 17.22 4294967295 483045022038b72c52176158ddd3de61eb7b86f905e8983a6c2c7fce35426cba359420649f022100a1fa8f55a281bc2f8440d561814a59be10b10193c2243c949de9574bed992a85014104d2777e3eb09b35a223ec2a5a0b0254282741046976f8765ab07bbee3715c46b8572e7b3143f0de5c615b3de6e24cd491bb52ec84bac50658bcc879a3d8e86bd5 4843801 336.3750694444
142760 1bcd0fa9ce0b1953364cb52015729e913c91f77774d43aa431667c32ae005e03 1 2011-08-27 05:29:51 400000000 34.6 1J5Z4mFHXQbLqehEkzcoa2Qgw2SGWyn6dw pubkeyhash 76a914bb56918ea3dc5b96343ac5b67843a09a1870c4a388ac 0 -1 152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 0 2011-11-11 00:05:55 11.48 4294967295 4830450221009877ac9a00d2182878aa86555e1675f9a939e4cbff68d98edb9f89698d41f4ae02201832f05f5dd2d26e67a7444c85959d53fa23bd761a9caa6e6d4c556f61a05f3b0141046f13594b5b3275d5acc8fc97ba642eda76595e1e891227560ddcd617cc12a7b3cff8081ea4f5c4625f12f20690c6df2594480822162e5fd7dc0606ff6d5747c0 6546964 303.1001851852
152758 4b28cbbfb472316b3ca32772cc2a1d05a87a37db91d951aab72b09a4d3811816 0 2011-11-10 22:58:57 761730212732 22851.9062 141utQD6sbepcubqvm7Xk8hWS71mojJg1Y pubkeyhash 76a91421148647f9b6aa60ab8161fa02cf8600e46b793e88ac 0 -1 152764 d81a2bad4b0dcec1e7fc79adb9143c85ab35c9347aa4b2c5660f0e1432c8e3dd 0 2011-11-11 00:05:55 21861.6562 4294967295 48304502207712beff0dc14506dd154a465f1560522e4577d96b4c16915c134cef4eeeed66022100c69776719e12208d0a33ff71f45bfe0d323ca3a6a7b7decb5ba61647fde41405014104b0807a7e5ab5d740f45c6f95421cfb8db0c5f1548f2bc0d10a15074a2e3691888cf90258257e6274c9a9e6033dfb001709035a52d0a7679c50a5276ade272fe2 4018 354.239814208
152754 b119c44a9c9bee8dec87492d5ce0d96a9784de1e14b26f83eaacbfccecafaaf5 0 2011-11-10 21:46:08 89089420 2.6727 18j82gzKFWJJWEiXx1RyGWUY69rYxCPTxr pubkeyhash 76a91454c037efc87d749eb4d30234c2e0dd73717ee7a588ac 0 -1 152764 d81a2bad4b0dcec1e7fc79adb9143c85ab35c9347aa4b2c5660f0e1432c8e3dd 1 2011-11-11 00:05:55 2.5569 4294967295 48304502206a86cbe79575735c709e303cab2380b6344c702b3b6d47ecb06be0f6ec840f78022100bd9c0510780afd4f14ea540eadf30f84e1897f623da628593ebd9cb61a88504601410490123b018833a24b864517617762eaef3cc80739a901120ec97f4ea02646debfd0b963bdcfb4a378e9ae1820a30cf2322e8d4ee172e76ea567b326897d8b7e93 8387 0.0864806673
152731 575b6a2694542ddc66e14d1c827aaa8924bdc95636869c94f09e70520d8ad4f7 1 2011-11-10 18:00:06 772000000 23.16 1QKaXk2hUDv61aRy5uQViq1Nr41CcttWCU pubkeyhash 76a914ffce5eab41b4213ebe134108331071337984959588ac 0 -1 152764 3815ece30a32cd879bf11fd4a4f6644695b63cf1521e1a1b1f7a9908bf0e8b54 0 2011-11-11 00:05:55 22.1564 4294967295 47304402203ee62439de32dd165c013302159772130792428135a9b4733e32bafafc32ca91022044c736b7883ed1b8c0af94c8393070effd75ee0fa8e62b61887dd99a29fe9336014104d9cd7d986ce7cbf9056ad41cc3bc4f56b6f93fb3a3eb07be16dffc80e521849631f69c8cf5e886682808d35059e27e4eca77fde73b8ebbcc0e33240e09a467eb 21949 1.9611837963

outputs:
Code:
block_id transaction_hash index time value value_usd recipient type script_hex is_from_coinbase is_spendable
152764 498ec7f88857d23bd19370252da439876960dad296a64534987ad54081a9cc39 0 2011-11-11 00:05:55 5001400010 143.5402 17aA19GvhzMHsq8xPwSXAPZutyr6kuzLEB pubkey 4104a8dd3f118a6122c2f8c6be3261670f7af76568cac8ff0ed95e5ef63238018e69b223dbdef9f9d5e94a053ff1afc390e230844c0b71f3648405807cd668979958ac 1 1
152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 0 2011-11-11 00:05:55 1199480 0.0344 1KPxwAbFVoDimPrVECF2zgiyfX9jGW9TCy pubkeyhash 76a914c9ca1b452087cdc6b89754c1090928d7a67ef23988ac 0 -1
152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 1 2011-11-11 00:05:55 990290000 28.4213 1AUzdKPJtPyFgrg88nK6fBH7jKkgPyphUj pubkeyhash 76a91468047787a9dce43acf61af61eb1867a9cb0724b688ac 0 -1
152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 0 2011-11-11 00:05:55 9940000 0.2853 1LVcc51n4Cdd8oKLc8GWEpu2wZzfg82Bpb pubkeyhash 76a914d5d3c21182c74b3c306e9ca9746b40d29971223388ac 0 -1
152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 1 2011-11-11 00:05:55 749060000 21.498 1ByQHoyjyMNn5NznRbSHwYcRf7DHXNL3ih pubkeyhash 76a914785bfe399c2cca278e45d76e34317dfa5941fe9f88ac 0 -1
152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 0 2011-11-11 00:05:55 74916000 2.1501 1JwtyTt2w7xZJgrgPg4rCyaV7DKEa4n5uH pubkeyhash 76a914c4dbfc4dac46dd1ca2cab8af4d431bd422a6d7d688ac 0 -1
152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 1 2011-11-11 00:05:55 525084000 15.0699 14rKLXhpX6Sukznqm5cykcgZMt4qZXamQB pubkeyhash 76a9142a3c10f240c9266f43c110242cbe877868f12ea088ac 0 -1
152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 0 2011-11-11 00:05:55 2824000 0.081 1AVntqwcaMtYVqYuH9f8zXYM1Gnz8QHVWe pubkeyhash 76a914682b16d8f9bad9fcb6993950233fac67f75c55c588ac 0 -1
152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 1 2011-11-11 00:05:55 397176000 11.399 1J9xrxAKgaRR5ddYtot559r7yDwQiS9dk1 pubkeyhash 76a914bc2c18bc245d5a4bc64567bca525741c0a8069d188ac 0 -1

transactions:
Code:
block_id hash time size weight version lock_time is_coinbase has_witness input_count output_count input_total input_total_usd output_total output_total_usd fee fee_usd fee_per_kb fee_per_kb_usd fee_per_kwu fee_per_kwu_usd cdd_total
152764 498ec7f88857d23bd19370252da439876960dad296a64534987ad54081a9cc39 2011-11-11 00:05:55 135 540 1 0 1 0 1 1 0 0 5001400010 143.5402 0 0 0 0 0 0 0
152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 2011-11-11 00:05:55 439 1756 1 0 0 0 2 2 991489480 28.4557 991489480 28.4557 0 0 0 0 0 0 819.4471476975
152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 2011-11-11 00:05:55 439 1756 1 0 0 0 2 2 759000000 21.7833 759000000 21.7833 0 0 0 0 0 0 598.80244212963
152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 2011-11-11 00:05:55 258 1032 1 0 0 0 1 2 600000000 17.22 600000000 17.22 0 0 0 0 0 0 336.37506944444
152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 2011-11-11 00:05:55 258 1032 1 0 0 0 1 2 400000000 11.48 400000000 11.48 0 0 0 0 0 0 303.10018518519
152764 d81a2bad4b0dcec1e7fc79adb9143c85ab35c9347aa4b2c5660f0e1432c8e3dd 2011-11-11 00:05:55 438 1752 1 0 0 0 2 2 761819302152 21864.2129 761819302152 21864.2129 0 0 0 0 0 0 0
152764 3815ece30a32cd879bf11fd4a4f6644695b63cf1521e1a1b1f7a9908bf0e8b54 2011-11-11 00:05:55 436 1744 1 0 0 0 2 2 10771000000 309.1277 10771000000 309.1277 0 0 0 0 0 0 140.63292604167
152764 09a4a4d5706d7123c13a8128040a50e00a79d4d369b3618f7b6140fe60e9c89c 2011-11-11 00:05:55 1519 6076 1 0 0 0 8 2 78001010386 2238.6289 78001010386 2238.6289 0 0 0 0 0 0 245.16466314468
152764 1b80bdad1902800146ef1a48665c4f13e409442feda0f9f442627aa6c3479297 2011-11-11 00:05:55 257 1028 1 0 0 0 1 2 43999811790 1262.7946 43999811780 1262.7946 10 0 38.9105 0 9.72763 0 0

block_hash_version_versionHex_merkleroot_time_mediantime_nonce_bits_difficulty_ chainwork_nTx_strippedsize_size_weight.tsv.gz:
Code:
block hash version versionHex merkleroot time mediantime nonce bits difficulty chainwork nTx strippedsize size weight
0 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f 1 00000001 4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b 1231006505 1231006505 2083236893 1d00ffff 1 0000000000000000000000000000000000000000000000000000000100010001 1 285 285 1140
1 00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048 1 00000001 0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098 1231469665 1231469665 2573394689 1d00ffff 1 0000000000000000000000000000000000000000000000000000000200020002 1 215 215 860
2 000000006a625f06636b8bb6ac7b960a8d03705d1ace08b1a19da3fdcc99ddbd 1 00000001 9b0fc92260312ce44e74ef369f5c66bbb85848f2eddd5a7a1cde251e54ccfdd5 1231469744 1231469665 1639830024 1d00ffff 1 0000000000000000000000000000000000000000000000000000000300030003 1 215 215 860
3 0000000082b5015589a3fdf2d4baff403e6f0be035a5d9742c1cae6295464449 1 00000001 999e1c837c76a1b7fbb7e57baf87b309960f5ffefbf2a9b95dd890602272f644 1231470173 1231469744 1844305925 1d00ffff 1 0000000000000000000000000000000000000000000000000000000400040004 1 215 215 860
4 000000004ebadb55ee9096c9a2f8880e09da59c0d68b1c228da88e48844a1485 1 00000001 df2b060fa2e5e9c8ed5eaf6a45c13753ec8c63282b2688322eba40cd98ea067a 1231470988 1231469744 2850094635 1d00ffff 1 0000000000000000000000000000000000000000000000000000000500050005 1 215 215 860
......
757434 000000000000000000029d11a2ff7d9446a7de7e575e50a79b0c83de37783f0b 805298176 2fffe000 c418d5a0e6322ed776887941f110cbf8b7a44cd2831b369413d193a76c3dfdf4 1665098543 1665096510 73455270 1708f9ae 31360548173144.85 000000000000000000000000000000000000000036907e07c3f7165e0e06d7a8 1464 386366 699282 1858380
757435 0000000000000000000136d2e367c81f36bd4c1e320231ed5d501b4e654151a1 1073733636 3fffe004 1fa72363a7ad010bcf30e56ce33ba558f23526d00117ab0793503eac2424418f 1665099208 1665097058 861948933 1708f9ae 31360548173144.85 000000000000000000000000000000000000000036909a8d92d25a922c67bb40 1680 529683 957930 2546979
757436 000000000000000000049c9f9176c2544b2d429d9ddc4d5461aeaa379fafcf8d 671080448 27ffe000 cd28bf0a6fca605d917ce3fee5dc0d97bcd44eeef8897e61b4dfe840d183cb8e 1665100292 1665097881 2098456665 1708f9ae 31360548173144.85 00000000000000000000000000000000000000003690b71361ad9ec64ac89ed8 2142 757447 1721126 3993467
757437 00000000000000000007d55451ffad86e079bf9df2db6314fcfa8eed609df8eb 541065216 20400000 091b86fb517f368ee8b1c4060b3908e5ff185db863aae9915c1c235584e7e7a3 1665100530 1665097963 426079596 1708f9ae 31360548173144.85 00000000000000000000000000000000000000003690d3993088e2fa69298270 1133 342043 658547 1684676

Bandwidth
This server is allowed 50 TB bandwidth per month. After that, I don't know what happens, but so far I'm far from this limit.

Credits
Blockchair Database Dumps has a staggering amount of data, easily accessible (at 10 100 kB/s) with daily updates. All data in this topic comes from Blockchair.

No spam please.
Self-moderated against spam. Discussion and questions are welcome.



Related topics
Bitcoin block data available in CSV format
List of all Bitcoin addresses with a balance
List of all Bitcoin addresses ever used
[~500 GB] Bitcoin block data: inputs, outputs and transactions
[800 GB] Ethereum data (see next post)
Pages:
Jump to: