Pages:
Author

Topic: Bitcoin block data (728 GB): inputs, outputs and transactions - page 6. (Read 3204 times)

legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Loyce do you know is any other webpage than blockchair.com for this?
Nope Sad
What data are you looking for exactly?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Loyce as I see the server for this is still unavaible?
Unfortunately, yes Sad

It looks like Chia mining killed cheap storage VPS servers.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
do you have any disk space or bandwidth/s quota on the server hosting this, and is it shared with your other projects?
I get 1 TB disk space and 10 TB bandwidth per month. It's not shared with my other projects (but it is the only server I have that can handle this much data).
So far, it's perfect for this project Smiley
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Be kind
Please only download with one thread at a time. I don't want to be kicked off another server.

Just curious, do you have any disk space or bandwidth/s quota on the server hosting this, and is it shared with your other projects?
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I was taking a look at input files to try to make some analysis. But the files are huge, which is a problem for my current set up
You can instantly remove the data you don't need, that makes it a bit smaller.

Update:
Files are complete now:
I've activated daily updates.
legendary
Activity: 2352
Merit: 6089
bitcoindata.science
That's assuming you already have the >320 GB blockchain downloaded by your own running node. Not everyone runs a full node and data like the one provided by blockchair.com and mirrored by OP is not supposed to be used by nodes or for anything security sensitive. It is just data useful for data analysis.

I was taking a look at input files to try to make some analysis. But the files are huge, which is a problem for my current set up (HD size and bandwidth).

My first attempt is to group inputs by transaction ID, then group addresses which were spend together. Creating wallets... Just like walletexplorer.com does (but it would be amazing if I could do this myself).

For example, I know that these 2 addresses belongs to the same wallet:


But this is a much more complicated job than I expected, as files are very large.

This analysis would expose privacy weakness. This kind of analysis is very easy to do for few addresses (if someone is being investigated), and people are not aware of that.
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
I just notice the dump have both inputs and outputs, but what's the difference? Which one is UTXO in this case?

I'll leave that to the reader. You're free to download and convert to whatever format best fits your needs.

Do not see any reasons to download blockchain from half-trusted third-party side. This is contrary to blockchain ideology.

I understand your point, but no one force you to use this dataset.

P.S. There's similar discussion at Sync your blockchain easily!.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I just notice the dump have both inputs and outputs, but what's the difference? Which one is UTXO in this case?
I didn't create this format Tongue I'll show some samples:
inputs:
Code:
block_id transaction_hash index time value value_usd recipient type script_hex is_from_coinbase is_spendable spending_block_id spending_transaction_hash spending_index spending_time spending_value_usd spending_sequence spending_signature_hex spending_witness lifespan cdd
141719 3c86cc88726f38d2f1e861850b143fe8d710e171a91291628ffe6418d8b60190 1 2011-08-20 06:12:23 990322200 112.8967 1L2wqJ4z4pjXpqP4EFudnn7TKE4sjksQQj pubkeyhash 76a914d0c893ee2fa4278a2542691e2f569247d833805988ac 0 -1 152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 0 2011-11-11 00:05:55 28.4222 4294967295 4830450220015c796fc6ddf0040c70115539d476426f5c69e1537bb8b7da90d4d671c564d302210094fc145108b617352db33880118a69c059055538af7472ae60a5f4b37d558a060141048396aebc72f8a7b26ea0f676c7c8b9379b0a93b487058787f78b2d256d2a4a6e37d7df1ac8314ee09c1f9a57d48f8c07fd1e4e3056bb17c0075ec703599cada9 7149212 819.4471476975
152763 5733d4a453dcea2690bdc8e8866f5788adadca82ba4639a3dada3d26f738ae9a 0 2011-11-10 23:53:08 1167280 0.035 1JFVo6cGL3cpyXZctC5Tw3HHvmAzuwkjSj pubkeyhash 76a914bd37ff9aa142d5ed87b1f9385c66b856b6c60e6088ac 0 -1 152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 1 2011-11-11 00:05:55 0.0335 4294967295 493046022100ba9fa562476163f5eaac9a6bbee123a0c3f1421377c35208af4b9606816603d1022100c1a797af43d8297f08011248c16f01ee9e95d03ff5d527c2ace2adb545eb88de0141047d949c30d6ffffe8e5ecd2ee774745bda2563e4842b7a9836bd0be8461786a2cceb1d844b0789afde927eace479955f9c27ff98e0e5afe71d4dde54e621aaa1f 767 0.0001036231
141326 5c51c88388d3452c0da455ad03ee50ca121ab4812d8790402527df74cf327db1 0 2011-08-17 11:03:42 700000000 77.35 17AqNueHmeFbk6rxnpvQM3QGFJM9oAs1XY pubkeyhash 76a91443ace1cda1b0aede06ca68bc0f60f4c11ecbfdc288ac 0 -1 152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 0 2011-11-11 00:05:55 20.09 4294967295 493046022100b848925df3d18baa10f26b169bd917927c0787aa5719d3e2811176ec66e8297f022100abf4688ed1b744fd3d64e2263137feb2b057ff93d35d8e8e8ddb13fee49c40780141049042d124b779c07a9c825fd572a5002c7a50efc1fea451ad390308196f75ea7b0747c86dbfa369400b0b8d7c1965999d3fe0ad2a4f87afd96c90a5d3ed2c5ecb 7390933 598.8024421296
152754 6b5cd836c52d17a1390e1a9533303941ad23e497f0b508922f1dc55c00d293f0 0 2011-11-10 21:46:08 59000000 1.77 1KGBGWUVbVpvW6Sw7GBVsjS2NS8uWbMsgM pubkeyhash 76a914c85113063d2c642f0566d605572581075dd7bd7188ac 0 -1 152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 1 2011-11-11 00:05:55 1.6933 4294967295 483045022100a4a8ce69ccbee08899a0c5693c1c43ddac5e4c187093fd7bd1efdcea70a92c3502200fe64020f94229976febf8427d4b1510187f500f95a004a4a7be498b0439c565014104175747a43f4f51bcb9f1753398157bf5b02cd9b239ea11b544d95b3f7c81881c46c0a86b57e445f0e803c630dc1f51e7b6f7d59f36b408509cd1b3cdda918625 8387 0.057272338
145520 f49804f14a44a7a478a870635662df8b6b089317c2e878b701c6ab4c1955cf2d 1 2011-09-15 22:35:54 600000000 33.3 1HxmXQjcU51CGuQuugjFBL4Aa2VvroqpsF pubkeyhash 76a914ba0e0eb50dce94d5979626f15745ab12e89369eb88ac 0 -1 152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 0 2011-11-11 00:05:55 17.22 4294967295 483045022038b72c52176158ddd3de61eb7b86f905e8983a6c2c7fce35426cba359420649f022100a1fa8f55a281bc2f8440d561814a59be10b10193c2243c949de9574bed992a85014104d2777e3eb09b35a223ec2a5a0b0254282741046976f8765ab07bbee3715c46b8572e7b3143f0de5c615b3de6e24cd491bb52ec84bac50658bcc879a3d8e86bd5 4843801 336.3750694444
142760 1bcd0fa9ce0b1953364cb52015729e913c91f77774d43aa431667c32ae005e03 1 2011-08-27 05:29:51 400000000 34.6 1J5Z4mFHXQbLqehEkzcoa2Qgw2SGWyn6dw pubkeyhash 76a914bb56918ea3dc5b96343ac5b67843a09a1870c4a388ac 0 -1 152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 0 2011-11-11 00:05:55 11.48 4294967295 4830450221009877ac9a00d2182878aa86555e1675f9a939e4cbff68d98edb9f89698d41f4ae02201832f05f5dd2d26e67a7444c85959d53fa23bd761a9caa6e6d4c556f61a05f3b0141046f13594b5b3275d5acc8fc97ba642eda76595e1e891227560ddcd617cc12a7b3cff8081ea4f5c4625f12f20690c6df2594480822162e5fd7dc0606ff6d5747c0 6546964 303.1001851852
152758 4b28cbbfb472316b3ca32772cc2a1d05a87a37db91d951aab72b09a4d3811816 0 2011-11-10 22:58:57 761730212732 22851.9062 141utQD6sbepcubqvm7Xk8hWS71mojJg1Y pubkeyhash 76a91421148647f9b6aa60ab8161fa02cf8600e46b793e88ac 0 -1 152764 d81a2bad4b0dcec1e7fc79adb9143c85ab35c9347aa4b2c5660f0e1432c8e3dd 0 2011-11-11 00:05:55 21861.6562 4294967295 48304502207712beff0dc14506dd154a465f1560522e4577d96b4c16915c134cef4eeeed66022100c69776719e12208d0a33ff71f45bfe0d323ca3a6a7b7decb5ba61647fde41405014104b0807a7e5ab5d740f45c6f95421cfb8db0c5f1548f2bc0d10a15074a2e3691888cf90258257e6274c9a9e6033dfb001709035a52d0a7679c50a5276ade272fe2 4018 354.239814208
152754 b119c44a9c9bee8dec87492d5ce0d96a9784de1e14b26f83eaacbfccecafaaf5 0 2011-11-10 21:46:08 89089420 2.6727 18j82gzKFWJJWEiXx1RyGWUY69rYxCPTxr pubkeyhash 76a91454c037efc87d749eb4d30234c2e0dd73717ee7a588ac 0 -1 152764 d81a2bad4b0dcec1e7fc79adb9143c85ab35c9347aa4b2c5660f0e1432c8e3dd 1 2011-11-11 00:05:55 2.5569 4294967295 48304502206a86cbe79575735c709e303cab2380b6344c702b3b6d47ecb06be0f6ec840f78022100bd9c0510780afd4f14ea540eadf30f84e1897f623da628593ebd9cb61a88504601410490123b018833a24b864517617762eaef3cc80739a901120ec97f4ea02646debfd0b963bdcfb4a378e9ae1820a30cf2322e8d4ee172e76ea567b326897d8b7e93 8387 0.0864806673
152731 575b6a2694542ddc66e14d1c827aaa8924bdc95636869c94f09e70520d8ad4f7 1 2011-11-10 18:00:06 772000000 23.16 1QKaXk2hUDv61aRy5uQViq1Nr41CcttWCU pubkeyhash 76a914ffce5eab41b4213ebe134108331071337984959588ac 0 -1 152764 3815ece30a32cd879bf11fd4a4f6644695b63cf1521e1a1b1f7a9908bf0e8b54 0 2011-11-11 00:05:55 22.1564 4294967295 47304402203ee62439de32dd165c013302159772130792428135a9b4733e32bafafc32ca91022044c736b7883ed1b8c0af94c8393070effd75ee0fa8e62b61887dd99a29fe9336014104d9cd7d986ce7cbf9056ad41cc3bc4f56b6f93fb3a3eb07be16dffc80e521849631f69c8cf5e886682808d35059e27e4eca77fde73b8ebbcc0e33240e09a467eb 21949 1.9611837963

outputs:
Code:
block_id transaction_hash index time value value_usd recipient type script_hex is_from_coinbase is_spendable
152764 498ec7f88857d23bd19370252da439876960dad296a64534987ad54081a9cc39 0 2011-11-11 00:05:55 5001400010 143.5402 17aA19GvhzMHsq8xPwSXAPZutyr6kuzLEB pubkey 4104a8dd3f118a6122c2f8c6be3261670f7af76568cac8ff0ed95e5ef63238018e69b223dbdef9f9d5e94a053ff1afc390e230844c0b71f3648405807cd668979958ac 1 1
152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 0 2011-11-11 00:05:55 1199480 0.0344 1KPxwAbFVoDimPrVECF2zgiyfX9jGW9TCy pubkeyhash 76a914c9ca1b452087cdc6b89754c1090928d7a67ef23988ac 0 -1
152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 1 2011-11-11 00:05:55 990290000 28.4213 1AUzdKPJtPyFgrg88nK6fBH7jKkgPyphUj pubkeyhash 76a91468047787a9dce43acf61af61eb1867a9cb0724b688ac 0 -1
152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 0 2011-11-11 00:05:55 9940000 0.2853 1LVcc51n4Cdd8oKLc8GWEpu2wZzfg82Bpb pubkeyhash 76a914d5d3c21182c74b3c306e9ca9746b40d29971223388ac 0 -1
152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 1 2011-11-11 00:05:55 749060000 21.498 1ByQHoyjyMNn5NznRbSHwYcRf7DHXNL3ih pubkeyhash 76a914785bfe399c2cca278e45d76e34317dfa5941fe9f88ac 0 -1
152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 0 2011-11-11 00:05:55 74916000 2.1501 1JwtyTt2w7xZJgrgPg4rCyaV7DKEa4n5uH pubkeyhash 76a914c4dbfc4dac46dd1ca2cab8af4d431bd422a6d7d688ac 0 -1
152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 1 2011-11-11 00:05:55 525084000 15.0699 14rKLXhpX6Sukznqm5cykcgZMt4qZXamQB pubkeyhash 76a9142a3c10f240c9266f43c110242cbe877868f12ea088ac 0 -1
152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 0 2011-11-11 00:05:55 2824000 0.081 1AVntqwcaMtYVqYuH9f8zXYM1Gnz8QHVWe pubkeyhash 76a914682b16d8f9bad9fcb6993950233fac67f75c55c588ac 0 -1
152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 1 2011-11-11 00:05:55 397176000 11.399 1J9xrxAKgaRR5ddYtot559r7yDwQiS9dk1 pubkeyhash 76a914bc2c18bc245d5a4bc64567bca525741c0a8069d188ac 0 -1

transactions:
Code:
block_id hash time size weight version lock_time is_coinbase has_witness input_count output_count input_total input_total_usd output_total output_total_usd fee fee_usd fee_per_kb fee_per_kb_usd fee_per_kwu fee_per_kwu_usd cdd_total
152764 498ec7f88857d23bd19370252da439876960dad296a64534987ad54081a9cc39 2011-11-11 00:05:55 135 540 1 0 1 0 1 1 0 0 5001400010 143.5402 0 0 0 0 0 0 0
152764 5cc203b9389f0f7eb50669eba04ac32d666892fc95f07c25603da8a6ed9316ae 2011-11-11 00:05:55 439 1756 1 0 0 0 2 2 991489480 28.4557 991489480 28.4557 0 0 0 0 0 0 819.4471476975
152764 e0105a83ce04598fe208f1910643a4f75951e9f95f40a0e88ac697d0ec72679d 2011-11-11 00:05:55 439 1756 1 0 0 0 2 2 759000000 21.7833 759000000 21.7833 0 0 0 0 0 0 598.80244212963
152764 6f2ddb0352071a02e89f9ed9bfaed33f0c1dc2fadd97b6f3a34e76bfe3ea6aed 2011-11-11 00:05:55 258 1032 1 0 0 0 1 2 600000000 17.22 600000000 17.22 0 0 0 0 0 0 336.37506944444
152764 0a3c1eb28c3003405ad745a5483ff25c583ac51f93bdbcd761e444ac8639f919 2011-11-11 00:05:55 258 1032 1 0 0 0 1 2 400000000 11.48 400000000 11.48 0 0 0 0 0 0 303.10018518519
152764 d81a2bad4b0dcec1e7fc79adb9143c85ab35c9347aa4b2c5660f0e1432c8e3dd 2011-11-11 00:05:55 438 1752 1 0 0 0 2 2 761819302152 21864.2129 761819302152 21864.2129 0 0 0 0 0 0 0
152764 3815ece30a32cd879bf11fd4a4f6644695b63cf1521e1a1b1f7a9908bf0e8b54 2011-11-11 00:05:55 436 1744 1 0 0 0 2 2 10771000000 309.1277 10771000000 309.1277 0 0 0 0 0 0 140.63292604167
152764 09a4a4d5706d7123c13a8128040a50e00a79d4d369b3618f7b6140fe60e9c89c 2011-11-11 00:05:55 1519 6076 1 0 0 0 8 2 78001010386 2238.6289 78001010386 2238.6289 0 0 0 0 0 0 245.16466314468
152764 1b80bdad1902800146ef1a48665c4f13e409442feda0f9f442627aa6c3479297 2011-11-11 00:05:55 257 1028 1 0 0 0 1 2 43999811790 1262.7946 43999811780 1262.7946 10 0 38.9105 0 9.72763 0 0

Quote
P.S. There's similar discussion at Sync your blockchain easily!.
That's different, the data in my topic isn't meant to be used by a wallet for transactions.
copper member
Activity: 1554
Merit: 489
Stop the war!
If I will need a blockchain in text format then I will easy to convert it from JSON format provided by native bitcoin RPC-API And I will be 100% sure of the security of this data
Can I challenge you to do it, and verify Blockchair's data?
Hmm. Just for fun?
Not. Only if I will see some benefit in it
legendary
Activity: 3472
Merit: 10611
Do not see any reasons to download blockchain from half-trusted third-party side. This is contrary to blockchain ideology.
If I will need a blockchain in text format then I will easy to convert it from JSON format provided by native bitcoin RPC-API And I will be 100% sure of the security of this data
That's assuming you already have the >320 GB blockchain downloaded by your own running node. Not everyone runs a full node and data like the one provided by blockchair.com and mirrored by OP is not supposed to be used by nodes or for anything security sensitive. It is just data useful for data analysis.
For example the ~800 MB file containing addresses with balance is useful in a lot of recovery cases as a database to search against where the user doesn't remember their address or not all of it.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
If I will need a blockchain in text format then I will easy to convert it from JSON format provided by native bitcoin RPC-API And I will be 100% sure of the security of this data
Can I challenge you to do it, and verify Blockchair's data?
copper member
Activity: 1554
Merit: 489
Stop the war!
I'll leave that to the reader. You're free to download and convert to whatever format best fits your needs.

Do not see any reasons to download blockchain from half-trusted third-party side. This is contrary to blockchain ideology.
If I will need a blockchain in text format then I will easy to convert it from JSON format provided by native bitcoin RPC-API And I will be 100% sure of the security of this data
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Can you create a sample of each file, with just a few MB, so people can see the that and how it is organized before downloading 500GB?
Lol Tongue You didn't click the links, now did you? There are thousands of files (one for each day), so pick a sample Wink

I think the best format is a relational database. SQLite for example.
I'll leave that to the reader. You're free to download and convert to whatever format best fits your needs.
copper member
Activity: 1554
Merit: 489
Stop the war!
I never saw the blockchain in CSV format, which is a easy to use format for data analysis.
Will you analize 500 GB data by-hand?  Shocked
I think the best format is a relational database. SQLite for example. But it is not exactly  Smiley
legendary
Activity: 2352
Merit: 6089
bitcoindata.science
The data
See blockdata.loyce.club
blockdata.loyce.club/inputs/ (159 GB - currently incomplete, growing 8 GB per day)
blockdata.loyce.club/outputs/ (117 GB)
blockdata.loyce.club/transactions/ (43 GB)

Hello LoyceV.

I am curious this file.

Can you create a sample of each file, with just a few MB, so people can see the that and how it is organized before downloading 500GB?

What is the problem in to make your own blockchain parser?
This is easier Smiley

I never saw the blockchain in CSV format, which is a easy to use format for data analysis. Is there anywhere else where you can download the blockchain in csv format other than in loyce.club and blockchair?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
What is the problem in to make your own blockchain parser?
This is easier Smiley
And I don't mind the fact that each file can be verified at an independent third party.
copper member
Activity: 1554
Merit: 489
Stop the war!
Quote
I've used Blockchair Database Dumps for several projects, but when I need it I don't want to wait weeks to download all files. So I created a mirror for Bitcoin data
What is the problem in to make your own blockchain parser?
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
October 7, 2022: I've consolidated a few posts here, so that I can remove Vod's post (sorry Vod) and have the second post in this topic for myself.

Credits
Blockchair Database Dumps has a staggering amount of data, easily accessible (at 10 100 kB/s) with daily updates. All data in this topic comes from Blockchair.
Spam discussion/question.    The page mentions 10 kb/s - is there any cost or requirement for the 100?

The page mentions 10 kb/s - is there any cost or requirement for the 100?
At some point they increased the speed limit, but didn't update their Readme.
The fool speed is a tad above my budget for this:
I just contacted Blockchair for an API key, which enables people to download at the fast rate, and a support rep told me they cost $500/month.

Bump:
blockdata.loyce.club/inputs/ (159 GB - currently incomplete, growing 8 GB per day)
This directory is now 349 GB in size. It's almost complete Smiley
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Ethereum data

I've added ethereumdata.loyce.club Abandoned! I'm out of disk space. I have a local backup, if you really need this: ask!
ethereumdata.loyce.club/calls/ (345 GB)
ethereumdata.loyce.club/erc-20_transactions/ (378 GB)
ethereumdata.loyce.club/transactions/ (71 GB)

All funded Ethereum addresses: ethereumdata.loyce.club/blockchair_ethereum_addresses_latest.tsv.gz (5.1 GB).  Blockchair doesn't updated this file regularly, so I don't do automatic updates.

Disclaimer
I don't like Ethereum, which is a centralized shitcoin that abandoned it's one unique selling point ("code is law") the moment it was convenient for the creator. Don't waste your money on it!
And since it's a shitcoin, I don't think it deserves it's own topic.

Missing files
Those 2 files are missing from my mirror because they're corrupted:
https://gz.blockchair.com/ethereum/calls/blockchair_ethereum_calls_20200113.tsv.gz
https://gz.blockchair.com/ethereum/calls/blockchair_ethereum_calls_20211110.tsv.gz
If anyone has the tools to rebuild those 2 files, I'd love to add them!
Pages:
Jump to: