Bitcoin Forum>Bitcoin>Project Development
Pages:
Author

Topic: List of all Bitcoin addresses with a balance - page 5. (Read 9201 times)

DeepComplex
newbie
Activity: 24
Merit: 9
Re: List of all Bitcoin addresses with a balance
April 18, 2022, 05:14:41 PM
#141
LoyceV,

Thanks for the list. The list is good.


Can you say how you've compiled it, scripts or Linux commands, etc?
did you pull it from the blockchain, utxo?

I would like to learn how to manipulate the data as you did.


Regards,


Quote from: LoyceV on April 17, 2022, 01:16:33 PM
It took me longer than I expected (because it was more work). Next time it's easy Cheesy

Quote from: DeepComplex on April 16, 2022, 01:22:55 PM
Based on my what I've seen on the block chain, p2pk addresses were used up to block 278,298.
Searching through the first 278,298 blocks (about 1821 daily "outputs" data dumps), I get 901,980 lines that contain the word pubkey. Without duplicate addresses, it's 203,283.
If I search all data, I get 3,669,383 lines. I don't think any address type ever stopped being used, it's still backwards compatible.
If I check how many of those addresses currently have a balance, I get 39,208 from the shortest list. The full list has 39,979 addresses. I'll continue with this one.

I'm not sure why the hash length varies this much:
Code:
367569  c59bd606c40e325841f17d1365d917f953507040ccb540b351967a402999a02e        2015-07-30 00:50:12     2450    0.0072  1aajpztNecNzGBDjJhGfkrsUJFgeyhBgY       pubkey  2103d68f90ba81455256cb7a0df14fb3930d6df61393207f2f3e71659414d296e0f0ac  0       1
367631  72f822c0a805d88706a5d696a1715d9e08006dcc2433a278a4a91f41db587b75        2015-07-30 12:50:57     70000   0.2048  1aajpztNecNzGBDjJhGfkrsUJFgeyhBgY       pubkey  2103d68f90ba81455256cb7a0df14fb3930d6df61393207f2f3e71659414d296e0f0ac  0       1
367679  3fecf7434af30ca54fdadf03dd7e12c52fcdc753f82afe1f545ebf5fc04388fb        2015-07-30 20:53:11     2435048 7.1252  1Kz5QaUPDtKrj5SqW5tFkn7WZh8LmQaQi4      pubkey  4104ffd03de44a6e11b9917f3a29f9443283d9871c9d743ef30d5eddcd37094b64d1b3d8090496b53256786bf5c82932ec23c3b74d9f05a6f95a8b5529352656664bac  1       1
367699  2bb1cf36f7004de8b93978e1b57a924e9e410ba5f1ac1fcf42abe64d9b51decb        2015-07-31 00:54:39     2340276 6.7798  1Kz5QaUPDtKrj5SqW5tFkn7WZh8LmQaQi4      pubkey  4104ffd03de44a6e11b9917f3a29f9443283d9871c9d743ef30d5eddcd37094b64d1b3d8090496b53256786bf5c82932ec23c3b74d9f05a6f95a8b5529352656664bac  1       1
Maybe it's the difference between compressed and uncompressed addresses?

Final result: balance_addy_pubkey.txt (6.5 MB). It's sorted by balance, highest first.
Sample:
Code:
323317181009 1PTYXwamXXgQoAhDbmUf98rY2Pg1pYXhin 4104633280c0a93b45217059013ddadab8d35b9a858336028fecdff64c6a5e068fadaf7d2b73bc22795fa160c2304703320516e1b0b20e43d613fa5975787c8287e4ac
87500062957 16mEzobs4wQPuAMq1C8QSQafcDHvzczVcs 410469119c7e7d8e8b9f404d70c36cc09cae033e86d259554fd945c9263560fdc9ddea108db86e5673b3894b0bf97807ff8ac6e322791cf8d4d22f7c55a0e812720aac
80000077848 1P2ZAuW9nUrFfwgVjfL2SA9sPXSruCfzp8 4104a9c9c642db7941be7d9d467786c05b66d13c4fdafeb81d66815fb336e7e19d42981f62f2624826898152444b99b590476c90ea7f2e15f50a00e726368d6ec7c1ac
75000064054 15UkFYLMs5nytwiKWqGgkkVo1fjLFAeJhs 41044da39d1810bb01f7039fa7cdfa3dca6de313bbf2d9db9f24170e87b0206f079adb64cccc5cdf6283a54826adaf265087b3f8eba02577153aeaa6cdea8165d6dfac
55000168944 15BKWJjL5YWXtaP449WAYqVYZQE1szicTn 41042e3d29ccb78680a0da35bac8dc7adf895feb2b8eb9dbc5058400d811f6d7ce46faf0099ef63a05f3a8340e7e95530515303d652f27c2d9b379174b41bfd7046bac
.........
.........
.........
1 13P4fnmSVrHa7g6SiYh6mYF8ucB273TqXR 210354e528107cddb35020d91d28549818cc6a0ac57627eec88d48e248ce381fed98ac
1 13DAZmeDhHreHYTqeY7ZdpZt8oW84BwXEQ 2102fd985d1b8352ac5ae0426d4037cfdc34b6d71043e765ef2879d0d6c00aa88285ac
1 12fBP1Dck3pLUqGG7czQqVKAGHNwUCBZxT 21038b9bdf9d80e0e5f40880fc26d425700cbd71de87b232b2cffb6fecf97fa70214ac
1 12f7dZYL1pTkB7smfU6jf6hNhzP3sTASso 21020454e3b33d1df7037ab7767665219bfe9a99ff4aea5672f3a1e9379ef9703a16ac
1 12d9tPargbPamzfhxEh4wdh5jauA5dtobM 210375106eb0bdf4ed504356e8debd010050c57f9c5a03cea95a8dc6d0d6f0cd48ccac
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
April 17, 2022, 01:16:33 PM
#140
It took me longer than I expected (because it was more work). Next time it's easy Cheesy

Quote from: DeepComplex on April 16, 2022, 01:22:55 PM
Based on my what I've seen on the block chain, p2pk addresses were used up to block 278,298.
Searching through the first 278,298 blocks (about 1821 daily "outputs" data dumps), I get 901,980 lines that contain the word pubkey. Without duplicate addresses, it's 203,283.
If I search all data, I get 3,669,383 lines. I don't think any address type ever stopped being used, it's still backwards compatible.
If I check how many of those addresses currently have a balance, I get 39,208 from the shortest list. The full list has 39,979 addresses. I'll continue with this one.

I'm not sure why the hash length varies this much:
Code:
367569  c59bd606c40e325841f17d1365d917f953507040ccb540b351967a402999a02e        2015-07-30 00:50:12     2450    0.0072  1aajpztNecNzGBDjJhGfkrsUJFgeyhBgY       pubkey  2103d68f90ba81455256cb7a0df14fb3930d6df61393207f2f3e71659414d296e0f0ac  0       1
367631  72f822c0a805d88706a5d696a1715d9e08006dcc2433a278a4a91f41db587b75        2015-07-30 12:50:57     70000   0.2048  1aajpztNecNzGBDjJhGfkrsUJFgeyhBgY       pubkey  2103d68f90ba81455256cb7a0df14fb3930d6df61393207f2f3e71659414d296e0f0ac  0       1
367679  3fecf7434af30ca54fdadf03dd7e12c52fcdc753f82afe1f545ebf5fc04388fb        2015-07-30 20:53:11     2435048 7.1252  1Kz5QaUPDtKrj5SqW5tFkn7WZh8LmQaQi4      pubkey  4104ffd03de44a6e11b9917f3a29f9443283d9871c9d743ef30d5eddcd37094b64d1b3d8090496b53256786bf5c82932ec23c3b74d9f05a6f95a8b5529352656664bac  1       1
367699  2bb1cf36f7004de8b93978e1b57a924e9e410ba5f1ac1fcf42abe64d9b51decb        2015-07-31 00:54:39     2340276 6.7798  1Kz5QaUPDtKrj5SqW5tFkn7WZh8LmQaQi4      pubkey  4104ffd03de44a6e11b9917f3a29f9443283d9871c9d743ef30d5eddcd37094b64d1b3d8090496b53256786bf5c82932ec23c3b74d9f05a6f95a8b5529352656664bac  1       1
Maybe it's the difference between compressed and uncompressed addresses?

Final result: balance_addy_pubkey.txt (6.5 MB). It's sorted by balance, highest first.
Sample:
Code:
323317181009 1PTYXwamXXgQoAhDbmUf98rY2Pg1pYXhin 4104633280c0a93b45217059013ddadab8d35b9a858336028fecdff64c6a5e068fadaf7d2b73bc22795fa160c2304703320516e1b0b20e43d613fa5975787c8287e4ac
87500062957 16mEzobs4wQPuAMq1C8QSQafcDHvzczVcs 410469119c7e7d8e8b9f404d70c36cc09cae033e86d259554fd945c9263560fdc9ddea108db86e5673b3894b0bf97807ff8ac6e322791cf8d4d22f7c55a0e812720aac
80000077848 1P2ZAuW9nUrFfwgVjfL2SA9sPXSruCfzp8 4104a9c9c642db7941be7d9d467786c05b66d13c4fdafeb81d66815fb336e7e19d42981f62f2624826898152444b99b590476c90ea7f2e15f50a00e726368d6ec7c1ac
75000064054 15UkFYLMs5nytwiKWqGgkkVo1fjLFAeJhs 41044da39d1810bb01f7039fa7cdfa3dca6de313bbf2d9db9f24170e87b0206f079adb64cccc5cdf6283a54826adaf265087b3f8eba02577153aeaa6cdea8165d6dfac
55000168944 15BKWJjL5YWXtaP449WAYqVYZQE1szicTn 41042e3d29ccb78680a0da35bac8dc7adf895feb2b8eb9dbc5058400d811f6d7ce46faf0099ef63a05f3a8340e7e95530515303d652f27c2d9b379174b41bfd7046bac
.........
.........
.........
1 13P4fnmSVrHa7g6SiYh6mYF8ucB273TqXR 210354e528107cddb35020d91d28549818cc6a0ac57627eec88d48e248ce381fed98ac
1 13DAZmeDhHreHYTqeY7ZdpZt8oW84BwXEQ 2102fd985d1b8352ac5ae0426d4037cfdc34b6d71043e765ef2879d0d6c00aa88285ac
1 12fBP1Dck3pLUqGG7czQqVKAGHNwUCBZxT 21038b9bdf9d80e0e5f40880fc26d425700cbd71de87b232b2cffb6fecf97fa70214ac
1 12f7dZYL1pTkB7smfU6jf6hNhzP3sTASso 21020454e3b33d1df7037ab7767665219bfe9a99ff4aea5672f3a1e9379ef9703a16ac
1 12d9tPargbPamzfhxEh4wdh5jauA5dtobM 210375106eb0bdf4ed504356e8debd010050c57f9c5a03cea95a8dc6d0d6f0cd48ccac
DeepComplex
newbie
Activity: 24
Merit: 9
Re: List of all Bitcoin addresses with a balance
April 16, 2022, 01:22:55 PM
#139
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
April 16, 2022, 10:44:32 AM
#138
Quote from: DeepComplex on April 16, 2022, 10:07:16 AM
I'm looking for all of the entries with script-type pubkey (P2PK) only and is unspent. Primarly between blocks 0 to 278,298. It is about ~34,300 entries. The fields I would like are: block_id    transaction_hash   time   value   value_usd   recipient   type   script_hex   is_from_coinbase is_spendable.
If I get this from "outputs", you get it for all outputs when they were created. So they're all "spendable" (unless they were used again in the same block), even though most of them were spent later on already.
If you need a list similar to what TheArchaeologist showed, you don't need most of the data you're asking for. Now the output looks like this:
Code:
160345  3abd1b599fe1e69147c6d375d0c85f47999ba99b5ddfbc090370bcb1d1775760        2012-01-03 00:38:50     1375309318      71.6536 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160345  ff0b0816051a55f0e50820f3551df083491114bff0048e214dca75adf52f7506        2012-01-03 00:38:50     2943300000      153.3459        1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160345  61622dff47f8f242ffeeae9f3b382fa975d309be45248af9f044e5350323b014        2012-01-03 00:38:50     1174309318      61.1815 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160345  a91453015e530f092013d24b063d821d46a739b6cddb913bd7f7fd30710b1971        2012-01-03 00:38:50     2780300000      144.8536        1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160345  bcec7c219b25a598750a85c136446d2992ba336f8e2c2c0a2e1c2248adeacef5        2012-01-03 00:38:50     1072309318      55.8673 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  c3d575d5773035585fcd65d9f8f3291e2dc0aa2f52e28f3a6c6c2ad11ffb4e29        2012-01-03 00:39:37     5001000000      260.5521        1CkjArEYChyvBhjrfw69SZXeMbCLooRvyD      pubkey  410408e007dd2c04b76beeb5a17c43ed81a8393174575bd80ba38b82097611f2ea0ff6bea432e69c22d5c67694462dc9d424477f130d55b35c595e8ec36ef774637cac  1       1
160346  7ea972403219e1b29db5b2dc7370ef3ef01317adca2520520f9934e0a35816ae        2012-01-03 00:39:37     2294300000      119.533 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  6f7a8a7733a44710b239f914b902d47013abf768c09a92c3c82448567dae0e57        2012-01-03 00:39:37     968309318       50.4489 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  5e434a0b0b670073b7c3b9983b73f6b71fccf4d88b200205451ada84da776ec0        2012-01-03 00:39:37     2187300000      113.9583        1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  80bdaf438a966115587de07654e271b89bd6c2be4f158d114ff23f9f09295c96        2012-01-03 00:39:37     845309318       44.0406 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  c2827d8e852bb0e0b0938fba7a214ea6387030773e42f524c805ebc622f9225a        2012-01-03 00:39:37     2159300000      112.4995        1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  caedb38e300073a08fa7cb04bf4da34018a8016506eb6c0fac15601ed375ccca        2012-01-03 00:39:37     745309318       38.8306 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  b53520a99ea46c541f8bcb2c3327d439a525ed897928a3771385550312fafdd1        2012-01-03 00:39:37     2142300000      111.6138        1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  0d11d8a4cd909bb7aba2b0e39fa3ee3f26a4253a6ffe34ffb1b948206826aaba        2012-01-03 00:39:37     741309318       38.6222 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  a0107298cf53abfeb699402def5949763721186ddac81e88e4f55154043bea91        2012-01-03 00:39:37     2025300000      105.5181        1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  7d5e8972f79b61ce3004a74fc5a1bda1e8d5531099ed5dfad3bb675c1982841c        2012-01-03 00:39:37     726309318       37.8407 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  f89d13dfcc18724e1d99a34e0149e7320277ff3e75ffacdd9008573d6c2ac8b2        2012-01-03 00:39:37     1834300000      95.567  1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  54e0dd5717434d69c0e761dbf3791b8df39d13fb8e8578f22e6f161475069280        2012-01-03 00:39:37     712309318       37.1113 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  588329950a7cd00e492ea40c1f2b79c9b3648767dd5c0d8be6b1e51030ace4a5        2012-01-03 00:39:37     1707300000      88.9503 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  4c7636192f99c3e49cba193615e0a9f9e679725f0a545b2ef53206d321b7674c        2012-01-03 00:39:37     556309318       28.9837 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  5960134323fef0e8a6f945306eb3d6d8a1283265de1e61d1068c041c5f5d3d9f        2012-01-03 00:39:37     1686300000      87.8562 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  559b2e1ccbe450d3ca276955dd76677226c00ba82a514397159646a0722c6d6a        2012-01-03 00:39:37     505309318       26.3266 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  aacfd9b0f72603d533bdce3c217f4c291be181b733a7930aa16876dba768270f        2012-01-03 00:39:37     1670300000      87.0226 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  40092b4c718d1078a20edb7e9ead9456314cdedb4daea9d868cab53d7a66f396        2012-01-03 00:39:37     358309318       18.6679 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
160346  5963ff2d7969842dd60a82b47b208037d09e6c47c6db2f4225167a4cade79b7d        2012-01-03 00:39:37     1585300000      82.5941 1VayNert3x1KzbpzMGt2qdqrAThiRovi8       pubkey  4104a39b9e4fbd213ef24bb9be69de4a118dd0644082e47c01fd9159d38637b83fbcdc115a5d6e970586a012d1cfe3e3a8b1a3d04e763bdc5a071c0e827c0bd834a5ac  0       1
I'm still not sure if the long hash at the end of each line is really what you need, but if it is, I could get only the address and hex from each line, remove the duplicates, match the addresses with the current list of addresses with a balance, and only show those. Would that be more useful? Otherwise you'll end up with a couple hundred GB of data.

Why end at block 278,298 if I you don't mind me asking?
DeepComplex
newbie
Activity: 24
Merit: 9
Re: List of all Bitcoin addresses with a balance
April 16, 2022, 10:07:16 AM
#137
Hi LoyceV,

I'm looking for all of the entries with script-type pubkey (P2PK) only and is unspent. Primarly between blocks 0 to 278,298. It is about ~34,300 entries. The fields I would like are: block_id    transaction_hash   time   value   value_usd   recipient   type   script_hex   is_from_coinbase is_spendable.

This is what I ultimately need.
Quote from: LoyceV on April 16, 2022, 03:02:34 AM
Quote from: DeepComplex on April 15, 2022, 08:30:25 PM
The only method I've found that gives me something close to the aforementioned is on Blockchair.com. Can you export that dataset to a  tsv/csv  file? The option "Get this dataset in .tsv format" on the site takes you to the Database Dumps.
I don't think I can easily get this data, other than clicking "Load more" 3000 times.
But even if you get everything: the address I checked hasn't sent funds yet, so if you're looking for the pubkey, it's not what you need, right?

Quote
I have no idea how to extract the data in that format from the outputs dump files. If you know some linux commands to parse them and give a similar output, that would be appreciated.
If you tell me exactly which part of the data dump you need, I can quite easily get it for you. It just takes a few hours to process all the data. Or I'll get you a short script that loops through the data, but to use it you'll need to download everything first.

If I take a random output, it has this data:
Code:
block_id	transaction_hash	index	time	value	value_usd	recipient	type	script_hex	is_from_coinbase	is_spendable
158078	968937595d4777356aa3dbd61148c4ddcf0aef44e6295a479731f34998ee392c	0	2011-12-19 00:08:29	5008750000	160.28	1J6iKnrn9RPM6D23VZ6o1z5muVLUjJ9NuT	pubkey	4104eacd3bfaf2d76524763c4416db596cc26e4ff20c13f12ac7d3cc5b5cec3ec88b53ff35b349fb47421976facdc25e63ea365416c3a4ace7acb7884caa6a6d7a52ac	1	1
158078	5c717d331dae3919337f11722e0e3f62b8c27230ee154f4c6c67b36a60bc4dd5	0	2011-12-19 00:08:29	1000000	0.032	1KSiuy6DeeDefyj2Ls8mnMvhRPb25xs6jx	pubkeyhash	76a914ca4fa68a872fb740f16ca1caaa414bd989362aa788ac	0	-1
158078	5c717d331dae3919337f11722e0e3f62b8c27230ee154f4c6c67b36a60bc4dd5	1	2011-12-19 00:08:29	119721000000	3831.072	1LaN6ooQJpCqC5nFb5TCinceDkCc24eetR	pubkeyhash	76a914d6b9bae958a79b931072b93a52e632f2464ae37e88ac	0	-1
158078	1c4e4dfa580566cb92ec3eb43e3650e542ba1ce7a35415b671f59794da702de4	0	2011-12-19 00:08:29	78812408397	2521.9971	1HT5UX8eSiT65fUZSskZYJ4nPPPL58Rzic	pubkeyhash	76a914b4709009d5e5844f643b02ecafa7d55d7acb3f7d88ac	0	-1
158078	1c4e4dfa580566cb92ec3eb43e3650e542ba1ce7a35415b671f59794da702de4	1	2011-12-19 00:08:29	100000000	3.2	17gN64BPHtxi4mEM3qWrxdwhieUvRq8R2r	pubkeyhash	76a914494294730abf03c846988654f15d1864469c737a88ac	0	-1
158078	bc9822da9fb69f43f1fe33682cd8007ad247deb75f5a6edb641751e456e18b93	0	2011-12-19 00:08:29	3578400000	114.5088	1KDE7sAtyhzJnLXcmJmAMYN9Xk3MRLsDxT	pubkeyhash	76a914c7c234d085dd3d5dbc342d59cca174335f554fa988ac	0	-1
Would you be looking for "4104eacd3bfaf2d76524763c4416db596cc26e4ff20c13f12ac7d3cc5b5cec3ec88b53ff35b349f b47421976facdc25e63ea365416c3a4ace7acb7884caa6a6d7a52ac"?
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
April 16, 2022, 03:02:34 AM
#136
Quote from: DeepComplex on April 15, 2022, 08:30:25 PM
The only method I've found that gives me something close to the aforementioned is on Blockchair.com. Can you export that dataset to a  tsv/csv  file? The option "Get this dataset in .tsv format" on the site takes you to the Database Dumps.
I don't think I can easily get this data, other than clicking "Load more" 3000 times.
But even if you get everything: the address I checked hasn't sent funds yet, so if you're looking for the pubkey, it's not what you need, right?

Quote
I have no idea how to extract the data in that format from the outputs dump files. If you know some linux commands to parse them and give a similar output, that would be appreciated.
If you tell me exactly which part of the data dump you need, I can quite easily get it for you. It just takes a few hours to process all the data. Or I'll get you a short script that loops through the data, but to use it you'll need to download everything first.

If I take a random output, it has this data:
Code:
block_id	transaction_hash	index	time	value	value_usd	recipient	type	script_hex	is_from_coinbase	is_spendable
158078	968937595d4777356aa3dbd61148c4ddcf0aef44e6295a479731f34998ee392c	0	2011-12-19 00:08:29	5008750000	160.28	1J6iKnrn9RPM6D23VZ6o1z5muVLUjJ9NuT	pubkey	4104eacd3bfaf2d76524763c4416db596cc26e4ff20c13f12ac7d3cc5b5cec3ec88b53ff35b349fb47421976facdc25e63ea365416c3a4ace7acb7884caa6a6d7a52ac	1	1
158078	5c717d331dae3919337f11722e0e3f62b8c27230ee154f4c6c67b36a60bc4dd5	0	2011-12-19 00:08:29	1000000	0.032	1KSiuy6DeeDefyj2Ls8mnMvhRPb25xs6jx	pubkeyhash	76a914ca4fa68a872fb740f16ca1caaa414bd989362aa788ac	0	-1
158078	5c717d331dae3919337f11722e0e3f62b8c27230ee154f4c6c67b36a60bc4dd5	1	2011-12-19 00:08:29	119721000000	3831.072	1LaN6ooQJpCqC5nFb5TCinceDkCc24eetR	pubkeyhash	76a914d6b9bae958a79b931072b93a52e632f2464ae37e88ac	0	-1
158078	1c4e4dfa580566cb92ec3eb43e3650e542ba1ce7a35415b671f59794da702de4	0	2011-12-19 00:08:29	78812408397	2521.9971	1HT5UX8eSiT65fUZSskZYJ4nPPPL58Rzic	pubkeyhash	76a914b4709009d5e5844f643b02ecafa7d55d7acb3f7d88ac	0	-1
158078	1c4e4dfa580566cb92ec3eb43e3650e542ba1ce7a35415b671f59794da702de4	1	2011-12-19 00:08:29	100000000	3.2	17gN64BPHtxi4mEM3qWrxdwhieUvRq8R2r	pubkeyhash	76a914494294730abf03c846988654f15d1864469c737a88ac	0	-1
158078	bc9822da9fb69f43f1fe33682cd8007ad247deb75f5a6edb641751e456e18b93	0	2011-12-19 00:08:29	3578400000	114.5088	1KDE7sAtyhzJnLXcmJmAMYN9Xk3MRLsDxT	pubkeyhash	76a914c7c234d085dd3d5dbc342d59cca174335f554fa988ac	0	-1
Would you be looking for "4104eacd3bfaf2d76524763c4416db596cc26e4ff20c13f12ac7d3cc5b5cec3ec88b53ff35b349f b47421976facdc25e63ea365416c3a4ace7acb7884caa6a6d7a52ac"?
DeepComplex
newbie
Activity: 24
Merit: 9
Re: List of all Bitcoin addresses with a balance
April 15, 2022, 08:30:25 PM
#135
@LoyceV,

The only method I've found that gives me something close to the aforementioned is on Blockchair.com. Can you export that dataset to a  tsv/csv  file? The option "Get this dataset in .tsv format" on the site takes you to the Database Dumps.

I have no idea how to extract the data in that format from the outputs dump files. If you know some linux commands to parse them and give a similar output, that would be appreciated.

Regards,

Quote from: DeepComplex on April 15, 2022, 11:53:15 AM
Hi @LoyceV,

I've perused your dump and indeed that info cannot be generated from them. Nevertheless, your blockchain dumps are cool and useful. Thanks again for that!

I was hoping that @albert0bsd or @TheArchaeologist could help with a script that can parse the blockchain.

I'll check the rusty-blockparser. If I do get something I'll share the same here.

Regards,

Quote from: LoyceV on April 15, 2022, 11:35:11 AM
Quote from: DeepComplex on April 14, 2022, 11:16:46 PM
How could I generate a dump like this from the blockchain?
I don't know how to do this. I don't think this data has the information, so you'll really have to get it from your own blockchain. If rusty-blockparser (see 2 posts up) can do it, I might be able to add it to my daily updates. Let me know if you find a way, I'm not checking it myself.
DeepComplex
newbie
Activity: 24
Merit: 9
Re: List of all Bitcoin addresses with a balance
April 15, 2022, 11:53:15 AM
#134
Hi @LoyceV,

I've perused your dump and indeed that info cannot be generated from them. Nevertheless, your blockchain dumps are cool and useful. Thanks again for that!

I was hoping that @albert0bsd or @TheArchaeologist could help with a script that can parse the blockchain.

I'll check the rusty-blockparser. If I do get something I'll share the same here.

Regards,

Quote from: LoyceV on April 15, 2022, 11:35:11 AM
Quote from: DeepComplex on April 14, 2022, 11:16:46 PM
How could I generate a dump like this from the blockchain?
I don't know how to do this. I don't think this data has the information, so you'll really have to get it from your own blockchain. If rusty-blockparser (see 2 posts up) can do it, I might be able to add it to my daily updates. Let me know if you find a way, I'm not checking it myself.
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
April 15, 2022, 11:35:11 AM
#133
Quote from: DeepComplex on April 14, 2022, 11:16:46 PM
How could I generate a dump like this from the blockchain?
I don't know how to do this. I don't think this data has the information, so you'll really have to get it from your own blockchain. If rusty-blockparser (see 2 posts up) can do it, I might be able to add it to my daily updates. Let me know if you find a way, I'm not checking it myself.
DeepComplex
newbie
Activity: 24
Merit: 9
Re: List of all Bitcoin addresses with a balance
April 14, 2022, 11:16:46 PM
#132
How could I generate a dump like this from the blockchain? Any script available?

Quote from: TheArchaeologist on December 14, 2021, 08:12:54 AM
Quote from: albert0bsd on November 06, 2021, 11:52:36 AM
Hi LoyceV since you already have some useful data, did you believe that you can publish some list like "Public keys with balance".

Not OP, but are you looking for something like this:
Code:
address                                  balance         pubkey
12higDjoCCNXSA95xZMWUdPvXNmkAduhWv       0.00010000      044d05240cfbd8a2786eda9dadd520c1609b8593ff8641018d57703d02ba687cf2f187f0cee2221c3afb1b5ff7888caced2423916b61444666ca1216f26181398c
17afxUJouat3fkaaQ9tZrDThxdkXGL4WrM       0.00010000      0428ae82b094ace0e2753b55ade744f65580effe4be3940601cd5a6c51d5af7c5a3747741abcfebf48085010fc7983107da4fbc3dff8b72401779fcfbaa05bd5d3
1PhUXucRd8FzQved2KGK3g1eKfTHPGjgFu       50.00673279     04a2b3590604543737ef6a5cbdf438e44ef66d584efab1565759e1bafc72462d6b5f547466e4cbd610f2dcd65f2c0d6a419cf0170c15d7e783edf7d0247fa74f3b
1DNML6RyHdsqRS7te5SpCkMGoYEkVJU6wD       0.00000890      04ab6c62bcdc3ee4a5ded010007e48c6404347997624af22582147bf4ca7f2b7c3b725d1278f19c87f85652adcc8a198104c233353683bc6c2d1f8149c53c3782b
15J5MZxBkbo88hfHbY96UEDdWWtTJKTBgj       0.00000890      04aeb58ff7e7109ad1d2e8f4f574c733833a8c9c394082cec0f691d63a352ce56d3a1ac62473b62b70b238495808b8c904efff6c02ae9fc97e442f153404b64af2
1EGDbCKGbizMFgwjn4WNHTvAthsvrsixDv       1.00000000      042dfb87e38d92220d20fc181d358dca0f3677f2eabf64425b355deda0ae34a057127b369a80dcb69fb652ddbcb5e91b1c1bd43634be8c00fd1f0c74d3c6b9b8d6
1PyNKEyqtDFB4HBG8ueDVxdPZfgwC6gAqw       0.00001000      046bfe6fa9d6ab1383378cb1f30fb913e9bb18bc7f8cc81aadd2ac70bb5177d4dd73cd7a59373d58738602ba850622e1f470aa8ea45972673bd06b8bdc119f7ef4
1JXFXUBGs2ZtEDAQMdZ3tkCKo38nT2XSEp       0.00001000      04044ee968e705cba538788fd49bdd95ec4b8d96d6fe52c3c0788c4289d6c6ec3305cb603ac45462ffd6093aaddebd0f1201945165d8b0a2b9bb9a70e392aa4936
18FAoviy2XGWRBvDaA1QSUrKuPH2ST8pA2       2.38000000      04d21667e5ab6f05072fec925c9f598af3fcf40aafd31dd89b8994a63f516a7426da3cceb9ff437678dcd02abd8451b51d61075c58f0cc7e072fa71dd8f74b603c
14ccB2QFEKZnd1gLUtgr7vWg6JrMeXaQXK       0.00001000      045551caaca4caa1535fdea7e3dd6646720869e5c0187077aecfeafe60741aa61a35b013eb0f55081e30fc1b3b03bbc39e2430b31073574c39dbe60d122edb8a1b
12u6hecWRHEPceLYZ9yhubZvhgKYuLGf1J       151.00001435    046bd20f8fe764c28f5fe1c4eba69d26e0af42bf1853213d593a5098e075344d6fdc998e9c1e469cad7d799a0891f3bc47ae1f6a1eb6caabd62890a7994f4b6d5f
1XPTgDRhN8RFnzniWCddobD9iKZatrvH4        0.00015529      0434417dd8d89deaf0f6481c2c160d6de0921624ef7b956f38eef9ed4a64e36877be84b77cdee5a8d92b7d93694f89c3011bf1cbdf4fd7d8ca13b58a7bb4ab0804
1BiDsroMqEwDhBmzkKLvCSAddGn8QGmLFf       0.00001000      040bc7100ef382290a250654c0b77841ea844b5ad09843f55bf79fe6bb3a320f178a899280cf6f0a92befe316d922bffc3e8d6f93a8f1b788ac18c7ee44cf2f15a
1JCjFJckVFvN5EFs7oUmzKjPVLq8XyPiwc       10.00000000     04200af342e1bb4b50ada09233ebdca598446d4c7c3c92c915b3ec4fa36ef9c81937a204fe1909509d5b5b429ab00e378a3a1bf7e20e783974e0b1d5263c3bd38b
15Z5YJaaNSxeynvr6uW6jQZLwq3n1Hu6RX       7941.06426123   047a51392bace353f4c3788c9c090ef4f635ec211159ec3b9f1bb7da7679517e126e98e0012bcb4d2b023c479afaaa1ad703ea1b24e1910e2cdad38744ba7aab8a
1JiZxcwHpivk35obqHUYbdNoE1BA8ty5TB       5.00000000      04cb9648aa9f91795d111ded927f1a175c44a53537c817da4616eb071a9ec83a2670430628d20e70ee7a45b23494fc217d1020784d4d56b2f932754d79e4c3b808
PawGo
legendary
Activity: 952
Merit: 1386
Re: List of all Bitcoin addresses with a balance
April 14, 2022, 03:19:36 PM
#131
Quote from: mrx23dot on April 13, 2022, 04:28:54 PM
Hello,
What's the RPC command to do the dump on local full node?

It is not RPC way, but I was using rusty-blockparser for doing dumps from local node. It uses local files, not rpc connection.
https://github.com/gcarq/rusty-blockparser
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
April 14, 2022, 06:36:42 AM
#130
Quote from: mrx23dot on April 13, 2022, 04:28:54 PM
What's the RPC command to do the dump on local full node?
You may be interested in Bitcoin UTXO Dump.
I haven't had the time yet to test it (use at your own risk).
mrx23dot
newbie
Activity: 2
Merit: 0
Re: List of all Bitcoin addresses with a balance
April 13, 2022, 04:28:54 PM
#129
Hello,
What's the RPC command to do the dump on local full node?

Is it possible to use remote RPC server for it, like electrum?

Cheers
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
March 27, 2022, 08:50:21 AM
#128
Quote from: PawGo on March 27, 2022, 04:19:10 AM
Open way to mine Monero on free cpu ;-)
It can produce $0.06 per day. I'm not turning electricity into heat for that, even if someone else pays for it.

Quote
Do you plan to run a full-node (with incoming connections) and maybe electrum server?
I've already almost filled it's disk, so that won't work.
PawGo
legendary
Activity: 952
Merit: 1386
Re: List of all Bitcoin addresses with a balance
March 27, 2022, 04:19:10 AM
#127
Quote from: LoyceV on March 22, 2022, 04:49:48 AM
The above post opened a new door: an anonymous Bitcointalk reader donated a dedicated server for this project! That means lots of storage (1 TB), lots of bandwidth (50 TB/month), and potentially more than once a day updates by parsing the Bitcoin blockchain instead of getting data from Blockchair.com!
I'm still setting it up now.

WOW, congratulations!
Open way to mine Monero on free cpu ;-)
Do you plan to run a full-node (with incoming connections) and maybe electrum server?
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
March 26, 2022, 08:14:04 AM
#126
Update: the new server is online, at the old location: addresses.loyce.club.
Because of the much larger disk space, I'll now keep more daily and monthly snapshots. It will take a couple of months to collect them all.
Enjoy the bandwidth Smiley
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
March 22, 2022, 04:49:48 AM
#125
Quote from: LoyceV on March 19, 2022, 07:51:55 AM
This project has already used up 1.5 out of it's 2 TB bandwidth for this month. To prevent the server from being suspended, I've scheduled to take down the Apache webserver in 5 days. I'll restart it on April 1st.
The above post opened a new door: an anonymous Bitcointalk reader donated a dedicated server for this project! That means lots of storage (1 TB), lots of bandwidth (50 TB/month), and potentially more than once a day updates by parsing the Bitcoin blockchain instead of getting data from Blockchair.com!
I'm still setting it up now.
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
March 19, 2022, 07:51:55 AM
#124
This project has already used up 1.5 out of it's 2 TB bandwidth for this month. To prevent the server from being suspended, I've scheduled to take down the Apache webserver in 5 days. I'll restart it on April 1st.

It's now the second time I have to take this precaution. I've paid this webhost for the next 1.5 years, and I'm at the maximum bandwidth for this package already. I don't really want to upgrade to another server yet (I use 4 different servers for 4 different projects).
If anyone's in urgent need for this data when it's offline: post here Smiley
LoyceV
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Re: List of all Bitcoin addresses with a balance
February 05, 2022, 08:19:06 AM
#123
Quote from: PawGo on February 05, 2022, 06:42:30 AM
You should check in accessLog how many requests you have from curl/wget type tools and how many from normal browser.
I never look at logs. Now that I did, it turns out logs don't get rotated, so it goes back more than a year.
Some counts:
Code:
"mozill": 688k
"chrom": 539k
"wget": 415k
I've wiped the logs now.

I assume people who use a normal browser will see the dates typed on addresses.loyce.club/.
PawGo
legendary
Activity: 952
Merit: 1386
Re: List of all Bitcoin addresses with a balance
February 05, 2022, 06:42:30 AM
#122
Quote from: LoyceV on February 05, 2022, 04:53:02 AM
Quote from: PawGo on February 05, 2022, 04:26:58 AM
If you want to keep "fixed filename", maybe add one more, small file with "update date" inside.
The update date is in the file's time stamp. It used to be easy to see, but since I added an index.html it doesn't show anymore.
I had to search a bit, Super User.com gave this command:
Code:
wget --server-response --spider  http://addresses.loyce.club/Bitcoin_addresses_LATEST.txt.gz 2>&1 | grep -i Last-Modified
  Last-Modified: Mon, 31 Jan 2022 00:57:38 GMT

Too technical. But maybe will be enough. You should check in accessLog how many requests you have from curl/wget type tools and how many from normal browser.
Pages:
Bitcoin Forum>Bitcoin>Project Development
Jump to: