How to read/parse blockchain and get bitcoin addresses having balance. - page 2.

btctousd81

sr. member

Activity: 434

Merit: 270

Quote from: amaclin1 on October 06, 2017, 02:03:50 PM

Quote from: btctousd81 on October 06, 2017, 01:26:41 PM

what changed in segwit ? i mean in block ? so i can try to change it in blockparser source.

Transaction format was changed in segwit.
These six words mean that your tool should be fully rewritten. Tongue

thanks
then looks like i am better off trying with other tools.,

edit1 :

i tried running it as it is, parsed around 50% blocks and got output file of 4 GB having format like this

Code:

---------------------------------------------------------------------------
   State of the ledger at block 420662 (minted : Thu Jul 14 08:18:22 2016)
---------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
   Balance Hash160 Base58 nbIn lastTimeIn nbOut lastTimeOut
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
   152955.81868182 e95dbb25283cc35d4a6aefa76c0382e63ce0fa36 3Nxwenay9Z8Lc9JBiywExpnEFiLp6Afp8v 56 Tue Jul 12 14:11:34 2016 11 Wed Jun 1 15:36:33 2016
   121295.29280322 b3b9f5025c397c07e7e37db7e5c9259ac95cd344 3J5KeQSVBUEs3v2vEEkZDBtPLWqLTuZPuD 387 Thu Jul 14 05:00:27 2016 196 Sun Jul 3 11:45:17 2016
   83000.01001479 e1223ee403b7250d54c08890679df36e9d6b4986 3NDQz8rZ3CnmsiBGrATk8SCpDXF2sAUiuM 6 Sun Jul 3 15:23:05 2016 1 Sun May 29 01:58:14 2016
   79957.11509537 a0b0d60e5991578ed37cbda2b17d8b2ce23ab295 1FeexV6bAHb8ybZjqQMjJrcCrHGW9sb6uF 85 Fri May 20 18:43:43 2016 0 Thu Jan 1 00:00:00 1970
   78172.33497629 c5464169a9aabad0e361ccf1d436d3e843708e7d 3Kg7Cmooris7cLErTsijq6qR1FH3cTiK2G 20889 Sun Jul 3 15:23:05 2016 4 Tue May 17 17:14:55 2016
   69370.10525051 b3dd79fb3460c7b0d0bbb8d2ed93436b88b6d89c 1HQ3Go3ggs8pFnXuHVHRytPCq5fGG8Hbhx 64 Sat Feb 13 06:40:01 2016 1 Thu Apr 23 14:10:25 2015
   66650.59620465 3d03002dbed5cb1dc10fc6bcb0886d2df32f2838 16ZbpCEyVVdqu8VycWR8thUL2Rd9JnjzHt 168 Fri Apr 29 10:29:14 2016 0 Thu Jan 1 00:00:00 1970
   66583.22391617 cd4b7b8f9db1b0c709fd0c9f0534fca6a9f40495 1KiVwxEuGBYavyKrxkLncJt2pQ5YUUQX7f 120 Sat Feb 13 06:40:01 2016 0 Thu Jan 1 00:00:00 1970
   66452.06624862 f9e6bbcdc83d8f351014e07495f386fe1067ec7b 1PnMfRF2enSZnR6JSexxBHuQnxG8Vo5FVK 114 Sat Feb 13 06:40:01 2016 0 Thu Jan 1 00:00:00 1970
   66378.80961189 6a6015e3793207af6dff7c48ee9e193d73547cdc 1AhTjUMztCihiTyA4K6E3QEpobjWLwKhkR 181 Thu Feb 18 17:09:28 2016 0 Thu Jan 1 00:00:00 1970
   66235.82427687 8b70193546504fa3623598722575f70b5b1c6455 1DiHDQMPFu4p84rkLn6Majj2LCZZZRQUaa 125 Sat Feb 13 06:40:01 2016 0 Thu Jan 1 00:00:00 1970
   66233.73850546 908b47d6a15d89c3ee34df1bc68a00ec4ff70c9b 1EBHA1ckUWzNKN7BMfDwGTx6GKEbADUozX 126 Sun Jun 5 18:31:03 2016 0 Thu Jan 1 00:00:00 1970
   66205.47375047 27ed35ebcf72744ec3b4a0f4dbb83d7620acf443 14e7XAZbepQp9MXXzjNG3fNLoAUpaBAXHW 132 Fri Apr 29 10:29:14 2016 0 Thu Jan 1 00:00:00 1970
   66170.69605917 f33a3af3695f041e2ea6271f1fe6309abac4dd58 1PB4xXUFyy4kSNqroCBVaQuCuw9VcN3be4 171 Fri Feb 12 04:57:18 2016 0 Thu Jan 1 00:00:00 1970
   66163.44524020 53f9818e8d2d23c3163c01a33d83e49ebfa16ac4 18f1yugoAJuXcHAbsuRVLQC9TezJ6iVRLp 124 Wed Mar 9 23:07:42 2016 0 Thu Jan 1 00:00:00 1970
   53880.05741758 d74de95f65799793f16b91ed8a152110652d3ec0 1LdRcdxfbSnmCYYNdeYpUnztiYzVfBEQeC 21 Fri Feb 12 04:57:18 2016 0 Thu Jan 1 00:00:00 1970
   50700.00172304 ffd32592cc07d25a8644d7dfd6919d348f9b1874 3R1hBCHURkquAjFUv1eH5u2gXqooJkjg4B 81 Fri Jul 8 14:53:10 2016 37 Sun Jun 12 14:26:28 2016

after around 505 lines, btcaddress is screwed .

Code:

2500.00111000 a0fc91baff4e19cb9f417cef2f00d2822106b5be 1FgDgWjeCMxrfiiaYnPdvnQG9DcYe6yX1r 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 d34e8a9a7786fdbbff59be2a5cc169729b2bbae0 1LGHfdXMoAh3rAqfZ5tRruVxWz4drjLGHQ 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 37a5e72afc3aa9ed12e51ee187c56a5d2e6f1ed9 165EuhCqScXdGJfDvYK5RZKN5SytnhwVjv 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 a0b5712a79778cc923e3ca0a54a8a442d8c24a7c 1FekUWTJ4S4UTYuj41yaJCuntvh2cszhsk 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 9519a19ef04031d40998d7a684c7878d4c5baa8e XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 51ccfe24e3577bb2b8f20ddf39ca69389dc3726c XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 9cfca6b08dc926b8903b58fde9c7d3db5f1b5007 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00111000 43c43187cc1bf92fd25b62477a2d772a7db679b4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2500.00005400 f90f808949b65ce2c01a0eacbf20febbf40b881e XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Tue Jun 21 03:52:29 2016 0 Thu Jan 1 00:00:00 1970
   2500.00000000 6a7266721acf340b77f84d8d0f677171ae7bb8a6 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Wed Jun 29 22:52:52 2016 0 Thu Jan 1 00:00:00 1970
   2500.00000000 b69398c2a4fe36e739ac765c4283d74323340afb XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 13 Sun Mar 6 03:46:10 2016 6 Mon Mar 28 01:11:36 2016
   2500.00000000 86d1cc562153dd9954e31e9c7f3a02390b1d4a3f XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Fri Feb 19 21:51:14 2016 0 Thu Jan 1 00:00:00 1970
   2500.00000000 2ac55d1a940deb5b5e684e500355f826b5381ee5 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Fri Nov 27 13:22:46 2015 0 Thu Jan 1 00:00:00 1970
   2500.00000000 64dac2d7527f9967a2c0b6f18d986f116cd89bfd XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Tue Jun 21 16:27:38 2016 0 Thu Jan 1 00:00:00 1970
   2499.99960000 f148343d37f0621eb00eef03ed191444ee8aaa30 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 2 Fri Jul 8 13:16:16 2016 0 Thu Jan 1 00:00:00 1970
   2499.99101000 74869025a6d4e202d1271b676ed5a374d92e94a2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 10 Thu Apr 2 17:52:36 2015 1 Tue Feb 11 17:17:54 2014
   2499.97356500 a4bc600d905f4465dd59a4dfbdb740231036dc71 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 7 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2497.99949660 1b490a2b5be46b6fd395c11ca0d6fd50f194fb7c XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Fri Oct 30 04:21:43 2015 0 Thu Jan 1 00:00:00 1970
   2492.25031000 bc8d40e64b33e2d99ff81e4643dbf8d80e2a5c9b XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 7 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2489.00121000 c8ca9fa680409b5d7375ad6d2fb5aac2e4405c89 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 5 Thu Apr 2 17:52:36 2015 0 Thu Jan 1 00:00:00 1970
   2483.80277466 b8f27aaedf87530547e61406f73947455371e6de XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 1 Fri May 27 20:27:01 2016 0 Thu Jan 1 00:00:00 1970

so unless you want hash160, or know how to convert hash160 to btcaddress., this tool is obsolete .

amaclin1

sr. member

Activity: 770

Merit: 305

Quote from: btctousd81 on October 06, 2017, 01:26:41 PM

what changed in segwit ? i mean in block ? so i can try to change it in blockparser source.

Transaction format was changed in segwit.
These six words mean that your tool should be fully rewritten. Tongue

btctousd81

sr. member

Activity: 434

Merit: 270

Quote from: LoyceV on October 06, 2017, 12:43:40 PM

Quote from: btctousd81 on October 06, 2017, 10:35:25 AM

i tried using https://github.com/znort987/blockparser

Code:

info: 481823 blocks, 295.351 MegaAddrs ,
fatal: failed to locate upstream transaction

I can't help you fix it, but I can tell you the reason: blockparser is older than SegWit, so it stops at the block in which SegWit got activated.
BTC.COM mines the 481,823rd block! Segwit is on stage!

If you get it to work, I'd love to have a complete list (in for example CSV-format) of all addresses with their balance. I'd like to play around with it a bit.

i can get a list before that block by stopping program.,

but i am looking for complete upto date list.
will post if i get it to work.,

what changed in segwit ? i mean in block ? so i can try to change it in blockparser source.

edit 1:

this is the code cauing the fatal error

Code:

auto upTXHash = p;
const Chunk *upTX = 0;
if(gNeedUpstream && !skip) {
auto isGenTX = (0==memcmp(gNullHash.v, upTXHash, sizeof(gNullHash)));
if(likely(false==isGenTX)) {
auto i = gTXOMap.find(upTXHash);
if(unlikely(gTXOMap.end()==i)) {
errFatal("failed to locate upstream transaction");
}
upTX = i->second;
}
}

i am not sure, will it give correct output. if i comment outt that line and recompile and rerun the code.,
will try anyways.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: btctousd81 on October 06, 2017, 10:35:25 AM

i tried using https://github.com/znort987/blockparser

Code:

info: 481823 blocks, 295.351 MegaAddrs ,
fatal: failed to locate upstream transaction

I can't help you fix it, but I can tell you the reason: blockparser is older than SegWit, so it stops at the block in which SegWit got activated.
BTC.COM mines the 481,823rd block! Segwit is on stage!

If you get it to work, I'd love to have a complete list (in for example CSV-format) of all addresses with their balance. I'd like to play around with it a bit.

Update: this post was Merited today, which reminded me to add this:

Quote from: LoyceV on June 11, 2020, 10:30:34 AM

Bitcoin block data available in CSV format
List of all Bitcoin addresses with a balance
List of all Bitcoin addresses ever used

btctousd81

sr. member

Activity: 434

Merit: 270

i am trying to get bitcoin addresses from my blockchain stored on server.

my bitcoin is up to date. index mode is not enabled, as i dont want UTXO database.

i tried using https://github.com/znort987/blockparser

compiled it succesfully.
strated like this

Code:

[root@blockparser]# env BLOCKCHAIN_DIR="/home/user2/" ./parser allBalances > allBalances.txt

info: mem at start = 0.000 Gigs
info: starting command "allBalances"
info: loading block chain from directory: /home/user2/.bitcoin
info: block chain size = 136.344 Gigs
info: initializing hash tables
info: estimated number of blocks = 1071.27K
info: estimated number of transactions = 276.71M
info: done initializing hash tables - mem = 0.000 Gigs
info: pass 1 -- walk all blocks and build headers ...
info: pass 1 -- took 48 secs, 488497 blocks, 136.34 Gigs, 2862.34 Megs/secs , 89430 early link misses, mem=0.000 Gigs
info: pass 2 -- link all blocks ...
info: pass 2 -- done, did 89430 late links
info: pass 3 -- wire longest chain ...
info: pass 3 -- done, maxHeight=488497
info: pass 4 -- full blockchain analysis (with index)...
info: computing balance for all addresses
info: hit ^C to interrupt and dump current, valid state of ledger
info: 1 blocks, 0.000 MegaAddrs ,
info: 141229 blocks, 1.738 MegaAddrs ,
info: 166873 blocks, 3.058 MegaAddrs ,
info: 181587 blocks, 3.939 MegaAddrs ,
info: 186771 blocks, 4.633 MegaAddrs ,

but it crashes in the middle., or some times at the end., i tried running it many times.,

previously i got memory related error., so freed up memory and increased swap on my server. then tried running it again

right now i am getting

Code:

info: 481823 blocks, 295.351 MegaAddrs ,
fatal: failed to locate upstream transaction

i counted and looks like it aborts when 2000 blocks are remaining.

i am not the only one getting this error..

https://github.com/znort987/blockparser/issues/65

so what you guys are using ? to get data extracted from blockchain files.

i have 64 gigs of ram, i7 , ssd .
centos 7 x64

is there any other tool ? optimized ? or any other fork

thanks for your time,.

Topic: How to read/parse blockchain and get bitcoin addresses having balance. - page 2. (Read 2018 times)