Get list of all addresses with a balance over x? - page 8.

erajaykumarrana

newbie

Activity: 2

Merit: 0

Quote from: starmyc on March 04, 2018, 01:50:58 PM

I've generated a new dump for all balances as of today.

Enjoy: https://transfer.sh/12aVfO/balances_20180304.out.gz

Cheers.

friend can you tell me how you generated this list, I have to generate it for dash coin....

I have seen your code on github... how can i compile it in visual studio 2017 on windows
do you parse chain state directory...

SoliDCoiNs

jr. member

Activity: 104

Merit: 1

Maybe someone has already done this and would like to share their results?
I think all of us would appreciate the effort!
Thanks!

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: P. on March 06, 2018, 03:24:28 AM

Thanks for explanation.
Another one is ..
there is one address that begins with 0e0 and few thousands of those starting with bc1. What does those mean?
T.

The 0e0 thing is something I didn't remove from output, and this is the obfuscation key of my database, and not an address. You can ignore it.

The bc1 addresses are bech32 addresses, introduced last year for segwit transactions. They are used in P2WPKH & P2WSH transactions.

Cheers.

P.

newbie

Activity: 2

Merit: 0

Thanks for explanation.
Another one is ..
there is one address that begins with 0e0 and few thousands of those starting with bc1. What does those mean?
T.

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: P. on March 05, 2018, 07:49:44 AM

Need to ask .. as second and third values are

1111111111111111111114oLvT2;43338925513
111111111111111111112BEH2ro;10940

if I check via bitref (or any other tool),

second key balance is 65.57999243
third 0.00010940

seems like greater numbers are flawed somehow?

You picked a unusable address (ripemd160 hash is 0x0000000000..., that's just lost coins) with more than 50000 utxos attached to it (I count 52057 utxos while blockchain.info only count 51302). Honestly, it will be hard to debug this one, as I don't know how blockchain.info fills its database.

By the way, I don't think there is a problem with big numbers. Other balances seems ok:

Code:

$ grep 1111116d87CjjDyP8SF5v1LTvUq22VFg balances.out 
1111116d87CjjDyP8SF5v1LTvUq22VFg;8957970093

$ curl -s https://blockchain.info/fr/rawaddr/1111116d87CjjDyP8SF5v1LTvUq22VFg|grep final_balance
    "final_balance":8957970093,

Another sample:

Code:

$ grep 336xGpGweq1wtY4kRTuA4w6d7yDkBU9czU balances.out
336xGpGweq1wtY4kRTuA4w6d7yDkBU9czU;4897336319144

$ curl -s https://blockchain.info/fr/rawaddr/336xGpGweq1wtY4kRTuA4w6d7yDkBU9czU|grep final_balance
    "final_balance":4897336319144,

Cheers

P.

newbie

Activity: 2

Merit: 0

Need to ask .. as second and third values are

1111111111111111111114oLvT2;43338925513
111111111111111111112BEH2ro;10940

if I check via bitref (or any other tool),

second key balance is 65.57999243
third 0.00010940

seems like greater numbers are flawed somehow?

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

I've generated a new dump for all balances as of today.

Enjoy: https://transfer.sh/12aVfO/balances_20180304.out.gz

Cheers.

rustychainz

newbie

Activity: 1

Merit: 0

anyone have an updated file? balances_498754.zip is quite old.

RebeccaCole

newbie

Activity: 37

Merit: 0

Can't you just sort the output?

aplistir

full member

Activity: 378

Merit: 197

Quote from: phucthinh11061997 on February 20, 2018, 08:59:31 AM

Who knows if the balances_498754.csv file shared above contains the compressed or uncompressed format of the bitcoin public address?

It is impossible to know if a bitcoin address is made using a compressed or uncompressed public key. Every bitcoin address has many compressed and uncompressed public keys that will generate that address.
There are 2^160 possible addresses and 2^256 public keys, which makes about 2^96 different keys for each address

But you can make an educated guess based on the date when the address was created. During the last 3-4 years >99% of generated addresses would use compressed public keys. And the oldest ones would probably use uncompressed keys.

The only way to be certain is if there is a spend action from that address. then you can look at the published public key and you will know if it is compressed or not.

phucthinh11061997

newbie

Activity: 19

Merit: 0

Who knows if the balances_498754.csv file shared above contains the compressed or uncompressed format of the bitcoin public address?

ivanxuu

newbie

Activity: 6

Merit: 0

Quote from: starmyc on February 20, 2018, 12:05:44 AM

I think I forgot to handle a case: When the chainstate is obfuscated. When it happens, a new record is added to database, hitting the assert condition and making the parsing go wrong.

I added the forgotten feature, but I'm not 100% sure it is ok. So far my tests were OK (for some reason, my main testing bitcoins db was clear of any obfuscation). Could you pull the source again from https://github.com/mycroft/chainstate and rebuild it to try it ? It would be really helpful.

Your new commit made it possible. Thanks. It's working fine now

Is anyone insterested in downloading the CSV file? I can upload it to internet if anyone is insterested.

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: ivanxuu on February 19, 2018, 08:25:32 AM

But now I'm getting:

user@localhost:~/Deveopment/chainstate $ ./chainstate
chainstate: chainstate.cc:45: int main(): Assertion `idx[0] == 'C'' failed.

Which is weird because if I add the following line before and comment the assertion like this:

~ 45 cout << "DBG: "<+ 46 //assert(idx[0] == 'C');

Then I get lines like:

user@localhost:~/Deveopment/chainstate $ ./chainstate
DBG:
Invalid output: 79656b5f65746163737566626f00 38c006 30080738760000
79656b5f65746163737566626f00;Invalid address or lost;30080738760000
last block: 06c03835a99fece306e4e25c7c7c7c34747a2ee3a3b177156a93b2ecdf7fd9a3
DBG: C
Invalid output: eacfdcd42b27112ab6c8b435abec20181d05b0ba5d4f1829c002cc3ef0000000 9729b525c0f657759ad3c11de5e525c596843cc30b84b975b5 4
eacfdcd42b27112ab6c8b435abec20181d05b0ba5d4f1829c002cc3ef0000000;Invalid address or lost;4

What could it be?

I think I forgot to handle a case: When the chainstate is obfuscated. When it happens, a new record is added to database, hitting the assert condition and making the parsing go wrong.

I added the forgotten feature, but I'm not 100% sure it is ok. So far my tests were OK (for some reason, my main testing bitcoins db was clear of any obfuscation). Could you pull the source again from https://github.com/mycroft/chainstate and rebuild it to try it ? It would be really helpful.

ivanxuu

newbie

Activity: 6

Merit: 0

Thanks a bunch! That made it!

Now, I've made a copy of ~/.bitcoin/chainstate dir, and move it to the path where i've compiled your program (~/development/chainstate):

$ cp -rp ~/.bitcoin/chainstate ~/development/chainstate/state

But now I'm getting:

user@localhost:~/Deveopment/chainstate $ ./chainstate
chainstate: chainstate.cc:45: int main(): Assertion `idx[0] == 'C'' failed.

Which is weird because if I add the following line before and comment the assertion like this:

~ 45 cout << "DBG: "<+ 46 //assert(idx[0] == 'C');

Then I get lines like:

user@localhost:~/Deveopment/chainstate $ ./chainstate
DBG:
Invalid output: 79656b5f65746163737566626f00 38c006 30080738760000
79656b5f65746163737566626f00;Invalid address or lost;30080738760000
last block: 06c03835a99fece306e4e25c7c7c7c34747a2ee3a3b177156a93b2ecdf7fd9a3
DBG: C
Invalid output: eacfdcd42b27112ab6c8b435abec20181d05b0ba5d4f1829c002cc3ef0000000 9729b525c0f657759ad3c11de5e525c596843cc30b84b975b5 4
eacfdcd42b27112ab6c8b435abec20181d05b0ba5d4f1829c002cc3ef0000000;Invalid address or lost;4
DBG: C
Invalid output: a59f2e72830363c38a438d12236a15d1ca3074ce980873bd7f3a1699f7010000 d139db52c05d8a0fb4bf2cc101c3916250098c52d03799b092 10000000000
a59f2e72830363c38a438d12236a15d1ca3074ce980873bd7f3a1699f7010000;Invalid address or lost;10000000000
DBG: C
Invalid output: 0118dd986e59473732239d39cb3b8890bf32677719dd8933b05f6614f4020000 83211539cb2ce3daeded5d37ada7692119b33dc8689a5058 1000000
0118dd986e59473732239d39cb3b8890bf32677719dd8933b05f6614f4020000;Invalid address or lost;1000000
DBG: C

What could it be?

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: ivanxuu on February 19, 2018, 03:53:09 AM

Would you mind uploading that CSV file? It contains all addresses, even with no balance, right?
I've alredy tried to compile your code but I'm getting errors related to gmp library. If you can upload that I wouldn't mind reuploading to my own http server so it can be accessed to others.

I can't for the moment & until the end of the week unfortunately.
If you have gmp librairies & headers installed, secp256k1 will use it. You'll have to edit the main Makefile and add "-lgmp" in the LDFLAGS line. It will build the binary using it.

ivanxuu

newbie

Activity: 6

Merit: 0

Quote from: starmyc on February 09, 2018, 11:27:14 PM

Hi everyone,

I've opensourced a small software to parse the chainstate & output all utxos with their addresses (including latest bech32) in CSV format. As for today, computation takes ~8 minutes & creates a 6 GB CSV file (there is 60M utxos).
Feel free to give it a try: https://github.com/mycroft/chainstate

Would you mind uploading that CSV file? It contains all addresses, even with no balance, right?
I've alredy tried to compile your code but I'm getting errors related to gmp library. If you can upload that I wouldn't mind reuploading to my own http server so it can be accessed to others.

btctousd81

sr. member

Activity: 434

Merit: 270

Quote from: Sakuraa on February 16, 2018, 06:07:54 PM

Quote from: btctousd81 on January 29, 2018, 09:46:03 PM

here it is latest dump

https://www.transfernow.net/51dss251ddg5

Friend, please upload a new version or update the link to the old one

i m not at machine., will need 2 days., watch this thread.

Sakuraa

newbie

Activity: 14

Merit: 0

Quote from: btctousd81 on January 29, 2018, 09:46:03 PM

here it is latest dump

https://www.transfernow.net/51dss251ddg5

Friend, please upload a new version or update the link to the old one

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: btctousd81 on February 12, 2018, 11:36:23 PM

Quote from: aplistir on February 12, 2018, 01:55:01 PM

Is there any way to get the extraNonce values of every mined block from 1 -> 150000 ?

I am interested in doing some blockchain archaelogy like this:
https://bitcointalksearch.org/topic/chain-archaeology-answers-from-the-early-blockchain-507458

what are you saying ?

satoshi secretly mined and has more than 50 btc which are from genesis block ?

On the first blocks mined, there were some input script on coinbase transaction (which is useless as it is not used at all in bitcoin's protocol). In this script, they were only 2 pushes in stack of 2 VLI: one for a timestamp (not sure), and another by an extranonce value. This value was used to alter the coinbase TX hash, thus the merkleroot hash, then the block hash value. I guess it was another variable which could be used to reach a valid block hash value (additional ot nonce & timestamp or re-ordering transactions).

This value is not existing anymore and recent blocks have a empty input script for coinbase transaction.

aplistir

full member

Activity: 378

Merit: 197

Quote from: starmyc on February 13, 2018, 05:14:24 AM

Can you check this file: https://transfer.sh/oZFQr/extranonces.final.gz ? Format is height;hash;extranonce value.

I've got values for ~180k first blocks. Didn't check if all coinbase input scripts had the same format, just exported 2nd value pushed in stack of coinbase input script using my export tools. If values doesn't match yours, please send me your values so I can fix my scripts, but so far, it seems to match values from the order thread.

Excellent! Thank you.

I have been looking for that information for some time now. I even tried some old blockchain parser, which could not do the job.

I checked the numbers against the ones I have, and the first 36288 extraNonce values are the same than the ones I had Grin

Topic: Get list of all addresses with a balance over x? - page 8. (Read 43401 times)