Get list of all addresses with a balance over x? - page 7.

bigman3912

newbie

Activity: 12

Merit: 0

Quote from: starmyc on March 20, 2018, 12:29:24 AM

Quote from: bigman3912 on March 19, 2018, 09:27:48 PM

starmyc, your chainstate parser can make all transactions db ? Or maybe you already have one?

The chainstate DB in Bitcoin Core only list the unspent transactions outputs (not even full transactions). So, answer is no, it can't.
If you want to list all transactions from a blockchain, you need a block parser (and I've mine too, but it is closed source).

Do you need something in particular ?

Yes, you right. I mean all unspent transactions, of course.

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: bigman3912 on March 19, 2018, 09:27:48 PM

starmyc, your chainstate parser can make all transactions db ? Or maybe you already have one?

The chainstate DB in Bitcoin Core only list the unspent transactions outputs (not even full transactions). So, answer is no, it can't.
If you want to list all transactions from a blockchain, you need a block parser (and I've mine too, but it is closed source).

Do you need something in particular ?

bigman3912

newbie

Activity: 12

Merit: 0

Quote from: starmyc on March 19, 2018, 06:49:54 AM

Quote from: starmyc on March 19, 2018, 04:57:19 AM

I could take a look at DASH, it would require me a few days however.

Ok, I was able to extract all current balances for Dash.
Have fun with this: https://transfer.sh/109sSU/balances.dash.out.gz

starmyc, your chainstate parser can make all transactions db ? Or maybe you already have one?

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: starmyc on March 19, 2018, 04:57:19 AM

I could take a look at DASH, it would require me a few days however.

Ok, I was able to extract all current balances for Dash.
Have fun with this: https://transfer.sh/109sSU/balances.dash.out.gz

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: erajaykumarrana on March 19, 2018, 04:42:54 AM

frnd... can you please tell me the dependancy package to compile your softwre in ubuntu linux....
I have fresh install of linux ubuntu.... but there are so many errors when I tried to compile it using make....
can you please PM me(I cant send you message as u block newbie....)....

Please help me, i Need all the unspent addresses of many coins.... i dont need balance I just need addresses....

I've sent you a PM with my email address so you can communicate privately, if you need to.

Which coins will you this information ?

The chainstate software is tied to Bitcoin Core (0.15+)'s chainstate format. If dash (or any other coin) is not using the same format, it won't work. Note that Bitcoin Core implemented this format in April last year, so it is most likely this software won't work.

I could take a look at DASH, it would require me a few days however.

erajaykumarrana

newbie

Activity: 2

Merit: 0

Quote from: starmyc on March 16, 2018, 09:58:41 AM

Quote from: bigman3912 on March 16, 2018, 09:40:43 AM

A few minutes? Whole core? That's really fast. When i tried to parse it with old "blockparser" i have 20% in 2 days and time was increased very fast every hour.

Can you share with me your chainstate parser? Wanna try it. Thanks.

The source code is opensource, on github: https://github.com/mycroft/chainstate
It doesn't parse the blockchain. It just read the chainstate from Bitcoin Core. https://en.bitcoin.it/wiki/Data_directory#chainstate_subdirectory
The chainstate is the result of parsing the blockchain, and contains all unspent transactions for new transaction validation purposes. When a new block is recieved by Bitcoin core, it will update this database to remove spent transactions and fills in new unspent transactions.

As for now, the chainstate is a 3-4GB leveldb database, with ~35 millions records (I didn't check recently). My tool will just read this database, and for each record (which are unspent transactions), decode it and find out which is the address & value (unspent amount of bitcoin).
Then, with a set of scripts, I sort unique addresses & compute their final balance. The whole process takes ~ 10 minutes (and it is not optimized at all).

I think I could do this. Mind to submit a patch ?
[/quote]

frnd... can you please tell me the dependancy package to compile your softwre in ubuntu linux....
I have fresh install of linux ubuntu.... but there are so many errors when I tried to compile it using make....
can you please PM me(I cant send you message as u block newbie....)....

Please help me, i Need all the unspent addresses of many coins.... i dont need balance I just need addresses....

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: dinhbiplus on March 18, 2018, 04:03:36 PM

Can't you just sort the output?

I think I could do this. Mind to submit a patch ?

dinhbiplus

jr. member

Activity: 42

Merit: 1

Can't you just sort the output?

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: dxlmz01 on March 16, 2018, 03:04:11 PM

Quote from: starmyc on March 16, 2018, 05:41:53 AM

New dump for all bitcoin balances, as of today: balances.out.gz

I think I'll remove small balances in the next version of this dump, as this file is really big and there are way too much really small wallets.

Have a nice day !

Thanks alot, I appreciate all the effort you're putting into this thread

can i ask for a favor ? if its not too much to ask, of course ...

I need a dump of addresses with balances until block #337000, not further.
can you find some spare time in your schedule to provide that ? I will be very thankful.

I can do this, but it would require a little work. Can you send me a PM about this (can't send you one, you're blocking messages from newbies like me Wink

) ?

dxlmz01

newbie

Activity: 7

Merit: 0

Quote from: starmyc on March 16, 2018, 05:41:53 AM

New dump for all bitcoin balances, as of today: balances.out.gz

I think I'll remove small balances in the next version of this dump, as this file is really big and there are way too much really small wallets.

Have a nice day !

Thanks alot, I appreciate all the effort you're putting into this thread

can i ask for a favor ? if its not too much to ask, of course ...

I need a dump of addresses with balances until block #337000, not further.
can you find some spare time in your schedule to provide that ? I will be very thankful.

bigman3912

newbie

Activity: 12

Merit: 0

Quote from: starmyc on March 16, 2018, 09:58:41 AM

Quote from: bigman3912 on March 16, 2018, 09:40:43 AM

A few minutes? Whole core? That's really fast. When i tried to parse it with old "blockparser" i have 20% in 2 days and time was increased very fast every hour.

Can you share with me your chainstate parser? Wanna try it. Thanks.

The source code is opensource, on github: https://github.com/mycroft/chainstate
It doesn't parse the blockchain. It just read the chainstate from Bitcoin Core. https://en.bitcoin.it/wiki/Data_directory#chainstate_subdirectory
The chainstate is the result of parsing the blockchain, and contains all unspent transactions for new transaction validation purposes. When a new block is recieved by Bitcoin core, it will update this database to remove spent transactions and fills in new unspent transactions.

As for now, the chainstate is a 3-4GB leveldb database, with ~35 millions records (I didn't check recently). My tool will just read this database, and for each record (which are unspent transactions), decode it and find out which is the address & value (unspent amount of bitcoin).
Then, with a set of scripts, I sort unique addresses & compute their final balance. The whole process takes ~ 10 minutes (and it is not optimized at all).

Understood. Will try it on my dual xeon server.

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: bigman3912 on March 16, 2018, 09:40:43 AM

A few minutes? Whole core? That's really fast. When i tried to parse it with old "blockparser" i have 20% in 2 days and time was increased very fast every hour.

Can you share with me your chainstate parser? Wanna try it. Thanks.

The source code is opensource, on github: https://github.com/mycroft/chainstate
It doesn't parse the blockchain. It just read the chainstate from Bitcoin Core. https://en.bitcoin.it/wiki/Data_directory#chainstate_subdirectory
The chainstate is the result of parsing the blockchain, and contains all unspent transactions for new transaction validation purposes. When a new block is recieved by Bitcoin core, it will update this database to remove spent transactions and fills in new unspent transactions.

As for now, the chainstate is a 3-4GB leveldb database, with ~35 millions records (I didn't check recently). My tool will just read this database, and for each record (which are unspent transactions), decode it and find out which is the address & value (unspent amount of bitcoin).
Then, with a set of scripts, I sort unique addresses & compute their final balance. The whole process takes ~ 10 minutes (and it is not optimized at all).

bigman3912

newbie

Activity: 12

Merit: 0

Quote from: starmyc on March 16, 2018, 09:17:13 AM

Quote from: bigman3912 on March 16, 2018, 05:49:21 AM

Thank you very much. so, i guess your chainstate parser still works?

Yup, it works great and until Bitcoin Core changes its internal db format, it will continue to work. It only takes a few minutes to generate & sort.

Quote from: bigman3912 on March 16, 2018, 05:49:21 AM

btw, every balance is matter, even it equals 1 satoshi.

file is not too big. it easily opens on a good pc.

This is not a file size problem, but a bandwidth problem. As for now, I'm using a "free" service, but I would rather host it on my own server, but BW is expensive for free stuff like this.

Cheers.

A few minutes? Whole core? That's really fast. When i tried to parse it with old "blockparser" i have 20% in 2 days and time was increased very fast every hour.

Can you share with me your chainstate parser? Wanna try it. Thanks.

LeonsBTT

newbie

Activity: 14

Merit: 0

Quote from: starmyc on March 16, 2018, 09:19:41 AM

Quote from: LeonsBTT on March 16, 2018, 09:15:10 AM

What if a person has more than 1 account?

I mean what is the value of these files?

Nobody can regroup all addresses of a single wallet.

If you don't value this information, you're free to not download it.

What I mean is that if I am a whale that hold 10% of total bitcoins for me it is very easy to have different accounts, adresses etc and divide my coins between them. In other words, What conclusions do you want to get using this data? Do you think that they will be reliable?

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: LeonsBTT on March 16, 2018, 09:15:10 AM

What if a person has more than 1 account?

I mean what is the value of these files?

Nobody can regroup all addresses of a single wallet.

If you don't value this information, you're free to not download it.

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: bigman3912 on March 16, 2018, 05:49:21 AM

Thank you very much. so, i guess your chainstate parser still works?

Yup, it works great and until Bitcoin Core changes its internal db format, it will continue to work. It only takes a few minutes to generate & sort.

Quote from: bigman3912 on March 16, 2018, 05:49:21 AM

btw, every balance is matter, even it equals 1 satoshi.

file is not too big. it easily opens on a good pc.

This is not a file size problem, but a bandwidth problem. As for now, I'm using a "free" service, but I would rather host it on my own server, but BW is expensive for free stuff like this.

Cheers.

LeonsBTT

newbie

Activity: 14

Merit: 0

What if a person has more than 1 account?

I mean what is the value of these files?

bigman3912

newbie

Activity: 12

Merit: 0

Thank you very much. so, i guess your chainstate parser still works?
btw, every balance is matter, even it equals 1 satoshi.

file is not too big. it easily opens on a good pc.

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

New dump for all bitcoin balances, as of today: balances.out.gz

I think I'll remove small balances in the next version of this dump, as this file is really big and there are way too much really small wallets.

Have a nice day !

starmyc

full member

Activity: 198

Merit: 130

Some random software engineer

Quote from: erajaykumarrana on March 13, 2018, 06:26:04 AM

friend can you tell me how you generated this list, I have to generate it for dash coin....

I have seen your code on github... how can i compile it in visual studio 2017 on windows
do you parse chain state directory...

I've used this tool https://github.com/mycroft/chainstate I worked on as you know, using Bitcoin Core 0.15.1's updated chainstate, on Linux.

Unfortunately, I've a bad news for you... First, I've no knowledge in Visual Studio and I'm pretty much sure you'll need to update the code a lot to make it compile.
The good news is that dashcore also merged the new chainstate db format (https://github.com/bitcoin/bitcoin/pull/10195) I'm parsing, so it would be possible that this tool works correctly against dashcore!

Topic: Get list of all addresses with a balance over x? - page 7. (Read 43401 times)