Author

Topic: Balances for accounts at a specific block (Read 258 times)

full member
Activity: 198
Merit: 130
Some random software engineer
Thanks for the advice. You guys have pointed me in the right redirection.

Are there any stats anywhere on how many addresses without zero balances are in existence?

I've exported those last week: http://bit.ly/BtcBalances20180427
I'm counting ~21.8 millions of addresses with balance currently.

This guy asks for money to share the dump.  Undecided

My curiosity about btc whales and monitoring the increase of active wallets doesn't go that far.

Bad evil guy who is asking money to pay bandwidth Sad

The tools I wrote to generate those are opensourced. You're free to use them to generate it and make it available for everyone!
newbie
Activity: 54
Merit: 0
Thanks for the advice. You guys have pointed me in the right redirection.

Are there any stats anywhere on how many addresses without zero balances are in existence?

I've exported those last week: http://bit.ly/BtcBalances20180427
I'm counting ~21.8 millions of addresses with balance currently.

This guy asks for money to share the dump.  Undecided

My curiosity about btc whales and monitoring the increase of active wallets doesn't go that far.
newbie
Activity: 54
Merit: 0
Thanks for the advice. You guys have pointed me in the right redirection.

Are there any stats anywhere on how many addresses without zero balances are in existence?

I've exported those last week: http://bit.ly/BtcBalances20180427
I'm counting ~21.8 millions of addresses with balance currently.

would you mind re-sharing the file? your link is down. Thanks
legendary
Activity: 3472
Merit: 4801
you could probably save balance of each address whose balance was changed by a transaction in a block at each height.

While I may not have spelled out the exact process, this is effectively what I intended.

If a balance hasn't changed, there is no need to store another record for it since you already have the balance and the block height where that balance was established.
full member
Activity: 198
Merit: 130
Some random software engineer
Thanks for the advice. You guys have pointed me in the right redirection.

Are there any stats anywhere on how many addresses without zero balances are in existence?

I've exported those last week: http://bit.ly/BtcBalances20180427
I'm counting ~21.8 millions of addresses with balance currently.
legendary
Activity: 3612
Merit: 5297
https://merel.mobi => buy facemasks with BTC/LTC
Thanks for the advice. You guys have pointed me in the right redirection.

Are there any stats anywhere on how many addresses without zero balances are in existence?

I've generated a list of all funded addresses for somebody on this forum a while ago... I used this script: https://github.com/cryptah/utxo-dump

IIRC, at that time, there were ~40.000.000 unspent outputs in the UTXO set, don't remember how many addresses were funded (multiple unspent outputs can fund the same address).
I can only estimate this number to have grown since then (about a month ago iirc)
newbie
Activity: 19
Merit: 0
Thanks for the advice. You guys have pointed me in the right redirection.

Are there any stats anywhere on how many addresses without zero balances are in existence?
legendary
Activity: 3612
Merit: 5297
https://merel.mobi => buy facemasks with BTC/LTC
Or, even easier...

Start at the beginning aggregating balances. As you encounter each new block, record the block height and current balances in a database.  Then when you want to know any historical balance, your can just query your database for the block height in question.  Meanwhile, your node can continue to remain synchronized and can continue to update your database as new blocks are received.

Wouldn't that require a huge database?
I think it wouldn't be practical to save all balances for all addresses that ever existed at each blockheight, but you could probably save balance of each address whose balance was changed by a transaction in a block at each height... That way you should be able to query the value of the sum of the unspent outputs funding address x at height y where the blockheight = the max blockheight for an entry for address x.

Something like this
Code:
CREATE TABLE balances (
    id int NOT NULL AUTO_INCREMENT,
    address varchar(45),
    balance_at_height int(15),
    height int(10)
    PRIMARY KEY (id)
);

For example, if my address was funded for the very first time at height 40000, then was funded for a second time at 401000 and i spent both outputs at 402000, only 3 inserts would be needed for my total history...

Code:
insert into (address, balace_at_height, height) values ("1MocACiWLM8bYn8pCrYjy6uHq4U3CkxLaa", 100000, 400000);
insert into (address, balace_at_height, height) values ("1MocACiWLM8bYn8pCrYjy6uHq4U3CkxLaa", 200000, 401000);
insert into (address, balace_at_height, height) values ("1MocACiWLM8bYn8pCrYjy6uHq4U3CkxLaa", 0, 402000);

Each new block you'd have to parse 1500? transactions, so if you'd only have to save the balances of addresses whose balance changed at each blockheight, you'd have to insert at least ~1500 changes/block * ~144 blocks/day = 216000 changes each day... That seems doable
legendary
Activity: 3472
Merit: 4801
Got it.

You said earlier I would have to aggregate it by following the blocks but would it not be easier to just modify the daemon to stop syncing at a specific block height and then use existing RPC api's to query the balances from that node?

Or, even easier...

Start at the beginning aggregating balances. As you encounter each new block, record the block height and current balances in a database.  Then when you want to know any historical balance, your can just query your database for the block height in question.  Meanwhile, your node can continue to remain synchronized and can continue to update your database as new blocks are received.
full member
Activity: 198
Merit: 130
Some random software engineer
Got it.

You said earlier I would have to aggregate it by following the blocks but would it not be easier to just modify the daemon to stop syncing at a specific block height and then use existing RPC api's to query the balances from that node?

Having already done this once, yes, it is way more easy to do this way.
newbie
Activity: 19
Merit: 0
Got it.

You said earlier I would have to aggregate it by following the blocks but would it not be easier to just modify the daemon to stop syncing at a specific block height and then use existing RPC api's to query the balances from that node?
legendary
Activity: 3612
Merit: 5297
https://merel.mobi => buy facemasks with BTC/LTC
That makes sense.

If it is always aggregated, how are balance lookups so fast?

Because when you install a node, it syncs the complete blockchain. During this syncing, the blocks are parsed and verified, and a node builds the UTXO set while syncing (it aggregates from block #1 up untill the current height). This syncing process takes hours, sometimes even days (depending on your node's version, the IO capacity, the memory speed, the cpu,...). After the initial sync, the UTXO set up untill a certain height is built, so as long as you keep your node running 24/7, or at least start it a couple times a week, it can keep up with all the new blocks pretty fast, your node just parses a new block when it receives it, and adapts its utxo set accordingly.

So, when you use a full client, your PC/server already did all these aggregations in the past, so you can now see your balance instantly. If you use an SPV client (like electrum, or most of the hardware wallets), this SPV client is connected to a full node, this node built the UTXO set in the past when it was syncing, so it can also reply instantly when an SPV client querys the unspent outputs funding a certain address.
newbie
Activity: 19
Merit: 0
That makes sense.

If it is always aggregated, how are balance lookups so fast?
legendary
Activity: 3612
Merit: 5297
https://merel.mobi => buy facemasks with BTC/LTC
Its possible to migrate a copy of the main ledger to a testnet and then reverse the data to a certain block for knowing what account balances were at that specific block?

Or would it be better to aggregate from the genesis block to the required block?

You always aggregate from the genesis block to the required block... The latest block does not specify all unspent outputs (which you can use to calculate the total value funding each of your addresses). Basically, a block contains a bunch of transactions. Each transaction just tells you which unspent output(s) from the UTXO set are used, and which new (unspent) outputs are generated. Offcourse, there is also some other data in a block (like the signature , the block header).

In order to generate the UTXO set (the set of unspent outputs that you can use to calculate your balance), you start from the genesis block and follow each unspent output. As soon as a transaction uses an unspent output from your current utxo set, it must be removed from the utxo set, and the output of the transaction will be added to the utxo set... The coinbase transaction also generates a new unspent output. If you do this upto height x, you automatically know all unspent outputs at height x, so you can use them to calculate the balances.
newbie
Activity: 19
Merit: 0
Its possible to migrate a copy of the main ledger to a testnet and then reverse the data to a certain block for knowing what account balances were at that specific block?

Or would it be better to aggregate from the genesis block to the required block?
Jump to: