What do you think about adding unpruneblockchain command?

gmaxwell

staff

Activity: 4242

Merit: 8672

Quote from: ercewubam on January 04, 2020, 04:58:40 AM

Quote

You can't fetch undo data right now, and you couldn't verify undo data as correct even if you could fetch it. So you'd have to store hashes of pruned undo data, and have some way to fetch it from peers.

Why undo data couldn't be re-generated by the client, based on downloaded blocks and UTXO set?

The only way to regenrate it would be to obtain and reprocess _ALL_ prior blocks and build a new utxo set from them. The undo data is the outputs removed from the utxo set by the block, so the data in it is not in the UTXO set after that point, nor is it in the removed block itself... it comes from prior blocks.

So if thats how you 'unprune' you are back to refetching the whole chain in order to do it.

ercewubam

newbie

Activity: 26

Merit: 62

Quote

Core will just stop from deleting the older blocks until it reached the new prune size.

Yes, I know. But the whole point is getting access to already pruned blocks. For example if I have all blocks from 572000 to the current one, it is now impossible to access block 570000 without redownloading everything. That is why I called it "unprune".

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

Quote from: ercewubam on January 03, 2020, 07:40:53 PM

-snip- But increasing it is difficult, because you have to redownload the whole chain even when you just need 2 GB more blocks when actually keeping last 50 GB.

If this still applies: https://bitcoin.stackexchange.com/a/46829, it will not redownload the whole blockchain.
Core will just stop from deleting the older blocks until it reached the new prune size.
So if you set an additional 50000MiB, it will not delete the existing blocks for about 9 months.

ercewubam

newbie

Activity: 26

Merit: 62

Quote

You can't fetch undo data right now, and you couldn't verify undo data as correct even if you could fetch it. So you'd have to store hashes of pruned undo data, and have some way to fetch it from peers.

Why undo data couldn't be re-generated by the client, based on downloaded blocks and UTXO set?

Quote

With 100 GB, in average rate of blockchain size increases annually, they will be able to store the blockchain transactions within past two years.

That is why I thought with 50 GB I will be able to store everything within past year. But it turned out that it is not enough to get some transactions from March or April 2019 and I ended up with around 9 months of history instead of 1 year.

tranthidung

legendary

Activity: 2310

Merit: 4085

Farewell o_e_l_e_o

Quote from: ercewubam on January 03, 2020, 07:40:53 PM

Not exactly. Disabling or enabling the prune is actually handled correctly. I thought about changing size of pruned blocks without redownloading everything. Changing pruning to lower value is easy, the client will just remove some blocks. But increasing it is difficult, because you have to redownload the whole chain even when you just need 2 GB more blocks when actually keeping last 50 GB.

Whenever you change your n parameter for prune=n, your Bitcoin Core will ask you to restart it with the message "Client restart required to activate changes".

In Pros and Cons of Bitcoin Node types (Full node and Prune node), I advice to set up the n parameter around 99 GB. That threshold tested by some others and personally, from the data and chart of bitcoin blockchain size, I felt that 100 GB is good enough for the one who want to run prune node.

With 100 GB, in average rate of blockchain size increases annually, they will be able to store the blockchain transactions within past two years. As always, if you don't have issues with data capacity, let's set up and run your Full node.
And if you don't have anything to do in-depth technically, I think using SPV wallets, such as Electrum is not too bad option.

gmaxwell

staff

Activity: 4242

Merit: 8672

Unpruning is not possible without changing the p2p protocol and storing more data, at a minimum.

You can't fetch undo data right now, and you couldn't verify undo data as correct even if you could fetch it. So you'd have to store hashes of pruned undo data, and have some way to fetch it from peers.

I doubt anyone would want the extra code or testing burden for supporting a 'half pruned block' state where some blocks have block data but not undo data.

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: ercewubam on January 03, 2020, 07:40:53 PM

But increasing it is difficult, because you have to redownload the whole chain even when you just need 2 GB more blocks when actually keeping last 50 GB.

i'm not sure if this command is needed that much. but it could be implemented in either one of these two ways in my opinion:
- when in pruned mode you keep the last n blocks which means when n+1st block is mined the block 0 should be discarded to keep the size at that fixed value. so one way of letting you increase the size would be letting the chain you store grow until it reaches the new value.
in other words, you don't download the history just let more blocks be stored from now on.

- the client could also store block headers (like any SPV client) which would take up an additional ~40 MB and then if it needed any part of the history it can just fetch the blocks with their heights and check them against the headers it already stores.

ercewubam

newbie

Activity: 26

Merit: 62

Quote

Do you mean is to disable the prune and download only selected heights?

Not exactly. Disabling or enabling the prune is actually handled correctly. I thought about changing size of pruned blocks without redownloading everything. Changing pruning to lower value is easy, the client will just remove some blocks. But increasing it is difficult, because you have to redownload the whole chain even when you just need 2 GB more blocks when actually keeping last 50 GB.

BitMaxz

legendary

Activity: 3374

Merit: 3095

BTC price road to $80k

What do you mean by unpruneblockchain? Do you mean is to disable the prune and download only selected heights?

I think much better read this article below.

- Bitcoin blockchain pruning – How to reduce Bitcoin core wallet file size
- Pro's and Con's on Bitcoin Block Pruning

I don't know if you can manually choose if what height you want to be pruned but you can set the pruned between 0 to 550. The 550 is the size of the pruned blockchain and "Automatically deletes previous block files to stay at 550gb target size. Then "prune=0" is "Disables pruning mode" and the "prune=1" is "Allows manual pruning via RPC"

I don't much have experience in using bitcoin core wallet but let me know why do you want to add "unprineblockchain command"?

ercewubam

newbie

Activity: 26

Merit: 62

Now, people can prune blocks manually by using pruneblockchain command. Is it acceptable to implement unpruneblockchain, which will redownload missing blocks up to selected height? This command would succeed only if all hashes still matches (meaning there was no blockchain reorganization in pruned blocks and all UTXOs are still correct).

Topic: What do you think about adding unpruneblockchain command? (Read 249 times)