Author

Topic: Why DB so big? (Read 293 times)

staff
Activity: 3458
Merit: 6793
Just writing some code
January 25, 2018, 11:02:41 AM
#13
But still, you will not be able to reuse your block files within a forked client to access forks faster for example.
You can always stop a node after syncing to a certain block height, backup the datadir, and then resume syncing. Now you will have a copy of the datadir before any forks and just copy and paste that datadir into the datadirs of forked clients to sync their forked blockchain. Since the pruned datadir is small, there isn't much additional disk space taken up by doing this, and way less than if you didn't prune the blockchain.
legendary
Activity: 1372
Merit: 1252
January 25, 2018, 10:13:01 AM
#12
Pruned mode is paradoxical because you need to be able to download the entire thing at least once... which kind of defeats the purpose. At best you would be able to save space afterwards. If we could engineer something that is able to download only the last GB which would be a reasonable amount since trying to go "1GB back in time" would require an huge amount of money for such a hashrate attack... but I don't see how that's even possible. You would need some epic wizardry engineering to pull it off without opening potential exploits.
Not really. Just to speak in my context, everyone in my country has unlimited bandwidth so it was never an issue about bandwidth. However, not everyone has enough disk space. Pruning solves this problem since it only store X mb of the blocks at any given time so you could theoractically have 550 MB of space before syncing and still be able to do it successfully. Whilst such a massive block reorg is rather uncommon, I really doubt anyone would be happy to see them needing to resyncing everything from scratch if it ever happens.

Aha that's great, I thought that pruning required you to have the required total space to sync at least once, but it seems that it never downloads an higher amount that the selected pruning amount, well then yes that's a solution.

But still, you will not be able to reuse your block files within a forked client to access forks faster for example. Im sure there are other use cases of owning the entire blockchain other than the security itself.
zaq
newbie
Activity: 2
Merit: 0
January 25, 2018, 07:01:49 AM
#11
Thank you all for exhaustive answers.
staff
Activity: 3458
Merit: 6793
Just writing some code
January 24, 2018, 07:44:22 PM
#10
Pruned mode is paradoxical because you need to be able to download the entire thing at least once... which kind of defeats the purpose.
No, that is its express and only purpose. Pruned mode is for the sole purpose of reducing disk space. It has nothing to do with bandwidth and was not designed for reducing bandwidth or computation requirements. It was literally designed to only reduce disk space usage.
legendary
Activity: 3038
Merit: 4418
Crypto Swap Exchange
January 24, 2018, 05:48:13 PM
#9
Pruned mode is paradoxical because you need to be able to download the entire thing at least once... which kind of defeats the purpose. At best you would be able to save space afterwards. If we could engineer something that is able to download only the last GB which would be a reasonable amount since trying to go "1GB back in time" would require an huge amount of money for such a hashrate attack... but I don't see how that's even possible. You would need some epic wizardry engineering to pull it off without opening potential exploits.
Not really. Just to speak in my context, everyone in my country has unlimited bandwidth so it was never an issue about bandwidth. However, not everyone has enough disk space. Pruning solves this problem since it only store X mb of the blocks at any given time so you could theoractically have 550 MB of space before syncing and still be able to do it successfully. Whilst such a massive block reorg is rather uncommon, I really doubt anyone would be happy to see them needing to resyncing everything from scratch if it ever happens.
member
Activity: 122
Merit: 10
Do what needs to be done.
January 24, 2018, 05:36:52 PM
#8
It's whole idea is to have everything validated from start in order to prove there are no faults. Therefore it all goes trough validation. Smiley
member
Activity: 210
Merit: 26
High fees = low BTC price
January 24, 2018, 05:33:46 PM
#7
Has to be big because it stores lots of data compression is double edged because it takes time to decompress
not that this is a magical cure for Bitcoin and the solution is to break the chain up into smaller chunks 

It's possible, but the software engineering to do it is extensive. Even then, I think mining nodes still need the complete tx history. The developers have a plan to design such a mode, but they can only do as much work as there is time to do it. I *think* Pieter Wuille had the most promising idea to do something like this, but I can't remember what the details are right now.

I like the full history on the block chain but maybe could live with roll up's from time to time but all the archive data
should be available
legendary
Activity: 1372
Merit: 1252
January 24, 2018, 03:14:45 PM
#6
Individually, you don't need to store the whole Blockchain. If you have no need for transaction information for addresses not associated with your wallet file, you don't have to use it. You can easily just prune the Blockchain. The disadvantage is that you can't access any transactions related to any other address that is not associated with your wallet.

You need to download the whole Blockchain though. It is unsafe to download only parts of it and expect it to be valid. At all times, a portion of the nodes must have the full Blockchain to facilitate their peers in synchronizing.

Pruned mode is paradoxical because you need to be able to download the entire thing at least once... which kind of defeats the purpose. At best you would be able to save space afterwards. If we could engineer something that is able to download only the last GB which would be a reasonable amount since trying to go "1GB back in time" would require an huge amount of money for such a hashrate attack... but I don't see how that's even possible. You would need some epic wizardry engineering to pull it off without opening potential exploits.

You can use wallet which not store all blockchain on your computer, look at Electrum

SPV wallets aren't safe... and I don't think we will ever be able to get past having to download the entire blockchain and validate it at least once.
full member
Activity: 714
Merit: 104
January 24, 2018, 10:55:28 AM
#5
You can use wallet which not store all blockchain on your computer, look at Electrum
legendary
Activity: 3038
Merit: 4418
Crypto Swap Exchange
January 24, 2018, 10:27:30 AM
#4
Individually, you don't need to store the whole Blockchain. If you have no need for transaction information for addresses not associated with your wallet file, you don't have to use it. You can easily just prune the Blockchain. The disadvantage is that you can't access any transactions related to any other address that is not associated with your wallet.

You need to download the whole Blockchain though. It is unsafe to download only parts of it and expect it to be valid. At all times, a portion of the nodes must have the full Blockchain to facilitate their peers in synchronizing.
sr. member
Activity: 322
Merit: 363
39twH4PSYgDSzU7sLnRoDfthR6gWYrrPoD
January 24, 2018, 09:55:19 AM
#3
Decentralisation.
If all transactions are stored then your node doesn't need to trust that a transaction is correct , it just verifies everything from the genesis block.

If you didn't store all transactions then nodes will have to trust that their info is correct, which is not really trustless.


Bitcoin has the concept of checkpoints though, which are hard coded to the client which prevents the rewriting of early blocks.
https://github.com/bitcoin/bitcoin/blob/c091b99/src/chainparams.cpp#L150-L162
legendary
Activity: 3430
Merit: 3083
January 24, 2018, 07:36:51 AM
#2
Would it be sufficient to keep unspent outputs only?

It's possible, but the software engineering to do it is extensive. Even then, I think mining nodes still need the complete tx history. The developers have a plan to design such a mode, but they can only do as much work as there is time to do it. I *think* Pieter Wuille had the most promising idea to do something like this, but I can't remember what the details are right now.
zaq
newbie
Activity: 2
Merit: 0
January 24, 2018, 07:07:49 AM
#1
Why do we need to keep all history on all nodes? Would it be sufficient to keep unspent outputs only?
Jump to: