Pages:
Author

Topic: Why do you need to download 7 years of chain block - page 2. (Read 9120 times)

full member
Activity: 315
Merit: 103
In the first place, there is no rationale to store 7 years of history for synced nodes. But how a new node could get into network if all the fullnodes have thrown away all the blocks(except last few ones to handle possible rollbacks)? Bitcoin has no answer for that.

I've developed a proposal to fix the issue and solve some important questions towards better scalability. The paper is just uploaded http://arxiv.org/abs/1603.07926 . I would be happy to get feedback.

This paper is a bit of a disappointment, sad to say.  First-- it makes a number of false claims about the capabilities of the existing Bitcoin system, e.g.

"To be rational it is needed to switch to an implementation of the Bitcoin protocol which does not hold the unnecessary data (such an implementation of a full node does not exist to the best of our knowledge)."

The whole reason that Bitcoin has state snapshots, as described in the paper, was because they were introduced to enable strong pruning (a more limited version is described in section 7 of the Bitcoin whitepaper).  Bitcoin has supported operating in a pruned fashion for _years_.


The primary suggestion the paper is making is to bootstrap nodes from state snapshots. The paper describes this as "trustless" but it is _not_, rather it makes the same strong assumption of hashpower honesty that SPV (lite wallet nodes do), implicitly trusting the history of the longest chain to be faithful.

This is not a new idea, its expression spans at least back to 2011:

https://bitcointalksearch.org/topic/merkle-tree-of-open-transactions-for-lite-mode-21995

and

https://en.bitcoin.it/wiki/User:DiThi/MTUT

(as is using random tests for data in the mining process,-- though the scheme you suggest is fully delegatable, so long as the miner doesn't mind giving temporary custody of his income to the centralized party holding the world's only copy of the blockchain. Smiley )

The state commitment proposals face a couple challenges, one is that if you are willing to strongly trust the hashpower-- then why not use the vastly more efficient SPV mode?  The other is that the initial efforts to implement continual state commitments found that the overhead (primarily IO) of maintaining them multiplied the full node operating costs at runtime by several fold.  The costs have made what would otherwise be another option for client security less attractive, supporting a more niche security model would be fine if it were inexpensive.

Alternative designs like mimblewimble try to make bootstrapping more efficient without adding a new form of strong trust towards miners.


Dear Greg, it seems you've missed a part of the protocol. Let me provide a quick description of Rollerchain, more precisely, modifications from the Bitcoin.

1. We authenticate the state and include root value into a blockheader. It is an old idea yes, and already implemented in Ethereum (and some other coins) as mentioned in the paper.

2. We then modify a Proof-of-Work function. A miner chooses "k" state snapshot versions out of last "n" a (sufficiently large) network stores collectively. In order to generate a block miner needs to provide proofs of possession for the state snapshots. On a new block arrival a miner updates k+1 states, not one, so full blocks (since minimal value in "k") are also needed.

Thus miners store a distributed database of last "n" full blocks AND state snapshots getting rewards for that activity. A new node could download not just a last snapshot, but from "n" blocks ago (or from somewhere in between). So it is not needed to "strongly trust the hashpower", but "hashpower" and also up to "n" last full blocks("n" considered to be >> than a deepest fork possible, say, 5000-10000 for Bitcoin) . And this is not about SPV, but about getting fullnode-going-from-genesis security(with probability of divergence going down exponentially with "n", see Theorem 2). Full blocks not needed in order to mine could be removed by all the nodes.

If you have concerns about this scheme please let me know, and I will try to formalize/address them.

Authenticated state root in a block header could be useful anyway for better SPVs. On efficient implementation, we're now working on that along with benchmarking. Let's see how it goes.


P.S. However, as PoW function modification is needed it is unlikely Bitcoin can go this way(miners will not throw away their ASICs).



full member
Activity: 215
Merit: 100
It reminds me when i first download the qt wallet, it said I need to download the blockchain for many days. LOL

The answer of why: the wallet needs to have full blockchain file for sync, we need to sync the wallet to see more transactions details, but you can still send bitcoin if the wallet is not sync
legendary
Activity: 3878
Merit: 1193
Will it ever be possible to enable pruning mode without having to download the entire chain the first time?

That depends on what you want to do. If you just want to verify a specific transaction, like when you receive a payment, you really only need to download all blocks that include all previous inputs. Then, any inputs that have been spent can be safely pruned.
legendary
Activity: 868
Merit: 1006
Will it ever be possible to enable pruning mode without having to download the entire chain the first time?
Isn't 1GB enough to guarantee safety? I mean who (attacker) is ever going to manage to go "1gb back in time?"
Of course full nodes is deal but I think people would use Core wallet more if you could enable pruned without download entire chain first time.
full member
Activity: 219
Merit: 102
ok, so I installed bitcoin core latest version and it downloaded over 80G of chain block. Let's assume, just for the fun of it, there are 1000 bitcoin core users out there. That's ~8T of wasted disk space. and considering bitcoin will live another 7 years and it will grow of couse, that's like ~20T of disk space (1000 users remember?) for what? couldn't be a centerlaize, maybe mirrored, location that the client will ask for the chain block from there? Why do we need to download it?

turn your point of view...
in a next futures, nearest than 7 years, 20Tb will be like today 20gb or maybe 20Mb..
just look here
http://arstechnica.com/gadgets/2016/08/seagate-unveils-60tb-ssd-the-worlds-largest-hard-drive/
give it time

Platitudes like this are speculative, unhelpful and infuriating.

Now we are being told we need two drives because the block chain gets corrupted. The current blockchain storage isn't fit for purpose and you are in denial.
hero member
Activity: 910
Merit: 511
ok, so I installed bitcoin core latest version and it downloaded over 80G of chain block. Let's assume, just for the fun of it, there are 1000 bitcoin core users out there. That's ~8T of wasted disk space. and considering bitcoin will live another 7 years and it will grow of couse, that's like ~20T of disk space (1000 users remember?) for what? couldn't be a centerlaize, maybe mirrored, location that the client will ask for the chain block from there? Why do we need to download it?

turn your point of view...
in a next futures, nearest than 7 years, 20Tb will be like today 20gb or maybe 20Mb..
just look here
http://arstechnica.com/gadgets/2016/08/seagate-unveils-60tb-ssd-the-worlds-largest-hard-drive/
give it time
staff
Activity: 4242
Merit: 8672
In the first place, there is no rationale to store 7 years of history for synced nodes. But how a new node could get into network if all the fullnodes have thrown away all the blocks(except last few ones to handle possible rollbacks)? Bitcoin has no answer for that.

I've developed a proposal to fix the issue and solve some important questions towards better scalability. The paper is just uploaded http://arxiv.org/abs/1603.07926 . I would be happy to get feedback.

This paper is a bit of a disappointment, sad to say.  First-- it makes a number of false claims about the capabilities of the existing Bitcoin system, e.g.

"To be rational it is needed to switch to an implementation of the Bitcoin protocol which does not hold the unnecessary data (such an implementation of a full node does not exist to the best of our knowledge)."

The whole reason that Bitcoin has state snapshots, as described in the paper, was because they were introduced to enable strong pruning (a more limited version is described in section 7 of the Bitcoin whitepaper).  Bitcoin has supported operating in a pruned fashion for _years_.


The primary suggestion the paper is making is to bootstrap nodes from state snapshots. The paper describes this as "trustless" but it is _not_, rather it makes the same strong assumption of hashpower honesty that SPV (lite wallet nodes do), implicitly trusting the history of the longest chain to be faithful.

This is not a new idea, its expression spans at least back to 2011:

https://bitcointalksearch.org/topic/merkle-tree-of-open-transactions-for-lite-mode-21995

and

https://en.bitcoin.it/wiki/User:DiThi/MTUT

(as is using random tests for data in the mining process,-- though the scheme you suggest is fully delegatable, so long as the miner doesn't mind giving temporary custody of his income to the centralized party holding the world's only copy of the blockchain. Smiley )

The state commitment proposals face a couple challenges, one is that if you are willing to strongly trust the hashpower-- then why not use the vastly more efficient SPV mode?  The other is that the initial efforts to implement continual state commitments found that the overhead (primarily IO) of maintaining them multiplied the full node operating costs at runtime by several fold.  The costs have made what would otherwise be another option for client security less attractive, supporting a more niche security model would be fine if it were inexpensive.

Alternative designs like mimblewimble try to make bootstrapping more efficient without adding a new form of strong trust towards miners.
full member
Activity: 219
Merit: 102
In the first place, there is no rationale to store 7 years of history for synced nodes. But how a new node could get into network if all the fullnodes have thrown away all the blocks(except last few ones to handle possible rollbacks)? Bitcoin has no answer for that.

I've developed a proposal to fix the issue and solve some important questions towards better scalability. The paper is just uploaded http://arxiv.org/abs/1603.07926 . I would be happy to get feedback.

Great to see people are working on this. I was beginning to think the "technology is cheap" brigade were winning. It was a bit much for me to take in in one go so I'll need to read it several times but there was once thing that struck a chord with me - the linked list. I think this is a key point to enable breaking the blockchain into resolvable chunks to enable both forward and backward discovery. You only need to find one chunk to be able to construct the whole chain by pregressively downloading the prev/next in the list..

I too have been working in that direction except using a variant of the bitorrent DHT. There are a couple of proposals whereby the DHT conveniently uses  merkle trees! and an arbitrary data field (1K) which I would increase to 1MB (current block size). The protocol would additionally have the prev/next  (linked list) of the blocks so the blockchain would be stored in the DHT and cached throughout the network. This would also work with the current chains without modification because you could just dump the entire chain and let the client use it.

I'm not sure about replication, as yet. The one thing that is lacking is a mechanism to ensure that at least one of each is available at all times so a slight modification of the rarity calculation might be needed.

I hope to see more research in this area.

full member
Activity: 315
Merit: 103
In the first place, there is no rationale to store 7 years of history for synced nodes. But how a new node could get into network if all the fullnodes have thrown away all the blocks(except last few ones to handle possible rollbacks)? Bitcoin has no answer for that.

I've developed a proposal to fix the issue and solve some important questions towards better scalability. The paper is just uploaded http://arxiv.org/abs/1603.07926 . I would be happy to get feedback.
sr. member
Activity: 462
Merit: 250
It's too big to download it. Sad story using tekcoin i need to sync it manually (can not from bootstrap) and it tooks around 2 weeks to complete it.
sr. member
Activity: 478
Merit: 250
ok, so I installed bitcoin core latest version and it downloaded over 80G of chain block. Let's assume, just for the fun of it, there are 1000 bitcoin core users out there. That's ~8T of wasted disk space. and considering bitcoin will live another 7 years and it will grow of couse, that's like ~20T of disk space (1000 users remember?) for what? couldn't be a centerlaize, maybe mirrored, location that the client will ask for the chain block from there? Why do we need to download it?
just because the basic concept of bitcoin, it's a decentralised ledger, no authority, no banks
member
Activity: 318
Merit: 10
with only under 10 mbps, it needs too long to download chain block Shocked
newbie
Activity: 7
Merit: 0
我来学习看看。
full member
Activity: 210
Merit: 100
 down loading of whole bitcoin transaction may necessary to validate in case if any transaction re verification etc . but only thing is as you said it occupies more storage on physical drive , but need to concentrate on compressing those files and reducing the so that you can save some disk space.
legendary
Activity: 1274
Merit: 1004
we need to download the whole bitcoin network because its will also help bitcoin system to verify each and every transaction and nothing can breake the chain and yes its wasting a lot of space
legendary
Activity: 3276
Merit: 2442
To me, Electrum and the other wallets like it are no different than holding your money on the online exchanges. You don't own your money. Somebody else does. They just let you to control your money as you like.

 The original wallet on the otherside, is the real deal. You actually own your money. (I assume there isn't anything shady in the code)
newbie
Activity: 50
Merit: 0
ok, so I installed bitcoin core latest version and it downloaded over 80G of chain block. Let's assume, just for the fun of it, there are 1000 bitcoin core users out there. That's ~8T of wasted disk space. and considering bitcoin will live another 7 years and it will grow of couse, that's like ~20T of disk space (1000 users remember?) for what? couldn't be a centerlaize, maybe mirrored, location that the client will ask for the chain block from there? Why do we need to download it?
The entire purpose of downloading the entire blockchain is to independently verify it and allow for the independent verification of every single block and transaction that occurs later. The only thing that a full node needs to "trust" is the genesis block, the very first block of the blockchain. This is actually hard coded into the software so that there is a defined starting point. After that, the node downloads all of the blocks from its peers. It will check each transaction in the block, and then check the block to ensure that it is valid. Because each block is chained together and transactions are also chained together, it is not possible to start from any point in the blockchain other than the beginning and still maintain trustlessness.

Thank you very much for this answer i too had the same question as the thread creator and this helps sum things up for me!
legendary
Activity: 1106
Merit: 1005
I think that in the future there wont be one blockchain downloaded for each user as it is today but more like one per household. A family will setup a computer and download the blockchain and keep it up to date while useing other wallets to point to that BC for verification and use. A family shares resources, there is only 1 fridge, 1 stove, 1 safe,  etc. Yes more can be had but typically only 1 per household. Same will happen with businesses and such.

The chain is growing pretty fast and maybe one day we will reach a point where its tough to download but technology is also fast moving. I remember using dialup to downlaod Photoshop years ago.... took 3 days, but it was worth the trouble. Bitcoin is the same, if you want to use it then you'll do whats needed.

you can even take that further, and have some kind of mini-societies of people who trust each other all point to one node one of the people in that mini-society owns.

Such as a family and some close friend of that family and family of that family.

Like mother-in-laws and brothers and sisters (who each have their own families).

As long as you trust each other there should be no problem.
sr. member
Activity: 378
Merit: 250
why do not we make a online storage which our software can be linked with i mean like a bootstrap script when we include in our webpage in the same manner we should include this

For one, this would be centralizing the script, where everyone takes information from the same source. This provider could change and edit the information and users will use it.

Server issues?
member
Activity: 85
Merit: 10
why do not we make a online storage which our software can be linked with i mean like a bootstrap script when we include in our webpage in the same manner we should include this
Pages:
Jump to: