self-pruning is bad, the entire idea behind bitcoin is having an eternal and intact ledger with all the info for reconstruction of all transactions ever executed.
The strengths of SHA256 will not be eternal.
Topic: Proposal for self-pruning Blockchain (Read 2444 times)
The strengths of SHA256 will not be eternal.
I'm not sure that what the OP is suggesting is what you think he is suggesting, but if he is then what is the benefit of his method over the method in the original whitepaper since both methods will need to store the all transactions that still belong to the UTXO and can remove all the rest of the transactions?
It seems that his method adds additional complexity with minimal (if any) benefit.
Removing the spent TXO locally is done by most (all?) clients. It doesn't help a new joiner unless there is some additional security. Basically, if I ask from an unknown party for his UTXO set, I cannot be sure that he returned the complete set. Surely I can detect tampering because I will verify the merkle tree but he may be omitting txs. On the other hand, if the snapshot is part of the blockchain, there will be POW associated. It seems that his method adds additional complexity with minimal (if any) benefit.
No need to be so finicky about the terminologies.
What the OP suggest is to introduce a snapshot block or blocks in the blockchain. Regular blocks are tx which are essentially delta changes to the UTXO set (he calls it accounts). Once in a while, he wants to flatten and write a snapshot. In the process, if some UTXOs are too small, he would discard them turning them into fees. Of course it's a protocol change but it's not impossible to do.
A couple of things to consider though:
- these snapshots will be huge. Even if you do it while blocks are being processed they will need to be mined and properly rewarded.
- they could be a downtime while these blocks are produced but you could mix snapshot UTXOs & normal tx though.
Not a big problem and it's done in a few applications but at the moment, the blockchain growth is not a big deal. Even if you think the block size grows linearly, the accumulated blockchain only grows quadratically - not exponentially.
Besides trimming spent tx introduces other problems for deterministic wallets.
Btw, it's not a new idea and has been mentioned a zillion times.
What the OP suggest is to introduce a snapshot block or blocks in the blockchain. Regular blocks are tx which are essentially delta changes to the UTXO set (he calls it accounts). Once in a while, he wants to flatten and write a snapshot. In the process, if some UTXOs are too small, he would discard them turning them into fees. Of course it's a protocol change but it's not impossible to do.
A couple of things to consider though:
- these snapshots will be huge. Even if you do it while blocks are being processed they will need to be mined and properly rewarded.
- they could be a downtime while these blocks are produced but you could mix snapshot UTXOs & normal tx though.
Not a big problem and it's done in a few applications but at the moment, the blockchain growth is not a big deal. Even if you think the block size grows linearly, the accumulated blockchain only grows quadratically - not exponentially.
Besides trimming spent tx introduces other problems for deterministic wallets.
Btw, it's not a new idea and has been mentioned a zillion times.
I'm not sure that what the OP is suggesting is what you think he is suggesting, but if he is then what is the benefit of his method over the method in the original whitepaper since both methods will need to store the all transactions that still belong to the UTXO and can remove all the rest of the transactions?
It seems that his method adds additional complexity with minimal (if any) benefit.
No need to be so finicky about the terminologies.
What the OP suggest is to introduce a snapshot block or blocks in the blockchain. Regular blocks are tx which are essentially delta changes to the UTXO set (he calls it accounts). Once in a while, he wants to flatten and write a snapshot. In the process, if some UTXOs are too small, he would discard them turning them into fees. Of course it's a protocol change but it's not impossible to do.
A couple of things to consider though:
- these snapshots will be huge. Even if you do it while blocks are being processed they will need to be mined and properly rewarded.
- they could be a downtime while these blocks are produced but you could mix snapshot UTXOs & normal tx though.
Not a big problem and it's done in a few applications but at the moment, the blockchain growth is not a big deal. Even if you think the block size grows linearly, the accumulated blockchain only grows quadratically - not exponentially.
Besides trimming spent tx introduces other problems for deterministic wallets.
What the OP suggest is to introduce a snapshot block or blocks in the blockchain. Regular blocks are tx which are essentially delta changes to the UTXO set (he calls it accounts). Once in a while, he wants to flatten and write a snapshot. In the process, if some UTXOs are too small, he would discard them turning them into fees. Of course it's a protocol change but it's not impossible to do.
A couple of things to consider though:
- these snapshots will be huge. Even if you do it while blocks are being processed they will need to be mined and properly rewarded.
- they could be a downtime while these blocks are produced but you could mix snapshot UTXOs & normal tx though.
Not a big problem and it's done in a few applications but at the moment, the blockchain growth is not a big deal. Even if you think the block size grows linearly, the accumulated blockchain only grows quadratically - not exponentially.
Besides trimming spent tx introduces other problems for deterministic wallets.
OP should go ahead and do it then. Once there is some code, it will be much easier to evaluate.
Terminology is important here because the proposal has to be precise, but if OP is really believes in this, code is even better.
:-)
No need to be so finicky about the terminologies.
What the OP suggest is to introduce a snapshot block or blocks in the blockchain. Regular blocks are tx which are essentially delta changes to the UTXO set (he calls it accounts). Once in a while, he wants to flatten and write a snapshot. In the process, if some UTXOs are too small, he would discard them turning them into fees. Of course it's a protocol change but it's not impossible to do.
A couple of things to consider though:
- these snapshots will be huge. Even if you do it while blocks are being processed they will need to be mined and properly rewarded.
- they could be a downtime while these blocks are produced but you could mix snapshot UTXOs & normal tx though.
Not a big problem and it's done in a few applications but at the moment, the blockchain growth is not a big deal. Even if you think the block size grows linearly, the accumulated blockchain only grows quadratically - not exponentially.
Besides trimming spent tx introduces other problems for deterministic wallets.
Btw, it's not a new idea and has been mentioned a zillion times.
What the OP suggest is to introduce a snapshot block or blocks in the blockchain. Regular blocks are tx which are essentially delta changes to the UTXO set (he calls it accounts). Once in a while, he wants to flatten and write a snapshot. In the process, if some UTXOs are too small, he would discard them turning them into fees. Of course it's a protocol change but it's not impossible to do.
A couple of things to consider though:
- these snapshots will be huge. Even if you do it while blocks are being processed they will need to be mined and properly rewarded.
- they could be a downtime while these blocks are produced but you could mix snapshot UTXOs & normal tx though.
Not a big problem and it's done in a few applications but at the moment, the blockchain growth is not a big deal. Even if you think the block size grows linearly, the accumulated blockchain only grows quadratically - not exponentially.
Besides trimming spent tx introduces other problems for deterministic wallets.
Btw, it's not a new idea and has been mentioned a zillion times.
UPDATED responses (in blue)
UPDATED (in blue)
You argue as if this was some sort of an isolated question. It is not. And I am not talking about opinion polls or anything like this. You just have to imagine what it means if Bitcoin is becoming unattractive for small payments. It means that some users will stop using Bitcoin and look for something else, which will relatively* hurt the value. The “majority”, whoever that is, will have an interest to maximize the value of their assets.
*) Even if the value is still rising and not dropping, it may rise less in case of fewer users than it would rise in case of more users.
*) Even if the value is still rising and not dropping, it may rise less in case of fewer users than it would rise in case of more users.
@Danny: Of course nobody knows what the future will bring. But the whole issue of size only makes sense under the assumption that it will.
I'm not sure what you mean by "the whole issue of size only makes sense under the assumption that it will"
If block size is limited, small transactions will be pretty much impossible.
Correct. That's a very real possibility, and I'm ok with that. Apparently a lot of other people are not. Since Bitcoin is a consensus system, changing the block size requires overwhelming acceptance from all the miners AND all the users. A simple majority isn't enough. There are quite a few developers and outspoken individuals that seem to think that the overwhelming majority exists. They better be right or we'll have quite a mess on our hands.
Personally, I find that it's safer to just leave the block size alone, but it will be interesting to see what the future brings.
A miner will always pick a larger transaction over a smaller, because for him it does not matter whether the user is giving a 1% fee or a 10% fee. Only the absolute amount matters.
Correct, a miner should always choose to confirm the transactions that pay the absolute largest ratio of bitcoins to bytes.
Of course most people don't want this to happen.
Well. Some people don't want it to happen. Some don't care one way or the other. And then some would prefer that it happen. Where the majority lies is perhaps debatable, but I suspect you are correct. The real question is:
"Is that majority overwhelming enough to force a consensus?"
@Danny: Of course nobody knows what the future will bring. But the whole issue of size only makes sense under the assumption that it will.
If block size is limited, small transactions will be pretty much impossible. A miner will always pick a larger transaction over a smaller, because for him it does not matter whether the user is giving a 1% fee or a 10% fee. Only the absolute amount matters. Of course most people don't want this to happen.
If block size is limited, small transactions will be pretty much impossible. A miner will always pick a larger transaction over a smaller, because for him it does not matter whether the user is giving a 1% fee or a 10% fee. Only the absolute amount matters. Of course most people don't want this to happen.
This is not a valid argument. When Bitcoin's popularity will grow,
You are assuming that Bitcoin's popularity will grow. It is entirely possible that usage of bitcoin will never be much larger than it is today, or that it will grow so slowly that there won't be any need to increase the block size for many decades, or that it will only become more popular with those that want to transact in VERY high values so that high transaction fees are acceptable, or that off-chain (or alternative chain) solutions will absorb all but the highest value transactions. There are many possibilities, it isn't a good idea to assume that one particular possibility that you personally like is the one that will actually come about.
there will be more transactions and that limit will have to be raised anyway, if you want to keep Bitcoin working.
This is not necessarily true. Bitcoin will work just fine without increasing the maximum block size. Regardless, I suspect that the maximum block size will be increased. There appears to be a significantly large majority of users that feel that very small transaction fees are a desirable feature even if it means increasing the block size.
This is not true. Bitcoin blocks are currently limited to 1 megabyte in size. Blocks occur on average once every 10 minutes. This means that the Bitcoin blockchain is growing linearly, NOT exponentially.
This is not a valid argument. When Bitcoin's popularity will grow, there will be more transactions and that limit will have to be raised anyway, if you want to keep Bitcoin working.
I agree with everything else you said.
You don't really cut size just by putting all the transactions into a single block.
It is true that you can remove some transactions, but even the number of transactions that CANNOT be removed is growing exponentially.
PS: Of course there is a limit, since there is a finite number of Satoshis. But that limit does not help us much now, and will not help has when Satoshis will be subdivided in the far future.
It is true that you can remove some transactions, but even the number of transactions that CANNOT be removed is growing exponentially.
PS: Of course there is a limit, since there is a finite number of Satoshis. But that limit does not help us much now, and will not help has when Satoshis will be subdivided in the far future.
Its amusing how often someone comes up with the great idea of reducing blockchain size, without searching forums or bitcoins documentation not only that development for pruning is already occurring but that it is briefly outlined in the original whitepaper itself. Also, how rushed people are to suggest the ideas without understanding the fundamentals of security involved, nor the actual data that would need to be maintained in a reduced blockchain (UTXOs)....
The idea, is that every client can do so in a decentralized manner, because this new genesis block will be behind 210,000 new blocks mined since, there will be no way to reverse transactions.
The only unsolved part for now, is what to do with older hashes, and merkle tree of the blockchain ? But maybe community can help here.
Adding to the excellent points DannyHamilton made, you're completely ignoring the problem of introducing a new node to the network. If the "genesis block" is rewritten and the history forgotten by everyone-- what basis do they have to judge the new system? You haven't provided a security objective/argument at all for that case. (Fortunately, as DannyHamilton points out, the original Bitcoin whitepaper addresses storage and does so in a way that doesn't make it unclear how a new node can be initialized. And Bitcoin core did 99% of the work to implement that storage reduction a couple years ago, in 0.8, it-- as mentioned-- just hasn't been a priority, due to the -- as mentioned-- linear growth maximum).The only unsolved part for now, is what to do with older hashes, and merkle tree of the blockchain ? But maybe community can help here.
If you're feeling that the response here is a bit to blunt you should realize that this proposal (and it's ilk) has been repeated here many times since 2011, seemingly by people who missed section 7 of the white paper and seldom proposing something even as good as the design Bitcoin already has. While it's great that you're enthusiastic and have ideas, when you make a public post you're consuming the time of all the readers and people who respond to you. It's helpful if you describe the research you already did when you first propose something in a community that you're new to, since when someone seems to have missed something obvious (like the whole section on the subject in the project's whitepaper) it gives an impression that the poster didn't care for the costs they imposed on others.
>Why do people insist on coming up with ideas without at least taking the time to read the whitepaper and understanding the basic concepts first?
I did read the whitepaper twice. And what ? It is way too short, and lack details.
I did read the whitepaper twice. And what ? It is way too short, and lack details.
Apparently you missed Section 7 titled: "Reclaiming Disk Space".
It starts with the sentence:
Quote
Once the latest transaction in a coin is buried under enough blocks, the spent transactions before it can be discarded to save disk space.
Satoshi even takes care of the math for you, demonstrating that your "petabytes!" is nothing but a bunch of FUD:
Quote
Old blocks can then be compacted by stubbing off branches of the tree. The interior hashes do not need to be stored. A block header with no transactions would be about 80 bytes. If we suppose blocks are generated every 10 minutes, 80 bytes * 6 * 24 * 365 = 4.2MB per year. With computer systems typically selling with 2GB of RAM as of 2008, and Moore's Law predicting current growth of 1.2GB per year, storage should not be a problem even if the block headers must be kept in memory.
rdponticelli commented on 14 Aug 2014:
https://github.com/bitcoin/bitcoin/pull/4701
Quote
This pull implements a new mode of operation which automatically removes old block files trying to maintain at most a maximum amount of disk space used by the node. This amount is configured by the user with the -prune switch.
There's also a lightweight sanity check which executes periodically during runtime to make sure the minimum block files required for the node to be operative are present.
This should allow to lower the amount of resources needed to run a node.
See the individual commits, about all the changes introduced.
There's also a lightweight sanity check which executes periodically during runtime to make sure the minimum block files required for the node to be operative are present.
This should allow to lower the amount of resources needed to run a node.
See the individual commits, about all the changes introduced.
https://github.com/bitcoin/bitcoin/pull/4701
>Why do people insist on coming up with ideas without at least taking the time to read the whitepaper and understanding the basic concepts first?
I did read the whitepaper twice. And what ? It is way too short, and lack details.
I did read the whitepaper twice. And what ? It is way too short, and lack details.
So everyone throws away all the old blocks from the most recent "genesis" block.
I want to spend a utxo from a block before that "genesis" block.
How do you verify that the transaction hash I am referring to is valid?
You can't.
So you'd be throwing away all bitcoins confirmed prior to each genesis block.
I want to spend a utxo from a block before that "genesis" block.
How do you verify that the transaction hash I am referring to is valid?
You can't.
So you'd be throwing away all bitcoins confirmed prior to each genesis block.
Since you've gone back and updated your original post (in blue), I'll post the updated responses here in blue:
This is not true. Satoshi never says anything about balances in the entire whitepaper. The reason for this is because the Bitcoin protocol does not deal with balances at all, it deals with inputs, output, and the associated transactions. What Section 7 describes is a method by which entire transactions can be pruned out of the blockchain once the transaction outputs have been spent. This means that when all the outputs created in a block have been spent, then the entire block can be pruned down to only 80 bytes. Meanwhile, all of the unspent outputs remain in tact (transactionID, offset, and all) so that they can be verified when someone secides to spend them.
But, the only reason for increasing the block size is to allow and encourage increased usage. Obviously there will be no incentive to increase the block size beyond the current technology's ability to contain the blockchain, since that will discourage increased usage. Therefore, if you assume that the maximum block size limit will be increased, then you can also assume that there won't be a significant need for your disastrous blockchain "self-recycling".
There are no "accounts" in Bitcoin at the protocol level, there are transactions, and transaction outputs. Something that is valued at "1 dollar-cent" might be worth much more (or much less) in the future. How will the software decide how much the transaction output is worth? If you allow transactions that spend someone else's bitcoins without a valid signature, then how do you keep an attacker from creating false transactions to steal larger amounts?
This is not true. Bitcoin blocks are currently limited to 1 megabyte in size. Blocks occur on average once every 10 minutes. This means that the Bitcoin blockchain is growing linearly, NOT exponentially.
This is not true. It is possible to prune data out of the blockchain. There hasn't been a lot of need for it yet, so there hasn't been much priority on enhancing Bitcoin Core to do so. Eventually pruning will be encoded into Bitcoin Core.
Unless the protocol is modified to increase the maximum block size, it will take nearly 20 years for the blockchain to reach 1 petabyte at 1 megabyte per block and 1 block every ten minutes. 20 years ago, IBM hadn't even released the 170MB Microdrive yet. Today, you can purchase a half terabyte on a SD card. Do you really think that a petabyte is going to be considered to be a large amount of storage in another 20 years?
No. You can just run an SPV node if you don't want to store the entire blockchain. It's explained in the original white paper. Furthermore, as I already stated, pruning is also possible.
Read the white paper.
Sounds like a bad idea to me. Especially since much better ideas already exist and were described in the orignal bitcoin whitepaper.
Why do people insist on coming up with ideas without at least taking the time to read the whitepaper and understanding the basic concepts first?
1. I read Satoshi whitepaper, and in chapter 7 he describes how to remove "intermediate" steps with zero balance.
This is not true. Satoshi never says anything about balances in the entire whitepaper. The reason for this is because the Bitcoin protocol does not deal with balances at all, it deals with inputs, output, and the associated transactions. What Section 7 describes is a method by which entire transactions can be pruned out of the blockchain once the transaction outputs have been spent. This means that when all the outputs created in a block have been spent, then the entire block can be pruned down to only 80 bytes. Meanwhile, all of the unspent outputs remain in tact (transactionID, offset, and all) so that they can be verified when someone secides to spend them.
2. I assume, that Bitcoin max. block size limit will be lifted, in order to allow Bitcoin to grow beyond Visa (2000/tps).
But, the only reason for increasing the block size is to allow and encourage increased usage. Obviously there will be no incentive to increase the block size beyond the current technology's ability to contain the blockchain, since that will discourage increased usage. Therefore, if you assume that the maximum block size limit will be increased, then you can also assume that there won't be a significant need for your disastrous blockchain "self-recycling".
3. Additional idea: every 210,000 blocks is to clean "dust accounts" (i.e. accounts with balances below a certain value, say, 1 dollar-cent), transferring it to miners. -- (not in genesis block, but regular block, as it will be forwardly-created)
There are no "accounts" in Bitcoin at the protocol level, there are transactions, and transaction outputs. Something that is valued at "1 dollar-cent" might be worth much more (or much less) in the future. How will the software decide how much the transaction output is worth? If you allow transactions that spend someone else's bitcoins without a valid signature, then how do you keep an attacker from creating false transactions to steal larger amounts?
Today, the Bitcoin blockchain is growing exponentially by leaps and bounds,
This is not true. Bitcoin blocks are currently limited to 1 megabyte in size. Blocks occur on average once every 10 minutes. This means that the Bitcoin blockchain is growing linearly, NOT exponentially.
without any way to cut down it’s size and remove unnecessary parts of it.
This is not true. It is possible to prune data out of the blockchain. There hasn't been a lot of need for it yet, so there hasn't been much priority on enhancing Bitcoin Core to do so. Eventually pruning will be encoded into Bitcoin Core.
In a few decades it could grow into a petabyte-size database due to exponentially accelerating adoption and skyrocketing number of transactions, hurting decentralization.
Unless the protocol is modified to increase the maximum block size, it will take nearly 20 years for the blockchain to reach 1 petabyte at 1 megabyte per block and 1 block every ten minutes. 20 years ago, IBM hadn't even released the 170MB Microdrive yet. Today, you can purchase a half terabyte on a SD card. Do you really think that a petabyte is going to be considered to be a large amount of storage in another 20 years?
But do we need _really_ need to keep all the data since the blockchain was born ?
No. You can just run an SPV node if you don't want to store the entire blockchain. It's explained in the original white paper. Furthermore, as I already stated, pruning is also possible.
Possibly not. Possibly, we can remove outdated parts of the blockchain.
But how ? What is the mechanism ?
But how ? What is the mechanism ?
Read the white paper.
I think it could be
- snip -
What do you think of it?
- snip -
What do you think of it?
Sounds like a bad idea to me. Especially since much better ideas already exist and were described in the orignal bitcoin whitepaper.
Why do people insist on coming up with ideas without at least taking the time to read the whitepaper and understanding the basic concepts first?
Jump to: