Topic: Proposed system to reduce Blockchain size - page 2. (Read 642 times)

well if core developers inteoduced schnorr signatures you would save 25% space

/kx

issues:
if you want to be a full node. then take the big boy pants, put them on and be a full node. if your just gonna leach the blockdata from someone else and then strip signatures, prune and filter to be left with pretty much just utxo's your no longer a full node. no longer able to seed the full data to others.
so just give up starting as a full node if next week your going to be a useless litewallet.
make the choice be a full node or a lite wallet.

solutions:
the main concern for full nodes is not the physical space. nor the actual time to download the full sync. but the fact that the main reference software is coded that you cant even see a estimated balance or send out a transaction until the sync is complete. thus making the sync appear slow because your sat there doing nothing twiddling your thumbs waiting for it.

but, you dont actually have to wait. so the solution is simple.
instead of starting as full node leaching off others to then downgrade to spv/litewallet. wasting others resources. do the opposite
get a UTXO set first to get users active and making transactions with a 'independently unverified balance' as a spv/lite wallet which is still safe because if your personal utxo is spent the network wont allow it to relay/confirm in a new block. then. the syncing to get the full archive and verify all transactions becomes a background task, thus no twiddling idle thumbs.

tl:dr
dont start as full and downgrade to spv
start as spv and upgrade to full

The whole point of Bitcoin is for it to be trustless. Someone still has to manage the files and everyone's definition of who is trustable is different. Who should be the one assigning the trusted people? It's fairly easy for that person to push an agenda through and manipulate the entire system. If I needed to trust someone, I might as well go for Paypal.
How would you get the chainstate without indexing the blockchain? Trusting another person to obtain the chainstate is considered fairly unsafe as the information inside can be manipulated. You need the entire blockchain to ensure the validity of the chain. If not, you might as well just use an SPV client.

SPV client would be a better alternative. If you don't validate the blocks fully, you are essentially an SPV client and you don't have to download any blocks at all.

I kind of knew this question would arrive and I was prepared for the same. The thing is we would not store it on one particular location but a group os 'trusted' locations.
Example: If we identify 50 or more trusted locations and store the archival there then it shouldn't be called centralized as the archival is still spread across trusted locations.

Yes, pruned blockchain does sort out the issue of storing the whole blockchain on your system however it still has to validate all the blocks until the genesis block.
Whereas my proposed system would eliminate the need to validate all the blocks until the genesis one and will only need to validate blocks until second genesis block/genesis block after archival whatever we name it.

Well that's a nice thing to start with. I haven't thought about it yet henceforth I can't comment on it. It's good that you have reached 15% compression and I hope you achieve more so that we all could get better at it.

If the size of the blockchain is the only concern in this discussion then instead of focusing on storing it elsewhere (which sounds more like centralization) focus on bitcoin transactions at byte level and see how you could "compress" them instead. FYI inside a transaction there are lots of wasted bytes that don't need to even exist. For your storage and even communication over P2P network you can come up with a compression technique that reduces the size by a lot.

I have actually been working on this recently and been able to "compress" a transaction up to 15%. That's just a start though, it needs a lot more work and study of other people's work to see what they have done, but you get the idea. With a block this can be more by simply dropping a lot of things and adding a "contract" in your compressor based on some factors such as block height. I expect the final result to be 30% compression.

My answer: "yes, we do"
Now, you have to believe me as I am the only one who answered, right!

What if someone else joins the conversation and replies: "no, we dont" ?
In this case you need answers from as many members as possible to know who is telling the truth.

The same applies to blockchain. Blockchain is more secure with more full nodes.
Also, full nodes make sure that miners are following the consensus.

Apart from that, it is always advised to run your own full node so you don't have to rely on other nodes.

It is worth mentioning that a full node is helping the network only if it accepts inbound connections.

Do we really need more? Is there a statistical study showing which is the ideal and necessary number? Perhaps we need much more of a diversity of locality where there are full nodes than necessarily a much larger number of them.

Apart from what khaled0111 already wrote, disk space is not the main bottleneck. 200 GB are not much, it's almost nothing for modern HDDs.

The main bottleneck is the propagation of new blocks and new transactions.

You can already prune your blockchain to less than 1 GB (AFAIK 550 MB was the minimum).

There was a proposal in 2014 called the "Mini-blockchain scheme", which is something very similar to what you're proposing. It has changed some rules to the code so that the fraction of the blockchain which has to be validated by full nodes is much shorter than in the current codebase, and from older blocks only the headers are stored. The proposal, however, hasn't got acceptance in the Bitcoin community, but there are altcoins using the proposed modifications, or variations of it.

Among the problems with this approach are:
- an 51% attack could not only double spend, but replace all complete blocks of the mini-blockchain. A "checkpoint" (issued in the code) would be necessary to make it usable again (see here)
- there is no way to validate most contracts, LN, for example, would be probably impossible.

What you are suggesting was proposed and discussed before, actually.
And the answer is no, this is not going to work nor solve the blockchain size problem.

First, if we are going to "store" an archived copy of the blockchain in a particular place then it can't be considered decentralized nor trustless system anymore.

Second, each block is cryptographically linked to the previous block. Thus, if we need to validate a block/transaction a full node has to go all the way back to the genesis block.

=> we need full nodes, and we need more of them.

This might a good useful idea but I have no idea how it can be achieved. I have the same problem like you. I am no expert in the Blockchain technology. :-(

To begin with, I don't have deeper technical knowledge about the blockchain as many others here have. Which is why I would like to get opinion for a solution which I think might be possible to reduce the blockchain size.
Again, I am no expert and just expressing my views to get some better reviews and understanding of the blockchain.

Considering that the size of blockchain has already crossed 200GB+ I think the simple solution here would be to archive this data from the Blockchain and store it on some particular locations from where the archival is easy to access.

Let us consider that we have archived all the blocks until 200GB and have stored on some secure place. Also, we have attached a hash/signature to the archive and modified the consensus such that every new block that is generated includes a hash/signature from the archive. So now instead of validating all the previous blocks, the miner will validate only the hashes of archived blockchain, previous block and the current block.

This will save the 200GB of data that a miner has to validate and reduce the size of the blockchain. I know there must be some flaws in this system. Let me know in the comments.