Topic: blockchain size Do we really need 20 gbs of data? (Read 2207 times)

I get a bunch of free $ / hrs from Microsofts Azure cloud program.  I run a few small spec VMs as nodes.  My way to contribute back a little.

I was wondering the same thing. Wouldn't it be the best of both worlds if you could download the full blockchain if you wish or a version that just contained "snapshots" of previous years? I know I would go back to using Bitcoin core then as opposed to third party wallets.

What would the harm be in taking all the transactions from 2009-2013 and hashing this result to give us H.  We now have a bitcoin reference client that can load a db that matches this has so we know it contains all unspent outputs as the starting point, we now just need to store the transactions/blocks since this point to keep a verified blockchain.  This way people can download the enitre history if they want to or just use a snapshot from a given year.

A year's blockchain history sounds sufficient for solid network integrity and minimizes the need to store an ever growing chain but perhaps I misunderstood how it works?

You can run a light weight client. I believe developers are already trying to solve this problem of blockchain bloat. It isn't necessary for you to run a full node since if you don't leave your computer on for 24/7 and port forward port 8333, it does nothing to the blockchain.

I used to use bitcoin core client and a blockchain.info wallet when I first started. Keeping the blockchain up to date is a real pain if you do not keep bitcoin core running 24/7. I changed to Electrum later on.

I think normal "consumer" user do not need all 20G of blockchain. A lightweight client will suffice. I believe we will see more web wallets and lightweight clients as more people adopt bitcoin because they are more user friendly. Downloading 20G of data is not what a newbie user would want to face.

do you really need 10 fingers?

Re: blockchain size Do we really need 20 gbs of data?

YES

On a bit unrelated matter.
There is enough money in mining already, for the miners to develop and start maintaining their own software; one that suits their needs.
I guess that will eventually happen.
Incentive is what drove bitcoin core devs (or its commit deciders) away from miners and also incentives are going to drive miners away from bitcoin core.

I may be mistaken, but I get the impression that a few members of The Bitcoin Foundation have significant influence over the individuals that decide which changes get pulled into the release candidates.

It won't be long before there will be another Bitcoin implementation suitable for mining: https://blog.conformal.com/btcd-beta-announcement/

As a point of clarification, the reference client is not the Bitcoin foundation's. It's an independent open source project which existed a long before the foundation.  The foundation currently funds some of the folks with commit access, though not all, and pays for some resources and tools.

Notwithstanding that, everything else you said was also perfectly fine.

Because, the Bitcoin Core team maintains the current reference client that implements the full protocol.


It isn't meant to be pretty.  It isn't meant to be crude.  It is meant to be a reference for how the protocol works.


There is nothing "official" about The Bitcoin Foundation.  They are a private club of people who have decided to give themselves a fancy sounding name and release their own reference implementation of the protocol.  Their client is as much a third party client as any other.

I could create my own organization called The Bitcoin Association, and then I could release my own version of the reference implementation.  My wallet would be just as "official" as theirs.


This is what wallets such as Electrum (the "official" client/server model wallet from the official organization "Electrum.org"), and MultiBit (the "official" SPV model wallet from the official organization "MultiBit.org") have already done.


There are already good solutions.  You just don't want to use them.

Of course it is a wallet idea.
All the bloom filters, that are the core of the SPV clients, are set up upon the content of a wallet.
Without a wallet there is no reason for an SPV client to exist in the first place.

The full node is a one that has all the blocks - can serve their content on request.

Other nodes only have the UTXO database - they can verify blocks and transactions, but cannot serve blocks, except a few last ones.

Maybe it has to be defined, but in the common sense, node is a creature that routes data.
SPV clients do not route any data - that is why I see no justification to call them nodes.

SPV isn't a "wallet" idea, it is a client idea.  A full node has no need for SPV functionality and an SPV client has no need for the full blockchain.  Including SPV functionality into a full node (any full node) is simply bloat.  There is now more code, more complexity, and more potential security issues.

The reality is MOST USER WILL NOT RUN A FULL NODE.   Period.  There is no conceivable future where a majority of users will run a full node.  If Bitcoin ever scaled to tens of millions of users probably less than 1% will ever run a full node.  Bitcoin-core should be the best reference implementation of a full node.   Most users will probably never use it and there is nothing wrong with that.  Having a full node & SPV client is like designing a toaster which also is a TV.   Why not just buy a TV or toaster depending on your needs?

Well I don't see why the core team wouldn't implement their wallet idea into the core client?
Would they intentionally keep the core crude? I wouldn't think so.
And yes if it isn't part of the official package it is third party unless the foundation calls it official.
Regardless of who is on the team. thats my take on it.  Make the install package different than; split the node and wallet off.
Allow the user to install one or the other. I don't know, I'm ignorant there are smarter people like both of you to come up with a good solution.

I don't think "node" has been formally defined.  You are free to use that definition is you like but then it kinda makes the term "full node" redundant.   The words "not much of a node" would imply there is more than a single specific set of characteristics for a node.   If my client is 3 blocks behind am I not a node?  The instant a miner finds a new block are all other clients on the network no longer a node?  Like I said feel free to define node to be whatever you want, that wasn't the point. 

Of course there are all kind of bitcoin clients / wallets.

But a bitcoin node, at the current protocol stage, must always keep a full copy of the blockchain (all 20+ megabytes).
Otherwise it couldn't be much of a node.

Ok fine use whatever terminology you want.  The point is there is no such thing as a third party client.  There are just clients which implement the Bitcoin protocol.  Some clients keep a full copy of the blockchain and some don't.

Here I'd like to say that "SPV node" is an oxymoron.

SPV clients are not nodes, because they only fetch data from the p2p network.
Nodes in p2p network should be also serving data to other nodes.

You do understand that in an open source decentralized network there is no such thing as a third party client.   Bitcoin-core is an implementation of a full node.  There are other implementations of nodes both full nodes and SPV nodes.  So the question is do you want to run a full node?  If so then there is a resource requirement to pay.  It just be software bloat to combine a SPV implementation and a full node implementation into the same software.   If the same dev team that is developing bitcoin-core also created an SPV client would you consider that a "third party" client?

Well you know, at the end of the day it all comes down to the argument that they deal with people who want to invest millions of dollars in bitcoin infrastructure - and who are you next to them?

It was inevitable that sooner or later the demand from multi-million businesses which discovered bitcoin as a new source of income would become more important for some people, than the demand of the community that actually had made this project a success in the first place.

But eventually everything is going to equalize - the miners are there to assure it.
IMHO, Bitcoin Core is a dead project already and it is just a matter of time before miners start using a different software. And then, to make changes like this, you won't have to talk to the bitcoin elite anymore.

you are right. I'm an ignorant ape from kokomo that can't contribute code.
but even this noob who can't code is aware there are some core issues that I shouldn't even have to say anything about.
and relying on 3rd party software to bandage the issue isn't a solution.
but that's fine if I'm ignored because that means the writhing mass that this coin is trying to penetrate is being ignored.

https://bitcointalk.org/index.php?topic=204283.new#new
There was a guy who was working on compression method, but looks like there are no updates about it from Jan 2014.

Of course I agree with you, but please be informed that saying things like that will likely move you from their noob list to their ignore list.

IMO, the chain compression for the nodes will get implemented sooner or later. Necessity is the mother of inventions so eventually things will change here or there and then it will get done.
The only question is: who is going to lead the solution through the community. Obviously it won't be people carrying a banner "I don't care about mining"

yes totally agree, the ability to do so has been around for as long as bitcoin. So why it hasn't been addressed yet is ignorant.
The other ignorant thing that has been bothering me is that the client should create backups automatically of the wallet.dat
But it doesn't for whatever reason. Everytime it starts, it should create a wallet.dat.bak-DATE(or whatever)
so if a HDD/controller shits itself while working on the blockchain it doesn't corrupt everything. (not that I would know)  
I don't know what the dev community has been doing for 5 years but they really need to get it in gear to make it (l)user friendly.
Nothing has changed at all in this time. Patches and stupid shit. mnsho
and devs I don't mean disrespect but seriously...

you must not have read the part where I said I shouldn't need to use a 3rd party client.
so put me in the paranoid camp but I haven't lost a btc yet *knock on wafers*

As it had been discussed here before, there are methods to compress the entire blockchain state on your disk (ATM to like 500MB), yet still allowing your node to have a secure copy of the entire UTXO db and thus all the possible address balances, as well as be usable for mining. Which SPV clients are totally useless for - they are just leeches of the bitcoin network and they should be banned, not encouraged, IMHO.

But to do this, you need to extend the blockchain protocol and to extend the blockchain protocol you need to change the reference software, and to change the reference software you need to talk to the people who have their own agenda - which have recently seemed to be quite contradictory to encouraging decentralization of nodes.

All it takes is just make a hash from serialized records of the current UTXO database and mine into the next block (each block, or once for awhile) - then enforce the protocol to verify this hash and reject blocks that would have a wrong hash. It isn't a rocket science - quite a simple feature actually, comparing to the rest of the bitcoin protocol.

Last time I moved this topic on this forum, I was told that they "way ahead" of me with the solution.
Though that was like over a year ago - now they obviously must be at least one year further ahead of me...

Anyway, don't expect it to happen within the next 10 years - not with this team, and ATM there isn't any other bitcoin software that would be suitable for mining.
And there isn't any solution for the disk space problem that could work securely without first upgrading the blockchain protocol (inside the mining software).

So if you want to keep running a bitcoin node like 5 years from now, better start monitoring hard disk prices already.
Then use the same rule as for bitcoins: buy when it's low.

Indeed. Use a SPV client unless you specially want to run a full node. Bitcoin Core's wallet doesn't have any advantages compared to other wallets.

I don't.  I expect Joe Blow will use a SPV client and never keep a full copy of the blockchain.  Enthusiasts, service providers, the paranoid, merchants, and researchers will keep a full copy of the blockchain.

How do you expect Joe Blow from kokomo to keep 20 + gbs of a blockchain on their computer for just a wallet?
This seriously needs to be addressed so a newbie can download the btc wallet and not worry about their
HDD exploding or downloading 4 years worth of data. Ok I'm exaggerating slightly and this is like beating a
dead horse. At least a 2 year old dead horse that hasn't seemed to be buried yet for whatever reason.
How many years more is it going to take to come up with a real solution?

Sure there are alternatives but that isn't acceptable when the "standard" is the client provided by the devs and it eats
a 1/3 of a hdd.
I shouldn't have to support a HDD corporation to keep a blockchain or dedicate a HDD to the blockchain
I shouldn't have to use a 3rd party "light wallet" either.
The BTC client is proven to be secure, I feel safer using it than a 3rd party wallet regardless at this time and moment.

Isn't it time that this is put to bed once and for all? 5 years to think this one over..