Topic: Scaling bitcoin: the elephant in the room (Read 3391 times)

For anyone interested, I've attached a draft whitepaper explaining all the details of the proposal to the first post in this topic.

New technologies such as blockchain have the potential to reduce cyber risks by offering identity authentication through a visible ledger

When Bitcoin was being developed on a university campus by academia it worked fine but now
we know that it won't scale and consensus might work on a dozen or so machines but now we
have 20,000 machines doing the "consensus" which leads to a lot of wasted effort

The internet runs on an array of specialized nodes providing specific services and the Bitcoin "Every node for himself"
philosophy is a disaster but then the Lightning Network goes the other way when it comes to off-block private ledgers
running on banking hubs so we end up with a single point of failure unless paying miners Tx fees to open up lots of
channel for our wallets.

Come on boys we had web-farms providing redundancy on the internet twenty years ago to spread the
load because it's starting to look to me like bitcoin is designed just to keep the miners happy and yes someone
needs paying to provide hosting services but lets start thinking about the public when it comes to design and
put the miners more to the back of the queue.

CPU-Wars and PoW to make a living, i don't know what they will think of next

In fact, the actual result of difficulty adaption is rather the opposite.  If you have "one bad actor with massive amounts of hashing power that can mine all blocks for himself", then the automatic difficulty adaption is *in the advantage* of this actor, not against this actor.  

Imagine a system where you have 100 "normal" solo miners that mine a block every 10 minutes.  Each miner has, on average, 1 chance in 1/100 to mine a block.  His average time of success is 1000 minutes.  His success series is a Poisson series with a time constant of 1000 minutes.  Each miner has an individual, independent Poisson stream of success with a time constant of 1000 minutes, and the total block rate is hence a Poisson stream (the union of these 100 individual Poisson streams) with an average of 10 minutes.

Now, imagine that our 'bad actor" arrives with about 100 times more hash rate.  Without adaption of difficulty, this doesn't influence the other Poisson streams.  Our new actor will make new blocks on average every 0.1 minutes, but our 100 other actors still have their block every 1000 minutes each.  

Come adaption: difficulty goes up x 101 (let us say, 100).  Or bad actor now has a Poisson stream of success of 10 minutes on average.  Our other actors are now only getting each a success every 100 000 minutes.  That's one block ON AVERAGE every 2 months.

It is not just that our bad actor has diminished revenue of our 100 normal miners: he also increased seriously the FLUCTUATIONS of their revenue.  Mining doesn't only become less profitable, it also becomes riskier.

So the difficulty adaption made life much worse for the "good and numerous" actors.

If you go from 100 to 1000 "good" actors at the start, they go from one block every 10 000 minutes (one block every week) to 1 million minutes (one block every 2 years).   Suppose that the average revenue still allows the good miners to make up for their PoW.  On average every week a block makes them get a relatively steady stream of income ; however, on average every 2 years a block is way, way too risky.

So, difficulty adaption favours the big "bad" actors.  It pushes the smaller miners into oblivion.  You limit the amount of independent solo miners to a small number.  in other words, you centralize, simply already by the effect of income fluctuations and hence, financial risk.  Add to that economies of scale, and you're done.

It is indeed. This proposal is radical, however it maintains the core principles of bitcoin, including the reason you state.

Of course, with a PoW consensus mechanism, you need rewards, to compensate for the economic loss in PoW.  The problem is triple:
1) that you then get convergence because of economies of scale (see bitcoin)
2) that you waste a lot of value, in other words, your system is a net value burner.  You need to waste a high amount, because that waste is your only protection.  If the waste is not gigantic, anyone can attack from the outside
3) that you split the system in a set of "wasters/PoW industry" and a set of "users", and both have different objectives.

But that's more a problem of wanting to use PoW based consensus, than an inherent problem of finding consensus.  

As I said, rewards give rise to strategies.  Strategies can then be different than the desire to come to honest consensus.  It is very difficult/impossible to find rewards such that the optimal strategy for those rewards is going to coincide with the desired other outcome, namely consensus.   The systemic difficulty in reaching consensus increases if you reward the one proposing it, instead of making it easier.  Normally, reaching consensus on "sufficiently old mem pool messages" shouldn't be that hard.  Of course, the *current mem pool state* of all active network participants is different, because of network delays.  But a list of "old enough mem pool transactions", broadcast at a certain moment by one of the participants, is easy enough to check and confirm.  The "difficulty" of having several of those lists being broadcast nearly simultaneously, and arriving at different order at different network participants, can also easily be solved by including a symmetry-breaking merit function, part of the protocol, that will assign different preferences of the broadcast lists.  After a reasonable network delay, most nodes can assume they received all of the list candidates, the merit function indicates which one is to be preferred, and all nodes will come to the same conclusion.  That's the consensus.  When the network propagates the message that consensus has been reached, nodes can start thinking of broadcasting a next consensus list, built on the previous one.
If all of this is done without reward, and just on the basis of "altruism" because one wants the system to work, there is no incentive to "game the system", to "make others accept YOUR consensus list" and so on.  There's nothing at stake, apart from contributing to the good functioning of the system.  From the moment you introduce rewards, it becomes strategic to be the one that gets HIS/HER consensus proposal and not your peer's one.  Instead of cooperating in the network, you compete.  

Such a system is totally opaque to any form of long-range roll-back attack, simply because no roll-back is possible.  Reached consensus is reached consensus, done real-time and on-line.   The price to pay is that verification is only on-line.  If you leave the network, you trust your network peers that they continue to build the consensus.  
You can think of ways to recover from the improbable "global network split", by accepting, when the network unites again, a merge consensus, that accepts all that happened on the two split histories.    If you discover nodes that have another prong, with another history from a given point in time, you accept as well their transactions, as those in your prong.  
A priori, these two prongs should be compatible: this is like in a DAG like coin: there can normally not be double spends of the same coin, or it would mean that there was a node, sharing both half-nets and doing double spends, one on both sides.  Splitting the network is already difficult, but splitting the network and being on both at once is even harder.  If ever that happens, one could accept the double spend exceptionally.  That's just some extra coin creation, no problem if it is rare enough.  The important thing is that no roll-back is possible.
Again, this merging is quite straight-forward if consensus DOESN'T come with reward.  Because merging the prongs doesn't mess with rewards that don't exist.


Well, if supply follows demand, price will stabilize.  And supply is value-controlled: if *making* a new coin has a fixed economic cost, you will ONLY make it if its market price is higher than that cost.  So as long as there IS a demand, this will converge to the set price.  You are right of course that if demand simply plummets, the lowest supply (namely NO supply) cannot go lower than zero.  You might introduce a systematic destruction of coins if you think that's a problem: a destroyed transaction fee, which is a given percentage of the transacted value.  It would indicate a "half life of coin": grossly the number of times you can transact a coin before it is gone entirely.  If you put that to, say, 1%, a coin's half life would be about 100 transactions, grossly.  I don't know if such friction is a good idea, though.  At least, it would make speculation impossible.  You put an upper cap on value.

The difficulty was brought in to prevent one "bad" actor with massive amounts of hashing power to mine all the blocks for himself and also to adapt to the technological improvement of processing power. So as soon as there are a massive spike in the total hashing power, then the difficulty will adjust to balance things out.

The difficulty is one of the core principles of the protocol. ^smile^

Rewarding consensus is completely and utterly key to the security and usability of a PoW cryptocurrency - without it, there is no way to bound transaction acceptability.

In addition to that, you cannot control the value of a currency by changing the PoW difficulty. Value is derived from supply and demand, changing the difficulty only affects the supply side.

This is a very good idea, but it shouldn't be used as a reward to consensus.  I think consensus should be done freely and then PoS is a good method (see other thread).  However, I think PoW is a good technique for coin creation (independent of consensus decision).  If coin creation is proportional to a weighted form of proof of work, where the weight is the economic cost of proof of work (technology dependent), then we have an automatic value control mechanism, and we are finally making a CURRENCY, not a speculative get-rich-by-hodling asset.  That currency would be created more if its market price rose (it is then more interesting to spend PoW to make it, than to buy it in the market).   Coin issuance would stop if the price goes below the cost of PoW.  In other words, PoW would work as the central bank regulating supply, and there's no problem of seigniorage, because it is wasted on PoW.

If a coin is set to cost about, say, $10 in PoW, then that coin will hover steadily around $10.  If it rises, more coins are created and this extra offer plummets the price.  If it drops, no more coins are created and its scarcity can make the price rise again.  Moreover, the expectation of constant price will avoid speculation, and promote currency usage.

But again, one should not reward consensus. 

In ethereum, uncle blocks contain redundant data. In my proposal, there is never any redundant data, because every transaction sent mines itself, so the analysis in that link doesn't apply.

Ripple, NEO, IOTA would fit the bill but I exclude ETH and the clones because they are much too slow

Single transactions leaves the wallet and it sure seems complicated how that simple transaction gets
written into the block-chain and it's also difficult to walk with one leg and a blind fold on a system that's
been designed to eat up CPU's

Scaling of the block-chain should had been in from day one never-mind later using this
as an excuse to pump up mining fees.

Global money ! You must be joking unless you think that trying to read, decode 200gb
block-chain on a Intel I3 has the processing power of a AS-400 machine

it's like trying to get Microsoft Access using Local MDB files to feed the world without linked tables!

 

I've been thinking about this more recently, because no-one has come up with a real replacement, or workable improvement for bitcoin yet...

The primary problem in my proposed design is the incentive for miners to reference orphaned blocks, or branches - in the initial proposal there is no incentive, and in fact it would be more profitable for miners to ignore uncles and just concentrate on generating large difficulty PoW blocks.

I propose as a solution to this problem that miners get rewarded for 'information gain', defined as the sum of block rewards of a previously unreferenced, or orphaned branches which a newly mined block includes in the main ordering via an uncle reference.

By using the block rewards of orphaned blocks as the reward, it isn't advantageous for a miner to purposefully generate a bunch of orphaned blocks, because the reward he will get from including them later is <= the reward he would get by just increasing the difficulty of his newly mined block.

In this way, it is advantageous for miners to add information to the main ordering, which will not only prevent orphaned, or stuck transactions, but also decrease the time it takes for a transaction to become 'confirmed'.

Comments?

Hi Fuserleer,

I largely agree with your assessment, but I think the only real problem here is making sure that nodes with old data don't end up posting transactions which are never included in the main ordering. There needs to be some incentive for miners to reference previously unreferenced blocks; perhaps something like ethereum's uncle reward.

The other issued you've outlined are minor, I think?

Cheers, Paul.

You just explained exactly a Block-Tree which I developed back in 2012, which the original eMunie project used.

It doesn't provide an sufficient scalability improvement over a blockchain and was subsequently dumped in 2014.

The overhead required to maintain the global state so that everyone knows there are n number of child blocks to a parent block at scale becomes quite extreme.  This kills performance past that point.  

If you don't keep all nodes consistent to the very latest information (CAP theorem gets in the way a bit here) then two problems arise:

1) Nodes may also not know what the main branch is because they don't have all the state information and reference parents in weaker branches by mistake.  If that happens, your main branch becomes weaker, because hash power is inadvertently distributed across many branches and lots of miners don't get rewarded.

2) New blocks that are children of old parents will be created and won't be included in the uncle list of the next real block of that parent due to it being created already.  

Your diagram that you put together shows exactly this (by coincedence I assume):



Block(X) is not reference by other blocks, maybe it came in late.  There is also no guarantee if or when it will be referenced by any future blocks either and therefore poses a double spend security risk.

You also have the problem that a dishonest miner can throw in a block on a recent parent with more POW than the other blocks referencing that parent and those which come after it.  That then becomes the branch with the most work.  Therefore, any inputs in a block higher up, can be represented in a block as a child of that new block.  If the input that is seen first is considered the legit spend, and those after it are considered double spends, well.....double spends can happen.

Could you write it down. Instead of having to go through questions and answers

Well looks like there's a plan now it's time to actually execute it. Maybe someone will jump on board as contributors if there's proof of concept to develop it further. 

I was just thinking of the SPV nodes /  mining a bit more and was reading / externding an idea that was about enriching SPVs with a minimum random part of other TX / blockchain parts.

If we would have enough such SPV + random nodes, the part's sum of those will do the job of many 'full' nodes.

Further: If mining could be done in such a SPV + random style as well, like:

You can only send (+  pre-mine) your own TX if you 'atomically' mine a couple of others TXs (at least 2) than mining could be fully decentralized as well and scaling / storage is no issue - at least if there are also enough merchants / miners with huge SPV + random nodes, where the random part might be very huge one for some bigger and riskier entities.

Any comments?  

Full nodes will likely be mining for profit, because they need up to date info on the latest blocks to make sure they get paid. The difference is that you wont need a mining farm to earn bitcoins anymore, even SPV clients can potentially earn a small block reward if they place their blocks on the longest chain and mine with sufficient difficulty.

So the more dedicated miners are the common miners we know yet?

What is the difference than to 0-conf TX ?