I think Erdogan's insight into the game theory behind the block size limit was correct.
But to recognize his insight, I think we need to stop thinking in terms of "valid blocks" and start thinking in terms of "valid transactions." All blocks that are composed exclusively of valid transactions are valid.
Now instead of thinking that only Core and XT exist, imagine that there are dozens (and in the future possibly hundreds) of competing implementations of Bitcoin. Each implementation has its own rules for what block size it will build upon. From this viewpoint, the "effective limit" is the size of the largest block that's ever been included in the Blockchain. If a miner wants to create a larger block (e.g., to collect more fees), then he has to weigh the chances that his block is orphaned with his desire to create a larger block. If we imagine that the block size limit across the network forms some distribution as shown in the chart labelled "NEW THINKING" below, then, since the miner can't be 100% sure what this distribution is, it is rational for him to use the tip-toe method to minimize risk.
I notice you did not reply or comment on my opinion of you guys solutions... Maybe you missed it, maybe you don't care to answer? Either way I'd be curious to better understand the logic behind your proposal.
Because if that's all this hinges on:
cost of mining a larger block = self regulating block sizes
We're in a world of problems.
VI. This is a clerical issue, because block propagation and other considerations incentivize miners to keep blocks small anyway. The 1MB is just a hard limit getting in the way of things, the marketplace of miners should be allowed to fix block size as it seems appropriate.
While this argument has been disingenuously brought by Gavin himself, the fact is that the proposed inverted bloom filters upgrade would allow all blocks to propagate in constant time, regardless of their size.
Some might still ask: why is that?
davout: gavinandresen: "oh, the IBLT stuff? yes, that’d make propagation O(1)" <<< so with that, there's no network bottleneck anymore, at least no real incentive for miners to keep blocks small, right?
gavinandresen:davout: Miners would only have the meta-incentive of “we can collectively maximize revenue if we make blocks THIS big”
Except miners are not a person. They are multiple, geographically diverse groups of interests each bounded by different resources, costs and infrastructure. I kind of happen to think that this is what is broken with the "nodes and miners should be able to decide on whatever block size they like" proposition. I can also see clear as day through the attempt of many here at rationalizing this behavior as "free-market decides best, how dare you propose centrally designed
SPAM CONTROL."
The assumption you seem to make is that miners & nodes (through the magic of the "invisible hand" I suppose) will arrive at an equilibrium of decentralization in some kind of benevolent act "because incentives & game theory". If we consider that the argument about cost of creating large block is moot, the rational then becomes: miners will act in an altruistic way to conserve trust of the network.
These points are not very clear to me. I don't imagine a scenario where several resourceful corporations do not turn this into an arms race that few will be able to keep up with. We are now only beginning to see mining and network infrastructure enter professional stage. If the incentive to mine Bitcoin increases the seemingly amateur and small scale set ups should soon be erased off the network and replaced by massive datacenters that should outnumber any of these small players so as to make their "voice" in the balance exercise of decentralization vs. block size worthless.
You might imagine that as "bitcoiners" realize this issue they will "protest" but I suggest that by this point A. you will not be able to actually become aware of the problem and B. there will be nothing to do about it as the network will have become "captured" because of "network ossification" and the general laziness of the herd which will prefer comfort and stability over change and doubt.