What if we had some type of patch that eliminated "silly sized blocks" and allowed miners to set any soft limit they want? Have it something like: allow any size as long as it's not more than x3 (or other number) the previous months average block size.
Market incentives will drive each mining pools soft limits. Increasing the limits will stimulate growth (increase centralization if done too fast compared to technology because of resources needed) while smaller blocks will increase fees. Some may like smaller blocks because of risk of orphans while others will want large blocks to grow the user base of people using bitcoins. Allowing a x3 size block means that as long as a minority are making small blocks, those who want larger ones can still help grow the "silly size" limit and those who have weak internet connections just have to download the blocks and can continue making small ones.
A spam attacker could only create large blocks that were bigger than all the pools soft limits only by mining it himself and the largest block he could create would be a x3 of the blocks on the system. And just like it's in their economic self interest to not get bigger than 51% it would also be in pools own self interest to not have a soft limit that would risk too much centralization of the system by making it too large for current technology and allows for the system to rapidly adapt at the time. The fact that most mining is done by pools means that the soft limit can be more fluid.
Ultimately if we let pools set any limit they want would they set limits that would allow bitcoin to thrive? If we assume the majority of pools act in the interest of a growing decentralized system would max block size be needed anymore other than something to prevent a "silly size" block?
Welcome to the thread!
I've seen ideas like "3x the average of the last N blocks" before. I don't think it's necessarily a bad idea. The problem I see, though, is that there's no hard limit at all in that case. I predict that people will then want another limit on top of this 3x dynamic limit in order to feel safe. Now we're back to the same place as we are now…but with increased complexity (dynamic + hard limit).
That being said, I do think the risk of miners publishing "ginormous" blocks if there was no cap at all is small. Based on real data, TradeBlock today estimated that it would take 137 seconds for a 8 MB block to propagate to half the nodes. By the time a 100 MB spam-block propagated, miners would have found another two and the spam block would be orphaned. The probability of success of such a spam attack is thus inversely proportional to the damage it can cause.