I think you may have slightly misunderstood my point. It isn't that spam itself needs to be limited, it is that the system is vulnerable to a sort of denial of service from having too much data (whether that happens to be spam or "real" data, although obviously spam is less desirable) stuffed into it. Satoshi made a judgement at one time that 1 MB was an appropriate limit based on that vulnerability, considering the state of the technology, storage, propagation, etc.. What is the evidence for that vulnerability being sufficiently smaller now or in the near future that a 8x or 20x or 1000x is appropriate instead?
I'm sorry but I just see a lot of wishful thinking going on. Of course, we all want it to scale, but instead of actually improving how it scales, the proposal today is to simply push back the safety limit and change not much else. (A bit like ripping out the airbags and safety belts from a car to cut weight and make it go faster.)
I like the car analogies they've worked well for me. So "spam " just being a term for denial of service or an overload of inappropriately priced transaction data, leaves me thinking it isn't quite wishful thinking making room for it.
Reflecting on the issue has lead real smart people to find solution that I think are nonsense (picking on Sidechains here) in my experience sometimes jumping in the deep end and being overwhelmed leaving one focused on solving the issue as they happen necessity is the mother of invention and I think allowing it is a great way to do RnD, something I feel is partly in keeping with the spirit of Bitcoin.
I mean heck, if the "spam" is paying full TX fees, how can you fault that? How do you even tell it's spam? Either way the miners will be made healthy.
"full TX fees" is a meaningless concept given that there is no set fee, it can be arbitrarily low. Just paying enough to motivate one (who happens to be the most accommodating) miner doesn't cover the entire cost to everyone on the network. Vitalik wrote about this once. Let me find it...okay here you go:
https://blog.ethereum.org/2014/02/01/on-transaction-fees-and-the-fallacy-of-market-based-solutions/
Adrian-x: I'll agree there is some merit to your idea of jump into the deep end and then figure out how to swim. Dangerous as fuck, but definitely a strong motivation to Get It Done.
I will agree that market based fees are a fallacy while block subsidies are so high, but we're talking Bitcoin not just the situation today. Vitalik isn't the best free market economics annalist I've ever read. Bitcoin fees are set up to be a market system and its not a fallacy. Miners today can ignore the market because there reward is subsidized by 25BTC, but once this diminishes significantly the fees become all important. I've also pondered long and hard as to why fees are a monetary unit and not a percent, and I've come to the same conclusion as satoshi that it needs to be a free market fee ranging from 0 to the total value of the economy.
Miners that ignore the macro economic data will not optimize and will either go bankrupt here are just the obvious ways fees will be optimized: mining no free transactions (blocking free transactions = less velocity and lover value fees), or mining too many free transactions (subsidizing the Bitcoin velocity = less income and marginal benefit in value of fees) or mining block with insufficient transactions (over priced transaction fees = blocks too small) or mining blocks with too many transactions (under priced on average = blocks too big and orphaned) the market will find an equilibrium and it will happen at a pace that miners can plan and test, basically miners have a constant view of the situation as the urgency for planing is kept in sight by the forced block halving every 4 years.