if it ain't broke, don't fix it.
Indeed. No matter the blocktime, a blockchain is never going to be real-time enough to make the difference for most applications. For example, NASDAQ's colored coin project is never going to try to do real-time trading on the blockchain, because they have to accommodate zillions of trades per second; even one second blocktimes would be way too slow.
Bitcoin isn't a payment system, although it's commonly described that way. It's a
settlement system, as well as a store of value system. Faster blocks might be better, but the level of risk in changing what isn't broken is pretty unjustifiable until we have more definitive evidence of both the usefulness and the safety of the change.
I think Gavin only brought this up to take a little break from the blocksize debate, as he and others were losing sleep over it. He had to know what effect it would have to say that even in a random comment. Changing the subject and proposing something more radical to soften up the opposition are time-honored negotiating tactics.