It's worth noting that a sudden steep increase or decrease of hashpower can require multiple 2016-block-periods to reach the 10 minute block time again, since the adjustment step is capped.
Also worth noting that we have never hit this limit, nor are we ever likely to now short of a major global catastrophe.The closest we came was an increase of 302%, when the difficulty went from 45 to 181 in 2010, and hashrate jumped from around 0.3 GH/s to 1.3 GH/s. The biggest drop was -28% in July last year, during the Chinese mining ban. To hit the limit now, we would either need to suddenly discover over 1 zettahash of new mining power (which will never happen), or we would need to lose 200 exahash in the space of a few weeks. Even with the largest mining country in the world banning mining, we didn't even come close, so to lose this much hash power at once would require some global catastrophe, such as a massive solar flare knocking out electricity for most of the world.
But even after that -28% drop last year, after the hashrate quickly recovered in the subsequent few weeks, the lowest the average block time came down to was ~9 minutes, which would be largely unnoticeable over a period of 10-20 minutes as you are waiting for your first confirmation. So if you are hoping for increased hashrate to speed up your transaction, then you are barking up the wrong tree.
