While it is irrelevant to BTC functionality if large portions of miners are shut down, I can see one risk. The amount of idle SHA256 capacity is increasing compared to the working one. Difficulty drops too.
If at one point there is 10x more idle miners sitting around than active miners, a 51% attack becomes more likely. Bad guys (public or private) could pick up alot of cheap idle mining capacity and break the blockchain.
Opinions?
It surely is a risk. But as you wrote, the hashpower of idle miners would have to be multiple times greater than active ones to become a real threat.
You don't even need a 'bad guys' buying the idle power (it wouldn't make sense financially for them to do it), there'd be a greater threat from large ex-mining farms that still hold unused miners.
But I don't see such scenario likely to happen. First of all, the mining hardware that would be switched off in the first place, would be the old-generation ones, so less efficient and less powerful. Second of all, it would make more sense for the mining farm, that ceased operations, to try to sell unused miners (even at low cost) to others (hobby miners, or other farms with lower electricity and data-centre costs etc).