Also say miners started pulling the plug temporarily because the price plunged. Would less mining in combination with the price being down also bring difficulty down even more so than price alone?
It's important to understand that there is NO algorithmic relationship between BTC price and difficulty. IIn my experience it's a feedback loop driven by the people that own and operate the mining hardware. When BTC price is high, or growing, people that own and operate mining hardware will try and acquire more Bitcoins. They do this by purchasing additional hardware (if it's available), or continuing to operate hardware that isn't all that efficient (but is likely all paid for).
If BTC price were to fall to $5000, withing a month you would likely see a decrease in difficulty. The folks with the lowest electricity price and most efficient hardware will just keep running and hope for an increase in BTC price and/or a drop in difficulty. Miners with high operating costs (i.e. high electricity prices) will be the first turn off thei hardware if it becomes to costly to operate. They may well wait a while to see if the price recovers, and then turn things back on. It's all driven by the psychology of 1000's of individuals (or companies), and what their current investment is in mining hardware. The total hardware investment in BTC mining is now huge, and nobody will quickly discard that hardware until they are convinced it won't pay off sometime (or they are forced to by economic circumstances).
Right now, the prevailing psychology for BTC price is "up, up, and away".