The argument that price causes difficulty is a simple one: Look, price went up, mining is profitable now, let's mine.
I don't know of such a simple argument for the other direction, that is also not an argument for "price increase causes more price increases".
(Your example actually shows decreasing supply, but that's close enough.)
Also, this argument can also be used in the case of "price is skyrocketing, I better hold onto my coins", which becomes self referential argument, and is the main reason why bubbles form, and also why investment scams and ponzi schemes exist.
So I can't really say that this backward looking (difficulty causes price), or self-referential logic (price causes price) doesn't work. It works for some time, until it stops working.
Okay, let's set BTC difficulty = 1 and see what happens to the price.
If someone makes the argument that difficulty doesn't (or hardly) affect price, then we should be able to set the difficulty to whatever we want and it won't make much of any difference whatsoever.
Here's a knockdown, simple argument for difficulty affects price -- get rid of the difficulty adjustment algorithm altogether and watch what happens to the price. If price isn't affected by difficulty, then it should be able to sustain itself in the total absence of difficulty adjustments.
If difficulty were 1, it would mean something like 7.143 MH/s are mining bitcoins. I would assume this would mean price would fall since only one person would be mining at that point. I don't think you can separate difficulty from the picture for any thought experiment. Difficulty to me is used as a field leveling mechanism. It's so no one person can monopolize. They would have to keep adding more equipment, power cost, time, and labor to get the same amount of BTC as before.
I think it is safe to say difficulty determines price and price determines difficulty. I think the argument is whether or when are they leading indicators or lagging indicators. What do I know though?