Here's another analogy. Let's say the company who make the locks on my doors release a new lock because the old one is defective. If I fail to replace my locks, should the company come to my house and burn all my belongings, because "Well, they were going to be stolen anyway"?
Just because coins haven't moved doesn't mean they are lost, and quantum computing is not suddenly going to hack all two million vulnerable coins at once. They will slowly trickle back in to circulation over a long period of time, meaning if we set a date to inactivate elliptic curve keys, then we will certainly be depriving some users of their coins. They could be in prison, be under house arrest, be unable to leave a country to reach their wallets/seed phrases, etc. Perhaps their bitcoin is locked in a trust for their descendents. Perhaps they had an inheritance plan to release it when their child reaches their 21st birthday. Perhaps there is a timelocked transaction waiting to be broadcast. The possibilities are endless.