It affects the people you send coins TO. It also increases the code complexity of all Bitcoin clients, which will ALL need to support the new code, in perpetuity. Optimistically, 50, 100, 200 years... Adding alternatives has to be done very carefully. We don't want this to turn into PGP.
Quantum computers capable of breaking ECDSA are a long, long way out. This isn't going to sneak up on us. We won't know if it's even possible to build such a machine for ten years.
Now IS the time to start working on the problem, but the work needs to be done in the wider crypto community to develop and test the techniques for quantum-resistance. Good crypto algorithms take a long time to bake.
The actual technical work to implement it is very easy once we settle on the signature algorithm. We can do it in a couple days and have it tested in a week or two.
That axiom leads me to the opposite conclusion: It's very easy for us to make the change to the code, but the blockchain is forever. We should not make format changes lightly. A proof of concept on the testnet would be fine just to check for unforseen problems, but fooling with the production net now would be seriously premature.
The real work is in creating better algorithms, and it's not being ignored.
As for Bitcoin's security, there are any of a dozen things that are much more urgent to work on. Just off the top of my head: key handling; cold storage; trust management; code auditing; refactoring.