Because what you write there is not known even public key to be cracked?
However, whenever you make a transaction in bitcoin, you must include the public key of whichever address the coins you are spending are stored on. This public key is then stored as part of your transaction data on the blockchain, and therefore is public knowledge which anyone can look up. Quantum computers provide an exponential speed up over conventional computers when attempting to reverse the ECDLP, which would potentially allow an attacker in the future to obtain a private key from knowledge of the public key.
If you never reuse an address, then this will not be a concern of yours ever. If you do reuse addresses, then you need to think about stopping doing that in maybe 20 years' time.