If you want complete privacy you have to start from scratch, which would mean that you will always use a TOR or a trusted VPN, and that all coins will be mixed. So first secure your internet privacy (mask your real IP), then move your coins to another wallet and reset Ledger, create a new seed and never connect to Ledger or any other server with your real IP. Then mix your coins again, and send them to Ledger.
I mostly got the mixing, IP masking and coin handling part. There definitely are some flaws I have sometimes missed, but now that I have a RYF-certified PC, I'll be fixing them as well. Tech and cybersecurity is a never-ending education.
Now the question arises as to how you will use these coins, because if you pay something online with this BTC you will reveal your identity in case you buy an item to be delivered to you - although there is a way not to use your real address.
My long-term mixed coins will be used P2P only, while my hot wallet will be a little more "vulnerable" privacy-wise. There are some workarounds for name and addresses as well that I've been successfully using for a while now. It feels a bit less secure than handing out your real personal information, but it's worth it in the end.
I'm not saying that Ledger has any bad intentions, but they have already shown considerable incompetence, which is just a warning that they should not be completely trusted. IP addresses combined with coin addresses could really be a big security issue, and the question is how Ledger handles them.
They're a company and they may have shown mostly good intentions in the first few years, but lately they've turned to the "darker" side in my opinion. As a company, no matter how much it looks like they are well intended, they may be influenced in the end the same way Google and almost all the other corporations of today have been - especially if large money comes into the game. The Secure Element is another thing that makes me question whether backdoors are a thing - hence why I previously searched for a way to use my HW
offline.
If you back up your private keys&seed on a airgapped PCs, then you don't have to worry about it becoming unusable and after a long time, it's something that should always be standardized way for recovery.
I wouldn't worry about losing the privkeys/seed as much as I'd worry about the fact it's somewhere stored on my drive. While Ledger has its own security and encryption by default, the other devices I own such as PCs are only as safe as I make them be. And since I'm not a security expert, there may be large flaws I'm missing.