Well, you should in theory have several different wallets for different purposes. So say that you have a hardware wallet for all of your Pornhub subscription payments (joke) ...then do not send any coins from that wallet to your friends and family. You can have an Electrum wallet for all the family and friends stuff... (with only a small amount of coins, if you want to fool them)
Another way to do this would be to use a different passphrase but the same seed phrase for each different wallet. That way, you don't have to resort to hot wallets and can have all your coins protected by the security of your hardware wallet, while still keeping them in entirely separate wallets.
Let me explain: Imagine someone gets 10 addresses from his ledger nano S, and he uses some of those addresses. Now, if someone else knows that address 1 belongs to that guy, can he somehow figure out the other 9 addresses belong to him? Does ledger themselves know? Can the government know about it? Can a hacker or a skilled programmer figure it out somehow?
In order: No, maybe, yes, and yes.
If I know one of the addresses from your Ledger wallet, and you haven't linked that address with any other addresses in a common transaction, then there is no way to link that address to any of your other addresses only through blockchain evidence, block explorers, etc.
However, if you use Ledger Live, then you connect to Ledger's servers and you send their servers a list of all the addresses in your wallet to receive the transaction history and updated balance, so Ledger can easily see all the addresses in your wallet. You can avoid this if you don't use Ledger Live, but if you substitute it with Electrum, for example, then whoever is running the Electrum server(s) you connect to will now see all your addresses in the exact same way. If you want to avoid this altogether, then you need to run your own node/server and connect to that instead.
If you connect to a third party server as above, be that Ledger, Electrum, or someone else, then that data can absolutely be shared with the government or any other third party. Also, if someone has infected your computer with malware, then they could similarly link your addresses when you connect your wallet.