Correct, if you generate a new address for EVERY transaction you ever receive, then when you spend BTC, it is possible that a few of those address will be linked together (though not necessarily all of them). You can use a wallet that allows coin control to gain better control over exactly which addresses get linked together during the send. You can improve your privacy substantially more if you use a "CoinJoin" system, where your inputs are blended with the inputs of other users in the same transaction.
If you are using multiple wallets, then you eliminate the chance that outputs received in one wallet will be used as inputs in the same transaction as outputs from one of the other wallets. However, if you "regularly send BTC back and forth", then it may become clear that the owners of the two wallets have a VERY close personal relationship. They may even be able to infer that it is the same person.