That's very interesting. But, unless you had the master xpub key, there's NO WAY to know all of my addresses; UNLESS you use them together, like:
Address A has 1 BTC
Address B has 2 BTC
And then you create a tx of 2.5BTC to Address C and 0.5btc to Address (change address) D.
Combining inputs from different addresses together in the same transaction is a strong indicator those addresses are owned by the same entity. In many transactions it is also possible to identify which output is probably the change (unless you actively take steps to obfuscate this information), and so again you can link this address to the same owner as the inputs.Address A has 1 BTC
Address B has 2 BTC
And then you create a tx of 2.5BTC to Address C and 0.5btc to Address (change address) D.
That way, you would be able to see that Address A, B, C and D are all connected, but other than that, there's no chance to see all other addresses 
That's not accurate though. There are many ways that people leak all the addresses in their wallet to third parties. The most common is by not running your own node. Any wallet which is not Bitcoin Core or is being pointed at your own node/server must connect to a third party server to query the history of the addresses in the wallet. When you do this, you send all the addresses in your wallet to the server and the server looks them up for you and send backs the balance and relevant history for those address. By doing so, the server can therefore link all the addresses in your wallet together. 
