Bitcoin was not designed to make transactions anonymous. Every transaction you make is linked to a public key. In fact, your wallet address is just the hash of your public key. Theoretically, somebody could connect the payments you make to other bitcoin addresses to construct a group of wallet addresses that you are likely to control. This method is called address clustering, and address themselves are usually analyzed instead of their public keys.
Example: One of satoshi's wallet addresses is 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa. If I feed this into Wallet Explorer, it will show you the transactions that were made with this address. Here it will also list the addresses associated with your wallet.
Since that wallet only has one address in it, I'm going to provide another well known address as an example. This address 1XPTgDRhN8RFnzniWCddobD9iKZatrvH4 was the one used by laszlo to purchase 10000BTC worth of pizza. When you insert that into wallet explorer, it will list all the other addresses associated with his wallet.
Now you're probably wondering at this point how someone could discover your identity from these wallets. The short answer is I'm not sure, the long answer is that there exists a company called Chainalysis that does these kind of address clustering activities. I think they also know the addresses belonging to exchanges. So if anybody that uses Chainalysis service, which is limited to governments and large institutions, gets some transactions linking one of your wallets with a known exchange wallet, and manages to see your documents, then they can store your personal information along with your wallet addresses in whatever database they are using.
Or you could look at the addresses of bitcointalk members and then wallet explorer could identify their other addresses, but not much information about their identity would be inferred since most of us are pseudonymous.
These methods of address identification fall apart completely if you use mixers to break the link between previous transactions and now.
Thanks for this lovely explanation. So BTC is not anonymous its pseudonymous. Since we can be tracked by our wallet address not by our physical ids.