Blockchain.info has zero control over your coins. All they provide is an interface for you to access your own private keys. It's the vulnerability of that interface to hacking at the user end that's the problem.
There is the additional risk that the service itself could be compromised. While their code currently generates wallets client-side, this could theoretically be changed trivially and without users noticing for quite some time (likely when there is a massive theft across the service). While unlikely, it's much better for peace of mind to have a clean binary with verified signatures -- after that, you never have to worry about compromise of a third party service. After hearing some of the horror stories about Blockchain.info over the years.... no thanks.