I've heard that increasing the block generation and retarget rates would produce blockchain forks too often because of latency problems.
No it won't.
Don't you think this will be an issue with blocks every 3 minutes and retargets every 240 blocks or is the problem being exaggerated?
It's being exaggerated. For instance ixcoin/i0coin and now SolidCoin all prove the p2p protocol works fine with blocks going by in the second(s) range. Now second(s) range isn't ideal, especially as transactions grow, however it shows that "block speed" being 10 minutes is mostly some magic number satoshi used which ended up not being the most optimal. Yes a few miners will get some orphans, a little percentage more than BTC, but that is fine, they are trying to win something. No one said that was going to be easy.
Also, assuming that the max capacity of blocks remains 1 mb, is there any worry that the blockchain could grow too big too fast from spam attacks?
A block every 3 minutes means less transactions per block on average given the same transaction load. When we start getting to the 1MB block limit we will need to redesign the network such that people (likely miners) will get paid more for processing transactions at great speeds. ie Unless you have a gigabit connection to the internet you won't be mining. I can go into this topic at another stage when I have more time.
A lot of people have this issue where they think about "current problems" too much and how something is running fine under the "current problems". I'm sorry to say that Bitcoin as it is will not survive if it becomes too much more successful, the network is going to clog and as we have seen recently with some BTC blocks taking upwards of 90 minutes, no business is going to want to rely on something which is so unstable.
Yes BTC was first, yes it has a lot more users, yes it's hard to get people to change. But whether SolidCoin surpasses it, or Bitcoin adopts some similar changes, is irrelevant, because in the end the user will win with a better network and more reliable cryptobanking.