True. But lots of people can stop using and those that aren't yet can be motivated to avoid it if it's continually associated with scams and is a pain in the ass to use.
* significant amount of bitcoins were stolen by hackers/scammers (my uneducated guess is that it's at least 10% of all coins at the moment) and can flood the market anytime they find suitable cash out method, causing market earthquakes/panics.
* I have to check every time my bitcoins aren't tainted before I deposit them to mtgox or similar place
I can manage around it because I follow the forums/irc every day. But for layperson any of this can be very nasty surprise. And all that assuming it's someone who knows how to securely use his/her wallet, don't let me even start about that!