The privacy problem is this: when you tell someone your public address, they can now associate you with that address, and monitor how much you have (at that address) and who you are sending money to (if they also have that information). They can also give the information to people or organizations that want to track people and their money.
I don't think there is a lot to be gained by using an address for every transaction since that just replaces the need to associate one address with the need to associate multiple addresses. Furthermore, when you spend the money at those addresses, there is a good chance that they will be used together in a single transaction, and thus be associated with each other and linked to you as a result.
Anyway - in most clients, the implementation is super-easy for the end-user, where you just scroll through a list of addresses you've received coins at, and select the ones you want to send from.