Due to the way bitcoin works you often have coins in address' you did not generate. For example, if you have a private key with 20 BTC, which you use to send me 10BTC, you get a new private key/public address for 10BTC of change, minus transaction fees. A wallet keeps track of all those addresses/private keys for you, usually after automatically generating them.
As an example, a bitcoin-QT wallet has a list of 100 generated public/private key pairs. As you spend and receive bitcoins the wallet app automatically uses those addresses and generates new ones to keep the "pool" at 100 addresses.
This is for Bitcoin-Qt wallet - in Multibit you have to generate address/private key manual but one time generating when you spend coins the rest of coins can back to every address in your wallet because is necessary to export and then import to the Bitcoin-QT all addresses of the wallet one by one.