My question is - once I have sent funds to this public key, how can I actually access the wallet to send bitcoins out? Is this done with the bitcoin client?
- snip -
Note, you generally send bitcoins to a Bitcoin Address, not a public key. The Bitcoin Address is generated from hashes of a public key. Websites such as the one you mentioned will generally give you the Bitcoin Address and Private Key.
To access the bitcoins and send them in the future, you will need to import the Private Key into a bitcoin client. The bitcoin client will then be able to generate the Bitcoin Address from the private key and will allow you to create transactions.
Be aware that depending on the client you choose to use, the bitcoins may move to a new bitcoin address that is not associated with your brain wallet when you create a transaction. You would need to create a new brain wallet and send any remaining bitcoins there (or specifically send all remaining bitcoins back to the original brain wallet address) if you want the bitcoins to be back in a brain wallet after sending some of them.