I think around $2,000 sounds like a reasonable investment to start with. I've made an account with
https://mtgox.com/, is there a better way to buy coins to start?
Also if there's any required reading on bitcoins that I don't know about, please share your wisdom anyone!
What country are you in? mtgox are good, but there are some better localized exchanges. If you're in the USA, I'd say mtgox is your best bet.
If you're going to invest any significant amount of money, familiarize yourself with Bitcoin security and learn how to keep your bitcoins safe, make sure you know exactly how to move your funds from place to place, familiarize yourself with the practices of any exchanges you intend to use and with the challenges you might encounter (e.g. identity verification, etc.), and familiarize yourself with how the Bitcoin protocol actually works.
This. Store your coins in a couple paper wallets, if you don't have a computer you will keep offline forever to use as a cold wallet. Getting familiar with security should be your first priority, by far.