At least use something like Blockchain.info where you could recover the coins in case they disappeared, had all funds seized, etc.
Exchanges really shouldn't be used for storage of coins. Fwiw, the bitcoin clients in the wild are pretty user-friendly. You might want to consider testing out a lite client like Electrum, which offers a good amount of security without needing the blockchain which is a huge PitA to download and maintain if you aren't a power-user (that said, Electrum does offer a fair number of power-user options, but you don't need to use them).
Just make sure you back up and encrypt your wallet, and you'll probably be fine. Ideally, if you have a substantial collection of coins, move everything BTC-related to an offline computer. Armory (and I'm sure others, by now) offers great options for offline wallets, though Armory does require DLing/maintaining the blockchain and a shit-load of RAM on your PC (8GB+ is essential - 6GB or less is risking it crashing unless you're using a very resource-lite OS without other programs running).
Multibit's a great compromise between a full and lite client because it well-utilizes a technology called SPV, but I'm not capable of explaining its differences well to you. You might want to check out the options available @
http://bitcoin.org/en/choose-your-wallet if someone else doesn't chime in with a more detailed explanation.
I'd very strongly recommend against keeping funds on Gox, though - whatever route you end up taking. You'd never want to hold a substantial number of coins there longer than absolutely necessary, and that goes for just about every exchange and web wallet you can't A) extract your private keys from - and B) verify doesn't have access to your coins