yenom, to use Armory or QT, you would probably have to download the entire blockchain; at least that's my understanding of Armory and QT, though I've never used either. If you have the bandwidth, then go ahead.
Or you could import your private key into a light client of your choice, based on the balance of convenience and security you desire, what operating system you use etc.
You could also do it without using a wallet at all.
Like you, I did a little experimenting with bitaddress.org, generating and funding a brainwallet address with a nominal amount of btc. To spend from that address, I used brainwallet.org.
I did it the quick easy way by going to the transactions page and entering my private key. This obviously involved some risk, because I was online when I entered the private key.
If you're willing to go to a little effort, you can do the whole thing without ever exposing your private key. It's called an offline transaction. You can do offline transactions from some wallets. I'm pretty sure you can do an offline transaction using brainwallet.org too, but I haven't done it yet, so I won't presume to give you instructions.
If you go the brainwallet.org route, I would suggest spending the entire contents of the address. See this thread:
https://bitcointalksearch.org/topic/where-change-goes-388977 
