Install Bitcoin somewhere.
1.) Give your portable disk a static drive letter (If thats not possible use the detach databases on shutdown option in bitcoin, otherwise changing paths will make problems with the DB)
2.) Then take the c:\programs\bitcoin dir and move it to your USB Stick/other partition.
3.) Make a shortcut to Bitcoin-qt.exe with the following commandline option:
datadir=E:\Bitcoindata
(assuming E:\ is your portable drive)
Just make sure the paths/driveletters wont change, then there should be no problems.
Or did i miss something? Tried that and it seems to work.
EDIT:
4.) Bitcoin stores its settings in the windows registry, but that shouldnt be a problem if you use a bitcoin.conf file, since it will just take those values then.