... with a bootable USB stick (Ubuntu inside)....
you are halfway to safety already.
do some changes to that USB stick OS:
- add persistent space to the Ubuntu inside
- change the USB disk's format to Linux specific so than windows can't recognize it.
- cut the internet by disabling network and don't allow it to connect to internet
- install a wallet that lets you easily set up cold storage an still use by signing and then broadcasting tx (I know Armory (full node) and Electrum (SPV) that does this)
- set password for the whole OS log in and also for the wallet and encrypt the folder it is on
- you can do many different types of encryptions and password setting if you like to be safer.
you can find how to do each of these steps by googling them, there are nothing complicated about them.