No, I'm prepared to hear any answer. I just am frustrated with incomplete answers. Telling me it's not safe despite all these precautions is an incomplete answer. I know it's not 100%, I already said this. I already said I'm willing to forgo complete security for some convenience but what you told me is that you wouldn't trust my setup at all which I just can't take as anything but bullshit. Yes I can lose my money, but here's a newsflash for you, you can also lose your paper wallet if a thief breaks into your safe. Perfect security does not exist and I'm not asking for it. All I'm asking is for a setup that is reasonable safe but you are telling me that my setup is inherently unsafe which I just cannot understand without any further explanations.
I already said that most of what I'm afraid of is a keylogger because I'm already very careful, have keepass and strong, uniquie passwords for any service I use, I have noscript installed, I have an antivirus running... My windows setup is already a lot safer than what most have but I'm not happy with it because I realize I'm actively browsing on this OS and a keylogging threat exists. All I wanted to do with my USB ubuntu setup is protect myself against that. Why? Because encryption takes care of the rest. And now you're telling me my USB Ubuntu setup will not even protect me against keylogging?
Modern computers are inherently insecure. They are slapped together quickly and cheaply. The prevelant debuging method employed is "ad-hoc" debugging, where the software or hardware is tweaked until it appears to work. Software and hardware is not proven correct, in part because it is perceived to be impossible. In truth, the halting problem only applies to Turing machines with infinite memory, which computers only imperfectly emulate.
I used to think that modern computers could be considered reasonably secure, if only they ran from Read-only memory. For over a year, I used a diskless computer booting from a live CD as my primary computer (a second computer acted as a file-server). For several more years my router was booting from a read-only floppy disk. Then I learned about an
attack on a Voting machine using read-only memory. They leveraged a stack overflow bug in one of the configuration menus into a full machine compromise. Because the machine was battery-backed, they were able to emulate the boot process. To get around to read-only memory limitations, they used a technique called return-oriented programming.
The implications for your laptop booting a "secure" USB key are obvious. A sufficiently skilled attacker may decide to emulate the boot process and prevent you from rebooting the machine; instead putting the machine in standby when you think you are turning it "off" (possibly adjusting LED behaviour in the process). You may think the battery is simply degrading with age. When you boot into Ubuntu, it may be running in a virtual machine, such that the hypervisor can record all of your keystrokes. The best part of return oriented programming is that if you do manage to do a hard-reset on the machine (by removing the battery), there may be no trace of the attack left on your hard-disk: simply because the binaries were never modified. The attacker would simply reinstall the malicious code the next time they come into contact with your machine.
There is a reason people advocate "cold storage" for large ammounts of money, commonly referred to as "savings". As the Armory author told you, it reduces your attack surface considerably.
From your description of this USB key, I get the impression that you are keeping only one copy. This is a security risk too. If your USB key gets lost or damaged, you would not be able to spend the funds. You really should consider some kind of paper backup in a safe somewhere.