...
As far as I understand, installing a clean OS is one of the best ways to guarantee you are not already compromised by trojans or keyloggers and, if your real OS gets compromised, they would have a hard time reaching inside the virtual machine and getting to the wallet.
...
Actually, you have it all backwards: a compromised host OS might very well have access to each and every file in the virtual machine - it's only a matter of what capabilities the malware has.
Your approach raises the bar by making your machine non-standard, but no VM vendor can guarantee that the clean virtual machine will remain safe when ran on a compromised system.
While an additional degree of security is certainly nice to have, you should not depend on it.
Virtual machines are usually deployed in exactly the opposite scenarios: to contain a threat and prevent it from leaking to the host OS.