The granularity of Bitcoin's passphrase encryption isn't at the file level, it instead is at the Bitcoin Address record level. There is one field in the Bitcoin Address record in the wallet.dat and that is what needs to be decrypted.
So after being decrypted, in memory is only the decrypted private key for Bitcoin addresses(es) being used for a transaction. But in memory at some point as well is the Bitcoin passphrase that you just entered which is then used for decryption. So like the other posts mention, the Bitcoin client still requires a secure computing environment to protect from keyloggers and other malware.