Why do you trust banks with your money? Oh wait, I don't, and I don't trust online or other wallets either.
I don't see what's wrong with a password protected Bitcoin core wallet. I think that is the most trustworthy, but that's just an instinctive reaction on my part.
But the OP question still remains.
Why do you trust the Bitcoin Core wallet?
Why do you trust the operating system that you are running your Bitcoin Core wallet on?
Why do you trust the hardware that your operating system is running on?
Why do you trust the mathematical functions of ECDSA and SHA?
I think the important question that the OP was asking is...
"If I do not have the technical expertise to know exactly what the software and hardware are doing, then how can I be certain that those who do have the technical expertise aren't lying to me?"
The security of open source relies on the concept that hiding the true nature of the thing requires that ALL TECHNICALLY CAPABLE people in the ENTIRE WORLD will collude to hide the behavior of the thing from ALL TECHNICALLY INCAPABLE people. If even 1 single technically capable person refuses to collude with all the other technically capable people, then that one person can reveal the true nature of the thing to all the technically incapable people.