Not crazy at all. Imagine how much silver you'd have to own for it to be equal to 1 bitcoin. 1 kg is $800, so 1 bitcoin at 80k would be 100 kg of silver.
How would you move that around? How would you take it with you if you had to leave the country? How would you even take it to an exchange? You'd have to take a car and hire someone to help you unload it, because even if you divide it into bags with 20kg each, that's 5 bags you have to carry from the car to the exchange.
What if I told you my investment in bitcoin is far greater than just 1 BTC. Why would I keep hundreds of kilos of silver?
Precious metals suck when you really have a lot of wealth to allocate.