So at some point the price of gold will start to go down and the price of BTC will keep going up.
I think so, this factor reduces the supply of Bitcoin circulating in the market because some users lost access to their wallets. I don't know how much Bitcoin can not be used because of this factor, but certainly this factor reduces the amount of supply in circulation and makes Bitcoin more special than gold