There are multiple exchanges which may somewhat differ in exchange ratio. There are also over the counter transactions that may differ in exchange ratio. The extent to which exchange ratios differ is limited by arbitrage opportunities between these markets. Overall, exchange ratio tends to converge between all exchanges. There can be fungibility issues (withdraw limits / delays) that limit arbitrage opportunities and can increase the difference between exchanges, but a bitcoin will still be a bitcoin once withdrawn.
So if there is a problem, it's a trust problem related to the ability of the exchanges to provide the Bitcoins on their sheets. You still can exchange Bitcoin in person to circumvent this trust problem.
Simply use an average price with data from all notable exchanges. Problem solved.
ya.ya.yo!
No, the problem is far to be solved using an average price of all the exchanges. The problem continues. Are every bitcoin in a exchange? No they dont, so there is a huge amount of bitcoins that are bypassing exchanges and they are not taking into account.
Even if we sum all the exchanges we are only taking into account a portion of the bitcoins offer or bitcoin demand so is fair to say that the actual price only correspond to that portion of bitcoins that are "in play" into the exchanges.
Every person who has Bitcoins can act as an exchange, thats the point and we only can measure, nowadays, the portion of demand and offer wich corresponds to the exchanges (like Bitstamp, Huobi and so on), no the global demand and global offer of Bitcoin (as a "product" itself).
Your problem only exists in theory. It's true that not every Bitcoin is "in play" solely on the exchanges. But in practice everybody uses the prices from the exchanges as reference price.
The same is true for precious metals like gold and silver. For sure there are many traders of these metals outside the major exchange. But all of them use the spot price as reference for doing business.
ya.ya.yo!