The judge represents neither parties and is supposed to be independent.
The problem with this is that it is only possible to show conflict of interest within a small sphere around the case. Without a thorough investigation into the financial and social lifes of the parties no deep assessment can be made about conflict of interest.
So for all intents and purposes you will always be at risk in an exchange. It can be called 'voluntary exhange' and you still have no way of preventing being screwed over.
Fraud/conflict of interest and voluntary exchange are not mutually exclusive.
What you claim would suggest that it is never possible to have fair exchanges of anything, regardless of whether there is law, government, or anarchy, and that we should either just live within those means and assumptions, or use some other system. So, is your point simply to point out that in all trade, free or government regulated, there will always be fraud (a blanket statement that doesn't actually provide any purpose), or do you know of some system whereby we can exchange goods and services without fraud that no one is aware of yet?
Well, almost.
You cannot know all circumstances or consequences of a deal in all cases, so you can never
guarantee a fair deal.
The more information you have and can process, the better you can be sure you get a good deal.
So certainty about fairness of a single unprotected voluntary exchange only goes so far.
My point is that you can only have relative fairness in society. You cannot have an absolutely fair system, in principle. Fairness is a human construction anyway. There is nothing fair about nature, for instance.
I see this as a big problem for both the resource based economy and a lot of the anarchistic ideas.
On one side, the resource-based-economy people wants to control eveyones resources.
It requires some godly computer running some godly software that makes no errors and has no exploits. It's almost religious if you think about it.
And it requires the current holders of resources to comply and share fairly (or at least, according to the computers demands). I don't see this happening any time soon because it will lead to a conflict around a different opinion of what is fair.
On the other side anarchist capitalists want free uncontrolled market states (preferably floating) and naively assume everyone will trade in a fair way (or else they pull out their guns) and that the market will automagically balance in a fair way.
Free markets, altho efficient, balance towards maximum profit, not towards where there is the most need for a good. This will lead to lots of friction which will lead to conflict. Not a stable substrate to run large societies on.
I find neither of these solutions particulary fair altho both systems were thought up with fairness as a goal.
An optimum fair solution probably lives somewhere between these two extremes of statism and freedom.