Nothing is stopping you from forming a moneyless commune with like-minded friends. It could be a great experiment.
You're naive if you think that infrastructure will get built if everyone does only the jobs that they want to do and which make them happy (not to mention the fact that the raw materials for building infrastructure need to be procured).
In defense of this notion, if the desire and bids for certain infrastructure become high enough, people will drop what they are doing and build said infrastructure regardless of their passion. Everybody has their price.
For instance, if nobody wants to clean toilets but one guy or two really want a clean toilet to shit in and/or they get tired of the stench emanating from poorly maintained toilets, they will likely be willing to pay a premium to get toilets cleaned well beyond a standard minimum wage.
