Taxes support the government. Not the infrastructure. Roads, sewage, everything that is actually needed can, by that very feature (necessity) be paid for privately.
Telling me that I have to support them because they provide something I need is much less impressive when they don't let anyone else provide it.
I'd like to point out that I believe you contradicted yourself. Specifically, you said that taxes don't support the infrastructure but that they support the government, and then you go on to say that the government doesn't let anyone else provide the infrastructure. So...then who is providing the infrastructure?
Yes, taxes absolutely support the infrastructure.
Could there exist a societal structure in which taxes are not needed. Yes. Does that structure exist currently? No.
Asking to not pay taxes because you'd prefer a society in which infrastructure is supported by private ventures is like asking for an honorary Ph.D because you think you have more knowledge in a particular field than someone who has paid the time and money to obtain a post-graduate degree. Sure, you may actually be more knowledgeable, and it might be a better idea to judge credentials based upon actual knowledge and merit rather than a piece of paper -- but too bad, that's not the reality we live in right now. Suck it up and deal with it.
When a large enough percentage of the people realizes that a certain type of society can exist that does not require taxes nor pieces of paper certifying academic credentials, then that society will take shape. But, the truth of the matter is that people either simply do not know how to make such a society a reality yet (it's a very, very complex problem), or they're pretty darn happy with the way things currently are.