People do have a moral loyalty to their states, their religions and their football teams. That's why a state can provide police and security more cheaply.
Your question about the men with guns misses the point about human nature. All that stands between us and chaos is the rule of law. Take away police protection and it takes an hour or so for looting to start. Take away border protection and you get invaded, many lives will be lost and all property owners may be dispossessed.
Every human society has to live with this reality. Given that mass violence is part of our nature, the only choice we have is whether we have men with guns that are tied up in rules and procedures and that work for us?
Saying they work for us is pretty much a joke when they force us to provide them their paychecks.
And given that, it becomes clear why, over time, governments act less and less as if they're "tied up" with any sort of limitations.
Yet you want to take a group of people with mass violence as part of their nature, and give them a socially-accepted monopoly on initiating violence.
Considering the millions that governments, even "democratic" governments have killed (especially in the 20th century) and continue to kill, I have a hard time seeing why you feel that a society without a centralized nation-state would be more dangerous
overall than one without.
Police and soldiers are employees. You choose to hire them and you choose what to pay them.
But that's just not true, either on an individual level or on a societal level.
If I see that the police in my city are corrupt, and specifically that Officers Adams, Baker and Crowley are guilty of abusing their powers and hurting innocent people, I can't decide to fire them, or to start paying them $0. Yes, I could
try to convince their bosses to do so, but the point is, it's not my decision as an individual.
And even if the entire society wants them gone, who's going to make it happen? As long as the people making the decisions an the people wielding the guns act in accord and protect each other, nothing can be done. In the U.S., over 90% of the populace was against the first round of bank bailouts. The rulers knew this. They ignored it, and nothing has happened to them. Even the voting is gamed. It may have been better in the past, and it may be better in other countries, but only because the ruling class wasn't inclined to be so brazen about flaunting their control, and the lack of the people's control. In any society, if the ruling class decides to stop pretending that the people have control, then the people will find out pretty quickly that they don't.
I agree that societies with centralised nation states are far more dangerous than those with no state. That is why I prefer to live in one with separation of powers, the rule of law and a decent border. The alternative is that you live at the mercy of foreign states. This is what you get if you don't have a strong state to defend you:
http://www.guardian.co.uk/world/2012/may/31/thousands-flee-extreme-violence-congoWhich do you choose?
If I were to have a true choice, it wouldn't be from the alternatives you offer... I'd rather choose a society with no government, but with people choosing voluntary protection services (or self-protection,) and with enough wealth to defend themselves from foreign states too. If Afgahnistan can do it, then a government-less U.S. could easily repel foreign states, and likely several other advanced nations could too.
Now, I honestly doubt that'll ever materialize in my lifetime, or anytime in the near future, but it's the moral ideal, and the most practically effective in terms of providing security and allowing for prosperity, so of course I try to choose options that lead closer to that ideal.
Given an already advanced and wealthy nation, and the option of continuing its government or disbanding it (or at least disbanding the higher levels,) I'll always opt for the latter. And while I believe it'll lead to greater personal benefit for me, that's not why I'd choose it... I'd choose it because I believe that that is the moral option, because I believe that initiating force against someone who has not done so is wrong, and is one of the greater wrongs that mankind is capable of.