Suppose my group of people banded together to defend our rights starts oppressing you, infringing your rights. This is how decentralization helps: you also have the right to defend yourself, to join together with others to do so, etc. Basically there would be no problem whatsoever with you and I, in the same territory, belonging to competing institutions that defend rights - you could call them governments if you want. And it would be a very good way to protect ourselves, to prevent any of these institutions from becoming oppressive. It's a fantastic "check and balance."
That is why any legitimate government should allow people to secede. Rather than forcing people to accept its rights-securing services, it should permit people to decline to participate and to participate in competing service providers. The reason governments as we know them don't allow this is because what they really want, rather than protecting people's rights, is power.
I agree with pretty much everything you say until you get to secession. It's a nice thought, but other than granting exit visas (i'm sure that's not what you mean), there are just no practical ways to implement that without wrecking the whole system.
If i'm understanding you correctly, there would emerge multiple city-states & even house-states in the middle of a country state? How would that work?
(maybe i misunderstand what you mean by "secede")