The people performing those actions are disagreeing with you.
So you know for a fact that a rapist knows that the person they are raping doesn't want to be raped because they themselves would want it? I know common sense would suggest that, but there is nothing common about sense. It only takes one crazy person to break unilateral consensus.
Euthanasia and rape fantasies are well documented, and while they're not technically murder and rape (in some places), some people feel quite strongly that even voluntary mercy killings are a form of murder - unlawful killing of another human being. Where is there unilateral consensus? What scope are we talking about? A man in the US has a brother in NL who takes advantage of it. The US man thinks the doctor has murdered his brother...a destructive act, but both the doctor and patient acted with NL laws. Is the US brother a neighbour to his dead brother? Should the doctor compensate the US brother?
Unilateral consensus is indeed necessary to generalize that something is "actually non-destructive", but even in extreme things like rape and murder there is contention. Even if one were to accept the three things you mentioned as exceptions, that's all they are. For everything else, there is contention. Instead of deforestation, if we used "remove all hydrogen cyanide" do we have a different picture? That's an extremely poisonous gas which most people would say "yay" to removing from the planet, but is removing it non-destructive? Everyone is happy, we know it will no longer kill us, but of course it's destructive...to the people who use it to kill rodents. They can find something else, they can do a million other things, and those are all irrelevant to the fact that nothing is non-destructive.
Umm. Every transformation by definition requires the destruction of a state. You are calling this "non-destructive" because the people affected are "happy" with the result. You're making a very strong case that destructiveness is a matter of opinion. Who are your neighbours, again? I'm still not clear on this. Anyone that is affected by an act? or anyone that is affected and can prove it?
So society cares. What if it doesn't know about it?
Is your ability to read minds a transferable skill?
I believe you'll find my answer to this above, here.
If I understand you correctly, your answer is "My neighbours are any human who might be affected by said deforestation and can prove it"?
While I respect your desire to take such responsibility, how will you seek damages from someone who has destroyed something of yours, and refuses to accept responsibility?
Well, in our current society, I would seek damages via the courts system. In an AnCap society, I would seek damages through arbitration, and should they refuse arbitration, they would be in violation of several of their contracts, resulting in far harsher consequences than I could ever levy.
I see, so they'll be forced to accept majority (or at least the strongest) rule either way.