This is like asking, "if I put snow tires on my car, and then take off one of the wheels, will I still be able to drive in the snow?" It's just a strange question.
Yes you can, in Electrum, Armory, Blockchain.info...
I said sane. It is nutty to allow users to shoot themselves in the foot by providing an interface to act on mistaken assumptions like these:
OP: Then I delete the address. Will money still be in the wallet? And I will be able to send money from 1st address? Without a trace?
I think it's nutty not to allow power users to do what they want. Obviously you should hide it and warn the user so the idiots don't use it, but still.