You only need to short it to start it up, you dont need to leave it in there while its running.
If you remove the ground from the PWR_ON pin, I'm pretty sure it turns off the PSU. I don't have one to test on right now, but are you sure on that?
I'm pretty sure. I've started PSUs up to see if they're not fried due to their owner's stupidity (hate tech support, hate hate hate) and they stayed on until I cycled the power.