I'm not sure why this should even be an issue. I don't walk out of a casino and leave my chips with the cashier, why would you leave SWC without withdrawing your BTC? Just like everything else in the BTC world, the responsibility of protecting the bitcoins is no ones except the user's. I personally feel that anyone that leaves their BTC in anyone elses control should expect them to be stolen.
You make a good point.
Yes, taking them back when you aren't using them is indeed the only way you will know your coins are safe.
I suppose if we are incarcerated it will not be possible to return them in a timely manner, but they won't get your coins by seizing the server, which is in Iceland and possibly non-trivial to do anyway.
We have db backups and should be able to return funds safely and relatively speedily without the current server, but a more detailed plan should be developed and announced.