Are you planning to merge mine with BTC? If you are, then those wallets would all be on the server as well, and there is no option to not send coins to them. Assuming you're setting up multiple coins/p2pool nodes, then you can start each like this:
./run_p2pool.py -a SOMEWALLETADDRESS ...
Where SOMEWALLETADDRESS is a wallet that lives somewhere else.
However, if you are merge-mining NMC, IXC, etc with your BTC p2pool, then the payouts for each of those coins would go directly into the server's wallet. There is no option to make payouts to other locations.
You could probably write a simple script to periodically check the balance of those wallets, and transfer funds to an off-server wallet.