does my wallet have to be online at the exact moment of payment to receive it?
No. Actually the payment never gets sent to the wallet, the wallet just gives you access to the coins on the network.
If the payment never gets sent to the my local wallet, does the amount leave the sender's wallet?
Or, does the network "remember" the payment, and the next time my local wallet is connected, the money is downloaded to my wallet?
The bitcoin is never sent "to your wallet".
When someone creates a tx which sends bitcoin to your addresses, the tx is broadcasted to the network, and later included in a block.
Your wallet contains private keys which enable you to spend the bitcoin on those addresses.
