Correct
Public key, in this context, means your bitcoin address (which is a hashed version of the public key).
If you use bitaddress.org to create paper wallet (or just to generate new address with private key etc) you'll see there are 2 parts: one marked as 'secret' - this is your private key (keep it safe and don't share this one), and the other, marked with green 'share' is your bitcoin address. You can use the latter to monitor your paper wallet balance.