A PGP public key can be used to encrypt messages to or verify signatures from the owner. To understand better how it works learn to do once RSA with pen, paper, and a calculator:
http://en.wikipedia.org/wiki/RSA_%28algorithm%29#OperationIt won't take long, and it's fun. This would also help you to understand Bitcoin better.
Download GNU Privacy Guard from
www.gnupg.org to generate your own keypair. It will generate for you two keypairs, one for signing, and one for encryption (for security reasons). Signing is the reverse of encryption. The hash is encrypted with you private key and decrypted with your public key (from the signing key).
GPG/PGP actually uses hybrid encryption, it encrypts the key (password) with RSA (asymmetric encryption / public key cryptography) , and the rest of the message with AES (symmetric encryption / password / key based). The reason for this is that the size of the message would be to large due to the modulo.
Symmetric encryption is more similar to the secret alphabets kid's play with, if you did that kind of stuff when you where a kid. Except a cipher like AES uses a different "alphabet" for each "letter", and then does a lot of permutations, well it works a bit different, but that's the gist of it. You can read more about it here:
http://en.wikipedia.org/wiki/Advanced_Encryption_Standard and
http://en.wikipedia.org/wiki/Substitution-permutation_networkGPG/PGP allows you to choose between different ciphers, but I recommend you stick with RSA/AES (future versions will also include ECDSA (what bitcoin uses). There quite a few user friendly front-ends, and browsers plug-ins.
In terms of security RSA4096 = ECDSA256 = AES128.
In short it's used for signing so that others can verify authenticity and encryption so that others can encrypt stuff to you.
Also think of the public key as a mailbox, people can put stuff in, but you need a key to open it, and only you can take stuff out.
