That will remove the bitcoin folder, but is important to remember that we have a bitcoin hidden folder with the configuration and the blocks file. That folder is on our home folder and to delete it we can use:
rm -rf .bitcoin
Or if we want to do it outside the folder we can use:
rm -rf ~/.bitcoin
If we don't delete that folder, the next time we install bitcoin and run it, it will keep the old configuration.
100% correct, but i assumed the OP didn't yet run bitcoind. But you are correct, i should have asked if he did already.
@OP: if you ran bitcoind, you have to follow seoincorporation's advice aswell (unless you ran bitcoind and sent funds to this wallet), if you did not run bitcoind my initial advice was sufficient.
EDIT: if you ran bitcoind, it never hurts to copy your wallet.dat BEFORE deleting ~/.bitcoin, it's always a good idear to keep any wallet you ever created in your archive... You never know if you passed along an address generated by a wallet.