If your new to Linux I would suggest a debian distribution like Blazr suggests: Ubuntu and its varieties are the simplest.
Personally I have always used Slackware as a linux distro, lately I have tried out Arch Linux which is awesome, but probably not what your looking for.
Its really easy to build your own Bitcoin wallet from source on Linux comparative to Windows/Mac even though Mac is easy with macports or homebrew.
Also forgot to mention Spazz, I wouldn't use one computer for both the mining "controller" and having the Wallet stored on the mining machine is a mistake in my opinion. Create the wallet on a computer with no network card or drivers/ back it up etc.
For a mining controller I had success using raspberry pi's with powered USB Hubs.