Free: MyBB however if you have any development experience and coding knowledge then Vanilla may be a good bet.
PHPBB is pretty much dead and always turns into a spam fest, SMF seems lagging so far behind so yea, others aren't really noteworthy; by that I mean you're going to want to go with a popular/most used one for the best support, especially if you're development experience is dusty.
Paid: XenForo or for a very large scale community IPB, I used to love vBulletin back in the days but with the release of 5 it went down hill in my opinion BUT they're great for support too and for larger communities, so depending on what features you're looking for, either of those may be a good bet.
Thanks for pointing out all the free & paid forums for me to check. I really didn't know about XenForo and IPB.
I second IPB, or Epochtalk, which is free.
I checked IPB and they are expensive if I choose cloud, and if I choose on-premise then their support is limited. But indeed the software is a quality thing.
Can you give 1-2 reference forum which is using Epochtalk? Is it still in development or it is offering the final product?
if you're interested in beta-testing the future forum software, then i'd suggest you to try epochtalk.
i personally like mybb too, but i think epochtalk is more elegant (if that works perfectly). anyway you can try and test several software on your local server and then decide which one is the best for your interest later on.
Not sure about beta-testing.
To test several software will be costly if I finally decide to go for paid ones, but it can be done with the free ones.
First time I read about epochtalk, but I am more convenient using smf because of my familiarity with this kind of software, I am a member of three forums that uses SMF and I find it very interesting and you can add a lot of features just like what Bitcointalk is doing.
I'm also much convenient using smf but I don't want to develop a forum with this software.
There are actually two reasons: (1) users accounts are vulnerable (2) I heard the software has not been updated for the last few years, not 100% sure though.
@akamit
Then start with SMF. Play with the admin panel. They have free themes. Install them. They have free modules. Add edit the features. If you don't find it satisfactory then try one that was suggested by users above.
Once I tested smf to understand how backends work. I like the core theme of smf, which is bitcointalk using but with some edits on design maybe. Still not sure which software to use