the main risk with going too fast is the heat - that's the only thing that can do a long-term damage to the board. And you'll need to go over 125C for that to happen, so I think it's relatively safe to say that it's rather difficult to damage the boards
(and a word of caution - having a good USB power supply is a must! If it ever goes over 5.5V that may actually do a irreversible damage to the components!)
So - feel free to push it to as fast as you like.
When you push it too much you'll see one of a few things happen:
- you'll start seeing a lot of hardware errors (over 5%)
- some boards may stop hashing completely (it's a matter of luck with the bitfury chip)
- the board will disappear completely from your computer - it will stop being recognized by the USB hub and PC. This may also be a sign of the power supply failing (if it goes below 4V that's the most likely outcome)
If you see any of those - that's a sign that you've gone beyond the limit and you should either slow it down a bit or get more cooling.
Luckily in bfgminer there is the Device Manager menu (press M to get to it) and you can adjust the number of bits (speed) for each individual board. You may find that one works just fine at 54, the other is more stable at 53, etc.
Have fun!