So for me it was like 10ms per op, which was still to long, so I decided to use the openssl implementation.
For a moment I even had a solution with a TCP server (written in C, with openssl) that was doing the ECDSAs for my actual app. That was pretty crazy, though it worked..
![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
Later I figured out how to combine the openssl lib with my go app and ever since then it works as fast as I needed.
Most of the languages these days allow creation of wrappers for native libs - that's probably the best way for you to go as well.