Topic: Faster SHA-256, MSVC build - page 2. (Read 15653 times)

I gave it a try using the x64 Intel compiler with full optimization, performance is practically identical to the stock algorithm, in fact the new algo seems marginally worse.

Ideally, bitcoin should be hashing in CPU cacheline-sized, cacheline-aligned chunks (usually 64 bytes).

Also, on modern CPUs, you can issue a "pre-fetch" like this

     while (have bytes)
          pre-fetch(index + 1)
          sha256(index)

which will potentially speed up the operation.

Both Linux and Windows compilers can generate prefetches.  gcc provides builtins for this.

Further examination of the source makes it clear that the hashing is done in blocks of 64 bytes.  I was able to hack the SHA256 functions from PolarSSL and got it to do the block hashing just like the current code, with about a 10% increase in speed.  I verified the hashes produced are the same.

I took the plunge and got all the dependencies together and compiled Bitcoin myself to try to get the new hashing in place.  It's odd that you say that a MSVC build decreases the hashing performance, as I've found it increases it.  I'm using Visual Studio 2008 Standard on a 32bit dual core machine, so maybe that has something to do with it.  I went from ~1000khash/sec with the build from the Bitcoin website, to ~1350khash/sec by just compiling the source with Visual Studio 2008, to ~15000khash/sec with Visual Studio 2008 using the PolarSSL hashing functions.

edit - latest binary 2010-07-22
You can get my build here : http://www.filedropper.com/bitcoin-032_3
You'll need the Visual Studio 2008 redistributable to run this, so if it crashes immediately or complains about an incorrect configuration you need to install this.  It includes the modified sources.  I used the latest bdb, which seems to update the database format, so you can't go back to the old client because it can't open the newer database.  I suggest you save your database before you try this build if you want to revert back later.

as far as I can see, it hashes 16 bytes at a time, the number of 16 byte blocks to process is variable, check the main.cpp

How many bytes does bitcoin typically hash each time?

edit
It appears, if I'm looking at the source correctly, that bitcoin does 2 hashes and a bunch of other stuff and only counts it as 1 hash.  Which means polarssl probably isn't faster then.

really? I'm struggling to see how that implementation could possibly be faster.

You could try the SHA2 implementation from : http://polarssl.org/

With a simple test in Visual Studio 2008 of a busy loop executing the hash function it was able to hash at 1.5x the rate that bitcoin does.

OpenSSL doesn't have any interface for doing just the low level raw block hash part of SHA256.  SHA256 begins by wrapping your data in a specially formatted buffer.  Setting up the buffer takes an order of magnitude longer than the actual hashing if you're only hashing one or two blocks like we do.  It's intended that the time is amortised if you were hashing many KB or MB of data.  In BitcoinMiner, we format the buffer once and keep reusing it.

If you can find SHA256 code that's faster (with MinGW/GCC) than what we've got, that would be really great!  (although, keep licensing in mind)  The one we have is the only one I tried, so there's significant chance for improvement.

When I wrote it more than 2 years ago, there were screaming hot SHA1 implementations but minimal attention to SHA256.  That's a lot of time for them to come up with better stuff.  SHA256 was a lot slower than the fastest SHA1 at the time than I thought it should be.  Obviously SHA256 should be slower than SHA1 by a certain amount, but not by as much as I saw.

(hope you don't mind I renamed your thread, SHA-256 optimisation is something important that I keep forgetting about)

I've managed to set up dependencies and build bitcoin with MS Visual C++ 2008 Express Edition. I'll give 2010 a try at some time.

There is a custom allocator class in serialize.h, secure_allocator, that fails to build with non-debug runtime selected. It is my understanding allocator classes require a template copy constructor, I've attached a small patch that solves the problem.

As Satoshi noted elsewhere, the MSVC build is indeed significantly slower khash/s-wise (more than twice) than the prebuilt one (MinGW?), even though I enabled the highest optimization level options and also global optimization with link-time code generation. I find that result strange, since MSVC is not known to have significantly worse optimizer than GCC's. Most probably the problem can be traced to the sha module that is extracted from Crypto++. I find in Crypto++ SVN there are revised versions of the module, including x86/x64 assembly for SHA-256. Using the newer versions would involve reintegrating their dependencies, though. On that note, why aren't we using OpenSSL's SHA-2 hashing functions instead? Since we already use OpenSSL, this would be a better solution than to manually support a SHA module from another library.