Topic: [ANN]: cpuminer-opt v3.8.8.1, open source optimized multi-algo CPU miner - page 162. (Read 444067 times)

cpuminer-opt v3.3.4 released.

https://drive.google.com/file/d/0B0lVSGQYLJIZVTFNSkZ0elRQZ2M/view?usp=sharing

very nice work
waiting for v3.3.4
thank you a lot

I'm starting to prepare v3.3.4 for release. I will check for new posts before I publish

As it stands there are two changes:

- removed incompatible SW build check to fix problems for users with Westmere class CPUs.
- optimized m7m AES +65%, SSE2 %59%

Edit: This is what v3.3.4 should look like when compiler for and run on westmere. It is not actual
program output because I don't have a westmere CPU.

 

I considered that but it's more than one user. And I downloaded the same binaries they did, and they stated they used
the westmere build, filename cpuminer.exe. So it is a copy of the same file not a reproduction. Also that is the only
build that does not support AES+AVX. Had the users tried the wrong file it would crash. Some did and reported same.

I don't know precisely how it was compiled but assuming I'm not being similarly misinformed by several users
my observations seem correct. The same file shows a different result for __AES__ on different CPU architectures.
I don't know which is correct but they should be the same.

Here is a bit of direct output of the variables used in the logic compiled for corei7


I've removed the warning messages otherwise no change from 3.3.3.

The result of the logic is not relevent in this example because the CPU has AES but the software check
is what's important. In this case sw_has_aes taken directly from __AES__ is false. This seems reasonable
for a westmere build and it agrees with my results from testing the CMB binary on the same CPU.

So I think I've gone in a complete circle and back to the same point. The CMB westmere binary displays a
different result for the __AES__ macro on different CPUs. Three users have reported he same problem with
actual westmere class CPUs because, for them, __AES__ returned false.

I am confident in the data reported by the users.
I am less confident about the nature of the CMB build. I don't know if they built native on westmere HW or
crosscompiled using -march=corei7. I could only verify indirectly by crosscompiling myself.

But even ignoring how the bin was built __AES__ should still produce the same result on any HW.

__AES__ does not change between processors if the binary does not change. Same for __SSE2__ and similar.

Observe: https://godbolt.org/g/WdT8c5

Try changing compile flags from -march=skylake to -march=core2 to -march=native or others from this list. You'll see that resulting assembler code changes in how many times print is called and what string constants are defined.

When __AES__ is not defined, then printf("AES\n") doesn't even make it into the binary. Same for others.

When the user said he's using the same binary as yours, he might said that by mistake -- you need to compare sha1sums of the binaries to make sure -- he might be running different .exe than yours.

No comments.

It's reverse logic. I'm actually checking to make sure it was not built with AES. An AES build is not
compatible on a non-AES CPU. A non-AES build is assumed to be SSE2.

But that logic had escaped me. In fact I never check sw_has_sse2 in the code.

Given that new information I still struggle with the fact that the westmere build for Windows shows
different results on my CPU (SSE2==YES), and real westmere/nehalem CPUs (SSE2==NO).

Given the actual code is testing for AES the result (SSE2==NO) actually means (AES==YES), the build
supports AES. Since an AES build won't work on a non-AES CPU the miner errors out with an incompatible
build.

The bottom line is the same build is being seen as AES on westmere but non AES on my skylake.
The difference with checking AES is that is the compiler macro that is now showing conflicting results.
 __AES__ returns true on westmere and false on skylake. And that's where I'm stuck.

Care to take another shot? Even when you miss the mark you make me think a little differently.

And another suggestion - since you have Windows support, can you add and icon to the compiled binary?

I made myself one that looks good IMHO on every resolution on windows.
I will be more than happy to provide the files that I used - icon.ico, icon.rc and icon.o

This my configure to add and the icon in binary:

Here is the screen:


The link and license for the icon
http://www.iconarchive.com/show/tulliana-2-icons-by-umut-pulat/k-cm-processor-icon.html

See anything out of place?

You say you're checking for SSE2, but in code you're checking for sw_has_aes.

As we wait for the new version with all things that you plan to include/fix, can you post which algos are AES-AVX1 and which are only SSE2,
both on Linux and Windows so everyone can contibute a faster source (if there is one) for every algo that needs AES-AVX1 optimisations?

Thanks. You're the third person with Westmere with this problem. The main problem is "SW built for SSE2..........NO."
When I use the same file on my Skylake it answers YES, as it should. It isn't the way it was compiled because
I'm using the same file downloaed from CMB.

The software build test is supposed to reflect the compilation, not the HW support, however, the same SW reports
different results on different HW. I can only conclude the CPU does affect the SW build test. Without a relaible way
to test the SW build I have no choice but to disable the test.

The plan for v3.3.4

All known issues will be resolved except 1.

hodl on Windows for non-AES CPUs is still not working and will remain disabled.

V3.3.3 seems to have finally solved all the low diff problems with hodl.

My attempt to help users by detecting when and under-oprtimized build is being used has
been a major flop. It's over. v3.3.4 will still display the detected CPU architecture and
expected performance but it will got give any advice or take preemptive action.
The capabilities check will be informative only, the miner will alway attempt to mine.

Also there are significant improvement to m7m algo thanks to Wolf0.

I intend to wait a while longer before releasing v3.3.4 in case there any more reports from
users about v3.3.3.

Just tested v.3.3.3 on Xeon E5649 (Westmere-EP, AES without AVX) and got same error like on v3.3.2

         **********  cpuminer-opt 3.3.3  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX1 extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0 and Jeff Garzik.

Checking CPU capatibility...
        Intel(R) Xeon(R) CPU           E5649  @ 2.53GHz
   CPU arch supports AES-AVX1...NO.
   CPU arch supports SSE2.....YES.
   SW built for SSE2..........NO.
Incompatible SW build, rebuild with "-march=native"

OK, Thanks. I can't explain why it displays "SW built for SSE2..........NO." on your CPU and YES on mine.
This is supposed to indicate how it was compiled.

I've had enough trying to get it to work. I'm removing those checks in the next release.

I used cpuminer.exe (I have Nehalem architecture), on cryptonight and on nist5. Same error on both.

Which exe did you use? Which algo?

Edit: I tried ccminer.exe on a skylake which has AES and AVX and the capabilities check said the SW was
built for AES. If you used the same file and it said it was not built for SSE2 then there is an inconsistency.
The same file was built for AES but not SSE2. When I compile with -march=native both are true.

It's becoming more clear that this check should be removed.

I tried this out and I get error:

Checking CPU capatibility...
        Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
   CPU arch supports AES-AVX1...NO.
   CPU arch supports SSE2.....YES.
   SW built for SSE2..........NO.
Incompatible SW build, rebuild with "-march=native"

Just to report, v3.3.3 seems good now. Haven't yet encountered any issues.

Edit: Just for reference
Tested with windows build on i7-4770k, i3-4160 & i5-2500k.

Second issue is hodl on CPUs without AES on Windows. It still doesn't compile.
It works fine on Linux.

I am not testing this on a real core2, just forcing a core2 compile on an i7. I don't
think this is a factor.

The problem is endian.h. It's missing when compiling on Windows and is needed by two
files used for non-AES hodl. I can't find any endian.h in any of the mingw libraries. I have
searched the net and didn't find any endian library for mingw.

I tried a few different variations of endian.h such as sys/endian.h and a file I found on the
net called portable_endian.h. Some didn't exist, some didn't have the necessary definitions
and some compiled but produced only rejects, none of them worked.

The endianness on Windows might be a factor.

I need to find an endian library for mingw that Works with Windows little endian. Any suggestions?

There are a couple of issues that I would like some advice on from the community. I will make seperate posts
so the discussion doesn't get garbled.

The first issue is compiling an optimum build for early CPUs with AES support. These are Intel CPUs between
core2 and Sandybridge including Westmere as well as similar AMD CPUs.

This is primarilly intended for users mining with such CPUs but may also be of interest to others.
 
I recently discovered the AES optimizations I have been using extensively use AVX1 instsructions as
well as AES. AES was implemented on CPUs before AVX1. The CPU capabilities check in cpuminer-opt
prior to v3.3.2 did not make this disctinction. It assumed if the CPU has AES it could use the AES code in
cpuminer-opt.

This results in users with these CPUs, particularly user using the CMB binaries, seeing the following message
on miner startup:


One user has reported that the Westmere precompiled binary appears not to be compiled for AES and AVX1.
It isn't clear why. That's where i need your help if you have one of these CPU. Linux, Windows, self compiled
or  prebuilt binaries, it doesn't matter. If you have a CPU in the Westmere family between core2 and Sandybridge
you can help a lot.

Please post:
- your CPU model and confirmation whether it has AES and or AVX (may also be called AVX1) if possible.
- whether and how you compiled or if you used a binary which one
- the miner output at startup showing the CPU capabilities check
- whether it mined successfully
- any other info you think is useful

Thank you in advance.

Edit: Another option is to abandon the rebuild check. It is intended to catch precicely the type of situation that
may currently exist with Westmere CPUs possibly getting lower than optimum performance. However, it seems to
be causing more grief than it's worth, partly because it was not correctly implemented. Users thought the rebuild
message was an error and meant the miner wasn't working.

Even if the rebuild check is working properly an attempt to use the existing Westmere Windows build will result
in complaints likely coming to me when they should be directed at the builder.

I'll wait to see the reaction is to v3.3.3, if there are still problems with the rebuild check, it's gone.

Yet another release coming, hopefully to fix the final hodl issue.
There isn't a good way to test the fix other than soaking it but it's
well understood so I'll get v3.3.3 out ASAP. There are no other changes.

Update:

https://drive.google.com/file/d/0B0lVSGQYLJIZQ1puZWRjWUdIRVE/view?usp=sharing

There is also a small change to build.sh.

Edit: v3.3.3 binaries

http://cryptomining-blog.com/wp-content/download/cpuminer-opt-3-3-3-windows.zip