Topic: [ANN]: cpuminer-opt v3.8.8.1, open source optimized multi-algo CPU miner - page 54. (Read 444067 times)

any xevan benchmark? using avx, my ivy only get 541kh 28,89kh

thanks

When there are no orders to mine Nicehash disconnects.

This is a problem for nacehash and not in the program
Этo пpoблeмa нaйca, a нe пpoгpaммы

Hello!

Please, fix it:

         **********  cpuminer-opt 3.7.3  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0, Jeff Garzik and Optiminer.

CPU: Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz
CPU features: SSE2 AES AVX AVX2
SW built on Nov 20 2017 with GCC 4.8.3
SW features: SSE2 AES AVX AVX2
Algo features: SSE2 AES AVX AVX2
Start mining with SSE2 AES AVX2

[2017-11-21 23:40:16] Starting Stratum on stratum+tcp://hodl.eu.nicehash.com:3352
[2017-11-21 23:40:16] 4 miner threads started, using 'hodl' algorithm.
[2017-11-21 23:40:16] stratum_recv_line failed
[2017-11-21 23:40:16] ...retry after 10 seconds
[2017-11-21 23:40:26] stratum_recv_line failed
[2017-11-21 23:40:27] ...retry after 10 seconds
[2017-11-21 23:40:37] stratum_recv_line failed
[2017-11-21 23:40:39] ...retry after 10 seconds

I commented about the benchmarks.

I had wanted to give a try at cuda dev but the learning curve was steep and resources (ie documentation) are expensive.
CPU mining isn't very lucrative but I can better transfer my skills when I can google anything that stumps me.

No problem. Hit me up if I can somehow help, I'm not that familiar with those technologies either but I'd like to help and learn
By the way, I posted a few benchmarks on GitHub to see from which algorithms the speed difference between Alpine and Ubuntu comes.

Just posting a link because I don't really want to re-create the table here: https://github.com/JayDDee/cpuminer-opt/pull/41#issuecomment-346017309

Thanks I'll look into it.

I don't develop using git. I develop in my own confort zone and upload to git when ready fo release. If I merge
a PR it messes up MY development environment. It's easier for me to import the changes to my env.

Also most everything, git, autoconf, x86, even to some extent c, are all foreign to me and I just hack my way through.
I'm learning as I go. Even my vector processing knowledge is based on 1980's tech (Cray). I even knew about OO in
the 80s before it was called OO. It took me a while to make the connection when OO became popular.

So in a sense I'm working in a foreign country with unfamiliar culture and different language. It's a fascinating
experience, I just have to manage my frustration with the many stupid mistakes I make.

Maybe this could be useful: https://gist.github.com/panzi/6856583.
Also, any particular reason you don't merge PRs?

The openssl version on Alpine is actually 1.0.2m, not 1.0.0, no idea why there is a conflict with an old version.

That hodl error is familiar. If works it may solve the issue that also required the custom version I wrote for Windows.
If Alpine is a special case I don't know how to test for it. I'll look into it for he next release.

The ssl warnings might be related to your old version of oenssl. My system has openssl-1.0.1. If you can you should upgrade to 1.1,
it will give better performance on algos that use sha256 even if it doesn't fix the warnings.

Yes there are a lot of warnings. Some are my own doing and I try to keep them under control. But there are also a bunch related to
 the hashing macros and i don't want to touch those macros. Fortunately 4way will make them obsolete and I will be able
to remove them.

Posted it on GitHub, but guess you're more active here? By the way, this can easily be fixed by adding "#include ", just need to know in which case ("#ifnfed") I need to include it.

Code:

algo/hodl/cpuminer-sha512_avx.o: In function `sha512Compute32b_parallel':
sha512_avx.c:(.text+0x369a): undefined reference to `htobe64'
sha512_avx.c:(.text+0x36c8): undefined reference to `htobe64'

Also:

Code:

/usr/lib/gcc/x86_64-alpine-linux-musl/6.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libssl.so.43, needed by /usr/lib/gcc/x86_64-alpine-linux-musl/6.3.0/../../../../lib/libcurl.so, may conflict with libssl.so.1.0.0
/usr/lib/gcc/x86_64-alpine-linux-musl/6.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: warning: libcrypto.so.41, needed by /usr/lib/gcc/x86_64-alpine-linux-musl/6.3.0/../../../../lib/libcurl.so, may conflict with libcrypto.so.1.0.0

But that's only a warning.
There are A LOT of warnings actually.

My super big mining machine ))):
         **********  cpuminer-opt 3.7.3  ***********
     A CPU miner with multi algo support and optimized for CPUs
     with AES_NI and AVX extensions.
     BTC donation address: 12tdvfF7KmAsihBXQXynT6E6th2c2pByTT
     Forked from TPruvot's cpuminer-multi with credits
     to Lucas Jones, elmad, palmd, djm34, pooler, ig0tik3d,
     Wolf0, Jeff Garzik and Optiminer.

CPU: AMD Athlon(tm) II X2 260 Processor
CPU features: SSE2
SW built on Nov 21 2017 with GCC 7.2.0
SW features: SSE2
Algo features: SSE2 AES AVX AVX2 SHA
Start mining with SSE2

[2017-11-21 09:11:31] Starting Stratum on stratum+tcp://mine.zpool.ca:6033
[2017-11-21 09:11:32] 2 miner threads started, using 'm7m' algorithm.
[2017-11-21 09:11:33] Stratum difficulty set to 2
[2017-11-21 09:11:36] Accepted 1/1 (100%), 23.84 kH, 12.00 kH/s, 43C
[2017-11-21 09:11:50] Accepted 2/2 (100%), 196.15 kH, 24.75 kH/s, 43C
[2017-11-21 09:11:51] Accepted 3/3 (100%), 126.72 kH, 24.71 kH/s, 43C
[2017-11-21 09:11:52] Accepted 4/4 (100%), 145.40 kH, 25.07 kH/s, 43C
[2017-11-21 09:11:53] Accepted 5/5 (100%), 33.15 kH, 24.74 kH/s, 43C
[2017-11-21 09:11:59] Accepted 6/6 (100%), 99.35 kH, 24.86 kH/s, 43C
[2017-11-21 09:12:03] Accepted 7/7 (100%), 222.69 kH, 25.16 kH/s, 43C

./autogen.sh && CFLAGS="-O3 -march=btver1" CXXFLAGS="$CFLAGS -std=gnu++11" ./configure --with-curl && make -j2 && strip cpuminer
Manjaro Archlinux
Linux impulse-pc 4.13.14-2-MANJARO #1 SMP PREEMPT Sat Nov 18 14:25:40 UTC 2017 x86_64 GNU/Linux

cpuminer-opt-3.7.3 is released.

git: https://github.com/JayDDee/cpuminer-opt

tarball: https://drive.google.com/file/d/1Nw-kHZ1bnkEjtuKlOdf3XSh-LNZReH4E/view?usp=sharing

Windows binaries: https://drive.google.com/file/d/1r6KLL_YBYvL3Lr0nfeVOEBlPXkATSIxw/view?usp=sharing

New in 3.7.3:

Added polytimos algo

Updated dockerfile

Introducing 4-way AVX2 optimization giving up to 4x performance inprovement
on many compute bound algos. First supported algos: skein, skein2, blake &
keccak. This feature is only available when compiled from source. See
RELEASE_NOTES for instructions how to enable 4-way during compilation
and which algos are currently supported.

Edit: It should be noted that not all algos have been fully tested using 4way.
Skein and keccak have both been tested but I couldn't find a pool for skein2 or blake.

For devs and geeks:

What is 4-way AVX2?

4-way AVX2 uses AVX2 instructions to hash 4 nonces in parallel per CPU thread instead of
just one yielding a theorectical 4x increase. The realized improvement will be lower due
to AVX2 operating at reduced clock speed and some overhead. 4-way AVX2 optimization
uses vertical vectoring similar to GPU miners instead of the existing horizontal vectoring
of some algos. Horizontal vectoring uses AVX2 instructions to hash one nonce faster.

Only CPUs with AVX2 will support 4-way.

Which algos can benefit?

This is the bad news. 4-way will help compute bound algos that use SPH functions the most,
the same algos that already have very efficient GPU miners available, or even ASICS.
They include the entire x11 family as well as more recent algos like hsr and polytimos.

Typical CPU algos will see little benefit as they don't usually rely on SPH or it is only a small
part of the algo.

Hashing 4 nonces in parallel creates the possbility of submitting multiple nonces per scan,
though I haven't seen this in my testing. It will likely be rare but is supported.

What overhead can reduce the gain?

As previously mentioned AVX2 code runs at a reduced CPU clock rate but there is also coding overhead
and workarounds that will affect some algos more than others.

Algos that are pure SPH chains will improve the most. The only overhead is to inlerleave the input data
then deinterleave the vectorized hash.

Some SPH functions work on 32 bit data for their 256 bit versions while other work on 64 bit data for both 512
bit and 256 bit versions. If an algo has a mix of 32 bit and 64 bit data extra interleaving/deinterleaving will be
required. This is not expected to have a significant impact.

Algos that use HW acceleration (AES or SHA) have additional challenges to implement vertical vectoring
and it may be very difficult or possible to implement vertical vectoring on these functions. These
functions will have to be run 4 times serially completely nullifying the benefits of 4-way for those
functions. Oher parts of such algos can use 4-way with intermediate interleaving/deinterleaving.

Algos that already use horizontal vectoring when rewritten for vertical vectoring are not likely to see
any further improvement.

What's the implementation plan?

Around 20 SPH functions need to be rewritten for 4-way but algos only become supported when the entire
function chain is optimized. The shorter chains will be implemented first and the longer ones last. There
is some flexibility in some instances and I can be convinced to modify my initial plan with some motivation.

The pure SPH algos will be done first as they receive the biggest gains. Algos with existing SIMD code or
HW acceleration will be done next with the existing optimizations run serially.

Next are algos that have a portion use SPH. Since SPH is only used for part of the algo the gains will be
limited.

Then I will try to rewrite the functions that use AES_NI to see if they can be improved with 4way.
SHA looks too daunting as it means rewriting openssl.

I may eventually look at rewriting the exiting SIMD optimized functions to work with 4-way. The gains
will be limited due to losing the existing optimizations. It's a matter of hashing 4 nonces in parallel using
vertical vectoring vs hashing 4 nonces serially using horizontal vectoring.

What about the future?

All of the functions that will support 4-way with AVX2 can easilly be extended to 8-way with AVX-512.

I tried a mingw cross compile a few weks ago but had problems. I gave up easily so it must have been serious,
possibly the fact that I have an old build env and was missing some packages or package versions.

However, I'm all in with the 4-way feature and it's going to be a massive amount of work. Keccak is only one of
around 20 functions I need to rewrite. And the keccac algo only needs the keccak hash function. Imagine x17
or xevan.

I will revisit the mingw cross compile but probably not until I upgrade my build system, probably buy a new PC with Ryzen.
In the meantime, if you're up for it, you could manage the Windows binaries. You can build them and host them yourself
or I could put your builds up on my gdrive. I'll certainly use them on my Windows PC.

Neoscrypt has some ASM options but I haven't dug into it. It would be nice to fix it but it's generally a poor performer
so it's not a priority for me.

Endian issues with hodl, I've seen this movie before. I had to write a custom one for Windows.

Can you show me the hodl error? I'd like to see it as it might help find a solution, if there is one.
If it's an Alpine only issue it may be hard to select.

I have a release ready to go, I can include a fix if one is quickly found. It will include the new docker file
regardless.

Got it to compile, you can try filoozom/cpuminer-opt:alpine.
It's quite a bit smaller but also a little bit slower. Also needs a fix in `algo/hodl/sha512_avx.c`.

More info: https://github.com/JayDDee/cpuminer-opt/pull/41#issuecomment-345828950
Help would be appreciated!

thats a VERY nice improvement!
Can't wait...

By the way, please consider releasing cross-compiled binaries for windows, they give much better performance that mingw on windows.
I have described the process earlier in the post, but sadly I have could not make neoscrypt work in that build (just crashes every time), probably something in the compile flags or algo settings.
If something is missing in what I posted and you want full instructions for the cross build, just say )

A little teaser of what I'm working on. It's only the beginning.

All you have to do is add "-t 4" at the end of your .bat file and it will use 4 threads. If you leave it blank it will use all your CPU has. You can specify anything from 1 to 8 threads this way.

*If you have an 8 core CPU with hyperthreading (or whatever AMD calls it these days) then you actually have 16 threads (AMD Ryzen 7 series for example).

how to config multi cpu thead. I have 8 cpu core but i want miner 4 core.