Topic: [ANN]: cpuminer-opt v3.8.8.1, open source optimized multi-algo CPU miner - page 176. (Read 444067 times)

That seems very worthwhile to do if it gets windows Working. If is requires extra work for devs to maintain a master list of
function declarations, so be it. I'll have to find a way to make it work.


I think there is. Even with "-march=core2 -DNO_AES_NI" the __AES__ var is still defined. It is based
on the architecture of the CPU, not the compiled app.

Thanks again for your contributions and I look forward to what you have planned next.

Thanks for the detailed explanation, it'll take a while to digest it. In the meantime you can take
a look at the latest release, I implemented some of your suggestions (forgot to credit you, need to fix that)
in v3.1.14. I do the initial __AES__ check in miner.h and you can see a good example of how I use it in
cpu-miner.c:cpu_capabilities_check.

One thing I immediately infered from your response is I need to use LTO to get Windows working
(and I sense you are working on that, if so thanks) and I need to clean up boost and implied function
declarations to get LTO working. That is plenty justification to do it. I look forward to seeing that cake
when it's fully baked.

This is link-time optimization. Same way what Microsoft does under the "whole program optimization" name.

This lets the compiler see how each function is used by other functions and optimize more aggressively since it knows about all functions, this can mean that the optimizer can skip some work done since it's not needed by the program as a whole.

__attribute__((used)) is needed to fix compilation errors when using LTO wih gcc -- gcc is late to the party when adding LTO/whole program optimization, so there are still some kinks to be ironed out -- the LTO optimizer tends to disregard pure-assembler functions, so they need to be marked as used by the program, otherwise they'll be thrown away at one of the passes before the linking happens. Same applies to static variables defined in headers.


I did remove the need for boost in hodlcoin after I posted that comment, I did not intend the changes to be visible by you (yet) -- The proper diff URLs are:
 * https://github.com/hmage/cpuminer-opt/compare/75db804...d01b290 -- UI
 * https://github.com/hmage/cpuminer-opt/compare/75db804...d01b290.diff -- raw diff


There's no need for that, you can just put -march=core2 instead of -march=native.



Again, you're seeing LTO problems mentioned above, for the reasons mentioned above. The proper diff URL builds cleanly on core2 and haswell.

That's part of changes to get rid of boost.

In C you have to tell the compiler what the function declaration looks like before you use the function, what happens next is undefined behaviour and thankfully on intel linux with gcc this doesn't break anything.

You have to put a function prototype in a header -- the dev needs to add a mention of the function in 3 places -- the header, the body, and the place that calls it. Since this doesn't happen often, I don't think it's a big deal.

Strange though that you didn't see any kind of these warnings before, probably MS Visual C defaults to not show this specific warning (and because MS has to worry only about making developer's sourcecode work only Intel Windows anyway).

There's no way to do what you want portably that works on both MSVC and GCC without requiring a function prototype in a header. Linux kernel relies on Linux-specific assembler code (.init) to do their module system without forcing everyone to list all function prototypes in one place.

cpuminer-opt v3.1.14 released.

https://drive.google.com/file/d/0B0lVSGQYLJIZaE5DYXA4SHl2WVk/view?usp=sharing

New in v3.1.14

Algos
     - cryptonight algo is now supported on CPUs without AES_NI.
       All algos now support both CPU architectures.
     - jane added as an alias for scryptjane with default N-factor 16

Build enhancements, see details in README.md (thanks to hmage)
     - build.sh now works for CPUs with and without AES_NI
     - it is no longer necessary to add -DNO_AES_NI CFLAG to
       configure command when building for CPUs without AES_NI.

     Note: Compiling requires some additional libraries not included
     in the default instalation of most Linux distributions: libboost-dev,
     libboost-system-dev, libboost-thread-dev.

UI enhancements
     - enhanced checks for CPU architecture, SW build and algo for
       AES_NI and SSE2 capabilities.
     - a warning is displayed if mining an untested algo.

Code cleanup
     - removed a few more compiler warnings
     - removed some dead code

Algo gate enhancements (for devs)
     - replaced algo specific null gate functions with generic null
       functions

I have a small issue with compiler warnings for using undefined functions. It haas to do with algo-gate.
The goal of algo gate was to make cpu-miner completely algo agnostic, meaning there is no algo
specific code anywhere in cpu-miner.c. That goal was achieved.

At this point there is only one long switch/case statement to register the selected algo with the gate
so cpu-miner can invoke its functions. The only base code that nees to be modified is the comand
argument definitions to support the algo.

The only drawback is a large number of compiler warnings, one while trying to register each algo.
That is because the algo's registration function is in its own source file whicj is not included by anyone.
The solution seems to be to make a cenrtal list of the gate registration functions of every algo.
That would add work to developpers wheh adding a new algo.

If anyone has a win-win solution, ie no warnings and no extra work, It would be welcomed. 

I really appreciate the work you did. I've been stumbling my way around the environment just trying to make
things work. It's been challenging to apply my experience to a completey foreign environment. Feel free to
comment on overall architecture. Being a foreigner in the land of GNU C I'm not aware of the local customs
and conventions.

I've done a first pass through your changes and I want to undertsand more so forgive my stupid questions.
I'm curious about some of the changes because things worked. Amd when things work I don't want to make
changes unless it makes them work better.

In general terms I ask why is it better, will it hash faster? Simpler compilation procedure is an obvious one
but other changes escape me.

I didn't know about the compiler support for __AES__. Good job.

I never heard of LTO before, what will it give me? Is that how boost now works with your changes?

Boost didn't work fo me until I added the linking options to teh makefile. Are your changes leading edge?
Will they work on older loads?

Can you explain the __attribute__ used stuff a bit more, that's a new concept for me. Again, it worked
before so how is this better?

Thanks again. I'll get the __AES__ implemented quickly, the other stuff can wait until I have a better understanding.

Update:

I've been spending a lot of time trying to implement another algo with little success. I get a nice multi page
dump for a buffer overflow, not just the typical segfault, but I digress. I put that aside for another day so I can
your user visible changes released.

I implemented __AES__ with a slight change, I allow configure to override automatic aes compile, not very useful
for users but it helps testing SSE2 on my i7.

There are still problems with build.sh. First groestl sse2 complained about ASM constraints so I removed USE_ASM
then a bunch of AES code failed to compile, then I stripped it down to "-O3 -march=native" and it works.

It's nice to get rid of the alloca warnings.

I didn't see a point in changing the way boost was linked, it doesn't make compiling any easier and it ain't broke.

I'll look into the other issues over the longer term.

Thanks again.

It'l be in the next release, hopefully in a day or so. It'l be handy for users not to have to know the architecture.

Exactly that, works for both non-AES CPU's and AES CPU's. GCC provides __AES__ macro if -march=native supports it.

Did not test on MinGW, though, only on Debian 8.4.

I presume you mean you made build.sh work out of the box?
That's great. I'll dig into it. If you left it -Ofast I'll change it to -O3,
that's what I always use anyway.

I had the same experience: Ofast may be slower than O2, but usually O3 is faster instead.

I've added some changes to make it work out of the box on non-AES linux builds:

Diff can be found here -- https://github.com/hmage/cpuminer-opt/compare/upstream...master
Raw diff -- https://github.com/hmage/cpuminer-opt/compare/upstream...master.diff

• On non-MSVC, if -march=native supports AES, then NO_AES_NI is not defined, otherwise it's defined.
• An LTO build did not work out of the box, some variables and functions needed to be marked as used, so they won't be removed by linker.
• Add -march=native into build.sh
• Fix a compiler warning, since builtin_alloca() is already defined as alloca() on GCC.
• Curiously, on my benchmarks, -O2 is faster than -Ofast on gcc 4.9.2.

For blakecoin you could use this pool: http://ny2.blakecoin.com/index.php?page=login

Thanks but scryptjane has been pool tested. When I said remaining algos I was referring to the ones that have
not yet been tested in a live pool. Specifically they are X17, blakecoin, fresh, cryptolight, and bastion. I have not been
able to find any pools offering to mine coins using these algos.

Sorry for the confusion.

You can test scryptjane at YACoin pool:

Miner config:

You can use 2 ports:
- 3433 difficulty 1
- 3434 difficulty 0.25

If you miner doesn't support difficulty less than 1 - use port 3433

cpuminer-opt v3.1.13 just released. Added support for a few dusty algos:
sha256d, scrypt, scryptjane and yescrypt.

https://drive.google.com/file/d/0B0lVSGQYLJIZRVJieTZsV2JTekU/view?usp=sharing

This just about completes the conversion from cpuminer-multi. Some numbers:

37 algos
31 algos pool tested
14 algos optimized with AES_NI support
6 algos benchmark tested only, need pool testing

See all the details in the first post of this thread.

If anyone  knows of anywhere to pool test any of the remaining algos please let me know.
I'll even test you your address.

There's nothing pressing in the pipeline so this will probably be the last release for a while,
barring bug fixes. I hope you all enjoy it.

Edit: cpuminer-opt can now mine all algos at hashpower, zpool and yiimp, all but whirlpoolx at nicehash,
all but hashimoto (ethereum), spreadx11 and m7m at suprnova, and all but scryptn at miningpoolhub.

Problem solved!

I decided to create a fresh VM using Ubuntu 14.04 LTS 64Bit.

After installation and running all updates i installed:

libboost-dev
libboost-system-dev
libboost-thread-dev
libssl-dev
libcurl4-openssl-dev
install g++

all had to be installed in order to compile successfully.

Next time i'll test Ubuntu 15.10.

Thanks for the help!

/usr/include/boost/config.hpp is missing. It could be a missing boost package or a problem with your system.
The three packages above are the only ones I manually had to install, I don't recall which dependecies
were installed with them. I don't know specifically which package cointains config.hpp. I suggest you poke around,
try installing other boost packages, it's gotta be there somewhere.

Hi,

ok i checked the libs.
I do have the 64 bit versions installed.
When i check the folder /usr/lib/x86_64-linux-gnu/ then i can see many files beginning with liboost.
I then tried and put this: LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/.

Nothing worked so far.

Here the error when compiling version 3.1.12:
 /usr/include/alloca.h:35:0: note: this is the location of the previous definition
 # define alloca(size) __builtin_alloca (size)
 ^
In file included from algo/hodl/hodl.cpp:9:0:
./boost/unordered_map.hpp:12:28: fatal error: boost/config.hpp: No such file or directory
compilation terminated.
Makefile:2996: recipe for target 'algo/hodl/cpuminer-hodl.o' failed
make[2]: *** [algo/hodl/cpuminer-hodl.o] Error 1
make[2]: Leaving directory '/home/moork/Desktop/cpuminer-opt-3.1.12'
Makefile:3157: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/moork/Desktop/cpuminer-opt-3.1.12'
Makefile:629: recipe for target 'all' failed
make: *** [all] Error 2

it cant access boost/unordered_map.hpp and boost/config.hpp

Hope we can progress from here.

I also replied to your pm. From the release announcement:

     Note: Compiling requires some additional libraries not included
     in the default instalation of most Linux distributions: libboost-dev,
     libboost-system-dev, libboost-thread-dev.

Hi!

there seems to be a PATH problem when compiling cpuminer-opt version 3.1.10 as well as 3.1.12 for the hodl algo.
This is what i am getting:


/usr/include/alloca.h:35:0: note: this is the location of the previous definition
 # define alloca(size) __builtin_alloca (size)
 ^
algo/hodl/hodl.cpp:9:29: fatal error: unordered_map.hpp: No such file or directory
compilation terminated.
Makefile:2841: recipe for target 'algo/hodl/cpuminer-hodl.o' failed
make[2]: *** [algo/hodl/cpuminer-hodl.o] Error 1
make[2]: *** Waiting for unfinished jobs....
mv -f algo/groestl/aes_ni/.deps/cpuminer-hash-groestl.Tpo algo/groestl/aes_ni/.deps/cpuminer-hash-groestl.Po
mv -f algo/.deps/cpuminer-c11.Tpo algo/.deps/cpuminer-c11.Po
mv -f algo/haval/.deps/cpuminer-haval.Tpo algo/haval/.deps/cpuminer-haval.Po
make[2]: Leaving directory '/home/moork/Desktop/cpuminer-opt-3.1.10'
Makefile:3002: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/moork/Desktop/cpuminer-opt-3.1.10'
Makefile:612: recipe for target 'all' failed
make: *** [all] Error 2


I have a relative fresh install of Ubuntu 15.10 x64. Can someone please advise where to place the files "unordered.hpp" and "thread.hpp" so that they can be found ?
Their current location is inside the cpuminer folder under algo/hodl/boost.
The hodl.cpp contains:
#include
#include

Help is appreciated!