Pages:
Author

Topic: Wolf's XMR/BCN/DSH CPUMiner - 2x speed compared to LucasJones' - NEW 06/20/2014 - page 17. (Read 547140 times)

sr. member
Activity: 378
Merit: 250
Curl 7.31.1 compiled with :
./configure --disable-shared --enable-static --prefix=/usr/local --disable-ldap --disable-sspi
make -j 4;make install;

miner still errors out:
./autogen.sh
./configure CFLAGS="-static'
checking for the version of libcurl... 7.37.1
checking for libcurl >= version 7.15.2... yes
checking whether libcurl is usable... no
configure: error: Missing required libcurl >= 7.15.2

Any ideas ?

./configure is looking at curl prefix in /usr (depending on the distro). You might want configure curl like:
./configure --disable-shared --enable-static --prefix=/tmp/curl --disable-ldap --disable-sspi
make ; make install

then the miner:
./autogen.sh
./configure CFLAGS="-static' --with-libcurl=/tmp/curl
make

legendary
Activity: 1092
Merit: 1000
Curl 7.31.1 compiled with :
./configure --disable-shared --enable-static --prefix=/usr/local --disable-ldap --disable-sspi
make -j 4;make install;

miner still errors out:
./autogen.sh
./configure CFLAGS="-static'
checking for the version of libcurl... 7.37.1
checking for libcurl >= version 7.15.2... yes
checking whether libcurl is usable... no
configure: error: Missing required libcurl >= 7.15.2

Any ideas ?
legendary
Activity: 1092
Merit: 1000
Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

I know it is, is there any other way to get it to run on centos ? If i compile on ubuntu and run on centos it complains about glibc version (2.14 required, centos runs 2.12). Any other way ? Is it possible to include glibc in the static binary ?

EDIT : Removing MADV_HUGEPAGE from cpu-miner.c fixes the problem but with a significant loss in speed. I need the extra speed!!

the static binary can include everything.
did you try:

./configure --enable-static

or

CFLAGS='-static'

?

Yes i have, i can't get past curl requirements even though i compiled curl from source with static flags.. I am not sure glibc is included in the static bin, can anyone confirm, dont want to waste time troubleshooting curl
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

I know it is, is there any other way to get it to run on centos ? If i compile on ubuntu and run on centos it complains about glibc version (2.14 required, centos runs 2.12). Any other way ? Is it possible to include glibc in the static binary ?

EDIT : Removing MADV_HUGEPAGE from cpu-miner.c fixes the problem but with a significant loss in speed. I need the extra speed!!

the static binary can include everything.
did you try:

./configure --enable-static

or

CFLAGS='-static'

?
legendary
Activity: 1092
Merit: 1000
Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??

You can't, because CentOS is garbage. I had to add in a flag to disable some optimizations for that shit OS.

I know it is, is there any other way to get it to run on centos ? If i compile on ubuntu and run on centos it complains about glibc version (2.14 required, centos runs 2.12). Any other way ? Is it possible to include glibc in the static binary ?

EDIT : Removing MADV_HUGEPAGE from cpu-miner.c fixes the problem but with a significant loss in speed. I need the extra speed!!
sr. member
Activity: 378
Merit: 250

really, because 10,000/528 = ~19.  19 is a "few machines?"

edit: and that diff is pathetically low.  444 is for a pentium 2 or someshit.

Yes, 19 is a "few machines". There are plenty more to use across different pools / coins.
Difficulty is assigned by the network and will scale up and down. 444 is the starting difficulty for that pool and its gotten considerably higher now.

Lighten up, you sound like you're having a bad day Wink
legendary
Activity: 1092
Merit: 1000
Looking for instructions on how to enable MADV_HUGEPAGE on Centos 6.5 , anyone ??
hero member
Activity: 644
Merit: 502
Why does it automatically set thread count to ~cpu count?

Using your formula:

[2014-08-03 13:19:33] accepted: 12/12 (100.00%), 514.93 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:34] accepted: 13/13 (100.00%), 513.02 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 14/14 (100.00%), 528.15 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 15/15 (100.00%), 525.17 H/s at diff 444 (yay!!!)

Perhaps this should be added to the original post? Its a BIG improvement.


To answer my own question, it appears as though the improvement is noticed only when the server has a large number of cpus / cache etc.
Older machines seem to perform better with a larger amount of threads, even when cpu count is lower.

I'm breaking > 10 000 H/s without even breaking a sweat across a few machines.


Thank you

really, because 10,000/528 = ~19.  19 is a "few machines?"

edit: and that diff is pathetically low.  444 is for a pentium 2 or someshit.
hero member
Activity: 644
Merit: 502
8 Core Xeon gets 370 hash/sec all day, all night with 8 threads in ubuntu 14.04 w/ hugepages=24.
hero member
Activity: 969
Merit: 1000
trying to compile for win 64 bit
with this instruction
https://bitcointalksearch.org/topic/m.6542760
tried many different ./configure
everytime it success compile but the compiled binary crash while trying to run also i tried to strip the exe too  Undecided
sr. member
Activity: 378
Merit: 250
Why does it automatically set thread count to ~cpu count?

Using your formula:

[2014-08-03 13:19:33] accepted: 12/12 (100.00%), 514.93 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:34] accepted: 13/13 (100.00%), 513.02 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 14/14 (100.00%), 528.15 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 15/15 (100.00%), 525.17 H/s at diff 444 (yay!!!)

Perhaps this should be added to the original post? Its a BIG improvement.


To answer my own question, it appears as though the improvement is noticed only when the server has a large number of cpus / cache etc.
Older machines seem to perform better with a larger amount of threads, even when cpu count is lower.

I'm breaking > 10 000 H/s without even breaking a sweat across a few machines.
Thank you
member
Activity: 81
Merit: 1002
It was only the wind.
Any updated miners ? The GPU now seems to have a massive advantage over CPUs now on monero

GPUs don't have a massive advantage yet - and when they do, there will be nothing I can do about it, seeing as GPUs are just better at this sort of thing.
sr. member
Activity: 378
Merit: 250
Don't set threads equal to number of CPU threads, that is wrong.
The general rule is threads = floor(L3 cache / 1024)
Use this: num_threads=$(($(cat /proc/cpuinfo | grep "cache size" | uniq | cut -d":" -f2 | cut -d" " -f2)/1024))
Should be 12 for X5650

You sir, are a legend. Why does it automatically set thread count to ~cpu count?

Using your formula:

[2014-08-03 13:19:33] accepted: 12/12 (100.00%), 514.93 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:34] accepted: 13/13 (100.00%), 513.02 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 14/14 (100.00%), 528.15 H/s at diff 444 (yay!!!)
[2014-08-03 13:19:35] accepted: 15/15 (100.00%), 525.17 H/s at diff 444 (yay!!!)

Perhaps this should be added to the original post? Its a BIG improvement.

Thanks! (Now to deploy this on a motherload of machines :/)
hero member
Activity: 616
Merit: 500
You should be getting atleast 400 H/s out of those CPUs (mine are hashing at 451 H/s)

How though? I've tried on a variety of *very expensive* hardware from Dell.
Each has 100gigs of ram, 24 cpus, etc..

Ubuntu 14.04:

# sysctl -w vm.nr_hugepages=72  (confirmed that it is set)
# git clone https://github.com/wolf9466/cpuminer-multi.git
# cd cpuminer-multi
# ./autogen.sh
# ./configure CFLAGS="-march=native"
# make
# make install
# screen -d -m -S minerd /usr/local/bin/minerd -o stratum+tcp://mro.pool.minergate.com:45560 -p x -u x
# screen -r (after a few hours of running)
[2014-08-03 06:05:25] accepted: 3952/7916 (49.92%), 288.94 H/s at diff 444 (yay!!!)

example of hardware: 24 CPUs - Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz
another example:       24 CPUs - Intel(R) Xeon(R) CPU X5650  @ 2.67GHz





Don't set threads equal to number of CPU threads, that is wrong.
The general rule is threads = floor(L3 cache / 1024)
Use this: num_threads=$(($(cat /proc/cpuinfo | grep "cache size" | uniq | cut -d":" -f2 | cut -d" " -f2)/1024))
Should be 12 for X5650
sr. member
Activity: 378
Merit: 250
You should be getting atleast 400 H/s out of those CPUs (mine are hashing at 451 H/s)

How though? I've tried on a variety of *very expensive* hardware from Dell.
Each has 100gigs of ram, 24 cpus, etc..

Ubuntu 14.04:

# sysctl -w vm.nr_hugepages=72  (confirmed that it is set)
# git clone https://github.com/wolf9466/cpuminer-multi.git
# cd cpuminer-multi
# ./autogen.sh
# ./configure CFLAGS="-march=native"
# make
# make install
# screen -d -m -S minerd /usr/local/bin/minerd -o stratum+tcp://mro.pool.minergate.com:45560 -p x -u x
# screen -r (after a few hours of running)
[2014-08-03 06:05:25] accepted: 3952/7916 (49.92%), 288.94 H/s at diff 444 (yay!!!)

example of hardware: 24 CPUs - Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz
another example:       24 CPUs - Intel(R) Xeon(R) CPU X5650  @ 2.67GHz



member
Activity: 81
Merit: 1002
It was only the wind.
Hello all,

Has anyone successfully compiled Wolfs cpuminer under centOS 6.4?

I tried for a couple hours yesterday to make it work, and even removed some of the values under the makefile but can't seem to get it to compile.

I'm stuck on this error..

./cpu-miner.c:1065: error: ‘MADV_HUGEPAGE’ undeclared (first use in this function)
./cpu-miner.c:1065: error: (Each undeclared identifier is reported only once
./cpu-miner.c:1065: error: for each function it appears in.)

Any help would be greatly appreciated!

CentOS apparently doesn't support hugepages - which sucks, because that's a lot of performance.
hero member
Activity: 979
Merit: 510
Each optimization I tested worked on my CPU - I don't have all CPUs, so I can't test. That much of a reduction is odd, though... are you sure the test wasn't affected by other factors?

It's quite a difference. I have 8 of those monster servers that are freshly installed (+updated) with Ubuntu 14.04.
They are all identical and unused (no running services, crons etc..) but the results are rather inconsistent across each so perhaps something else could be influencing things.
I'll dig a bit deeper and see if anything comes of it.

Going to check this out as well, running it without the optimizations.


Without optimizations:

[2014-08-03 00:58:53] accepted: 4/4 (100.00%), 195.91 H/s at diff 5000 (yay!!!)
[2014-08-03 00:59:04] accepted: 5/5 (100.00%), 195.29 H/s at diff 5000 (yay!!!)
[2014-08-03 00:59:07] accepted: 6/6 (100.00%), 194.45 H/s at diff 5000 (yay!!!)
[2014-08-03 00:59:08] accepted: 7/7 (100.00%), 194.69 H/s at diff 5000 (yay!!!)


With:
[2014-08-03 01:00:28] accepted: 3/3 (100.00%), 369.24 H/s at diff 5000 (yay!!!)
[2014-08-03 01:00:36] accepted: 4/4 (100.00%), 369.24 H/s at diff 5000 (yay!!!)
[2014-08-03 01:00:36] accepted: 5/5 (100.00%), 369.23 H/s at diff 5000 (yay!!!)
[2014-08-03 01:00:52] accepted: 6/6 (100.00%), 369.14 H/s at diff 5000 (yay!!!)

Crazy how efficient those optimizations are.
Though sadly needs to be around 440+ or so for current difficulty to break even.
hero member
Activity: 616
Merit: 500
Btw.. Just thought i'd mention this:

Latest CPUMiner compiled from git: (24 x Intel(R) Xeon(R) CPU X5650 @ 2.67GHz)

Over a 20 minute duration:

[2014-08-02 20:25:25] accepted: 191/234 (81.62%), 230.74 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:29] accepted: 192/235 (81.70%), 230.63 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:29] accepted: 193/236 (81.78%), 230.61 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:31] accepted: 194/237 (81.86%), 230.16 H/s at diff 980 (yay!!!)
[2014-08-02 20:25:40] accepted: 195/238 (81.93%), 231.01 H/s at diff 980 (yay!!!)

Then i removed all of the "optimizations" from the Makefile:

From:

am__append_3 = -Ofast -flto -fuse-linker-plugin -funroll-loops -fvariable-expansion-in-unroller -ftree-loop-if-convert-stores -fmerg
e-all-constants -fbranch-target-load-optimize2 -fsched2-use-superblocks -maes

to :

am__append_3 = -maes

The result?

[2014-08-02 20:26:44] accepted: 189/231 (81.82%), 289.83 H/s at diff 1384 (yay!!!)
[2014-08-02 20:26:45] accepted: 190/232 (81.90%), 290.09 H/s at diff 1384 (yay!!!)
[2014-08-02 20:27:03] accepted: 191/233 (81.97%), 291.41 H/s at diff 1384 (yay!!!)
[2014-08-02 20:27:07] accepted: 192/234 (82.05%), 290.47 H/s at diff 1384 (yay!!!)

There are numerous posts/warnings on-line about over-aggressive compile-time "optimizations".
Just thought I'd post my results/findings which concur.

Regards,
Cami

You should be getting atleast 400 H/s out of those CPUs (mine are hashing at 451 H/s)
member
Activity: 81
Merit: 1002
It was only the wind.
hero member
Activity: 979
Merit: 510
Each optimization I tested worked on my CPU - I don't have all CPUs, so I can't test. That much of a reduction is odd, though... are you sure the test wasn't affected by other factors?

It's quite a difference. I have 8 of those monster servers that are freshly installed (+updated) with Ubuntu 14.04.
They are all identical and unused (no running services, crons etc..) but the results are rather inconsistent across each so perhaps something else could be influencing things.
I'll dig a bit deeper and see if anything comes of it.

Going to check this out as well, running it without the optimizations.
Pages:
Jump to: