Pooled/Remote Mining - Open Source - Updated 2010-12-24 - page 2.

doublec

legendary

Activity: 1078

Merit: 1005

Quote from: puddinpop on December 13, 2010, 09:09:47 PM

Where was the trouble when you attached the debugger? I tried to duplicate this here, but I don't have enough hash rate locally to overload the server. It sounds like a network issue to me, but it's possible it could be something else. Are you sure you have enough upload bandwidth to the server? Is your host throttling it?

I saw nothing obvious in the debugger. It still seemed to be stepping through the clients ok. Is it possible for a slow or malicious client to cause the server to block or slow down?

I should have enough bandwidth. I have a similar server that serves a lot more traffic than this one running the pooled miner. That said my host has been killing the server for excessive CPU usage recently so anything is possible.

Is anyone else willing to run/take over running the pool that might have a more reliable server or connection?

puddinpop

member

Activity: 103

Merit: 17

Quote from: doublec on December 12, 2010, 11:49:02 PM

The server got stuck idling and producing zero hashes again. I've temporarily fallen back to the previous server, version 1.2.1, until the problem is resolved. Note that 1.2.1 is averaging about 60% the CPU usage of 1.2.2.

Where was the trouble when you attached the debugger? I tried to duplicate this here, but I don't have enough hash rate locally to overload the server. It sounds like a network issue to me, but it's possible it could be something else. Are you sure you have enough upload bandwidth to the server? Is your host throttling it?

kaspar

newbie

Activity: 2

Merit: 0

Some idea just came to my head:

We could write a "wrapper"-client to the pooling-server which revealsthe standard rpc-API to other clients. This would enable us to use every BC mining client out there for remote-mining. Or do I miss some obstacle here?

kaspar

newbie

Activity: 2

Merit: 0

What's the best way to finetune the opencl implementation?

If I run DiabloMiner with -w 256, I get about ~250mhash/s, which seems to be maximum, as the Catalyt Control Center displays almost 100% GPU Usage. However, if I use the OpenCL remoteminer, I peak at 100mhash/s. (-aggression 12, everything else above 12 crashes the miner)
I tried to play with the -gputhreads and -gpugrid parameters, but could not achieve any better results.

Do you have any ideas to get a similiar performance from opencl-remoteminer?

EDIT: Also the remote-opencl-client consumes a lot more cpu power (33% on 4x2.4GHz). Same is valid for the CUDA client, which runs on a different video card. Why do those versions consume such a high amount of CPU power? In contrast: DiabloMiner's CPU load is at 2%.

Current system configuration:

OS: Win 7 Pro x64
CPU: Q6600 (4x2.4GHz)
Video #1: Radeon HD 5850
Video #2: Gefore 9600 GT

doublec

legendary

Activity: 1078

Merit: 1005

The server got stuck idling and producing zero hashes again. I've temporarily fallen back to the previous server, version 1.2.1, until the problem is resolved. Note that 1.2.1 is averaging about 60% the CPU usage of 1.2.2.

doublec

legendary

Activity: 1078

Merit: 1005

puddinpop, some timestats.txt for the last few hours:

Code:

section	count	time
BitcoinMinerRemote Handling Messages	3538540	6125980632
BitcoinMinerRemoteServer::AddDistributionFromContributed	7444	12161303
BitcoinMinerRemoteServer::BlockToJson	7444	37761410
BitcoinMinerRemoteServer::DecodeBase64	1137964	7965160
BitcoinMinerRemoteServer::EncodeBase64	14888	110513
BitcoinMinerRemoteServer::GetAllClientsCalculatedKHashFromBest	990	12888072
BitcoinMinerRemoteServer::GetAllClientsCalculatedKHashFromMeta	990	5102865
BitcoinMinerRemoteServer::GetOldestNonVerifiedMetaHashClient	1684875	13319578
BitcoinMinerRemoteServer::SendWork	7444	218351595
BitcoinMinerRemoteServer::Step	3538541	1986327641
MetaHashVerifier::Start	398	112139
MetaHashVerifier::Step	78488	1280946445
RemoteClientConnection::GetCalculatedKHashRateFromBestHash	104395	12802790
RemoteClientConnection::GetCalculatedKHashRateFromMetaHash	104395	5005396
RemoteClientConnection::GetNewestSentWorkWithMetaHash	1606785	8341113
RemoteClientConnection::GetSentWorkByID	1137964	3100622
RemoteClientConnection::MessageReady	327210958	585449940
RemoteClientConnection::ProtocolError	327210958	575656559
RemoteClientConnection::ReceiveMessage	1140525	300611249
RemoteClientConnection::SendMessage	20680	17677026
RemoteClientConnection::SetWorkVerified	384	1151
RemoteClientConnection::SocketReceive	753356	11741667
RemoteClientConnection::SocketSend	19110	1143503950
VerifyBestHash	1001002	6195863

doublec

legendary

Activity: 1078

Merit: 1005

The server got stuck in an idle state again. It was using 0% CPU and not processing requests. I'm building in debug mode now and I'll attach with a debugger if it happens again.

adv

full member

Activity: 168

Merit: 100

When i try build only remote miner, i write in CMakeLists.txt:

Quote

OPTION(BITCOIN_ENABLE_CUDA "Enable CUDA miner" OFF)
OPTION(BITCOIN_ENABLE_OPENCL "Enable OpenCL miner" OFF)
OPTION(BITCOIN_ENABLE_REMOTE_SERVER "Enable remote miner server" OFF)
OPTION(BITCOIN_BUILD_GUI "Build GUI (bitcoin)" OFF)
OPTION(BITCOIN_BUILD_DAEMON "Build Daemon (bitcoind)" OFF)
OPTION(BITCOIN_BUILD_REMOTE_MINER "Build remote miner (bitcoinr)" ON)

And get error:

Quote

make
[ 11%] Building CXX object cmake-bitcoinr/CMakeFiles/bitcoinr.dir/__/src/remoteminermain.cpp.o
In file included from /home/adv/bitcoin-remote-20101211-src/src/remote/remotebitcoinheaders.h:35,
from /home/adv/bitcoin-remote-20101211-src/src/remoteminermain.cpp:19:
/home/adv/bitcoin-remote-20101211-src/src/remote/../serialize.h: In function ‘void Unserialize_impl(Stream&, std::vector&, int, int, const boost::true_type&)’:
/home/adv/bitcoin-remote-20101211-src/src/remote/../serialize.h:475: error: there are no arguments to ‘min’ that depend on a template parameter, so a declaration of ‘min’ must be available
/home/adv/bitcoin-remote-20101211-src/src/remote/../serialize.h:475: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
/home/adv/bitcoin-remote-20101211-src/src/remote/../serialize.h: In function ‘void Unserialize(Stream&, std::map&, int, int)’:
/home/adv/bitcoin-remote-20101211-src/src/remote/../serialize.h:588: error: ‘pair’ was not declared in this scope
..... and many cutted....

What i do wrong?

Sources from bitcoin-remote-20101211-src.zip
uname -a: Linux home 2.6.32-5-686 #1 SMP Thu Nov 25 18:43:34 UTC 2010 i686 GNU/Linux
g++ --version: g++ (Debian 4.4.5-8) 4.4.5

BitLex

hero member

Activity: 532

Merit: 505

nice that there seems to be an OpenCL miner available now,
sad though, that it doesnt work (for me on Nvidia at least).

Code:

...snip...
:140:3: error: call to 'rotate' is ambiguous
sharound(E,F,G,H,A,B,C,D,R(36),0x650A7354);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:47:123: note: instantiated from:
#define sharound(a,b,c,d,e,f,g,h,x,K) {t1=h+(rotateright(e,6)^rotateright(e,11)^
rotateright(e,25))+(g^(e&(f^g)))+K+x; t2=(rotateright(a,2)^rotateright(a,13)^rot
ateright(a,22))+((a&b)|(c&(a|b))); d+=t1; h=t1+t2;}

^
:45:30: note: instantiated from:
#define rotateright(x,bits) (rotate(x,32-bits))
^~~~~~

:2967:24: note: candidate function
ulong __OVERLOADABLE__ rotate(ulong, ulong);
^

:2966:23: note: candidate function
long __OVERLOADABLE__ rotate(long, long);
^

:2965:23: note: candidate function
uint __OVERLOADABLE__ rotate(uint, uint);
^

:2964:22: note: candidate function
int __OVERLOADABLE__ rotate(int, int);
^

:2963:25: note: candidate function
ushort __OVERLOADABLE__ rotate(ushort, ushort);
^

:2962:24: note: candidate function
short __OVERLOADABLE__ rotate(short, short);
^

:2961:24: note: candidate function
uchar __OVERLOADABLE__ rotate(uchar, uchar);
^

:2960:23: note: candidate function
char __OVERLOADABLE__ rotate(char, char);
^
:141:3: error: call to 'rotate' is ambiguous
sharound(D,E,F,G,H,A,B,C,R(37),0x766A0ABB);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:47:46: note: instantiated from:
#define sharound(a,b,c,d,e,f,g,h,x,K) {t1=h+(rotateright(e,6)^rotateright(e,11)^
rotateright(e,25))+(g^(e&(f^g)))+K+x; t2=(rotateright(a,2)^rotateright(a,13)^rot
ateright(a,22))+((a&b)|(c&(a|b))); d+=t1; h=t1+t2;}
^
:45:30: note: instantiated from:
#define rotateright(x,bits) (rotate(x,32-bits))
^~~~~~

:2967:24: note: candidate function
ulong __OVERLOADABLE__ rotate(ulong, ulong);
^

:2966:23: note: candidate function
long __OVERLOADABLE__ rotate(long, long);
^

:2965:23: note: candidate function
uint __OVERLOADABLE__ rotate(uint, uint);
^

:2964:22: note: candidate function
int __OVERLOADABLE__ rotate(int, int);
^

:2963:25: note: candidate function
ushort __OVERLOADABLE__ rotate(ushort, ushort);
^

:2962:24: note: candidate function
short __OVERLOADABLE__ rotate(short, short);
^

:2961:24: note: candidate function
uchar __OVERLOADABLE__ rotate(uchar, uchar);
^

:2960:23: note: candidate function
char __OVERLOADABLE__ rotate(char, char);
^
:141:3: error: call to 'rotate' is ambiguous
sharound(D,E,F,G,H,A,B,C,R(37),0x766A0ABB);

...snip...

the CUDA client doesnt show its best hashes anymore,
loads the GPU a bit more (50%->65%) what makes it a bit faster hashing (gtx260: 18M->25M),
but it also loads the CPU more than before (50%->75%).

doublec

legendary

Activity: 1078

Merit: 1005

Something odd happened with the server. I noticed it was using 0 CPU as if no clients were connected. I connected a miner and I just got this repeated:

Code:

Requesting a new block 1292156535151
Requesting a new block 1292156540258
Requesting a new block 1292156545364

So it was constantly 'requesting a new block'. Any thoughts on what might cause this? I restarted the server and the miner connected and worked fine.

Azetab

newbie

Activity: 19

Merit: 0

is the OpenCL or CUDA client better for a nVIDIA GPU?

doublec

legendary

Activity: 1078

Merit: 1005

Is it possible that the new server code uses more CPU? Prior to the update I was averaging 60%, now it's 95%. It's hard to tell if it's because of more clients suddenly joining or the update (about 10 new clients joined). Maybe they're OpenCL clients and the server is just busier. It'd be great if the server could have stats about number of OpenCL, Cuda and CPU clients are connected.

puddinpop

member

Activity: 103

Merit: 17

Quote from: jgarzik on December 11, 2010, 08:22:46 PM

What is the remote protocol, so that other miners may participate?

A few bytes of header info and a json object. The source is the best resource to look at if you need specific info.

Quote from: doublec on December 11, 2010, 09:29:23 PM

Quote from: puddinpop on December 11, 2010, 04:38:38 PM

The first post has been updated with a new release.

cmake-bitcoinr/CMakeLists.txt seems to be missing a line for remoteminerthreadcpu.cpp. There's a link error unless I add this.

Yeah, and the GPU sources need remoteminerclientgpu.cpp changed to remoteminerthreadgpu.cpp.

Quote from: doublec on December 11, 2010, 09:36:15 PM

In the debug.log I get lines like this a lot (tens of thousands of lines) from some IP addresses:

Code:

Client x.x.x.x sent metahash for block we don't know about!

What causes this? Note that this is with the 20101206 release.

The clients I posted before my last post had a bug that would cause that, so if someone is using those clients you'll see that. It could also happen if you just restarted the server, and clients are still hashing blocks from before the restart. It should go away after a few minutes.

doublec

legendary

Activity: 1078

Merit: 1005

In the debug.log I get lines like this a lot (tens of thousands of lines) from some IP addresses:

Code:

Client x.x.x.x sent metahash for block we don't know about!

What causes this? Note that this is with the 20101206 release.

doublec

legendary

Activity: 1078

Merit: 1005

Quote from: puddinpop on December 11, 2010, 04:38:38 PM

The first post has been updated with a new release.

cmake-bitcoinr/CMakeLists.txt seems to be missing a line for remoteminerthreadcpu.cpp. There's a link error unless I add this.

doublec

legendary

Activity: 1078

Merit: 1005

Quote from: puddinpop on December 11, 2010, 04:38:38 PM

The server will now send the contribution type to the clients, and the clients will display it upon connecting to the server. The server will also save timing stats to a text file periodically so that we can narrow down what the bottlenecks are. If you are running a server, please post your timing stats after the server has been up for a while.

I'll update the server I run today and post some stats when it's been running for a while.

kwukduck

legendary

Activity: 1937

Merit: 1001

I added the line to the remotebitcoinmineropencl.cl file

Code:

build LOG:C:\Users\Administrator\AppData\Local\Temp\OCLD5E9.tmp.cl(20): error: can't
enable
   all OpenCL extensions or unrecognized OpenCL extension
  #pragma OPENCL EXTENSION cl_khr_byte_addressable_store : enable
   ^

the rest of the log is the same, still resulting in 0 khash/s

jgarzik

legendary

Activity: 1596

Merit: 1100

What is the remote protocol, so that other miners may participate?

puddinpop

member

Activity: 103

Merit: 17

kwukduck

legendary

Activity: 1937

Merit: 1001

The OpenCL client seems to run but it reports back 0 khash/s

i tried changing -gpu- value, didn't change anything

this is what it tells me:

Code:

Client will start 1 miner threads
Attempting to connect to 173.255.205.10:8335
Connected to 173.255.205.10:8335
1 OpenCL platforms found
OpenCL finding best kernel configuration
1 OpenCL GPU devices found on platform 0
Setting OpenCL device to first device found
Create context rval=0
Create command queue rval=0
Creating program with source
Building program
Got message 2 from server.
Server version 1.2.1
Build program rval=-11
build STATUS:■
build LOG:C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(19): warning: type

qualifier specified more than once
__constant const uint f1=0xFF000000;
^

C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(20): warning: type
qualifier specified more than once
__constant const uint f2=0x00FF0000;
^

C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(21): warning: type
qualifier specified more than once
__constant const uint f3=0x0000FF00;
^

C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(22): warning: type
qualifier specified more than once
__constant const uint f4=0x000000FF;
^

C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(23): warning: type
qualifier specified more than once
__constant const uint f5=0xFFFFFFFF;
^

C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(74): warning: unrecognized
#pragma
#pragma unroll 1
^

C:\Users\Administrator\AppData\Local\Temp\OCLCADE.tmp.cl(269): error: write to < 32
bits via pointer not allowed unless cl_khr_byte_addressable_store is
enabled
metahash[(myid*loops)+it]=((uchar *)&A)[0];
^

1 error detected in the compilation of "C:\Users\Administrator\AppData\Local\Temp\OCL
CADE.tmp.cl".
Dcl_khr_icd=1 -Dcl_amd_fp64=1 -Dcl_kh
Create kernel rval=-46
Done allocating OpenCL resources for (16,16)
Finding best configuration step end (16,16) 5ms prev best=9223372036854775807
Done allocating OpenCL resources for (16,32)
Finding best configuration step end (16,32) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (16,64)
Finding best configuration step end (16,64) 2ms prev best=9223372036854775807
Got message 4 from server.
Done allocating OpenCL resources for (16,128)
Finding best configuration step end (Address 1MVUHYbcuuR5Dx6D9A6BzxsE9wtHqJRChx
not found in block being solved
16,128) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (16,256)
Finding best configuration step end (16,256) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (32,16)
Finding best configuration step end (32,16) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (32,32)
Finding best configuration step end (32,32) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (32,64)
Finding best configuration step end (32,64) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (32,128)
Finding best configuration step end (32,128) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (32,256)
Finding best configuration step end (32,256) 3ms prev best=9223372036854775807
Done allocating OpenCL resources for (64,16)
Finding best configuration step end (64,16) 1ms prev best=9223372036854775807
Done allocating OpenCL resources for (64,32)
Finding best configuration step end (64,32) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (64,64)
Finding best configuration step end (64,64) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (64,128)
Finding best configuration step end (64,128) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (64,256)
Finding best configuration step end (64,256) 3ms prev best=9223372036854775807
Done allocating OpenCL resources for (128,16)
Finding best configuration step end (128,16) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (128,32)
Finding best configuration step end (128,32) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (128,64)
Finding best configuration step end (128,64) 2ms prev best=9223372036854775807
Done allocating OpenCL resources for (128,128)
Finding best configuration step end (128,128) 3ms prev best=9223372036854775807

Done allocating OpenCL resources for (128,256)
Finding best configuration step end (128,256) 4ms prev best=9223372036854775807

Done allocating OpenCL resources for (16,16)
Got message 10 from server.
Server Status : 113 clients, 208372 khash/s
1 blocks generated since 2010-12-09 23:03:20 UTC
Server reports my khash/s as 0

Topic: Pooled/Remote Mining - Open Source - Updated 2010-12-24 - page 2. (Read 59088 times)