Pages:
Author

Topic: CKPOOL - Open source pool/proxy/passthrough/redirector/library in c for Linux - page 14. (Read 124153 times)

legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
As I already implied, the software is full of 64bit maths and that will produce a noticeable difference, as will having double the registers in general code.

... and you will find that 5 year old link quite outdated now ... comments about not everyone having 64bit processors and "fewer issues" with 32bit are now wrong.
It's the other way around, fewer issues with 64bit and I'm not sure you can even find 32bit processors online any more.
If you can, you've been given old dodgy hardware.

Now the most relevant point ... if you want to run a pool, and you are trying to skimp on hardware, be well aware that you may lose blocks.
You need a bitcoind, and running one on a small vps is a problem waiting to happen.
full member
Activity: 179
Merit: 131
I'm really not sure how a 32bit OS can be more efficient on 64bit hardware ...

Edit: ... and I'll add that mining requires 64bit math quite a bit ... since 4billion really isn't a big enough number Smiley
So all that code would be slower.

I am not a programmer and I know very little about 32bit vs 64bit programs. However, some years ago when I started to rent VPS for my private web site, I did some readings and testing myself. I have made the conclusion that Linux i386 is more efficient in using the RAM than Linux amd64. Somebody did more thorough tests than me and posted the results on this web page, if you wanted to know that.
full member
Activity: 179
Merit: 131
Hi Con,

I don't believe that you clearly explained the requirement to use Linux amd64 for your CKPool. The only error I saw from the previous posts is the "unknown type name sa_family_t" from /usr/include/linux/un.h.

Could you please explain in detail which parts on your CKPool require Linux amd64?

The main reason I asked is that, for small VPS it is more efficient - in term of the RAM usage - to use Linux i386 instead of amd64.  So it would be helpful to know the impact of compiling and running your CKPool on Linux i386.

Cheers,

Anto
The answer is I don't care so I've not bothered to make sure the code is fine on 32bit architectures and I don't want to care. You have your priorities wrong if you are worried about that on a pool.
I clearly didn't ask or demanded that you should make sure your code run on i386. I just asked which parts rely on amd64. So never mind then.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
I'm really not sure how a 32bit OS can be more efficient on 64bit hardware ...

Edit: ... and I'll add that mining requires 64bit math quite a bit ... since 4billion really isn't a big enough number Smiley
So all that code would be slower.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
Hi Con,

I don't believe that you clearly explained the requirement to use Linux amd64 for your CKPool. The only error I saw from the previous posts is the "unknown type name sa_family_t" from /usr/include/linux/un.h.

Could you please explain in detail which parts on your CKPool require Linux amd64?

The main reason I asked is that, for small VPS it is more efficient - in term of the RAM usage - to use Linux i386 instead of amd64.  So it would be helpful to know the impact of compiling and running your CKPool on Linux i386.

Cheers,

Anto
The answer is I don't care so I've not bothered to make sure the code is fine on 32bit architectures and I don't want to care. You have your priorities wrong if you are worried about that on a pool.
full member
Activity: 179
Merit: 131

I thought I would see if one of the small project boards would work as a proxy so I compiled it on an Odroid C1 (fedora). Compiled easily with only signed/unsigned warnings and connects just fine to kano.is but I get the following error when a miner finds a share and tries to submit it:
32bit processor? There's no 32 bit support in ckpool code.

Hi Con,

I don't believe that you clearly explained the requirement to use Linux amd64 for your CKPool. The only error I saw from the previous posts is the "unknown type name sa_family_t" from /usr/include/linux/un.h.

Could you please explain in detail which parts on your CKPool require Linux amd64?

The main reason I asked is that, for small VPS it is more efficient - in term of the RAM usage - to use Linux i386 instead of amd64.  So it would be helpful to know the impact of compiling and running your CKPool on Linux i386.

Cheers,

Anto
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
I just set up the proxy to migrate away from p2pool without having to update each miner to a new URL. There seems to be an issue if the username on the miners contains p2pool difficulty settings such as "/256+256" tacked on to the end of the username. The miners using this technique see the ckproxy as dead.
The slash is illegal in the username and pointless to use with p2pool anyway. The + should work though.
member
Activity: 98
Merit: 10
Please, it's 2015. This is 64 bit code only, but if it's working ignore the error as it's just something to do with your distribution.

Compiled on another system (32-bit too - so the bitness doesn't meter) - now works like a charm :-)

Ok... this explains the problems I have had... Whether it works on a 32 bit system depends on how you use it.


I thought I would see if one of the small project boards would work as a proxy so I compiled it on an Odroid C1 (fedora). Compiled easily with only signed/unsigned warnings and connects just fine to kano.is but I get the following error when a miner finds a share and tries to submit it:
32bit processor? There's no 32 bit support in ckpool code.

Understood - but I thought I saw a message earlier that someone had compiled it on a 32bit processor and it worked. I expected it wouldn't compile if that were the issue but it does and connects to kano.is with no problem. Just can't submit any shares.

I'll drop the experiment. Thought a simple, cheap, power efficient proxy would be good to run vs having it on one of my servers. It would be good to have a redundant, backup proxy.

I managed to compile and run using debian jessie  and
Code:
./configure --without-ckdb
with the intention of running it on an old laptop as a proxy and hoping to get some nice logs.

Did not succeed in getting the "-p" proxy to run

It would almost work but the following would appear in the logs and eventually the hashrate on the pool ( kano.is ) would drop off and it would fail to submit shares.

Code:
Invalid json line:1 col:4 pos:4 text: NULL string argument from stratifier.c submit_share:2965
Attempted to send null message to generator in send_proc

The passthrough proxy "-P" does work well in 32 bit. The hashrates add up and each worker shows up on the pool. I have used it for several weeks to combine the connections from miners on different machines.

Code:
 Failed to set json object from connector.c connector_stats:782
 Invalid json line:1 col:5 pos:5 text: NULL string argument from connector.c connector_stats:784
 Failed to set json object from connector.c connector_stats:787
 Passthrough:{"runtime": 593}

My advice to anyone ckpool/ckproxy with a 32 bit system is to experiment, test, check the logs and verify that the hashrates  correspond at all ends. Difficulty adjustments can complicate the testing.

It would be nice if everything worked without problems on 32 bit systems. Then again, I should probably ditch the old laptops. For the record, I'm running my small ckproxy on an old Compaq Evo N410, Pentium III, 1200MHz ca. 2003.  Grin
member
Activity: 110
Merit: 10
Question, can i use CKPOOL files to set up my own private pool on a raspberry pi to mine Peercoin solo with rent and home hash?
No idea - we don't support any sort of alt-coin mining at all and no point asking questions about it here.

However, any mining needs a full fledged, high performance, hacked bitcoind ... so yeah that, oddly enough, wont run on an rpi.
Yea ok np, was just thinking its a sha256 coin. But can i run a bitcoin-qt wallet on a rpi ...
I very much doubt it, but if you could, you'd be silly to mine to it coz you'd have a VERY high chance of losing blocks you mine.

Quote
However, any mining needs a full fledged, high performance, hacked bitcoind
Okie dokie i trust you kano. Thanks
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
Question, can i use CKPOOL files to set up my own private pool on a raspberry pi to mine Peercoin solo with rent and home hash?
No idea - we don't support any sort of alt-coin mining at all and no point asking questions about it here.

However, any mining needs a full fledged, high performance, hacked bitcoind ... so yeah that, oddly enough, wont run on an rpi.
Yea ok np, was just thinking its a sha256 coin. But can i run a bitcoin-qt wallet on a rpi ...
I very much doubt it, but if you could, you'd be silly to mine to it coz you'd have a VERY high chance of losing blocks you mine.

Quote
However, any mining needs a full fledged, high performance, hacked bitcoind
member
Activity: 110
Merit: 10
Question, can i use CKPOOL files to set up my own private pool on a raspberry pi to mine Peercoin solo with rent and home hash?
No idea - we don't support any sort of alt-coin mining at all and no point asking questions about it here.

However, any mining needs a full fledged, high performance, hacked bitcoind ... so yeah that, oddly enough, wont run on an rpi.
Yea ok np, was just thinking its a sha256 coin. But can i run a bitcoin-qt wallet on a rpi and mine solo on 127.0.0.1?? any way to set a stratum url for rent mining to home QT? 
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
Question, can i use CKPOOL files to set up my own private pool on a raspberry pi to mine Peercoin solo with rent and home hash?
No idea - we don't support any sort of alt-coin mining at all and no point asking questions about it here.

However, any mining needs a full fledged, high performance, hacked bitcoind ... so yeah that, oddly enough, wont run on an rpi.
member
Activity: 110
Merit: 10
Question, can i use CKPOOL files to set up my own private pool on a raspberry pi to mine Peercoin solo with rent and home hash?
member
Activity: 111
Merit: 10
how do you know if u find a block solo mining solo.ckpool, my ant says it found a block but how do i confirm this?
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4

I have the same problem building (with ckdb) on Debian :

Error: Required library gsl-dev not found. Install it or disable support with --without-ckdb

What changes are needed to enable the build ?

Thanks
I updated the README a week ago and it shows the ubuntu requirements for gsl.
You'll have to work out how to translate that to debian if debian has different package names.
ldp
newbie
Activity: 11
Merit: 0

I have the same problem building (with ckdb) on Debian :

Error: Required library gsl-dev not found. Install it or disable support with --without-ckdb

What changes are needed to enable the build ?

Thanks
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
Building a pool on Debian;

Found one issue with building on Debian 8 --- I needed to modify the configure script to get the PostGres support to configure/compile.
...
Seems the GSL library in the Debian 8 repository no longer has main() ; configure needed to be told to also link in gslcblas (which is tested afterward) in order to work.... bit of a kludge on my part to get things to configure.
Then see how -ck did it for postgresql and do something similar and submit a change for configure.ac

*however* looking at the close bindings between ckpool, ckdb and the web code is it even possible to split these onto separate machines ?

The idea is to keep everything loosely coupled to allow any component to be grown/shrunk/replaced; makes for a quick way to tear down and replace any machine that gets compromised.
They use linux sockets, so yeah not really possible with ckdb <-> ckpool with the current code.

The bitcoind needs to be REAL close since block changes need to be as fast as possible.
Block changes and sending out new work on a block change is so important on any pool, though it would seem that a lot of people can't understand that and mine on slow block change pools ... ... ... their loss.

Sooner or later (in the distant future? Smiley ) I'll write a separate (tiny) program as a standard socket that sits in front of ckdb, then it's only a matter of changing some of socket.php to talk to ckdb 'remotely' - i.e. web can be anywhere then.

Quote
Also, is any work being undertaken on the web code ? -- there are some tweaks I'd like to make but I don't want to duplicate effort.
? I don't commit often enough ??

Quote
Lastly, are you open to any tweaks being submitted as new branches / pull requests on the Bitbucket respository...

Regards,
Nick
Of course, but also you are best to have your own git fork (as I'm sure you do already) so that if you submit changes that aren't what I want, you still have your own version.

As I mentioned in the link before, asking in IRC is best regarding lots of questions.

There is a very long list of changes that I keep adding new ones to the list (that I write down any time I think of something) - for all sorts of reasons for each - some minor and some major Smiley
member
Activity: 193
Merit: 10
Hi Kano,

Building a pool on Debian;

Found one issue with building on Debian 8 --- I needed to modify the configure script to get the PostGres support to configure/compile.
Works as is (of course) on Ubuntu and Fedora - no idea what the difference would be on debian.
Seems the GSL library in the Debian 8 repository no longer has main() ; configure needed to be told to also link in gslcblas (which is tested afterward) in order to work.... bit of a kludge on my part to get things to configure.

Quote
Lastly, should ckpool (in pool mode) support connecting to an upstream pool as backup if the local bitcoind fails --- I tried using kano.is as backup but this did not work...
The pool is a pool, not a proxy.
What you are describing is setting up a proxy that points to both your pool and my pool.
As for when it fails over, well that depends on what happens to the pool.
I didn't think this one through properly... of course it needs to be another bitcoin server for the failover.


I have everything built and working on a single machine now..

The Systems Architect in me wants to break it all up into separate (virtual) machines;

I'd would like to setup:-
  • a VM for (walletless) bitcoin
  • a second (walletless) bitcoin server VM (if/when I go 'live') for failover
  • ckdb + postgres server
  • warm standby ckdb + failover postgres server
  • a ckpool server
  • a web server
  • a data backup server (literally just software and storage to backup to)

*however* looking at the close bindings between ckpool, ckdb and the web code is it even possible to split these onto separate machines ?

The idea is to keep everything loosely coupled to allow any component to be grown/shrunk/replaced; makes for a quick way to tear down and replace any machine that gets compromised.

Also, is any work being undertaken on the web code ? -- there are some tweaks I'd like to make but I don't want to duplicate effort.

Lastly, are you open to any tweaks being submitted as new branches / pull requests on the Bitbucket respository...


Regards,
Nick
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
Building a pool on Debian;

Found one issue with building on Debian 8 --- I needed to modify the configure script to get the PostGres support to configure/compile.
Works as is (of course) on Ubuntu and Fedora - no idea what the difference would be on debian.

Lastly, should ckpool (in pool mode) support connecting to an upstream pool as backup if the local bitcoind fails --- I tried using kano.is as backup but this did not work...
The pool is a pool, not a proxy.
What you are describing is setting up a proxy that points to both your pool and my pool.
As for when it fails over, well that depends on what happens to the pool.
member
Activity: 193
Merit: 10
Building a pool on Debian;

Found one issue with building on Debian 8 --- I needed to modify the configure script to get the PostGres support to configure/compile.

The VM running CKProxy crashed overnight; I'm investigating what caused this....

Would it be of use if I were to provide my notes on getting a fully functional node working for inclusion in the git repo ?

Lastly, should ckpool (in pool mode) support connecting to an upstream pool as backup if the local bitcoind fails --- I tried using kano.is as backup but this did not work...

Pages:
Jump to: