Pages:
Author

Topic: [ATTN: POOL OPERATORS] PoolServerJ - scalable java mining pool backend - page 12. (Read 31168 times)

sr. member
Activity: 266
Merit: 254
legendary
Activity: 2618
Merit: 1007
Wow, sounds great so far! Smiley

What about Namecoin support? Then you could even quite fast test if your "winning solution relay" works (15 times faster than Bitcoin...).
sr. member
Activity: 266
Merit: 254
Although I don't care much about java, this seems to have some very unique features Smiley.

What about the stales, do you have some data about that?

Java's not so much of a feature, just a fact.  Although it does give the benefit of a single binary for all platforms.

I haven't done any solid testing on stales but it delivers work back to longpoll requests as fast as it can get them in from the work sources after a block change.  Using multiple daemons should speed this up.  The only instances of stales I saw were in testing when I was bringing the server up and down constantly so the server was losing it's work-worker map and the miner kept humming along with old work.  After I implemented the work-worker map dump to disk on shutdown I haven't seen one.
sr. member
Activity: 266
Merit: 254
Quote
No Derivative Works — You may not alter, transform, or build upon this work.
Might be a bit hindering... a lot of pools have rewritten parts of pushpoold for example.

As it says on the download page I plan to fully open source it in the near future, as soon as the larger project is released.

Quote
Some ideas I had for pool backends:
  • Shares evaluation (stale/valid/"winning") done by backend, not bitcoind and only shares that are >=current_difficutly actually get relayed to bitcoind
  • "metapool"/"inceptpool" mode that requests getworks from other pools like a miner and relays them
  • certain share counts of users as event trigger

backend currently does evaluate share status, until a real live instance has found a winning block I've got it set to force all shares upstream just in case but that is just a switch and when it's proved it's evaluating a winning solution I will turn it off.

Event triggers are easy to implement, the question is the notification mechanism.  Currently it's just a http request to a URL of your choice but there's got to be a less ugly way.  Open to suggestions.

Re: metapool mode, actually it can already do that.  I just haven't released the doco for it yet.  Coming soon...

full member
Activity: 228
Merit: 100
Wow,

i definitely will test it.
Although I don't care much about java, this seems to have some very unique features Smiley.

What about the stales, do you have some data about that?


regards, talpan
legendary
Activity: 2618
Merit: 1007
Quote
No Derivative Works — You may not alter, transform, or build upon this work.
Might be a bit hindering... a lot of pools have rewritten parts of pushpoold for example.

Some ideas I had for pool backends:
  • Shares evaluation (stale/valid/"winning") done by backend, not bitcoind and only shares that are >=current_difficutly actually get relayed to bitcoind
  • "metapool"/"inceptpool" mode that requests getworks from other pools like a miner and relays them
  • certain share counts of users as event trigger
sr. member
Activity: 266
Merit: 254
Pushpoold Vs PoolServerJ performance test results now published

As a necessary component to a larger project I've had to write a complete pool server in java.  The rest of the project is still in progress but the pool server is ready to release into beta.  The key problem I needed to solve and part of the reason I rolled my own rather than just using pushpoold is scalability.  Pushpoold is great at what it does but the difference I needed was to get around the bitcoind bottleneck.  PoolServerJ solves that problem rather neatly by caching work and also allowing the pool operator to run as many bitcoin daemons as they want to and feed them all into the poolserver.

A few of the features of PoolServerJ:

    * Native merged mining support.  Will work with standard bitcoind 0.5
    * SCrypt blockchain support (e.g litecoin)
    * Customisable coinbase transaction (pay block rewards to any wallet perhaps an offline secure wallet)
    * Internal work generation (more than 10x faster than rpc getwork)
    * Work caching - caches work from bitcoind or internal WorkMaker to handle short term spikes in requests
    * Database resilience.  Server can keep running if database goes down and shares will be sent to database when connection is reestablished
    * DoS resilience with QoS support to ensure workers who've submitted valid work are serviced with priority
    * Capable of pulling work from multiple bitcoin daemons to get around the getwork bottleneck and also to provide some redundancy
    * Notify of block change via HTTP to a user configurable URL to support event triggered share processing
    * Cached database handling (optional) to reduce round trips.
    * Supports Mysql, Postgresql, sqlite3 (JDBC based so others can be added easily - currently only tested on mysql)
    * Runs as a Windows service
    * Longpolling support
    * Integrated block monitoring using all available bitcoin daemon work sources
    * Dump shares to Database, log file or stdout
    * Safe restart - shares issued to client are dumped to file on shutdown and reloaded on startup so your miners won't get stale shares if you restart a live server
    * Only dependency is a Java 6+ JDK.

Performance

Extensive performance testing against pushpoold has shown PoolServerJ capable of more than 4 times the capacity for share submits and 3 times the capacity for getwork requests.  This is the limit of what could tested on limited available hardware however it's architecture allows for significant scaling and theoretically these limits could be significantly exceeded.

update: Since the release of 0.4.0 WorkMaker Edition getwork capacity has improved by 4 times and share submit capacity by 2 times over the original performance test results.

Please give it a go, feel free to send me any feedback, rip into it if you like.

You can find it here

The webserver may be a bit slow, I'm testing out EC2 micros to see how they handle basic web server duties so sorry about that.  Will have a test pool up and running soon (as soon as I finish the front end). <-- update: this never happened and probably won't... I'm too busy working on the software and there's plenty of pools using it now so I've got enough guinea pigs Wink
Pages:
Jump to: