Topic: [ATTN: POOL OPERATORS] PoolServerJ - scalable java mining pool backend - page 2. (Read 31150 times)

[ANNOUNCE] Poolserverj WORKMAKER EDITION RELEASED - 0.4.0rc1

This is such a major release I thought it deserved it's own thread 

https://bitcointalksearch.org/topic/announce-poolserverj-workmaker-edition-released-040rc1-51226

Apologies to anyone experiencing stability issues with the 0.3.1 release.  I based it off a commit that further down the line that the 0.3.0 tag and it turns out it included a merge with a refactored branch.

I've redone using the exact 0.3.0.FINAL commit as the base.  The changes themselves are very minor and have been tested in production by a pool that builds from source so now that's it based of the right level stability should be identical to 0.3.0.FINAL.

I've uploaded the new pckage as 0.3.1.01 and I'll remove 0.3.1.

Following

I've just posted two new versions of psj binaries on the website.  These contain a hotfix for a potential security issue.  I have reason to believe this issue has been exploited already.

The stable pre-merged mining version has been posted as 0.3.1
The merged mining branch contains the hotfix in .11 mini-binary.

There are two parts to this fix, one was fixed in the merged mining branch some time ago and I thought it was in main branch but was mistaken.  The was just discovered a couple of hours ago.

I'm not going to explain the issue in detail until pool ops have had a chance to update.  If you want to know please PM me or ask on IRC in private chat.  So long as I know you're a pool op that's running psj I'll give you the rundown.

I will post full detail here in a few days.  I have not committed these changes to the public repository.  I will do this in a few days also.  If anyone has concerns about this I'm happy to send you the new source code privately if you don't want to wait.

I've added a new 'Guide to setting up poolserverj in Eclipse 3.7' page to documentation section of the website.  If you want to fork or make mods for your own pool this should get you going pretty quickly with a decent build environment.

http://poolserverj.org/documentation/guide-to-setting-up-poolserverj-in-eclipse-3-7/

Thank you!  I have a much better idea of what is going on now.  It seems that early teething problems with merged mining have created quite a few Bitcoin blocks with a stale Namecoin element and these blocks spawned a rumour about their likelyhood to which I fell victim.

By the way, thanks for the work on PoolServerJ.  I'm not a pool operator myself but I've been donating.

Just had another look at the cgminer thread, looks like conman made a patch a couple of days ago.  I'm not sure if it's been released as a binary yet and even if so it'll probably be a while before the majority of cgminer user upgrade but at least now if pools get complaints to about the issue they can point to a solution that doesn't force miners to abandon their mining client of choice.

In fact the total (valid + invalid) shares of btc and nmc rounds are the same if a "double block" has been found before and we are still hashing for new blocks. BTC share count on the website is 600 ahead because these are the valid shares. I will clarify on the pool website that these numbers are total shares - invalid shares.

The reason why nmc invalid shares are currently higher is, that cgminer don't care about longpoll on new nmc network blocks as far as i understood.

A share that solves a block of difficulty n will always solve for all chains where difficulty is < n so your intuition is right so long as bitcoin difficulty remains higher.

The only instance where this might not occur would be where the miner is working on a work that is valid for only only the bitcoin block.  This is how it can happen:

a namecoin block is found but difficulty was below bitcoin so no bitcoin block is found.  All work previously issued is now no longer valid for namecoin but remains valid for bitcoin.  So if a bitcoin block is found very soon after before miners have refreshed work then it may be stale for namecoin but valid for bitcoin.

Under normal circumstance this should be very rare.  After a block is found LP should take less than a second or two to get the miners working on fresh work that is valid for both chains.  So you'd have to have to find a namecoin block then a bitcoin block (from a different getwork) within 1-2 seconds.

In practice this can actually happen more often.  cgminer ignores longpoll unless the prev_block_hash has changed.  This will only change when you find a bitcoin block.  So it will ignore longpolls that come straight after a namecoin-only block is found and continue working with old work.  This can go on for anywhere up to a minute before it refreshes it's work.  On average though call it 30 seconds.  So if namecoin block is found then within 30 seconds a bitcoin/namecoind block is found by a cgminer then you might see a bitcoin only block on a mm pool.

I think if you take a look at the yourbtc.net site, you'll see that the shares were not the same as the bitcoin share for this round are about 600 ahead of the namecoin round. I do find it quite interesting that we seem to find these blocks so close together though.

Hi,

Just a simple question about when a Bitcoin block and Namecoin block are found simultaneously.

My original intuition was that a hash for a merged mining header which Generated a Bitcoin block would also be able to generate a Namecoin block due because the difficulty of Namecoin was lower than that of Bitcoin.

I was later told that this was in fact false.  While merged mining it is quite possible to find a Bitcoin block without a Namecoin block and that dual Bitcoin-Namecoin blocks, while possible, were relatively rare.

However, since urstroyer set up PoolServerJ for yourbtc.net all of the Bitcoin blocks (4 now) have come at the same time as Namecoin blocks.

Can anyone explain this?  Does PoolServerJ somehow do something clever with the header to guarantee that Bitcoin blocks always come with Namecoin blocks?  Is this just coincidence?

Yep no one tops Shadders! He is a kick ass developer.

I'm not quite sure what shads did, but here is proof that psj is by far the best pool server.



For the quick math impaired, that a total of 129,601 shares submitted with 2 stales. Unbelievable!! 

I am donating all of my "world's best rig builder" winnings to psj.

PSJ takes work from the upstream and makes sure it's not duplicate work to put in it's cache right?

So, if that work expires and new work is requested would that not mean it wasted valid work that a miner could have found POW?  I understand that transactions could be occurring frequently and could be added into new work for a solution but I would think this would be harder on the miners to find a solution.

Just a guess.

Turns out this bug doesn't actually exist *sigh of relief*... the pool in question was just having a looooong streak of bad luck.

if you mean the pre mm versions (0.3.0.FINAL) then no.  If the problem even exists it's only in mm edition and probably only in the last couple of versions.

would this bug affect normal poolserverj?

Urstroyer I came across something last night that seemed a bit odd and Davinci's suggestion that your cache size might be too high could be on the money.

Somehow, I don't know when, the code that checks work age in cache has dissappeared.  When retrieving work from the cache it's supposed get retrieving and discarding until it finds one that's no expired.  That wasn't happening.  So this will only happen if your work cache size is quite large but you could have been passing out work that was so old the namecoin network was rejecting it.  It's just a guess.

In any case I've made a fix for the problem and released .10 on the website now.

As a backup could you do an sql dump of your solutions this round and send to me via email.   I'll run a script over them and check if any meet difficulty for either chain which should tell us if there's a problem detecting them or not.

The only other tip I can give you is that I have noticed luck improves if you have the right work cache and timeout  as it will take work that never gets done.

We already talked about it, couldn't find any unusual. I seems to be just bad luck or variance. PSJ is running smooth like always!