Guess I'll move my miners over to Bitminter until this is sorted out. Hope I didn't lose 1/4 of a day here.
No need.
Guess the cat is out of the bag sooner than I had hoped. Anyway... Abridged version: In the confusion surrounding last night's emergency I made a mistake that allowed the reward system to overpay some miners.
No miners have lost any income due to this and it is business as usual.
Details: The database issue happened right around the time we mined block #277343. Since the CPPSRB code couldn't read the database at the time, it wasn't able to count the payouts in that block's coinbase transaction. CPPSRB went into fail-safe mode pending my intervention.
I was automatically notified very soon after of the database issue. I saved state and shutdown CPPSRB while it was in fail-safe mode without paying attention to the fact that I hadn't allowed it to tally this one block's earnings yet.
Then I proceeded to correct the hardware issue, etc, etc........ and I completely forgot CPPSRB was in fail-safe when I saved state and shut it down. Without thinking when I started it back up, I ignored its cries about fail-safe and allowed it to load its state and continue...... without it tallying the fail-safe'd block.
I actually realized shortly after that I had made this error. No harm done yet. I went to shutdown CPPSRB to correct it and the moment I did we found block 277361..... containing a set of payouts remarkably similar to those in block 277343. Some people had then officially been double paid.
After the last issue I had with a double payment months and months ago (resulting in a final roughly 2 BTC loss that I covered at the time), I had coded in some handling of negative balances, allowing miners to "work off" an overpayment. However, I didn't think about how the stats code would react, since they assume a balance can never be negative. This seems to manifest on the stats as a messed up maximum reward graph and never having any balance stick until the over-payment is covered.
Unfortunately, the Bitcoin community has not been known for its honest users, overall. So, my apologies in having to withhold this information, and I hope the reasoning is understood. I'm sure if I had immediately posted something about this that most people who were double paid would have took off with the over-payment never to return. It is sad, but true. Fortunately the majority of affected miners continued to mine as usual and a good portion of this error has corrected itself.
In any case, I know there are going to be a bunch of people who now will do exactly as I initially feared and just flee with whatever part of their over-payment they have, but I kindly ask that anyone affected please have some respect, realize that I am a fallible human being that makes mistakes, consider any mistaken payment as a 0% interest payday loan, and simply continue to mine as normal so that the overage is covered because any remaining overage will end up coming out of donated funds as I don't have a huge pile of income from pool fees to fall back on for something like this.
I sincerely apologize for the issue and confusion caused by it. Regardless of how much of these funds are unrecoverable, I won't allow any miners to be adversely affected by this mistake and will ensure that it is covered. (As of this writing about 80% of the over-payment has been repaid through mining so far... I expect that to increase further when more shelved shares are paid.)
I have also put in code that will yell at me in the future if I try to dump CPPSRB's state while it is in fail-safe mode to prevent this particular oversight from ever happening again.
if (!safe_mode) {
dump_full_state();
} else {
printf("************************************************************************************\n");
printf("*** Refusing to dump state while in safe mode. Sorry, lost enough coins this way! *\n");
printf("************************************************************************************\n");
}
Long story short.... business as usual. Pool is up and running, shares are tallied, blocks are being found, everything is working as expected. For the few miners affected by this, please accept my apologies for the confusion and I hope you will do the right thing and continue to mine.
Thanks for mining.
-wk
Edit: Clarification: Miners/share log shares were credited for block 277343 as normal. Also, I will correct the stats to show block 277343 soon.