Topic: [EXPERIMENTAL] poclbm-autohop: Yet Another Automatic Pool Hopper - page 3. (Read 10144 times)

The main thing deciding which pools I support is the pool support in pident. I could add them myself, but it seems like there are new ones popping up every day, so keeping track and writing the parsers might be more than I want to take on. Practically, I probably wouldn't bother adding a pool that's average block solve time is longer than a week or so.

Before adding any pool to the actual miner, I have to review it and determine (as best I can) the reward system used and its expected utility. (For instance, I don't currently support Slush in the miner because I haven't yet figured out a way to calculate its utility. I haven't tried very hard yet, either, but it may not even be worth the effort.) So, reviewing the ones you mentioned:

• PolMine: I can't actually in a quick search verify their reward system. If I can figure that out, I'm not opposed to adding it, but I probably wouldn't sign up there myself.
• MainframeMC: Claims to use a cheat-proof scoring system. I haven't investigated further. At best, it'd be a good backup pool. Not opposed to adding it, if I can confirm the reward system.
• bitcoinpool: Last time I checked, they use a hopping deterrent (halving the value of shares if you mine less than 50% of the round). This should still be hoppable (but less so), but I'd have to come up with a good utility function. Since rounds shorter than an hour aren't subject to this, it's not quite as cut and dry as halving the utility of a normal prop pool (but that could be a good starting point). By the way, the hop point in the "halve the rewards" scenario is ~10%.
• bloodys: Not currently in pident, I don't think.

Also, my long-term rejected shares seem to be somewhere in the neighborhood of 4-5%. This is definitely more than mining a single pool with poclbm, but presumably the hopping bonus makes up for it. It's still worth working on, however. For me, most of them seem to happen right after switching pools, so something's probably going wrong there.

I'm also checking my efficiency on BTC Guild to see if the new hopping method is giving positive results, but it'll be several days before there's enough data to even consider. Successfully defeating the one hour delay will be necessary, otherwise more pool operators may consider it as a solution, rather than changing to a non-broken reward system.

I noticed PolMine, MainframeMC and bitcoinpool are in your data-feed, but not in the pools.py code.  Are those not ready for prime-time or were they omitted for a reason? I've had good luck with PolMine.

Another one to consider adding is pool.bloodys.com, though they are fairly small. Is there a lower limit on the GH/s that you consider reasonable?

Okay, thanks for the explanation! I'm very interested in any pool using pident, and if you need help to hack it (like, where/how are the scores calculated ), ask me on IRC or by email

The only reason it isn't released anywhere is because it's not up to my own standards for sharing (it has a bit of site-specific code in it, and isn't very well organized, because I changed how I implemented it halfway through). If I have a chance, I'll try to refactor it a bit and fork the pident repository on github. However, even if I do, it probably wouldn't be too mergeable. I tried to work entirely on top of pident, rather than modifying it, for two reasons:

• First, this made it easier to merge any changes you made.
• Second, I gave up trying to figure out exactly where in the code the scoring was happened. I think I figured it out since then, but as you'll see below, I'm not sure the work I've done is directly applicable to improving the scores themselves.

Regardless, I can at the very least attempt to describe my current thought process and algorithm.

The first instinct is to just take whichever pool has the highest score and assume they found the block. As you've said, however, the accuracy of that method isn't incredibly high. For the purposes of pool hopping, we don't want to guess which pool found the block. Rather, we want to take all the available information, and assign probabilities to each pool. Using these probabilities, we can calculate the utility of mining at each pool.

At first, I just used each pool's percentage of the overall network, which is accurate, but I figured it should be possible to do better with additional information, namely pident's scoring. The trick is figuring out how to massage the scores pident produces into probabilities. To that end, I sampled an equal number of blocks from each pool's set, producing a block universe where each pool is equally likely to find a block. pident's scoring works based on identifying unique characteristics of each pool's blocks. It (to my knowledge) has no idea that some pools are simply more likely to find blocks, so modeling on this sampled block universe makes sense to me.

Treating the scores, then, as relative confidences that each pool found the given block, we rescale them so they sum to 1, so they at least have a chance of representing probabilities. Once I did this, and plotted the likelihood that each range of scores meant a pool did find the block, there was a definite pattern. (In the particular data set I looked at, a rescaled score of 0.06-0.07 implied a ~13.5% chance that the pool found that block.) By the time we reach scores of 0.1, the probability is almost 100%. Ultimately, I modeled the data using the atan function (which matches the curve). (Does this make theoretical sense? Who knows, but as long as it works practically, I'll go with it.) The idea here is to scale the scores so that a score of 0.5 implies a 50% probability. I eyeballed the function, and it's not very forum friendly, but it matches the data fairly closely.

The end result of all this massaging of the original scores produced by pident is a rough probability that the block belongs to each pool, but only in the magical "every pool is equally likely" universe. To get back to the real world, we multiply each score by the pool's percentage of the overall hash rate. The last step is rescale these final values so they again sum to 1. That's the final probability I use in calculating the data in the API.

The next step is to test these numbers and make sure they at least come close to matching the ultimate results. I'll probably wait a few days so I have some out-of-sample data before I embark on that quest, however.

Other next steps:

• Getting the code postable. I have no desire to keep it that close to my vest.
• Deciding on the best way to implement the hopping strategy implied by the API. poclbm is kind of a pain. I contemplated modifying cgminer, but that's quite a chore. If recent improvements to bitHopper have made it more usable (e.g. unbroken LP support, and not spamming pools with high-frequency getwork requests), it might be best to implement it there. I'd have to compare its average reject percentage with poclbm.

Hello. I see you are using pident for choosing where to hop. I feel obliged to remind you that accuracy is fairly low ( http://pident.artefact2.com/accuracy ).

But hey, you said you tweaked it so it gathers more data… If you managed to improve the accuracy, could you care to share the source so that I can merge it? I know the WTFPL doesn't require you to do that, but it would be nice. If your public domain miner relies on a closed-source web API, then it's of no use

Maybe the pool disabled N time rolling. and I don't really know why "nonces_left += 0xFFFFFFFFFFFF" is required, so I removed these 2 parts of code. one getwork, one response expected. that's the normal behaviour in phoenix miner.

Add: I still using a heavily modded poclbm/2011.b7. No plan to catch up to the new version ...

I have seen that, and I have no idea why it happens. poclbm is such a mess in general, that it's hard to do any real sort of debugging. When it's happened to me, I've noticed that I can make it go away by running a CPU-bound process in the background (like a CPU miner). I also haven't the foggiest idea why that would help, but without knowing the reason for the rejected shares, it's hard to debug.

I utilize poclbm's built-in multi-server support to handle the multiple servers, and I don't think it handles switching very well, sometimes leading to additional rejects. However, fixing poclbm itself isn't very fun for me.

Also, I just noticed your message about hash rates. I haven't personally noticed too much of a drop, but my Windows miner has never done very well at staying at peak, even using stock poclbm. Linux seems fine with both, in any case.

Thanks for the update.  How are your stale shares?  In the Windows version that I built the program usually gets into a mode where it continuously produces stale shares after running for a few minutes.

I have modified the block probability API to at least partially score the blocks to determine the probability that each pool produced it, rather than simply using each pool's estimated hash rate. However, both factors are considered.

This should increase overall efficiency by some unknown degree, but since I have no easy way to measure such a thing (other than creating new accounts on all the pools and mining using only this miner), we may never know for sure. The most likely result of this change is that BTC Guild and bitcoins.lc should see increased time as the pool with highest utility.

Since this change is on my backend, no upgrade is necessary. Clients will automatically pick up the new data.

you're just arguing semantics

"it does reward every share equally" != "because every share has the same chance of being inside and outside the window"

That's like saying every lottery ticket is a winner because every lottery ticket has a chance to win

or

Every candidate is hired because every candidate has a chance to be hired

or

Everyone is a millionaire because everyone has a chance to be a millionaire

it does reward every share equally because every share has the same chance of being inside and outside the window

most *PPS variants are vulnerable to BLOCK OF DEATH problem where a block or several long blocks could delay or otherwise affect the payouts for new members which discourages new people from joining the pool since the pool has to pay the old members back for their work despite being in the hole

PPLNS doesn't suffer from this problem, because when a block is long, the older members just don't get paid - they didn't get lucky
this means when a new member joins he has the same chance as anyone else to get paid and get paid promptly/in full

Finally got it running on Windows 7.  I like it.  However, my hash rate is not as high as poclbm.exe Windows native executable.  My hash rate dropped to 305 from 316 MH/s.  So probably will not use your auto-hopper unless you can get the hash rate a little higher.

You've only submitted 10 shares, so the averages haven't really had time to stabilize. Here's a brief explanation of the various fields. Some are in stock poclbm, and some are from luke-jr's version.

The second section is your estimated hash rate based on accepted shares in some recent time period. Over time, it should be roughly equal to the first field, unless you're getting a lot of rejected shares. Eff stands for efficiency and refers to your recent ratio of shares per getworks. On some pools this will be low, and on others it may be 5 or greater.

The next section tells you the number of rejected and total shares you've had. (In this case, you've submitted 10 shares, all accepted.)

The final section tells you the number of getworks the client has issued and the overall efficiency. Better efficiency is primarily better for the pool, rather than for the miner, but it doesn't hurt, either.

Can you explain what the different fields mean on the last line of the display?
Am I only getting an effective rate of 294 MHash/sec?

So when does hopping get its own section under mining?

This is getting interesting, may the most efficient hopper win

Yes, it's working now, thanks

*SMPPS is broken/high risk of not returning 100% of expected value per share as soon as the pool hits negative, which should happen ~50% of the time.

PPLNS can choose arbitrarily large Ns (the highest useful N is 5 000 000 000, awarding each share with 1 Satoshi when blocks are found) if people really feel left out or are not able to get that they have a chance to get paid multiple times, equalizing the chance of not getting paid at all.
All that is needed is a proper interface, that shows percentages instead of "expected payout", as PPLNS looks forward to define rewards, not so much backwards as prop.


That said, as soon as pool operators start delaying LongPolls they really start to hurt their own profit + their own miners. Just think of the fact, that every millisecond in delay lets the miners compute 100 000 hashes per 100 MH/s in vain and will create more stale shares.
Also, if you just have to submit a valid solution every 10 minutes or so to count as an "active miner", one might be able to just pull a getwork from there every 10 minutes, solve + submit it, to not get throttled. The shorter this window, the more miners the operators hit + screw over, the longer, the easier it gets to fake a CPU miner.

The Geometric method refers to Meni's scoring system.