Topic: bitHopper: Python Pool Hopper Proxy - page 26. (Read 355813 times)

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.



nothing to see here... move along. Let me guess you didnt want to read a 200 page thread? LOL.
Yeah a lot of pools still hoppable, some even like us, just wish we would stay longer. Some do use anti-hopping techniques. If you do use the proxy, you will want to keep up to date with your pools. They change, some add anti hop techniques, some change payout systems, some break, a few are buggy. The best anti hopping technique is a fair reward system. The rest have various levels of effectiveness.

I have been mining on a PPS pool, but this proxy has gained my interest.

What pools are still hopable?  Obviously proportional.  But don't some use anti-hoping techniques?  

Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

Actually, the manual correction is probably not only different for every miner out there based on their location, but also depends on how many other workers are currently connected to the various pools. It's probably a very unreliable method to determine a block finder.

Maybe it's possible to hook into bitcoind and see in which order connected nodes send the new block information. There might be patterns, but I suspect they are not very constant and need to be re-learned often. The learning would probably take a few hours at the begin of every session to recognize patterns of deepbit, BTC guild, and other pools, if there is any difference.

atm it is worth mining deepbit and yes you can have various pools with mine_deepbit role at the same time, this role is very sensible regarding you geographic location to pools so that's the reason it doesn't work with a 100% accuracy


@deepceleron nice analysis man, I'm with this method (normalizing LP responses from pools) and in the future some automated learning would be awesome. I think manual correction you have posted are really different for every bH miner out there.

edit: dunno if it's a viable idea... could't we rely on a pident service to automatically have bH learn how to make it's LP adjustments ?

I haven't been keeping up with this thread lately so I figured i'd just post and ask.

Is it worth using mine_deepbit with deepbit and/or BTCGuild?
At the moment it sounds like it's completely random and would be better off being disabled until it's been further refined.

Also can you use mine_deepbit with deepbit and btcguild at the same time?
Thanks

I did some in-depth analysis of Bithopper Long-Poll timing method for identifying block-solving pools, initially to see how pool-delay corrections can be used to make round-finding results more accurate, I used the console log output of BH, which after some grepping, looks like this:

(round 141030 - found by deepbit)

[1295][23:39:34] LP Call mining.mainframe.nl:8343/LP
[1326][23:39:35] LP Call pit.deepbit.net:8332/listenChannel
[1333][23:39:35] LP Call mineco.in:3000/LP
[1334][23:39:35] LP Call eu1.triplemining.com:8344/LP
[1341][23:39:36] LP Call arsbitcoin.com:8344/LP
[1344][23:39:37] LP Call uscentral.btcguild.com:8332/LP
[1355][23:39:38] LP Call pool.bitclockers.com:8332/LP
[1356][23:39:38] LP Call pool.bitp.it:8334/longpoll
[1358][23:39:39] LP Call bitcoins.lc:8080/LP
[1392][23:39:42] LP Call su.mining.eligius.st:8337/LP

We see above, the first pool to respond is NOT the finding pool!

I have workers logging into 16 different pools; however only 11 of those reply with new block LP pushes. Current Bithopper logic simply guesses that the first LP is the finding pool; as we can see in above, the first LP was not from deepbit in this case. As I had seen this behavior before, I requested the addition of an lp_penalty option - a per-pool option in Bithopper, where you can "delay" the LP reponse time on fast-reporting pools.

Now, we must classify what that delay should be. I used the LP timestamps from the console log as shown above. I was originally expecting the finding pool to clearly stand out, with a definitive earlier LP push after correction, but even this turned out not to be the case, so I should have modded the output to be tenths of seconds for better analysis before I started..

Here is 11 rounds of data, with the actual finding pool verified with digbtc.com and pident.artefact2.com. The block delays for each pool and each round are normalized against the time of the finding pool, if present:


The time-delay table is the top, different analyses are in blue, and the actual correction I am using is on is on the far right. Note the lp_penalty setting would be the opposite of this correction - eligius might get lp_penalty:0 and mineco: 4.

The bottom table is the time delays after applying the best correction I could hand-tweak. We see that still the finding pool almost never is the fastest (highlighted in red). (This might improve with 1/10th second stats resolution).

However, we see that if we average all pool LPs after learning a delay correction, the finding pool does begin to stand out.

The conclusions I have to make:

-Per-pool LP delays are more random than one might expect; and some pools have more variation than others (likely related to number of miners, and their p2p distance from block-solving pool)

-Deepbit's block-find LPs do not clearly stand out from other non-finding LPs received from deepbit,

-The first reporting pool, even when testing many delay correction formulas, will not necessarily be the finding pool.

-Only with a time delay correction for each pool, and then comparison of each pool to the average of all pools can we hope to reliably identify the stand-out block-finding pool. We also need a confidence factor, so if no pools stand out, we can conclude none of LP-returning pools we check found the block.


We have also seen --p2pLP come to prominence, where LP block-finding information is shared on IRC, and then a vote helps further refine results. However this still is very rudimentary; and as nearly all bH users get LPs from deepbit vs only some getting other pools, deepbit tends to win votes undeservedly (and these biased voting results would be more wrong if it wasn't for DB being the largest pool).


So to improve BH LP block finding to the point of reliability, we need:

- Self-learning of per-pool LP delays (LP delay learning must remove the finding pool however, or else we bias deepbit's delays because they find so many blocks)

- Publishing of all local corrected new block LP results to IRC, not just winning pool (perhaps JSON format, like "** block 3fe33a: {"triple": 0.0, "bitp": 0.3, "deepbit": 0.5, "mineco": 1.3, "bclc", 1.7}"

- Averaging (not voting or first response) all IRC results (including pools we don't actually check ourselves), to determine if there is a clear LP block winner; a confidence level can be assigned.

For the super-advanced, it may even be able to profile a 'signature' of block delays to determine who found the block, since pools also seem to have a different LP delay depending on their p2p distance from the finding pool.

i'll ever have a backup server. i have seen to many days with all pools above 100% cdf. so i just don't care about "even out"-graphs

i saw two differences with abcpool to other pools:

abcpool is lagging out very fast (only with bh)
and my hashrate has gone crazy (but seems feasible if they are counting invalids for their hashrate calculation)

i do think it is about bh's delag call; but i dont know how yet. it seems to be identic to a normal getwork. the delag calls are send every ten seconds; and thats really not much.

Backup? We don't need no steenking backup!

 

need to add IOcoin support.. next.

I think I have read about the "DoS-like appearance" in some other thread before, can't remember where though.
But I don't think I have seen invalid shares being part of it, aggressive pulling/getwork I think it was when I read about it before.

The big question, is the massive amount of invalid shares only occurring on the ABCpool?

the owner of abcpool just let us know not to use them atm:
https://bitcointalksearch.org/topic/m.457579

there are problems with counting invalids (you get your payment though), but for them bh behaves a little dos-sing.

we don't know why yet; but we'll see. after that: it seems to be a really good pps pool as a backup

I was responding to deepceleron, not you bb. My response to meni: https://bitcointalksearch.org/topic/m.457483

Edit: and now thanks to Meni I've sped up the simulator hugely by generating random numbers directly by using geomtrically distributed random number generator (rgeom) instead of using rpois and then a 'find block' loop.

back to good ol' bithopper, after done having fun with ixcoin.

look at the new changes.
niiiiiiice.

I plotted your sample:



and a sample I produced:



Please excuse the missing labels. x-axis is number of shares (in percentage of difficulty), y-axis is number of occurances (summing over windows of 10000 shares).

The results differ slightly, most significantly at around 0.

Btw, I got the latest c00w last night and have been mining with it since. Still no share at deepbit:



?

I have found 0.5 to work quite nice as well

My simulator uses randomly generated round shares that are generated with simple algorithm mimics block finding procedure. (poisson process) To get statistically meaningful result, I used high quallity Mersenne Twister as a base pseudo-random generator.

pseudo code:

table of generated 120k round shares : http://cl.ly/8wg3

The described method does not result in uniformly distributed pool shares.

Also in case you missed it, Meni replied to you here.