Pages:
Author

Topic: bitHopper: Python Pool Hopper Proxy - page 25. (Read 355813 times)

donator
Activity: 2058
Merit: 1007
Poor impulse control.
August 16, 2011, 02:19:33 AM

In all cases the standard deviation of "daily" average was comparable to the to the average share value, which I'm not exactly sure how to interpret. Does it even make any sense?


Wow, real difficulties, hash rates etc.... I applaud you. Kudos. I'm way too lazy to do that. I just use floating point shares to make calculations easier. It means that I can only get comparative efficiencies easily, if I want real world stuff I have to go through and change values which slows everything down.

Anyway, if mean ~ sd it usually indicates a wide spread of values (depending on median). I just ran byteHopper to get some results to post using 3 pools, hash speed ratios of 0.01, 0.1 and 0.1  and no hop off point. The first is over twenty weeks, the second daily estimate over twenty days, and the final one over one day.

slow pool:
mean =  1.673466    1.422362      3.225511
sd = 0.5091813       0.7734413     5.912161
median = 1.841146   1.51612       1.590691

mean =  1.706857       1.747548   4.391248
sd = 0.2108296           0.668324   4.405089
median = 1.701362      1.634362   2.275186

fast pool
mean =  1.573833    1.700709        2.903397
sd = 0.09222334      0.2546248      2.972965
median =  1.576645  1.745717        1.701818

combined:
mean =   1.646658    1.644637      3.895539
sd = 0.1879099        0.4111877     5.067339
median =  1.616157   1.587286       1.874025

As you can see, as sampling intervals decrease the standard deviation (and hence variance) increases - slowly for the slow pools and more rapidly for the fast pools. If you want to model the variation you get as a miner, then use days as a time interval. If you want as close to a theoretically correct result as possible, use more. I usually use about a thousand loops of a million rounds for that.

Now, a request: chartporn please!
donator
Activity: 2058
Merit: 1007
Poor impulse control.
August 16, 2011, 01:31:07 AM
Create a new role called "mine_slice". This would be for the small pools with low hashrate (user selectable). The faster pools would stay "mine".
My idea is to run the slice scheduler inside of the default scheduler. So the behavior would be: If there were "mine" pools below threshold, you would be mining them under OldDefault rules. If there were no acceptable "mine" pools, all of the "mine_slice" pools would be selected running under slice rules as a group, but only if there was one or more below threshold. If all of these options were exhausted, we would then resort to "backup".

I thought this was an interesting idea. At first look, by minimising low pool shares you should minimise variance a bit compared to olddefault without being completely egalitarian (a la slice) so efficiency should be improved c/w slice, but a little less than old default. I could be wrong about this.

What makes it more interesting is that you can test it - just put all your small pools onto mine_slush but with a .25 penalty so they can mine to .43 if they have to. Mine_slush seems to work on a backup-ish fashion - it doesn't seem to jump there much if there's a good role:mine around.

My new sim is designed to be as modular as possible so once it's done I'll try to write a sim of your suggestion. You may be waiting a while though. Try it yourself with oDS and mine_slush penalty 0.25 in the meantime.
newbie
Activity: 39
Merit: 0
August 16, 2011, 01:20:12 AM
Some numbers from simulation I mentioned above:
- Miner submitted total of 70,000,000 shares. Used pools with share submission ratios: 210, 165, 11, 55, 5, 2.5 times higher the tested miner ratio.
- Miner submissions were not accounted towards the block completion (it might have had some impact the smallest pools).
- Difficulty: 1890362. Share value as in PPS: 2.644996e-5.
- Standard deviation and variation calculated on the average share value for batches of shares of size 21600. (1GH/s * 1 day)
- OldDefaultScheduler: Average share value: 5.453634e-5. Standard deviation: 6.54271e-5. Variation: 4.2807e-09
- DefaultScheduler: Average share value: 5.316599e-5. Standard deviation: 5.39707e-5. Variation: 2.91284e-09
- HybridScheduler (<20% OldDefaultScheduler, > 20% DefaultScheduler): Average share value: 5.02669e-5. Standard deviation: 4.86141e-5. Variation: 2.36333e-09

In all cases the standard deviation of "daily" average was comparable to the to the average share value, which I'm not exactly sure how to interpret. Does it even make any sense?
member
Activity: 102
Merit: 10
August 16, 2011, 12:29:35 AM
@ooc : Had you considered an approach like I mentioned in the below post? I know c00w was planning a scheduler re-write, however I'm not sure what his priorities are. Please check it out, as I definitely think it could be helpful given the current stats of hoppable pools.

https://bitcointalksearch.org/topic/m.443792

@everyone : I haven't given up on streamlining the whole stats/skin process. I have something rudimentary, however I still don't have everything quite ready. Smiley
donator
Activity: 2058
Merit: 1007
Poor impulse control.
August 15, 2011, 10:10:04 PM

Did anyone run simulation on some sample pools with OldDefaultScheduler and DefaultScheduler to see what is the different in efficiency and variation/standard deviation ?


I'm in the middle of a rewrite to do a simple slice atm, suggested by joulesbeef. I'm approaching slicing as following:

  • 'Shares' ('shares' are just a placeholder, the value of share is determined later) are distributed to equally to pools under *hop point* up until the *hop point* limit of any particular pool.
  • Equal share slicing (favouring no particular hoppable pool): Divide shares contributed as above by number of pools to which they were distributed to get the 'value' of each share.
  • Equal time slicing: divide shares by number of pools to which they were distributed, and then multiply by a pools' fraction of the combined hash speed of all pools the to get the 'value' of each share.


    Of course this is an idealised version of slicing and assumes no loss of time hopping from one pool to another and

    If I'm wrong about this please let me know before I go any further with the rewrite!

full member
Activity: 168
Merit: 100
August 15, 2011, 09:14:54 PM
And a side question:
Since we cannot identify the owner of block in any way other than checking the stats on a pools website or by guessing, then what prevents the pool operators from not admitting that their pool actually found a block and take the income to themselves?

a) that happens
b) those pools don't last or they produce enough for their miners that they don't miss an occasional stolen block.
newbie
Activity: 39
Merit: 0
August 15, 2011, 08:51:34 PM
And a side question:
Since we cannot identify the owner of block in any way other than checking the stats on a pools website or by guessing, then what prevents the pool operators from not admitting that their pool actually found a block and take the income to themselves?
newbie
Activity: 39
Merit: 0
August 15, 2011, 08:43:04 PM
A slice is simply a amount of time allotted to mining on a given pool, so its slices time, keeps track of slices and uses that to switch between 2 eligible pools

Are slices size calculated based on amount of shares the pool already have or they are just equal?
Did anyone run simulation on some sample pools with OldDefaultScheduler and DefaultScheduler to see what is the different in efficiency and variation/standard deviation ?

I tried to run a simulation on it, but the efficiency numbers I got (I will share them when I get back home) for those schedulers were very close - and the difference was smaller than standard deviation on average daily efficiency). That result is actually surspised me since, I'd think that mining in a pool which is 20% instead in a pool which is 2% done should have a big difference in the estimated share value. The deviation was something like 2x smaller for the DefaultScheduler compared to OldDefaultScheduler.

If my numbers are wrong and in fact there is a big difference in efficiency for those schedules, it might be interesting to simulate a hybrid scheduler like this:
a) If there is a pool with number of shares less than < some_fixed_ratio * difficulty (i.e. some_fixed_ratio = 20%) then use OldDefaultScheduler
b) otherwise, use DefaultScheduler
In theory it should have smaller variance than OldDefaultScheduler and better efficiency than DefaultScheduler. In practice, well, who knows Smiley
member
Activity: 84
Merit: 10
August 15, 2011, 08:25:32 PM
What would be the best way of going about tweaking the lp_penalty so that I can get bithopper to make the most accurate guess that it can on who solved a block?  Does anyone have a method that is semi simple?
legendary
Activity: 2450
Merit: 1002
August 15, 2011, 05:48:54 PM

I'm guessing "--altslicesize" has a part in this, but I don't understand what a "slice" is, is it shares?

(The more I add to this post, the more confusing it gets, so I will just press "Post" now)  Huh

A slice is simply a amount of time allotted to mining on a given pool, so its slices time, keeps track of slices and uses that to switch between 2 eligible pools
newbie
Activity: 42
Merit: 0
August 15, 2011, 04:50:13 PM
A crazy idea just popped up while I was messing around with some bash-scripts.
 
Would it be possible to have bh running multiple pools simultaneously?
And splitting the getwork requests of those based on a on-every-X-getwork.

If one sets a pool to 'on-every-X-getwork: 10' it would do a getwork every 10th getwork, getting 10% of the total hashspeed.

Why would anyone want something crazy like that?
Direct a constant portion of ones hashingpower towards a pool that one might like to support despite hopping or whatever reason.

Getting bh to re-read the user.cfg either by command or by internal filechange-checking would be a nice, but not required, addition to this.


Another idea came up while writing this post, but it can be shot down quickly if someone has the data:
Would it be favorable to mine, for example, two(things change if there are more?) sub 10%shares pools simultaneously, instead of one until it reaches the threshold and then jump to one of the others, if it/they are still below threshold, or is it better to try and get lucky as early on in a round as possible?

I'm not quite sure how the hopping is determined atm.
Will bh constantly try and find the pool with the lowest share-count and then jump to it?
or will it stay with a pool it has jumped to until the threshold is reached, then jump to the pool with the lowest share-count?
I'm guessing "--altslicesize" has a part in this, but I don't understand what a "slice" is, is it shares?

(The more I add to this post, the more confusing it gets, so I will just press "Post" now)  Huh
member
Activity: 84
Merit: 10
August 15, 2011, 03:59:47 PM

yeah, the recent changes in bithopper basically make you a 24/7 miner at deepbit. Which is fine if you want to 24/7 mine at deepbit. Otherwise, though, it's best to just not even bother with deepbit.


About half of all of my shares are going to deepbit right now.  What would be awesome is if bithopper could check with pident about once an hour and confirm that solved block was actually from the same pool that bithopper received the LP from, and if not have bithopper auto adjust the penalties a little bit at a time until a certain accuracy is reached.  Of course like most ideas I am sure this is easier said than done.  

Where do we put the lp_penalty value for a pool?  Is it in user.cfg or pools.cfg?
legendary
Activity: 1526
Merit: 1002
Waves | 3PHMaGNeTJfqFfD4xuctgKdoxLX188QM8na
August 15, 2011, 03:59:35 PM

Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.

yeah, the recent changes in bithopper basically make you a 24/7 miner at deepbit. Which is fine if you want to 24/7 mine at deepbit. Otherwise, though, it's best to just not even bother with deepbit.

I'm still on the tuesday 9 august version Tongue
member
Activity: 102
Merit: 10
August 15, 2011, 03:59:06 PM
My bH, on my server, will only pull lp for one round. At home, on my pc, it keeps working regularly.
full member
Activity: 168
Merit: 100
August 15, 2011, 03:54:45 PM

Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.

yeah, the recent changes in bithopper basically make you a 24/7 miner at deepbit. Which is fine if you want to 24/7 mine at deepbit. Otherwise, though, it's best to just not even bother with deepbit.
legendary
Activity: 1764
Merit: 1006
August 15, 2011, 02:37:59 PM
Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.
same here.

the pools my hopper hops into was bclc & mtred pps, aside from db
member
Activity: 84
Merit: 10
August 15, 2011, 02:32:12 PM
Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.
bb
member
Activity: 84
Merit: 10
August 15, 2011, 02:15:51 PM
Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Oh. I thought deepbit was working for everyone already. Especially with the IRC.
legendary
Activity: 2450
Merit: 1002
August 15, 2011, 01:56:47 PM
Actually, the manual correction is probably not only different for every miner out there based on their location, but also depends on how many other workers are currently connected to the various pools. It's probably a very unreliable method to determine a block finder.

Maybe it's possible to hook into bitcoind and see in which order connected nodes send the new block information. There might be patterns, but I suspect they are not very constant and need to be re-learned often. The learning would probably take a few hours at the begin of every session to recognize patterns of deepbit, BTC guild, and other pools, if there is any difference.

Off subject here but...um dude your name confused the hell outta me when I first saw it. I was like "I didnt make that post...WTF?!?!"   oh, his name is missing the r...haha
legendary
Activity: 924
Merit: 1004
Firstbits: 1pirata
August 15, 2011, 01:55:33 PM
I have been mining on a PPS pool, but this proxy has gained my interest.

What pools are still hopable?  Obviously proportional.  But don't some use anti-hoping techniques?  

hy, please edit your user.cfg and you will find info on every pool too in there
Pages:
Jump to: