Geometric method: New cheat-proof mining pool scoring method

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: btcmonkey on July 29, 2011, 08:41:56 AM

I have not had a lot of success getting your proof to render for me. Can you describe how you came up with the decay factor, r? Isn't it really a growth factor for any reasonable c?

It's growth in the score of new shares, but a decay in the value of old shares. It's the unique value that makes the sums come out right. In fact you could choose r first and then find c, the average score fee, in terms of r.

Quote from: btcmonkey on July 29, 2011, 08:41:56 AM

Is it true that for a very low number of shares ( < 1000 ) at the current difficulty, the total fee gets really large ( > 50% ) when c = 0.001? My implementation seems to show this. Does this mean that a really lucky block find would mean bad news for pool members, or is my implementation flawed?

Yes, the fee is large for short rounds. This is because there aren't many participants to receive a reward, otherwise early miners would get a disproportionate reward.

Quote from: btcmonkey on July 29, 2011, 08:41:56 AM

Expanding on this, what impact would having the score start at some high arbitrary number (e.g. r^10000) instead of 1 have? It seems it could enable setting a max value for what fee would be taken, but I'm not sure how doing this would effect the cheat-proofness of the system and expected fee calculations.

If you do this and keep the score fee as stated, it will be like decreasing the score fee, which means that this is no longer hopping-proof.

Quote from: btcmonkey on July 29, 2011, 08:41:56 AM

For difficulty 2 and difficulty 3 shares is p simply 2/difficulty and 3/difficulty respectively?

Yes.

All in all the method was designed for everything to be 100% accurate in expectation, though this means relatively high variance and some counterintuitive situations.

btcmonkey

newbie

Activity: 20

Merit: 0

This is a really interesting score implementation. I have a few questions.

I have not had a lot of success getting your proof to render for me. Can you describe how you came up with the decay factor, r? Isn't it really a growth factor for any reasonable c?

Is it true that for a very low number of shares ( < 1000 ) at the current difficulty, the total fee gets really large ( > 50% ) when c = 0.001? My implementation seems to show this. Does this mean that a really lucky block find would mean bad news for pool members, or is my implementation flawed?

Expanding on this, what impact would having the score start at some high arbitrary number (e.g. r^10000) instead of 1 have? It seems it could enable setting a max value for what fee would be taken, but I'm not sure how doing this would effect the cheat-proofness of the system and expected fee calculations.

For difficulty 2 and difficulty 3 shares is p simply 2/difficulty and 3/difficulty respectively?

Inaba

legendary

Activity: 1260

Merit: 1000

Thank you Meni, I think I understand now. I will probably have some more questions tomorrow Wink

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: Inaba on June 19, 2011, 11:47:59 PM

Thanks for responding. So just to confirm, MAX is being set to the score of the last share, which is arbitrary in so far as it could be anything depending on when the round ends?

Yes. And it's also arbitrary in the sense that it's just used for numerical stability, you can use another value for it as long as it's used consistently in all parts of the calculation.

Quote from: Inaba on June 19, 2011, 11:47:59 PM

I think I understand what's going on now as far as lscore - max. Is the share table then intended to contain one row per share each with a score as opposed to one row per user with an aggregate score that is increased for each share submitted? I think that may be where I went off track and why it wasn't making sense.

Correct, one row per share.

Quote from: Inaba on June 19, 2011, 11:47:59 PM

Is each share worth a certain amount, regardless of who submitted it or does the value of a share differ depending on who submitted it? By this, I mean if person A has submitted 500 shares in the past hour and person B has submitted 200 shares in the past hour, person A's 500th share is worth more than person B's 200th share or A's 500th and B's 200th are worth the same if they are submitted at the same time (well, one is worth slightly less than the other depending on which order it was submitted)?

Yes, the value of a share depends only on when it was submitted and not on who submitted it. The total payout of a worker is just the sum of the payouts for all of his shares.

Inaba

legendary

Activity: 1260

Merit: 1000

Hi Meni,

Thanks for responding. So just to confirm, MAX is being set to the score of the last share, which is arbitrary in so far as it could be anything depending on when the round ends?

I think I understand what's going on now as far as lscore - max. Is the share table then intended to contain one row per share each with a score as opposed to one row per user with an aggregate score that is increased for each share submitted? I think that may be where I went off track and why it wasn't making sense.

Is each share worth a certain amount, regardless of who submitted it or does the value of a share differ depending on who submitted it? By this, I mean if person A has submitted 500 shares in the past hour and person B has submitted 200 shares in the past hour, person A's 500th share is worth more than person B's 200th share or A's 500th and B's 200th are worth the same if they are submitted at the same time (well, one is worth slightly less than the other depending on which order it was submitted)?

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

The formula you quote is used only when the round ends to calculate the payouts.

lscore is the logarithm of the score for a given share.

"share" is a table that contains information about all shares. "max" is the lscore of the last share (note the "order by id desc limit 1;" part). So only for the last share lscore - max will be 0, for earlier shares it will be negative. You sum over all shares.

The later the share was submitted, the higher its lscore, as specified in the algorithm.

Inaba

legendary

Activity: 1260

Merit: 1000

So I have been trying to figure out the last part of this and from everything I've read in the thread and from some example code I've looked at for both PGSQL and MySQL, I can't quite grasp what's going on here:

let totscore := sum(exp(share.lscore - max)) + exp(round.los - max)

In the pseudo code, PGSQL code and MySQL code, the bolded part would always seem to evaluate to zero? Is this correct? If not, it seems that lscore will be an arbitrary number based off the last user to submit a share for that block (presumably the person who found the answer). So one person can have an lscore of 80, having been there the entire round, and another person can have an lscore of 1, but find the block, thus giving the completely random and arbitrary value of lscore in that scenario.

So, my question is can someone explain which value lscore is and why? If it's the former, what's the point of having an expression that always evaluates to zero? If the latter, what's going on with the formula that it can take an essentially random number and use it as a valid value for calculating score?

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: simplecoin on June 08, 2011, 05:48:35 PM

Quote

0.001%(or less) fee!

If you use f=0, c=0.001 then it's not 0.001% fee, it's 0.1% fee. And it's the average - on any round it can be much higher. You can also consider using negative f to make the expected fee 0.

so using
$c = 0.001;
$f = (-$c)/(1-$c);
should result .1%? or should that be closer to 0?
Is there a way to get to 0% without possible losses?

With these parameters the expected fee is 0, and for any round and it can be as low as -0.1001% (negative) or as high as 100%.
There's no way to have expected fee of 0% without a risk of losing out on a round. Note that the losses will be very small though.

simplecoin

sr. member

Activity: 406

Merit: 250

Quote from: Meni Rosenfeld on June 08, 2011, 03:04:01 PM

I should emphasize that mid-round estimates are fairly meaningless. It is always almost certain that round end will be far enough in the future that all current shares will be worthless. In particular, calculating the expected reward for already existing shares will be close to 0, while calculating the reward if the round ended now will be much higher.

However, the numbers you write might indicate a problem with the calculation. Please post all the values involved, as well as the lscore values of the last few shares - both in general and for the particular worker.

It seems to be behaving now, I think it was just the earlier implementation. It's off, just not quite as badly.

Quote

Also, in your thread you say

Quote

0.001%(or less) fee!

If you use f=0, c=0.001 then it's not 0.001% fee, it's 0.1% fee. And it's the average - on any round it can be much higher. You can also consider using negative f to make the expected fee 0.

so using
$c = 0.001;
$f = (-$c)/(1-$c);
should result .1%? or should that be closer to 0?
Is there a way to get to 0% without possible losses?

Quote

So, if you're not trying to pool-hop, you will be paid slightly more than those who are.

Those not trying to pool-hop will, in expectation, earn exactly as much as those who are, per share submitted.

Got it.

martok

full member

Activity: 140

Merit: 100

It's been some time since I've used MySQL but I expect that might give you some trouble implementing this method. Not to say it can't be done but you might have to be careful with it. Unless MySQL has a serializable mode IE select * from round for update blocks other threads trying to score a share, there is a possibility of bad data getting in:
thread1: select * from round; lastscore = 1
thread2 select * from round; lastscore = 1.
thread 1 scores current share at 1+r
thread 2 does the same thing
thread 1: update round set lastscore=lastscore+r
commit
thread2 update round set lastscore=lastscore+r
but thread2's score is wrong because 1's update dwasn't accounted.

MySQL has transactions these days but I wonder how it handles this where shares depend on previous data.

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: simplecoin on June 07, 2011, 01:39:45 PM

using (1-rd.f)*(1-rd.c)*p*rd.b*sum(exp(lscore-lastlscore)) to calculate an estimated earning, I'm getting wildly incorrect results.

For example my account says 11.x, when I run the full round calc it's closer to 6.

Additionally, the sum of estimates is >88, when it should be < 50.

I'm looking at a round with ~1mil shares.

I should emphasize that mid-round estimates are fairly meaningless. It is always almost certain that round end will be far enough in the future that all current shares will be worthless. In particular, calculating the expected reward for already existing shares will be close to 0, while calculating the reward if the round ended now will be much higher.

However, the numbers you write might indicate a problem with the calculation. Please post all the values involved, as well as the lscore values of the last few shares - both in general and for the particular worker.

Also, in your thread you say

Quote

0.001%(or less) fee!

If you use f=0, c=0.001 then it's not 0.001% fee, it's 0.1% fee. And it's the average - on any round it can be much higher. You can also consider using negative f to make the expected fee 0.

Quote

So, if you're not trying to pool-hop, you will be paid slightly more than those who are.

Those not trying to pool-hop will, in expectation, earn exactly as much as those who are, per share submitted.

simplecoin

sr. member

Activity: 406

Merit: 250

using (1-rd.f)*(1-rd.c)*p*rd.b*sum(exp(lscore-lastlscore)) to calculate an estimated earning, I'm getting wildly incorrect results.

For example my account says 11.x, when I run the full round calc it's closer to 6.

Additionally, the sum of estimates is >88, when it should be < 50.

I'm looking at a round with ~1mil shares.

simplecoin

sr. member

Activity: 406

Merit: 250

1 final question.

How do I calculate estimates while using the logarithmic scaling?

(Update, nvm, think I got it)

simplecoin

sr. member

Activity: 406

Merit: 250

Got it!

I haven't had to wrap my head around this much math in a while.

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: simplecoin on June 07, 2011, 10:35:48 AM

Quote from: Meni Rosenfeld on June 06, 2011, 11:24:23 PM

Quote from: simplecoin on June 06, 2011, 04:15:15 PM

I'm getting close Cheesy

seems like r^i is huge! even a 53digit double gets rounded eventually.

You'll need to use either periodic rescaling or logarithmic scale, as discussed in this thread.

I'm going for logarithmic, since the other seems hackish.

So, I'm close...... it looks like max is the previous row lscore value, is that right?

Yes. The exact value used for max doesn't matter, as long as it's used consistently and it's about the right size. Its role in the calculation is just to shift the scale to a reasonable location to avoid under/over flowing the exp.

simplecoin

sr. member

Activity: 406

Merit: 250

Quote from: Meni Rosenfeld on June 06, 2011, 11:24:23 PM

Quote from: simplecoin on June 06, 2011, 04:15:15 PM

I'm getting close Cheesy

seems like r^i is huge! even a 53digit double gets rounded eventually.

You'll need to use either periodic rescaling or logarithmic scale, as discussed in this thread.

I'm going for logarithmic, since the other seems hackish.

So, I'm close...... it looks like max is the previous row lscore value, is that right?

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: simplecoin on June 06, 2011, 04:15:15 PM

I'm getting close Cheesy

seems like r^i is huge! even a 53digit double gets rounded eventually.

You'll need to use either periodic rescaling or logarithmic scale, as discussed in this thread.

simplecoin

sr. member

Activity: 406

Merit: 250

I'm getting close Cheesy

seems like r^i is huge! even a 53digit double gets rounded eventually.

Meni Rosenfeld

donator

Activity: 2058

Merit: 1054

Quote from: simplecoin on June 06, 2011, 01:14:35 PM

I could really use the help getting this setup on my opensource pool frontend.

It's based on mysql, php and pushpool.

Any help would be greatly appreciated, as well as benefit the community with an opensource solution for their own implementation.

I'll sit this one out because I haven't the slightest idea how to set it up (if I had, I probably would have done so myself a long time ago). I hope martok will be willing to compare notes with you.

Dusty

hero member

Activity: 731

Merit: 503

Libertas a calumnia

(watching)

Topic: Geometric method: New cheat-proof mining pool scoring method (Read 24409 times)