[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 620.

cabin

sr. member

Activity: 604

Merit: 250

Quote from: organofcorti on November 17, 2012, 08:11:05 AM

The correlation coefficient between hashrate per round and shares per round on D is 0.08. That means no significant correlation at all. Hashrate does not correlate with round length in a linear way. Plotting one against the other doesn't really show any pattern at all. It looks like a cloud of mosquitos.

Unfortunately the data only goes back to August - is there access to all rounds somewhere?

This looks promising:
http://p2pool.info/blocks?from=0

organofcorti

donator

Activity: 2058

Merit: 1007

Poor impulse control.

The correlation coefficient between hashrate per round and shares per round on D is 0.08. That means no significant correlation at all. Hashrate does not correlate with round length in a linear way. Plotting one against the other doesn't really show any pattern at all. It looks like a cloud of mosquitos.

Unfortunately the data only goes back to August - is there access to all rounds somewhere?

cabin

sr. member

Activity: 604

Merit: 250

Quote from: kano on November 17, 2012, 01:20:18 AM

Quote from: forrestv on November 17, 2012, 12:59:24 AM

Quote from: kano on November 17, 2012, 12:31:52 AM

Quote from: forrestv on November 16, 2012, 08:56:58 PM

Quote from: gyverlb on November 16, 2012, 08:49:31 PM

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

So it doesn't include all the 1-diff shares generated by the miner ... ?

No... pool hash rate is only determined by actual shares, which is accurate enough.

So back to my original statement then.
The pool hash rate reported is below the actual hash rate of the miners (like all pools)

However, since p2pool has 60x the number of LP's compared to other pools, the actual hash rate is quite a bit higher than reported - whereas with a normal pool it shouldn't be more than about 1% higher (I typically get <0.4% for GPU/BFL/ICA and ~0.7% for MMQ)

Yet in those ignored shares, if there is a valid Block, it will of course get through, so rather than the number of blocks being representative of the number of valid shares, it is in fact representative of the number of all work generated by all p2pool miners which is of course higher than the reported hash rate.

So when people are reporting that p2pool is getting 110% luck consistently (which of course isn't possible - the probability of getting 110% over even 1 month is excessively unlikely) they are in fact not comparing the correct numbers.
Yes p2pool may well now be averaging 100% as expected, but those > 100% numbers are the result of not comparing the correct numbers

One thing that might mitigate some of this is that p2pool does send a header that tells the miner to submit all work, even if it is considered dead right after a long poll. This work should show up in the number of DOA shares the pool reports. So we might be counting most of the hash rate that others pools miss right after a long poll.

Also +/- 10% over the course of a month will happen fairly regularly with a pool of this size.

mdude77

legendary

Activity: 1540

Merit: 1001

Quote from: sharky112065 on November 16, 2012, 11:31:11 PM

Quote from: Syke on November 16, 2012, 10:41:33 PM

Quote from: sharky112065 on November 16, 2012, 06:03:40 PM

Yes our 90 day moving average is impressive and it is being used to sugar coat or slick over the issue we are trying to bring up. Just because our overall luck is good does not mean we don't have a scalability problem. When we get those bumps in hash speed, the person/s that jumped on board usually see that their payouts are low because of the bad luck during the period they hopped on and they leave. That drops us back down in hash rate back to where the supposed scalability issue is no longer in play.

So until we have a run of good luck in the 350+ GH range I and probably some others will still have doubts.

That's just ridiculous. Take a look at the last week. Hashrate has been climing to over 400 GH/s, and the 7-day average is still over 115%.

As you said... "has been climbing" So for part of the week we were below 350 GH. and 7 day average is now down to 110%

I'm done (as in, not gonna argue with you anymore), no one will take a serious look into the issue as long as people are sugar coating it with ... see look our overall luck is good. What I and others are saying is we are noticing that blocks are being solved by the pool less frequently when we are over a certain hash rate.

If you look at my posts, I used to be saying the same thing, except I was saying 300 GH was the magic number.

M

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: kano on November 17, 2012, 01:20:18 AM

Quote from: forrestv on November 17, 2012, 12:59:24 AM

Quote from: kano on November 17, 2012, 12:31:52 AM

Quote from: forrestv on November 16, 2012, 08:56:58 PM

Quote from: gyverlb on November 16, 2012, 08:49:31 PM

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

So it doesn't include all the 1-diff shares generated by the miner ... ?

No... pool hash rate is only determined by actual shares, which is accurate enough.

So back to my original statement then.
The pool hash rate reported is below the actual hash rate of the miners (like all pools)

However, since p2pool has 60x the number of LP's compared to other pools, the actual hash rate is quite a bit higher than reported - whereas with a normal pool it shouldn't be more than about 1% higher (I typically get <0.4% for GPU/BFL/ICA and ~0.7% for MMQ)

Yet in those ignored shares, if there is a valid Block, it will of course get through, so rather than the number of blocks being representative of the number of valid shares, it is in fact representative of the number of all work generated by all p2pool miners which is of course higher than the reported hash rate.

So when people are reporting that p2pool is getting 110% luck consistently (which of course isn't possible - the probability of getting 110% over even 1 month is excessively unlikely) they are in fact not comparing the correct numbers.
Yes p2pool may well now be averaging 100% as expected, but those > 100% numbers are the result of not comparing the correct numbers

you can find a block with an orphaned share too!
think of it as: with a 1diff share being stale on a normal pool u still can get a block (in relation to p2pool)

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: forrestv on November 16, 2012, 08:56:58 PM

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?
With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.

What might be underestimated too is the amount of work from nodes which stop mining after stale shares.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: forrestv on November 17, 2012, 12:59:24 AM

Quote from: kano on November 17, 2012, 12:31:52 AM

Quote from: forrestv on November 16, 2012, 08:56:58 PM

Quote from: gyverlb on November 16, 2012, 08:49:31 PM

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

So it doesn't include all the 1-diff shares generated by the miner ... ?

No... pool hash rate is only determined by actual shares, which is accurate enough.

So back to my original statement then.
The pool hash rate reported is below the actual hash rate of the miners (like all pools)

However, since p2pool has 60x the number of LP's compared to other pools, the actual hash rate is quite a bit higher than reported - whereas with a normal pool it shouldn't be more than about 1% higher (I typically get <0.4% for GPU/BFL/ICA and ~0.7% for MMQ)

Yet in those ignored shares, if there is a valid Block, it will of course get through, so rather than the number of blocks being representative of the number of valid shares, it is in fact representative of the number of all work generated by all p2pool miners which is of course higher than the reported hash rate.

So when people are reporting that p2pool is getting 110% luck consistently (which of course isn't possible - the probability of getting 110% over even 1 month is excessively unlikely) they are in fact not comparing the correct numbers.
Yes p2pool may well now be averaging 100% as expected, but those > 100% numbers are the result of not comparing the correct numbers

forrestv

hero member

Activity: 516

Merit: 643

Quote from: kano on November 17, 2012, 12:31:52 AM

Quote from: forrestv on November 16, 2012, 08:56:58 PM

Quote from: gyverlb on November 16, 2012, 08:49:31 PM

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

So it doesn't include all the 1-diff shares generated by the miner ... ?

No... pool hash rate is only determined by actual shares, which is accurate enough.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: forrestv on November 16, 2012, 08:56:58 PM

Quote from: gyverlb on November 16, 2012, 08:49:31 PM

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

So it doesn't include all the 1-diff shares generated by the miner ... ?

Syke

legendary

Activity: 3878

Merit: 1193

Quote from: sharky112065 on November 16, 2012, 11:31:11 PM

As you said... "has been climbing" So for part of the week we were below 350 GH. and 7 day average is now down to 110%

So what you're saying is, when you flip a coin less than 350 times a day you get 50/50, but when you flip it more than 350 times in a day you get 60/40? Claim it all you want, it makes no sense.

Let's look at this week day by day (which is a bad idea as there's too much variation). 2 blocks or less is below expected, 3 blocks or more is above average.

1-day ago. 2 - Below
2-days ago. 3 - Above
3-days ago. 4 - Above
4-days ago. 4 - Above
5-days ago. 4 - Above
6-days ago. 1 - Below
7-days ago. 0 - Below

4 days above average, 3 days below average. Where's the problem???

Quote from: sharky112065 on November 16, 2012, 11:31:11 PM

I'm done (as in, not gonna argue with you anymore), no one will take a serious look into the issue as long as people are sugar coating it with

Go for it. Take a serious look.

https://github.com/forrestv/p2pool

And when ASICs hit you'll see that you're imagining patterns where there are none.

sharky112065

sr. member

Activity: 383

Merit: 250

Quote from: Syke on November 16, 2012, 10:41:33 PM

Quote from: sharky112065 on November 16, 2012, 06:03:40 PM

Yes our 90 day moving average is impressive and it is being used to sugar coat or slick over the issue we are trying to bring up. Just because our overall luck is good does not mean we don't have a scalability problem. When we get those bumps in hash speed, the person/s that jumped on board usually see that their payouts are low because of the bad luck during the period they hopped on and they leave. That drops us back down in hash rate back to where the supposed scalability issue is no longer in play.

So until we have a run of good luck in the 350+ GH range I and probably some others will still have doubts.

That's just ridiculous. Take a look at the last week. Hashrate has been climing to over 400 GH/s, and the 7-day average is still over 115%.

As you said... "has been climbing" So for part of the week we were below 350 GH. and 7 day average is now down to 110%

I'm done (as in, not gonna argue with you anymore), no one will take a serious look into the issue as long as people are sugar coating it with ... see look our overall luck is good. What I and others are saying is we are noticing that blocks are being solved by the pool less frequently when we are over a certain hash rate.

Syke

legendary

Activity: 3878

Merit: 1193

Quote from: sharky112065 on November 16, 2012, 06:03:40 PM

Yes our 90 day moving average is impressive and it is being used to sugar coat or slick over the issue we are trying to bring up. Just because our overall luck is good does not mean we don't have a scalability problem. When we get those bumps in hash speed, the person/s that jumped on board usually see that their payouts are low because of the bad luck during the period they hopped on and they leave. That drops us back down in hash rate back to where the supposed scalability issue is no longer in play.

So until we have a run of good luck in the 350+ GH range I and probably some others will still have doubts.

That's just ridiculous. Take a look at the last week. Hashrate has been climing to over 400 GH/s, and the 7-day average is still over 115%.

forrestv

hero member

Activity: 516

Merit: 643

Quote from: gyverlb on November 16, 2012, 08:49:31 PM

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: kano on November 16, 2012, 08:03:54 PM

So - how is the pool hash rate calculated?

It includes orphans and dead on arrival shares as they are seen by every node.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: gyverlb on November 16, 2012, 07:20:06 PM

Quote from: kano on November 16, 2012, 06:32:36 PM

Quote from: K1773R on November 16, 2012, 06:10:48 PM

...
i dont mind the variance, u still get more than all other pools, just not in a steady way

It does depend on how the pool hash rate is calculated ...

The pool hash rate is actually somewhere around 5% or greater above the share rate.

So if the share rate is used to determine the pool hash rate

No. kano this is beginning to look like pure trolling: please install p2pool and use it or read the code instead of spreading FUD. I know using the forum to look for this kind of information is difficult, but it has actually already being discussed at lengths in this very thread.

So - how is the pool hash rate calculated?

rav3n_pl

legendary

Activity: 1361

Merit: 1003

Don`t panic! Organize!

Stale/doa shares depends on miner config and p2pool server load.
Orphan shares are just luck.
Shares are ONLY to calculate payout.
All shares are tested for possible block found.
Share diff is scaled to pool ratio/share about every 10s.
So stale and orphan ratio are NOT responsible for pool luck/block finding ratio.

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: kano on November 16, 2012, 06:32:36 PM

Quote from: K1773R on November 16, 2012, 06:10:48 PM

...
i dont mind the variance, u still get more than all other pools, just not in a steady way

It does depend on how the pool hash rate is calculated ...

The pool hash rate is actually somewhere around 5% or greater above the share rate.

So if the share rate is used to determine the pool hash rate

No. kano this is beginning to look like pure trolling: please install p2pool and use it or read the code instead of spreading FUD. I know using the forum to look for this kind of information is difficult, but it has actually already being discussed at lengths in this very thread.

kano

legendary

Activity: 4592

Merit: 1851

Linux since 1997 RedHat 4

Quote from: K1773R on November 16, 2012, 06:10:48 PM

...
i dont mind the variance, u still get more than all other pools, just not in a steady way

It does depend on how the pool hash rate is calculated ...

The pool hash rate is actually somewhere around 5% or greater above the share rate.

So if the share rate is used to determine the pool hash rate, then yes it is lower than the true hash rate by quite a bit and thus the blocks found would appear to be above the expected 100% ... when in reality the share rate doesn't include the VERY high stale rate ... whereas the block rate does since the blocks are not lost, just the shares.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: sharky112065 on November 16, 2012, 06:03:40 PM

Quote from: Syke on November 16, 2012, 01:44:23 PM

Quote from: sharky112065 on November 16, 2012, 04:51:30 AM

OK, show me 5 or 6 blocks in a day when we are over 350 GH for the entire day and I will believe you. I know what variance is, but variance to the good when we are over 350 GH for 24 hours and I'm sold. It doesn't even need to be 5 or 6 blocks, just positive luck for that day.

The odds of having a 6-block day, when the hashrate is high, is quite low and simply due to variance. Not finding one isn't going to prove anything. Our 90-day moving average is over 110% of expected payout. That's impressive.

Yes our 90 day moving average is impressive and it is being used to sugar coat or slick over the issue we are trying to bring up. Just because our overall luck is good does not mean we don't have a scalability problem. When we get those bumps in hash speed, the person/s that jumped on board usually see that their payouts are low because of the bad luck during the period they hopped on and they leave. That drops us back down in hash rate back to where the supposed scalability issue is no longer in play.

So until we have a run of good luck in the 350+ GH range I and probably some others will still have doubts.

i dont mind the variance, u still get more than all other pools, just not in a steady way

sharky112065

sr. member

Activity: 383

Merit: 250

Quote from: Syke on November 16, 2012, 01:44:23 PM

Quote from: sharky112065 on November 16, 2012, 04:51:30 AM

OK, show me 5 or 6 blocks in a day when we are over 350 GH for the entire day and I will believe you. I know what variance is, but variance to the good when we are over 350 GH for 24 hours and I'm sold. It doesn't even need to be 5 or 6 blocks, just positive luck for that day.

The odds of having a 6-block day, when the hashrate is high, is quite low and simply due to variance. Not finding one isn't going to prove anything. Our 90-day moving average is over 110% of expected payout. That's impressive.

Yes our 90 day moving average is impressive and it is being used to sugar coat or slick over the issue we are trying to bring up. Just because our overall luck is good does not mean we don't have a scalability problem. When we get those bumps in hash speed, the person/s that jumped on board usually see that their payouts are low because of the bad luck during the period they hopped on and they leave. That drops us back down in hash rate back to where the supposed scalability issue is no longer in play.

So until we have a run of good luck in the 350+ GH range I and probably some others will still have doubts.

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 620. (Read 2591920 times)