Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 708. (Read 2591919 times)

It's certainly possible, but I wanted to add that p2pool.info now also asks blockchain.info for a list of all blocks (including orphaned blocks) that include the p2pool donation address.  I trust that blockchain.info is much more likely to have seen all orphaned blocks (excluding the recent downtime) given that they connect to almost every publicly accessible node.  I think the likelyhood of unnoticed  orphaned blocks is very low at this point.


I believe forrest just commited this change to the git tree last night.

Thanks for doing that twmz.  Sadly that isn't what I wanted to hear.

While 80% over 90 days is plausible looking beyond 90 days the likelihood of such a low % becoming increasingly unlikely.

Since inception the pool has found 372 blocks.  Stats for the first block are unknown so we will exclude that.  

# Blocks: 371
Estimated shares in blocks:  619,152,160
Expected shares in blocks:  547,529,631
Lifetime Luck 87.9%
Shares per block relative to expected.  1.13x

I calculate
Actual # of blocks: 371
Expected # of blocks: 422

Mining is a possion distribution.
The odds of having 371 or less events when 422 are expected is 0.71%
http://www.sbrforum.com/betting-tools/poisson-calculator/

There is only a 0.71% chance we are just facing bad luck and a 99.29% chance something is resulting in less blocks found.

I am certain that "something" isn't malicious intent by developer however something is causing us to find less blocks than expected.

A number of theories:
a) some nodes are accidentally not submitting blocks.
b) something in the code is causing valid blocks to not be detected.
c) someone with ~40GH/s is intentionally withholding blocks to damage the network.
d) the stats are inaccurate and twmz observation is coincidental (I don't believe it just included it for completeness)
e) we are finding more blocks but they are being orphaned at a higher rate (>10%) and those orphans aren't being seen by p2pool.info node.
f) there is some bias in SHA-256 hash (unlikely but not impossible) such that low nonce values are less likely to find blocks that higher ones, due to LP p2pool searches more low nonce values
f) someone has found a method to "cheat" the p2pool reward algorithm

Note not all of these are equally likely I am just kinda brainstorming and putting down any possibility no matter how remote.

A couple suggestions:
a) found blocks should be forwarded to all nodes regardless of their staleness (this will aid in troubleshooting if nothing else)
b) some mechanism of recording the exact # of shares per round would be useful.

That being said I can no longer afford to take a 20% haircut on revenue so I will be temporarily leaving p2pool.

On edit:  I will leave one rig on p2pool using new address and attempt to gather detailed stats on shares vs payout.

There have been on and off discussions of if the p2pool.info luck calculations are somehow wrong (i.e. too low) because of some error in the estimated hashrate, etc.

I did an experiment with independent data to see how it compared and here are the results.

Instead of looking at anything from p2pool, I looked at my payments over the past 30 days.  My personal p2pool hashrate has been unchanged during the past 30 days and I have a detailed record of every share I have found, so I am certain what my hashrate was during that duration.  Note, I did happen to be the block finder for one of our blocks in the past 30 days, and so I did even receive a bonus payment to counter balance all the other blocks where I didn't receive the bonus payment, so that effect is mostly washed out.

Taking the difficulty changes into account, I calculated what my expected payment should have been over the past 30 days given my hashrate.  I compared that with what I was actually paid.  Over the past 30 days, I received 78% of amount that I was expected to have received given my hashrate.  The luck calculation from p2pool.info (which is based on average pool hashrate and blocks found) calculates that the past 30 day luck is 77%.

77% global luck vs 78% personal luck.

That makes me pretty comfortable that the p2pool.info luck calculation approach is not fundamentally broken and off by 10% or something. 

If anyone else sees a different pattern in their own personal payouts, that would be interesting.

yeah this morning it was terrible. 5m blocks. just plain suck. lol

Oh god, all is lost! I ruined it last time and now it's happened again... New P2pool thread rule: NO TALKING ABOUT THE LUCK!

So I checked my namecoin balance recently (something I tend to forget to do)  and noticed some coins.  So I dug through the log file and came up with this:


$grep submittal ~/p2pool/data/bitcoin/log
2012-04-27 16:34:07.646242 > Merged block submittal result: False Expected: True
2012-04-29 09:16:43.527938 Merged block submittal result: True
2012-04-30 20:00:57.030804 Merged block submittal result: True

Is that first one a namecoin block that was orphaned or invalid somehow? I'm guessing its not orphan, since as I understand it you wouldn't know about that right at submittal.  I have only ever found a few namecoin blocks so I'm not sure what the log usually looks like.  Just curious really, anyone know?

Ah shit oops.

No, ah!  Shouldn't have said anything. 

Heh, just kidding.  Luck does seem to be getting a bit better, plus I just found a block (178015) which always makes me feel lucky =)  I think that is four blocks ever found for me, unless I missed one somewhere. 

Its nice to see luck increasing a little.

I posted a pseudo-spec on what would be required to mine on P2Pool with BFL hardware, thought this was relevant.

https://bitcointalksearch.org/topic/p2pool-and-bfl-hardware-78540

Thanks !

P2Pool gave you a tip for finding it
Not sure why more pools don't do that ...

Lololol

Variance.  It explains all.

To take a break from the serious conversations going on in this thread right now...


I just want you to know, I'm the one who found the block that broke the 23h 49m without a block drought.  No thanks necessary. But tips are, pay me bitches! 

What is going on?  It seems very strange that we don't solve a block all day and then solve three blocks in a row!?

It likely does just as if the pool is unaware of a orphaned share (which isn't a block) but the effect would be negligible.  

Even at avg diff of 800 only one in 700 shares is a block solution so the amount that not knowing about that share can affect stats is <1%.

If a node is unaware of any orphaned share it will affect things like luck calculations (but not payouts).

A block whose share is stale is no different than any other orphaned or DOA share with the exception that it is very rare.


p2pool doesn't need to know that it found a block because that doesn't affect
a) its work
b) its estimate on hashrate
c) reward split

sites like p2pool.info keep track of found blocks because miners like to know when to expect the next payment.  If your node was never notified on any p2pool block via the p2pool network it could still operate just fine (it will always be notified by bitcoind just like it is on any other found block).

Still I would think found blocks who's shares are oprhaned/DOA sould be replicated to every single p2pool node.  Getting the block broadcast from as many nodes as possible can only help block orphan rates.  Right?

So the fact that such a block doesn't make it to the share chain for most P2Poolers doesn't skew the statistics in any way?
... again in relations to the low stats people have been discussing ...

There's no difficulty looking through the block chain to find all the recent P2Pool shares,
(not sure about identifying blocks early on when P2Pool first started of course)
but does P2Pool itself record blocks that didn't get into the share chain - that it isn't notified of via the share chain?
(I've no idea - that's why I ask)

well, I thought the page was showing the ping times from his location since it was his page I was looking at.

Oh well.

If it is possible to lift the version # of the p2pools in the list, might be good to have it there, if you wanna pick a pool that is current.

-- Smoov

Here is the explaination forrest has given in IRC in the past...  If a share is stale/doa but still meets the current bitcoin difficulty requirements (aka it is a block), it is sent to the local p2pool node's bitcoind.  It is also sent to all of the node's peers.  Each of the peers also send the block to their own local bitcoind instances (in order to increase the speed of propagation through the bitcoin network).  However, each of the peers does not relay the stale share to any of their peers.

So it propagates partially through the p2pool p2p network and should be sent to at least 11 bitcoin nodes (1 bitcoin node from the finder of the block and the bitcoin nodes of the block finder's 10 outgoing p2pool connections).  If the block finder has any incoming p2pool connections, it will go to those p2pool peers as well.

A side effect of the fact that the stale share is not fully propagated through the p2pool network is that stale share/valid blocks are sometimes not announced on IRC because the IRC bots sometimes never hear about the stale share (because they are not a direct peer of the block finder).  We have seen this side effect several times (p2pool blocks have been found and not announced in IRC by any of the half dozen or so p2pool nodes that are connected to IRC and announcing blocks).

So, you can confirm that a 10 second share that is effectively late (stale/rejected) will still make it all the way to bitcoind if it is a valid block?
So P2Pool already has this 2 way path?
A share that isn't valid for the share chain will still pay out to everyone and go to bitcoind if it is a block?
(or the share chain can also store invalid shares?)

If forrestv will confirm this - then that's good.