Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 85. (Read 2591928 times)

I suppose i'll just dirty up the code a bit . . .

I see Classic 0.12.1 isn't supported. This just for BIP 9 compatibility or what?

For the Bitcoin network, yes. For other P2Pool networks, no requirements were changed.

Which coin daemons are supported this release? Just core 0.12.1+?

P2Pool release 16.0 - commit hash: d3bbd6df33ccedfc15cf941e7ebc6baf49567f97

HARDFORK - Upgrade URGENTLY required in the next few days.

Windows binary: http://u.forre.st/u/wanckfqm/p2pool_win32_16.0.zip
Windows binary signature: http://u.forre.st/u/wqjnuihh/p2pool_win32_16.0.zip.sig
Source zipball: https://github.com/forrestv/p2pool/zipball/16.0
Source tarball: https://github.com/forrestv/p2pool/tarball/16.0

Changes:
* CSV compatibility
* Requires Bitcoin >=0.12.1

Several BIPs will take effect in the next few days and in order for P2Pool to continue working without producing invalid blocks, everyone needs to upgrade. At 50% of our hashrate upgrading, P2Pool instances will start displaying a warning saying that an upgrade is required. Reaching that point as quickly as possible is very important. And then, at 95%, users that have not upgraded will be excluded. If non-upgraded users aren't excluded before the Bitcoin changes takes effect, P2Pool users will be subject to paying other users for invalid work - effectively a withholding attack.

So, please upgrade to 16.0 now and also tell everyone else to.

  binaries isn't here ... https://github.com/p2pool/p2pool/releases/tag/16.0

You should upgrade. Do a git pull.

I have not yet upgraded my nodes. Once I do, due to the size of my mining operation, almost all of the network hashrate will be on v16, and v15 shares will start to be rejected. It would be a good idea to upgrade before I do.

Recently got a warning that over 50% have voted for an upgrade to v16?

If you can specify how you think the system is flawed, I'm all ears. However, simply stating that you think it's flawed without going into any detail or even demonstrating that you understand the system is not very helpful.

This reminds me very much of the arguments I used to have 5 years ago when people denied that pool hopping worked at all.

It was tedious then and it is tedious now. The problem is that logical falacies involving probability are subtle and difficult to explain convincingly if you lack a formal background.

I may return to the discussion later, or I may not. Either way, I'm pretty sure you are incorrect, but I'm not the one working on a project based on a faulty model, so its not really my problem. It may also be fixable without drastic changes (your later post that showed up while I was typing this may be sufficient, or close enough), but it will almost certainly be more complicated and probably still vulnerable to something. It is also possible, somewhat, that I'm still misreading what you are proposing. I don't think so though.

BTW, I was always assuming a constant difficulty for each share for simplicity of description, but that doesn't change anything in general.

It's actually quite simple to fairly order shares within a level, as long as you don't care whether the ordering is chonological. For example, you can sort the shares by their hash -- lowest hash (closest to mining a block) comes first. Fair, simple, easy, and unnecessary.

It's also trivial to make a DAG with a constant M/N ratio. Instead of allowing shares to specify as many uncles as they want, you can force each share to specify exactly K parents, and create a rule where the parent with the lowest hash is the canonical parent, and only the canonical parent's parents are counted as grandparents. Fair, simple, easy, and unnecessary.

The best solution is just to make sure that the portion of the share chain that's used for calculating rewards always represents the same amount of work. This puts the fewest restrictions on the DAG structure, and ensures that the ratio of the numbers that actually matters -- the amount of work used to calculate the size of the payout, and the amount of work done by others that would have made the payout -- remains a constant 1.0. As in the PPLKD scheme I described earlier.

Shares are paid in proportion to their difficulty. You take the difficulty for a share, divide by the summed difficulty of all shares among the last M (or N for vanilla p2pool), and multiply by the block reward for that block (including fees), and that gives you the reward a share would receive for that block if it were found.

In periods where M is higher due to random variation, the probability of a share getting rewarded by another share finding a block increases, but the payout size decreases. These effects cancel out.

For example, if M is 4 (and all shares are the same difficulty), then if I mine a share, it will have 4 chances of probability 1/D to earn 1/4 of a block. If M is 256, then my share will have 256 chances of probability 1/D to earn 1/256th of a block. In either case, the expected revenue is the same.

The problem comes not when M is higher or lower, but when M is rapidly changing.

A share will be payable for a considerable amount of time, and M might change during that time. For example, if N is 4, my share will have its potential reward calculated 4 different times as it travels through different levels of the DAG. Let's say that we start with 1 share per level in all 4 levels of the DAG when my share is mined (my own share being the top level). After this, 1 more level is mined with 1 share per level, and then a level with 5 shares is mined, followed by another two levels with 1 share. The first chance I have to get paid comes from my own share, where the DAG looks like this:

Round 0:
1*
1
1
1

I get one 1/D chance of getting a 1/4-block reward here. (The asterisk denotes my share.) Next round:

Round 1:
1
1*
1
1

1/D chance of getting a 1/4-block reward.

The next round is a bit tricky. Five shares are mined at once, but since they aren't yet aware of their siblings, each one sees the top level of the DAG as having only one share:

Rounds 2a-2e:
1
1
1*
1

Five 1/D chances of getting a 1/4-block reward. Finally:

Round 3:
1
5
1
1*

One 1/D chance of getting a 1/8-block reward.

Total expected reward = 2*(1/4D) + 1*(5/4D) + 1*(1/8D) = 15/8D blocks. The fair reward would be 1/D blocks.

The above was a rather contrived and extreme example. I wrote a script to simulate the rewards for a N=2016 PPLNS system with between 1 and 10 shares per level, and the typical revenue per share varies by less than 1%, with the maximum I've seen being 2.6%.

Again, you can't hop based on this, because it depends on events that happen after you mine a share, which can't be predicted. It's a selfish mining vulnerability, but not a hopping vulnerability.

And also again, this variation is completely eliminated by the PPLKD scheme I described earlier.

Are all shares paid equally, or is this a scoring-based system?

I think if I am reading correctly, all shares are paid equally. That means during periods that M is higher due to random variation, the value of mining on the pool is lower. During periods that M is lower, the value of mining on the pool is higher. The steady-state average is not what matters here because the miner can choose to participate only during certain system states.

If you attempt to apply a score to correct for this, you can do that, but then you have the issue of how to score multiple shares at the same level. To do this fairly and unexploitably, you need an order. Back to square one.

By "on the next attempt" do you mean on the next share, or on the next level of the DAG, or something else? If you mean on the next share, then you need to multiply by M/N to correct for the expected number of shares per DAG level. According to PPLNS, your share only gets kicked off after a certain number of DAG levels, so you get multiple "attempts" per level.

If you're referring to the situation where you mine a share, then the number of shares per level changes abruptly, then yes, there will be anomalies in expected revenue. I addressed that issue in the part of my post that you didn't read. It doesn't lead to hopping, though, because it depends on events that happen after a share is found, whereas hopping requires the miner to make decisions based on events that happen before a share is mined.

It does happen with PPLNS, because the difficulty of the last N shares is not constant. As I mentioned in my earlier post. And it's easy to fix. As I mentioned in my earlier post.

No it doesn't. The probability that a block will be found on the next attempt (ignoring various constant scaling factors) is 1/D. It is independent of M. As M grows then mining becomes less profitable and it is more profitable to hop elsewhere.

The reason this doesn't happen with PPLNS is that N is constant. M is not.

Sorry, I didn't read the rest of your post because it seemed premised on an error. Once we get past the initial error the rest of the discussion may be worth another look.

I am not seeing any nodes come up on the page... Did you remove the ones you had listed?

Ditto!  Mine are up and running well too.

Everyone, lets update to the latest version from the main repo (version 16).  Nodes are reporting we are around 30% there.

To clarify, expected reward for a share is independent of the work done since the last block was found. If there are more shares per DAG level/height, that means that your share will stay in the last N height for a longer amount of time, so it will have a greater probability of being rewarded, but the reward will be smaller.

Let M equal the number of shares in the last N levels of the DAG. The average number of shares per level is M/N. Assume all shares have equal difficulty for now, and assume steady-state conditions, where M stays roughly constant for a single share's transit through the last N levels. Should a block be found, each share's reward will be 1/M. The probability that a block will be found depends on the amount of work done among those M shares. Since all shares have equal difficulty, that means that the probability of a block being found will be M/D, where D is the ratio of the network difficulty and the difficulty per share. Under steady state conditions, the expected reward for a share is equal to the probability that a block will be found in the next N levels times the reward should a block be found, and will equal (M/D) * (1/M) = 1/D. This means that the expected reward for a share is independent of M as long as the equal-difficulty and steady-state assumptions hold.

And what if they don't hold? Both can bring up a potential problem: a malicious or selfish entity can change their mining behavior in order to flush out or prolong the lifetime of a cohort of shares. If Alice has a burst of good luck, and mines 10 shares out of 20 when she normally should only mine 2 shares out of 20, then she can change her subsequent mining behavior in order to keep that burst of good luck in the last N levels for as long as possible. She can do this either by working on extra-high-difficulty shares or by intentionally mining siblings instead of children on the DAG. On the flipside, when Bob has a spell of bad luck, he can erase it faster by working on low-difficulty shares and trying to always mine DAG children. Note that this vulnerability already exists in p2pool, as individual nodes can specify different share difficulties. The DAG does not create the issue; it's simply an issue with PPLNS with variable difficulty. The issue also does not promote hopping, as hopping only makes sense when the expected reward for a share depends on the past mining results, whereas this issue makes revenue dependent upon the future behavior of other miners.

In any case, it's simple enough to fix. Instead of doing PPLNS where the window is based on the number of shares (or the height in the share DAG), we can do PPLKW (pay-per last K work) where the window is based on an amount of work performed. We set some work threshold (e.g. enough to find 3 blocks on average), then go back through the DAG one level at a time and count all the shares in each level until the total work counted exceeds our work threshold. Thus, each share will always have the same proportion of the total work regardless of future behavior of other miners. The only way to get an enemy's shares out of the DAG faster is to add hashrate and increase the rate at which blocks that reward them are found.

It is not. With the scheme as described the split factor for block rewards is not constant (ignoring difficulty changes) as it is for PPLNS (i.e. 1/N). More shares at the same height means rewards split more ways, which reduces the expectation for a winning share. This is in some way analogous to PPS, and will uncover similar problems.

The shares are partially ordered. Each share has a height associated with it determined by the share's parent. In a share chain, each share that makes it into the final chain has a different height. In a share DAG, multiple shares can have the same height. With both, for PPNLS you reward all shares where share_height >= chain_tip_height - N.

Yes, the analysis of incentives is more complex. I haven't seen any ways in which DAGs could incentivize hopping, as it's still PPLNS and expected reward is independent of the work done since the last block was found. However, there are some potential issues regarding selfish mining on the share DAG with some DAG reward algorithms. I think they aren't too difficult to solve, but it's true that care is required.