Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 58. (Read 2591920 times)

Block!

Also, am I misunderstanding something here? Overall pool efficiency is sitting at 87%. http://p2pool.org/stats/index.php sits at exactly this efficiency.

Why? http://uk.p2pool.science:9332/static/ has a current listed efficiency of 103%. Why would anyone chose to lose almost 20% of their hashing power?

I understand that connectivity and processor limits apply here - nodes need to be well connected and low-latency. But why would someone use one that... Isn't? "just" to have control?

New P2Pool node: http://p2pool.halvors.org:9332/

We have waves, gentleman : https://www.youtube.com/watch?v=r_E7yrCAq58

Thank you for the detailed explanation.

Right now p2pool shares can contain up to 50 kB of new transactions, plus an unlimited amount of previously-shared transactions. The way p2pool works is that it takes a block template from bitcoind, then iterates through the transaction list. It adds each transaction to the share one-by-one, noting whether that transaction has been included in previous shares or not, until 50 kB of the transactions in the share are being included in a share for the first time. Once the 50 kB limit has been reached, p2pool stops adding transactions to the share, and truncates the block that would be mined. This means that it takes about 20 shares (or around 10 minutes) before p2pool is capable of mining a 1 MB block.

If we use the current 50 kB limit on new transactions per share, and if Antpool mines a block, then we mine 3 non-block shares followed by a block (i.e. about 2 minutes after them, what we'll see in our share sizes is something like this:

1. Antpool block (clears out our mempool)
2. 50 kB share mined (50 kB new)
3. 100 kB share mined (50 kB new)
4. 150 kB share mined (50 kB new)
5. 200 kB share mined (50 kB new) -- block!
6. 50 kB share mined (50 kB new)
7. ...

When I say "100 kB share mined", I mean that the block for that share would have been 100 kB, not that 100 kB of data were sent over the network. Network messages are much smaller than block sizes.

If we allowed 100 kB and increased the mempool size, we might see something like this:

1. Antpool mines a block (partially clears out our mempool)
2. 400 kB share mined (100 kB new, 300 kB reclaimed from mempool)
3. 520 kB share mined (100 kB new, 20 kB reclaimed from mempool)
4. 640 kB share mined (100 kB new, 20 kB reclaimed from mempool)
5. 760 kB share mined (100 kB new, 20 kB reclaimed from mempool) -- block!
6. 120 kB share mined (100 kB new, 20 kB reclaimed from mempool)
7. ...

I want to either get rid of the limit entirely or increase that limit to 1000 kB of new transactions, which means we would see something like this:

1. Antpool mines a block (partially clears out our mempool)
2. 990 kB share mined (690 kB new, 300 kB reclaimed from mempool)
3. 990 kB share mined (20 kB new, 970 kB reclaimed from mempool)
4. 990 kB share mined (20 kB new, 970 kB reclaimed from mempool)
5. 990 kB share mined (20 kB new, 970 kB reclaimed from mempool) -- block!
6. 990 kB share mined (890 kB new, 100 kB reclaimed from mempool)
7. ...

While the second scenario is an improvement over the status quo, I consider it still broken. I think miners have an obligation to users to mine blocks with as many transactions as feasible, and the second scenario does not do that. A 100 kB limit might result in something like 1/4 of our blocks being under 990 kB.

As I believe that I am ethically bound as a miner to process as many user transactions as possible, I personally am not willing to continue mining under scenarios 1 or 2. I think that p2pool truncating a block template from bitcoind is simply unacceptable.

The third scenario is the only one that would ensure that every block mined by p2pool is as large as the node trying to mine it wants it to be. This scenario means that the first share after each new block is published will include a relatively large number of new transactions, which means that the network size of the share might be large. This could cause propagation delays for that share, and might increase the disparity in share orphan rates between nodes with different hashrates or different hardware speeds.

However, I suspect most of the network data will be transmitting transactions between the node's bitcoind's p2p port (localhost:8333) and p2pool, or sent in advance of the share as a transaction announcement via p2pool's p2p network. If I understand the network protocol correctly, share messages over the network are just a set of hashes, which means that a 990 kB share with 890 kB of transactions in it that were being included in a share for the first time should have close to 0 transactions that hadn't already been shared over the p2p layer. If I'm right, the network message size of such a share should be on the order of 32 bytes * 2500 transactions = 80 kB. If this is true, then raising the limit to 1000 kB won't change typical orphan rates much or at all. I intend to do some packet sniffing after I hard fork and increase the max new share size to see how big the network messages actually become.

I hear what your saying, I was thinking from the viewpoint of trying to keep our variance as low as possible, up until about a week ago it was getting pretty brutal

My understanding so far is that you want to double the p2pool share size and increase mempool, fixing this will fix the issue you identified with p2pool producing blocks smaller then Bitcoins available mempool?

Bitcoind updated.  It was previously running v0.12

Warning has been cleared from that node.

forrestv, I think you might be confusing min and max here. The current code allows for people to signal for SegWit if they want, but it requires that people signal for BIP9 (that is, if 0x20000000 is greater than bitcoind's reported version, then we use 0x20000000). The code you are proposing would *require* p2pool users to signal for SegWit, even if they're running a version of bitcoind that doesn't support SegWit, and even after SegWit is activated (which itself is a violation of BIP9).

It might be best to change that line of code to this, and end the editing of this line once and for all:

It's totally understandable, and I appreciate your presence and contribution to the fullest as well, thank you very much! Do what you gotta do

The beauty of a clean hard fork is that the existing version will continue to work normally, just with less hashrate. Both versions should be able to run in parallel without interfering with each other. In order to make that happen, though, I need to find a change that I can make that will cause existing nodes to reject my shares and also cause my nodes to reject the network's shares. I think *reducing* my share version to e.g. 14 or below would work, as long as I also change my code to require a share version number less than or equal to my new share version for received network shares. (I think there's a lambda in networks/bitcoin.py I can use for that.) There might be a better way to achieve the fork, though, and I'm open to suggestions.

This isn't a "screw you guys, I'm going home." It's a "this issue needs to be fixed, and it takes a hard fork to fix it cleanly, so I'm just going to do it, and you can join me if you want to."

I know you have received some opposition on the PR on github, but I'd ask that you give it a little time before forking. I'd love to have the time to review and understand the proposed change better, and I imagine as it's been less than a day others would as well 

Either way your contributions to p2pool have been appreciated, thanks.

I mined two of them.

I'm probably going to fork off of p2pool soon in order to increase the limit of 50 kB of new transactions per share. That limit has been severely curtailing p2pool's average block sizes and fee revenue. It looks like we're only getting about 90% of the revenue we could be getting if our block sizes didn't suck.

I'll keep my miner off for awhile.

Whom ever is doing the rain dance for the block gods, thank you and KEEP DANCING!! This is pretty damn awesome

As mentioned on github, the 100 kB of that PR is better, but insufficient.

Rain of Blocks.

I've already included an increase in my segwit PR.

I've opened an issue on github about p2pool generating small blocks. https://github.com/p2pool/p2pool/issues/341

I think it's due primarily to shares being limited to at most 50 kB of new transactions per share, which means that it takes 20 shares (about 10 minutes) before a 1 MB block can be generated by p2pool. Exceeding this limit will cause a p2pool share to get punished in case of an orphan race, and so the p2pool code prefers to remove transactions from the block template. As an exacerbating factor, it seems that the p2pool mempool is limited to 2.5 MB; we may want to increase this as well.

 
 Go P2Pool Go !!!!


                     

      http://imgur.com/DzMH9eg

Thank you for your answer and support.
 For me is not a political issue.
 It's just a good node that can handle high dificulty shares with low doa share %. As long as they are valid.