Author

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 408. (Read 2591964 times)

hero member
Activity: 798
Merit: 1000
I've gone ahead and set up a public node for BTC to go along with my other couple coins.

http://us-east.royalminingco.com:9332/static/

The fee is .5% to pool and .5% donation to forrestv. This node has my patches related to helping small miners get less variance, and uses the interface which shows average "time to share" per-miner so you can see what's going on. Pseudo share vardiff is also set reasonably per miner so no ADDR+ tweaks are needed just for graph purposes. Next I want to fold in iongchun's miner-specific stats page, and add some extra charts/history/info to that.

It isn't like the world needs more public p2pool nodes, but I wanted to have at least one that I knew was using the changes I worked on for the alt coin pools. Smiley

What's the possibility of merging your tweaks, especially for the smaller miners, back up into forrestv's master branch?
full member
Activity: 134
Merit: 100
I have a dedicated VPS node running bitcoind and p2pool and getting 5.8% DOA on local rate on 150Gh/s.  Everything running smooth and am now awaiting my first p2pool block.
legendary
Activity: 2968
Merit: 1198
I've gotta say an average share time of 5h40m hours compared to 1d8h block times (at the time I post) is better than I expected. At 6 shares per block found, and a SPREAD of 3, I'd average 18 shares at any given time once I've been mining for a few days. That'll drop if the pool hash rate grows, but atm, ~18 average blocks seems like plenty to make sure I'm paid on every block and my payments don't vary wildly. I'll need to actually see it in action first hand a few weeks, but personally I don't feel that is unacceptable variance at all.

It'll be nice if/when I can get some sort of miner-specific graph page up to plot this stuff over time and let you see visually at a glance your projected and real share values.

The absolutely biggest issue we have with variance is the block time. We've had four-day periods with NO blocks. That's just terrible for everyone. Thankfully our hash rate is growing faster than the network, so at the moment at least, this is becoming a slightly smaller problem, and hopefully this trend continues. Given that the new ASIC miners work so well with p2pool it seems like we have a real shot to survive. It was looking questionable for a while. People just would not tolerate rounds lasting a week or two, which is where we were headed.


sr. member
Activity: 434
Merit: 250
I've gotta say an average share time of 5h40m hours compared to 1d8h block times (at the time I post) is better than I expected. At 6 shares per block found, and a SPREAD of 3, I'd average 18 shares at any given time once I've been mining for a few days. That'll drop if the pool hash rate grows, but atm, ~18 average blocks seems like plenty to make sure I'm paid on every block and my payments don't vary wildly. I'll need to actually see it in action first hand a few weeks, but personally I don't feel that is unacceptable variance at all.

It'll be nice if/when I can get some sort of miner-specific graph page up to plot this stuff over time and let you see visually at a glance your projected and real share values.
legendary
Activity: 2968
Merit: 1198
I've gone ahead and set up a public node for BTC to go along with my other couple coins.

http://us-east.royalminingco.com:9332/static/

The fee is .5% to pool and .5% donation to forrestv. This node has my patches related to helping small miners get less variance, and uses the interface which shows average "time to share" per-miner so you can see what's going on. Pseudo share vardiff is also set reasonably per miner so no ADDR+ tweaks are needed just for graph purposes. Next I want to fold in iongchun's miner-specific stats page, and add some extra charts/history/info to that.

It isn't like the world needs more public p2pool nodes, but I wanted to have at least one that I knew was using the changes I worked on for the alt coin pools. Smiley

More public node are a very good thing. People get less local latency and are able to compete better with dedicated nodes, lowering the barrier to entry, so more p2pool participants.

Nice to see p2pool steadily growing in hash rate and network share BTW. We are back down to about one day per block, after getting close to two days per block a few weeks ago.
sr. member
Activity: 434
Merit: 250
I've gone ahead and set up a public node for BTC to go along with my other couple coins.

http://us-east.royalminingco.com:9332/static/

The fee is .5% to pool and .5% donation to forrestv. This node has my patches related to helping small miners get less variance, and uses the interface which shows average "time to share" per-miner so you can see what's going on. Pseudo share vardiff is also set reasonably per miner so no ADDR+ tweaks are needed just for graph purposes. Next I want to fold in iongchun's miner-specific stats page, and add some extra charts/history/info to that.

It isn't like the world needs more public p2pool nodes, but I wanted to have at least one that I knew was using the changes I worked on for the alt coin pools. Smiley
legendary
Activity: 3430
Merit: 3080
Is anyone having issues with Bitcoind/Bitcoin-QT v0.9.0?

I updated my P2Pool to v0.9.0 and I am getting an error now after restarting the server that it's lost connection with Bitcoind. Yet up until this point, I upgraded it a day after v0.9.0 came out its been running fine and transactions have been flowing.


It's working fine. I only notice that bitcoind getblock latency grows from 300ms to 800 - 900 ms.

default -minblocksize has been altered to 750,000, if you override that you'll maybe get your lower latency back
newbie
Activity: 4
Merit: 0
Is anyone having issues with Bitcoind/Bitcoin-QT v0.9.0?

I updated my P2Pool to v0.9.0 and I am getting an error now after restarting the server that it's lost connection with Bitcoind. Yet up until this point, I upgraded it a day after v0.9.0 came out its been running fine and transactions have been flowing.


It's working fine. I only notice that bitcoind getblock latency grows from 300ms to 800 - 900 ms.
legendary
Activity: 1540
Merit: 1001
Is anyone having issues with Bitcoind/Bitcoin-QT v0.9.0?

I updated my P2Pool to v0.9.0 and I am getting an error now after restarting the server that it's lost connection with Bitcoind. Yet up until this point, I upgraded it a day after v0.9.0 came out its been running fine and transactions have been flowing.

working fine here.

M
hero member
Activity: 630
Merit: 501
Is anyone having issues with Bitcoind/Bitcoin-QT v0.9.0?

I updated my P2Pool to v0.9.0 and I am getting an error now after restarting the server that it's lost connection with Bitcoind. Yet up until this point, I upgraded it a day after v0.9.0 came out its been running fine and transactions have been flowing.

 

legendary
Activity: 1540
Merit: 1001
You can imagine how big this effect becomes between a large mining farm and a single 180GH Antminer S1 miner.

TLDR: Sorry.

Thanks for the detailed explanation.  I knew most of it already.  The part I definitely didn't know what that share size on the alt chain does matter.  That was one of the conclusions I'd reached when thinking about how changing your pseudo share size could help things.

M
sr. member
Activity: 434
Merit: 250
To elaborate on my illogical response...

things that don't make sense to me:

- You adjust your pseudo share size to adjust the share difficulty on the pool.  That means you get less shares, which means you keep less payout.  Why do that?  Wouldn't you be better off pointing 10% of your hashrate to the pool and the other 90% somewhere else?

Pseudo share size has nothing to do with share difficulty on the pool or payouts. Those are the little Accept X/Y your miner prints every few seconds or so. It just lets the pool know you are alive and plot graphs, etc. To actually get paid you need to submit a share where the difficulty is higher than the minimum share difficulty for the pool (or higher than your share difficulty target if it's higher than the minimum).

To get more or less pseudo shares you use +. For instance, if you run a local node you could spam it with tiny shares by adding +1 to your payment address. Has no effect on income.

To get more or less real shares you use /. Any /DIFF set below the network minimum is ignored, and your target diff to get on share chain is the minimum. Any /DIFF higher than the network minimum means you'll get on the share chain less often, but the shares are more valuable (your target share difficulty is the value stored to calculate your payout, all shares on share chain do not have equal value, maybe this is your confusion?). The maximum target diff you can set is 30x the network minimum. If you set it higher, it'll adjust every time the minimum target adjusts.

- Unless sharesize matters when you adjust your pseudo share size.  If sharesize does matter, then how can it not affect the share difficulty if 6 small shares an hour or one large share every hour has the same result?

In a normal PPLNS pool, the payment window is all of the shares of work submitted (what p2pool calls pseudo shares), and there is only one type of share. All work submitted is logged and is in the payment window for a certain amount of work N. N is usually some multiple of the coin's difficulty.

In p2pool, the shares a normal pool would count are ignored and only shares of a certain difficulty target are counted. This is because there's no central database to record all of the info. We're decentralized and nodes can't trust each other. So the payment database, instead of being MySQL or similar, becomes a bitcoin style block chain, we call it the share chain. The share chain contents are the PPLNS payment database for p2pool. It has a specific target speed just like bitcoin. The share chain target is 30 seconds. The minimum difficulty to get a share onto the share chain is adjusted similar to bitcoin's difficulty algorithm, so the share chain grows by one share (block) every 30 seconds.

So the minimum difficulty for shares to get on the share chain is soley a factor of time. How long is it taking to add shares to the share chain? Just like bitcoin itself, the diff adjusts so the target rate is hit. The contents of the shares on the share chain don't matter.

(Side note: It's endlessly confusing that "share" might be a pseudo share, a real paying share that is recorded on share chain, or a "block" on the p2pool sharechain.)

So your question above, if you are talking about pseudo shares, it doesn't matter.

The issue is that the share chain has a finite number of "shares" (blocks) available. When it's full the older ones are expired. Even if the luck has been bad pool hash rate has dropped to where we aren't finding N work before the share chain length maximum is hit, opposed to the PPLNS window on a normal pool would still be counting those shares. This is a separate issue/concern of mine.

To even get on the share chain at all, you need to find a share (in your client) with a difficulty that is higher than your target share difficulty. I'll give an example below.

- If the time between shares affects share difficulty, that means it's based entirely on luck!  If the pool hashrate remains constant, and luck goes sour and people start getting less shares, does the share difficulty suddenly decrease?  Likewise, if luck increases, does share difficulty increase?

Yes, and this is exactly how bitcoin works as well. Difficulty is set based on actual shares found. Real hash rate doesn't matter, it's only the observed hash rate from actual found blocks that changes the difficulty target.

So why bother with /DIFF and using a higher one than you need to? Take a small coin with a p2pool share chain target of 1 minute, and two miners:

If Miner A has speed S and miner B has speed 99*S and are both mining on the p2pool node with the same difficulty targets (for simplicity ignore there being a vardiff, say both miners use /1, and the minimum share diff is a tiny number like on a scrypt coin) then A will find 1 out of every 100 shares and B will find 99 out of every 100 shares in some period of time. We want to find one share per minute for the share chain, so the minimum network difficulty will adjust so that shares come in once a minute. Thus, miner A finds 1 share every 100 minutes and miner B finds 99 shares every 100 minutes. As the p2pool network speed (or just miner B's speed) grows, miner A's rate of finding shares will get slower and slower. These shares all have a value of "1" for our example.

If a paying block is found when we have these 100 shares in the share chain:
Miner A is paid 1 * 1 / 100 of the block reward = 1%
Miner B is paid 99 * 1 / 100 of the block reward = 99%

(The math is # of shares * value of shares / total share chain value, assuming each of a miner's shares have same value, it's actually a sum of each individual share value in practice.)

Now let's say miner B feels bad for miner A and wants to help reduce his variance on when he gets paid. So whereas miner A might be mining diff 1 shares, miner B is going to change over to ADDR/99 to mine diff 99 shares. There is a 30x cap, so p2pool will actually make /30 the maximum it will use.

Now, in a specific period of time, miner A will find S shares  and miner B will find 3.3*S shares (since his diff target is 30 times higher vs his 99x faster speed). With the network difficulty from up above, A is still finding 1 share every 100 minutes but now B is only finding 3.3 shares every 100 minutes instead of slightly less than 1 per minute. This means the share chain is way too slow. Network minimum difficulty will drop until both miners combined average 1 share per minute. Once adjusted, in this 100 minutes of much lower minimum difficulty, miner A will find about 23.25 shares for every 76.75 shares miner B finds (please pretend we can have fractional shares to make the math clean).

The advantage is that miner A now finds shares on the share chain every 4.3 minutes (100/23.25) instead of one every 100 minutes. You can see that's a massive drop in his wait time. Miner B only goes up to about 1.3 minutes per paying share. What happens to the payouts?

The total value of shares on the share chain are 23.25 * 1 + 76.75 * 30 = 2325.75

Miner A is paid 23.25 * 1 / 2325.75 of the block reward = 1%
Miner B is paid 76.75 * 30 / 2325.75 of the block reward = 99%

So the end result is that Miner B can increase his time per share from 1.01 minutes to 1.3 minutes in order to let Miner A drop his time per share from about 100 minutes to 4.3 minutes. The miners still make the same average income over time.

You can imagine how big this effect becomes between a large mining farm and a single 180GH Antminer S1 miner.

TLDR: Sorry.
hero member
Activity: 924
Merit: 1000
Watch out for the "Neg-Rep-Dogie-Police".....
Hey how long are the p2pool shares valid for?

3 days I think  Smiley
legendary
Activity: 1540
Merit: 1001
I don't see a reason to use any parms to control the pseudo difficulty sent to your miner.  I'm pretty sure that has zero affect on the pool share difficulty, that is, the minimum share difficulty required to get on the p2pool altchain.  I agree with you, with vardiff, chances are you'll get bumped up to a decent pseudo share size pretty quickly.

I'm talking about using / which adjusts actual share difficulty to find shares on the sharechain, not + which are the pseudo shares. It does have a rather huge impact on small networks. If I go the opposite way and mine with /1 so I'm always at the minimum, I can skyrocket the minimum share difficulty for the network. Noticing my impact on it when I first set the node up is what prompted me to always max out when I join in.

really?  that's interesting.  I would think the min sharechain difficulty is based on bandwidth or size of shares.. not time of shares. 

that's a bit illogical imho.

To elaborate on my illogical response...

things that don't make sense to me:

- You adjust your pseudo share size to adjust the share difficulty on the pool.  That means you get less shares, which means you keep less payout.  Why do that?  Wouldn't you be better off pointing 10% of your hashrate to the pool and the other 90% somewhere else?
- Unless sharesize matters when you adjust your pseudo share size.  If sharesize does matter, then how can it not affect the share difficulty if 6 small shares an hour or one large share every hour has the same result?
- If the time between shares affects share difficulty, that means it's based entirely on luck!  If the pool hashrate remains constant, and luck goes sour and people start getting less shares, does the share difficulty suddenly decrease?  Likewise, if luck increases, does share difficulty increase?

It makes more sense for it to be based on pool hashrate.

Maybe I'm missing something...

M
sr. member
Activity: 476
Merit: 250
Hey how long are the p2pool shares valid for?
legendary
Activity: 1540
Merit: 1001
I don't see a reason to use any parms to control the pseudo difficulty sent to your miner.  I'm pretty sure that has zero affect on the pool share difficulty, that is, the minimum share difficulty required to get on the p2pool altchain.  I agree with you, with vardiff, chances are you'll get bumped up to a decent pseudo share size pretty quickly.

M

I'm talking about using / which adjusts actual share difficulty to find shares on the sharechain, not + which are the pseudo shares. It does have a rather huge impact on small networks. If I go the opposite way and mine with /1 so I'm always at the minimum, I can skyrocket the minimum share difficulty for the network. Noticing my impact on it when I first set the node up is what prompted me to always max out when I join in.

really?  that's interesting.  I would think the min sharechain difficulty is based on bandwidth or size of shares.. not time of shares. 

that's a bit illogical imho.

M
sr. member
Activity: 434
Merit: 250
So a followup to my earlier questions. From IRC forrestv mentioned the 30x cap is a hard coded limit that other nodes will enforce, so I can't raise it on my own node without the other nodes rejecting my shares. Also, the dust threshold works so that each share you find will be above the dust threshold, not based on the average actual payout you'll receive based on average # of shares in chain.
sr. member
Activity: 434
Merit: 250
I don't see a reason to use any parms to control the pseudo difficulty sent to your miner.  I'm pretty sure that has zero affect on the pool share difficulty, that is, the minimum share difficulty required to get on the p2pool altchain.  I agree with you, with vardiff, chances are you'll get bumped up to a decent pseudo share size pretty quickly.

M

I'm talking about using / which adjusts actual share difficulty to find shares on the sharechain, not + which are the pseudo shares. It does have a rather huge impact on small networks. If I go the opposite way and mine with /1 so I'm always at the minimum, I can skyrocket the minimum share difficulty for the network. Noticing my impact on it when I first set the node up is what prompted me to always max out when I join in.
legendary
Activity: 1540
Merit: 1001
So I was working on one of my nodes this evening. I noticed the share targets were massively too high on my Uno node. Turns out dust threshold was set too high for a coin with a (currently) .25 block reward. I tweaked that around some so it was working better, and noticed some other things. This is my node:

http://us-east.royalminingco.com:9655/static/

I'm mining with about 1/4 of the pool power, I have my diff set to /64000 to help keep minimum diff low for smaller miners. The smaller guys on there at the moment as I post, from my debugging related to the dust issue, have their share targets set to hit a share every 30 minutes. I've seen from other alt coins that the networks.py settings basically end up where miners all get set (if they have the hash power to avoid dust issues) to the same time per share. I've never researched what formula feeds that yet though.

With the 30x diff cap, my target drops to a small fraction of the /64000. I'm finding, at the moment, 1 share per 1.5 minutes. The pool will find a block once per hour. So I'll have about 40 shares in the time we find a block. The other guys will have about 2 shares.

I'm wondering a couple things. Is there any problems if I just up or remove the 30x cap on my node? I honestly wouldn't find averaging 2-3 shares per block like the other smaller miners, which would reduce the minimum share diff even further. In fact, if I remove the cap totally, I bet the vardiff would target me to 30 minutes per share just like the other 2 miners, meaning I'd be about 20x higher diff target than I am right now (which is 30x cap, so about 600x the minimum difficulty). I'm curious why that cap was put into place originally.

Also, when the expected value is generated for the dust calculations, is that based on the expected value of a single share or the expected value of the average # of shares a miners will have at any given time in the share chain? I assume the latter, just want to be sure without reverse engineering how the math is calculated in that function. Wink

I don't see a reason to use any parms to control the pseudo difficulty sent to your miner.  I'm pretty sure that has zero affect on the pool share difficulty, that is, the minimum share difficulty required to get on the p2pool altchain.  I agree with you, with vardiff, chances are you'll get bumped up to a decent pseudo share size pretty quickly.

M
sr. member
Activity: 434
Merit: 250
So I was working on one of my nodes this evening. I noticed the share targets were massively too high on my Uno node. Turns out dust threshold was set too high for a coin with a (currently) .25 block reward. I tweaked that around some so it was working better, and noticed some other things. This is my node:

http://us-east.royalminingco.com:9655/static/

I'm mining with about 1/4 of the pool power, I have my diff set to /64000 to help keep minimum diff low for smaller miners. The smaller guys on there at the moment as I post, from my debugging related to the dust issue, have their share targets set to hit a share every 30 minutes. I've seen from other alt coins that the networks.py settings basically end up where miners all get set (if they have the hash power to avoid dust issues) to the same time per share. I've never researched what formula feeds that yet though.

With the 30x diff cap, my target drops to a small fraction of the /64000. I'm finding, at the moment, 1 share per 1.5 minutes. The pool will find a block once per hour. So I'll have about 40 shares in the time we find a block. The other guys will have about 2 shares.

I'm wondering a couple things. Is there any problems if I just up or remove the 30x cap on my node? I honestly wouldn't mind averaging 2-3 shares per block like the other smaller miners, which would reduce the minimum share diff even further. In fact, if I remove the cap totally, I bet the vardiff would target me to 30 minutes per share just like the other 2 miners, meaning I'd be about 20x higher diff target than I am right now (which is 30x cap, so about 600x the minimum difficulty). I'm curious why that cap was put into place originally.

Also, when the expected value is generated for the dust calculations, is that based on the expected value of a single share or the expected value of the average # of shares a miners will have at any given time in the share chain? I assume the latter, just want to be sure without reverse engineering how the math is calculated in that function. Wink
Jump to: