Author

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 190. (Read 2591920 times)

sr. member
Activity: 434
Merit: 250
So basically what you are saying is that people with more mining power will be happy to have MUCH higher difficulty than p2pool's already very high difficulty, so that others can have lower difficulty, and thus people with more mining power will also have higher variance than most people mining on p2pool so the others can all have lower variance?
Unfortunately "people will do what is best for others at their own detriment" usually doesn't pan out too well.

Yeah maybe I'm being naïve. Don't know. It just doesn't seem like the largest miners need to have potentially 150 shares in the share chain at a given time, when they could have 15 shares worth 10x as much instead. There isn't a real 'detriment' here as you'll still earn the same amount of income on average.

Also a big miner shouldn't view p2pool share difficulty as "very high".

I don't have the math to come up with a confidence interval on the # of shares someone will have in the share chain for a given hash power off the top of my head. However, for sake of example, lets say the huge miner with 150 share on average has high confidence to be in the 130-170 range. Likewise, if his diff target was ten times higher, he'd be 13-17 with 15 on average. If each share of normal difficulty paid out .01 BTC for simplicity (.1 BTC for the 10x higher target shares), then his income averages 150 * .01 or 15 * .1, 1.5 BTC either way. The question is if the steps within this range are steps of .01 BTC each or .1 BTC each.

When I used to run sha256 alt coin p2pool nodes and would mine on them with my S1, I'd set /DIFF as high as I could to not shut out GPU miners. I know, few people likely do that in the p2pool BTC world, which is why it makes more sense for p2pool to automatically scale the vardiff beyond just minimizing network traffic and be smart regarding the fact the sharechain is a fixed size.

I guess my point is the # of shares in the sharechain is a resource that needs to be managed as intelligently as possible for the health of the network. For each big miner with 100+ shares in the chain at any one time, there will be far more miners struggling to keep just 1 share active at all times.

Edit: On a sidenote I sent an email to windpath with various old links and github patches in case he wants to apply some nice front end improvements from last year. It seems p2pool-node-status went quiet before they moved from -devel to the normal branch, and the alt coins that also used the interface all faded away. So the cool improvements to show things like estimated time per share and miner-specific difficulty targets have been forgotten to the sands of time. Smiley
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...
Of course it has the problem of abuse by larger miners splitting up their 'farm' to multiple addresses and thus acting like multiple smaller miners - so that would have to be resolved some way I've got no idea how Smiley

It wouldn't be in a large miner's financial interest to push out/hurt smaller miners. Just makes p2pool network speed lower for everyone. Granted, the community is full of vandals though.

Edit: Correction, the hard code in p2pool was to make a miner's minimum difficulty target even if they said /1 to be such that they can only find 1.67% of the total share chain's shares. This is far too many shares for 1 miner to have, of course...

So basically what you are saying is that people with more mining power will be happy to have MUCH higher difficulty than p2pool's already very high difficulty, so that others can have lower difficulty, and thus people with more mining power will also have higher variance than most people mining on p2pool so the others can all have lower variance?
Unfortunately "people will do what is best for others at their own detriment" usually doesn't pan out too well.

In fact there's a very obvious example with the recent blocks where 2 were mined by people who put almost no transactions in them.
No transactions confirmed is bad for bitcoin in general, but anyone short sighted may well only care about the BTC reward and not the bitcoin network ... or p2pool's reputation.
legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
Very excited about the blocks we have been finding lately...

...img snipped...

Not so excited about the frequency of 0 transaction blocks....

If your "that guy" please PM me, I'll keep your identity private, and may be able to help you make your node more efficient and include some transactions in your blocks as well.

0 transaction blocks reflect poorly on our community, and with current transaction volume slow down the whole network.

Do you mean zero transaction blocks bring no extra transaction fee with block reward when it is found? and will the transaction fee be shared to all miners or only block founder?
How do you insert transactions in blocks? any reference to shed some light about this concept?
Zero transaction blocks (well, technically they're reported as 1 transaction because of the block reward distribution) have zero transaction fees.  How could they have any fees since there are no transactions in the block?  In a block that actually does have transactions - like that last one p2pool found 353366 - there were 0.02990026BTC of fees.  Those fees are combined with the 25BTC block reward and distributed to all miners according to the shares the miners have on the payout list.

The transactions are inserted into the block by the node on which you're mining.  Each node runs its own copy of the Bitcoin Core.  If a node has setup their configuration to do something like setting a max block size of 10kb, or their mintxfee to something like 100BTC, then if they happen to find a share that satisfies the network difficulty the chances are exceptionally good the block will contain no transactions other than the block reward.
full member
Activity: 706
Merit: 100
Very excited about the blocks we have been finding lately...



Not so excited about the frequency of 0 transaction blocks....

If your "that guy" please PM me, I'll keep your identity private, and may be able to help you make your node more efficient and include some transactions in your blocks as well.

0 transaction blocks reflect poorly on our community, and with current transaction volume slow down the whole network.

Do you mean zero transaction blocks bring no extra transaction fee with block reward when it is found? and will the transaction fee be shared to all miners or only block founder?
How do you insert transactions in blocks? any reference to shed some light about this concept?
sr. member
Activity: 434
Merit: 250
Maybe p2pool could consider some (more complex) version of that in the share chain to force miners to use a diff related to their hash rate and thus allow smaller miners to reduce their variance.

Been looking around the past month of posts and saw this. I think I recall a conversation once that it'd be nice to scale the difficulty target for miners to target some preferred # of shares in the sharechain. It does this already in some regard as I think there's a cap the target for a hash rate can't be so low that you'll find more than 5% of the full share chain's % of shares on average. (Going by memory and haven't touched p2pool since last April.) However that's realistically way too high. For low variance payouts for a large miner, you only need to keep a few shares in the chain at all times. This is a reason "friendly" large miners could use /DIFF to kick their difficulty targets way higher... they'd have less shares in the share chain (leaving more room for smaller people and reducing their variance), but the shares they do have are worth that much more during payout.

Actually I used google and spent some time trying to find what I'm thinking of. I did. It's here, the bottom portion of this post:

https://bitcointalksearch.org/topic/m.4830990

with small followup:

https://bitcointalksearch.org/topic/m.4831519

What I provided was a different way to calculate target difficulties for large miners such that they'd always have shares in the sharechain (amount of payment would vary, but they'd almost never miss a payment totally). For large miners it might drop the # of shares in the sharechain they were gobbling up by a factor of 10-15. Which means the sharechain difficulty is lower, making it easier for smaller miners to find shares.

Of course it has the problem of abuse by larger miners splitting up their 'farm' to multiple addresses and thus acting like multiple smaller miners - so that would have to be resolved some way I've got no idea how Smiley

It wouldn't be in a large miner's financial interest to push out/hurt smaller miners. Just makes p2pool network speed lower for everyone. Granted, the community is full of vandals though.

Edit: Correction, the hard code in p2pool was to make a miner's minimum difficulty target even if they said /1 to be such that they can only find 1.67% of the total share chain's shares. This is far too many shares for 1 miner to have, of course...
sr. member
Activity: 434
Merit: 250
I believe there's also work being done to use the proxy to be able to balance miner hashrate.  The idea being there to try to mitigate the effects of "swamping" a node if a large miner comes onto a node with a lot of smaller miners, they could be proxied to a different node where their hashrate is more appropriate.

I've been away a year so maybe you are referring to something else, but if you are talking about small miners having their difficulty targets skyrocket when a large miner joins the same public p2pool node, that is avoided with this pull request:

https://github.com/forrestv/p2pool/pull/174

Basically, it assigns difficulty targets to miners based on their payment address instead of the total hash rate of the local node. This way the difficulty target your miner is working on is identical to running your own private node, even if some other far larger miner is on the same public node with you. This is for the true difficulty target to get shares, a different pull request was submitted related to pseduo-shares.

Just been checking out ckpool and ckproxy, I think those came out after I left, quite cool. I'd have been all over ckproxy if I knew about it back then. I attempted to learn enough Haskell to run proxypool with p2pool for BTC mining, but it wasn't worth the pain. Smiley
sr. member
Activity: 434
Merit: 250
I may sound a little dumb for writing this.... with the p2p pool are you your own "private" pool e.g you keep the entire 25BTC apart from a little fee or is it a very big mining pool and you are supporting the pool by having your own node close to you? Im guess then the 25BTC would be then shared?

Thanks

The 25BTC is shared throughout all of the p2pool network. It's like a distributed pool, where you mine off your own node (or a public node) with the advantages that can give you (zero latency if you run it local to your miners) but you also benefit from sharing hash power with the rest of the p2pool network to reduce variance.
member
Activity: 103
Merit: 10
I may sound a little dumb for writing this.... with the p2p pool are you your own "private" pool e.g you keep the entire 25BTC apart from a little fee or is it a very big mining pool and you are supporting the pool by having your own node close to you? Im guess then the 25BTC would be then shared?

Thanks
sr. member
Activity: 507
Merit: 253
Code:
Local: 1946GH/s in last 5.8 minutes Local dead on arrival: ~37.2% (35-39%) Expected time to share: 4.0 hours
Why is my "Local dead on arrival" so high?

Update: It just now improved:
Code:
Local: 2017GH/s in last 10.0 minutes Local dead on arrival: ~1.8% (1-4%) Expected time to share: 3.8 hours
Was it just "adjusting" or something?
member
Activity: 103
Merit: 10
Link to the P2Pool Node list?
legendary
Activity: 1258
Merit: 1027
Very excited about the blocks we have been finding lately...



Not so excited about the frequency of 0 transaction blocks....

If your "that guy" please PM me, I'll keep your identity private, and may be able to help you make your node more efficient and include some transactions in your blocks as well.

0 transaction blocks reflect poorly on our community, and with current transaction volume slow down the whole network.
donator
Activity: 2058
Merit: 1007
Poor impulse control.
...
I guess also there needs to be clearly stated that anti-hop functionality is deployed on some pools - slush for example - that means you will not be paid your fair share if you disconnect from the pool either on purpose or due to an outage.

The expected loss is related to the amount of time you disconnect vs the % of the average block find time that the pool pays (whatever that may be)
If you simply just disconnected and don't come back, then it would be related to a % of the average expected loss.

You could compare that to a 'lock-in contract' where you lose out by leaving, except you don't gain much by staying.
There is possibly a very small gain when others lose out by leaving - the amount lost by anyone who disconnected may be distributed to everyone who didn't - so if the pool is 5PH and someone with 5TH lost 100% of their payout, everyone might gain about 0.1% more on that particular payout on their expected 100% less orphans less fees ... of course that depends on how the payout is calculated.

I'm not sure anyone has proven that to be the case, Kano. From your description you could as easily be talking about PPLNS with a very small N, which is still a fair reward method, and for any fair system expected value of a share should always be B/D.

The expected value of shares submitted over a period of time including disconnections should be the sum of the shares submitted.

Well just to use the simplest argument based on one assumption: that the pool's anti-hop deterrent does work:
The expected payout is E if you mine 24/7
The expected payout if you hop is expected to be E-e lower by some 'e' if the hopping deterrent works.
So E-e is less than E if e>0 Smiley


Bah! Humbug! Redefining expectation, that's cheating.

Moreover, that's what I said when I caught Bitclockers out. You shouldn't use a man's words against him.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...
I guess also there needs to be clearly stated that anti-hop functionality is deployed on some pools - slush for example - that means you will not be paid your fair share if you disconnect from the pool either on purpose or due to an outage.

The expected loss is related to the amount of time you disconnect vs the % of the average block find time that the pool pays (whatever that may be)
If you simply just disconnected and don't come back, then it would be related to a % of the average expected loss.

You could compare that to a 'lock-in contract' where you lose out by leaving, except you don't gain much by staying.
There is possibly a very small gain when others lose out by leaving - the amount lost by anyone who disconnected may be distributed to everyone who didn't - so if the pool is 5PH and someone with 5TH lost 100% of their payout, everyone might gain about 0.1% more on that particular payout on their expected 100% less orphans less fees ... of course that depends on how the payout is calculated.

I'm not sure anyone has proven that to be the case, Kano. From your description you could as easily be talking about PPLNS with a very small N, which is still a fair reward method, and for any fair system expected value of a share should always be B/D.

The expected value of shares submitted over a period of time including disconnections should be the sum of the shares submitted.

Well just to use the simplest argument based on one assumption: that the pool's anti-hop deterrent does work:
The expected payout is E if you mine 24/7
The expected payout if you hop is expected to be E-e lower by some 'e' if the hopping deterrent works.
So E-e is less than E if e>0 Smiley
donator
Activity: 2058
Merit: 1007
Poor impulse control.
The title of this thread includes "Hop-Proof". While I expect that is really just hyperbole, I wondered what it would mean to be "Hop Proof"? Contracts with the pool members to never use another pool? Kind of like getting married?  Smiley

I don't think anyone ever properly answered your question so I'll give it a shot.

In this context, "hop" is a technical term related to ways you can exploit pools running certain older payment methods by changing between pools in a way that will increase your income at the expense of other miners on those pools who remain loyal to the pool and stay there all of the time. Eventually some alternate payment methods were developed which were hop-proof in the sense a miner can't enter/leave the pool in a systematic way in order to boost their earnings. The most popular is PPLNS, although to me the most interesting is DGM (by Meni Rosenfeld). (I ran a Terracoin pool for over a year that used DGM.)

So when p2pool says it is Hop-proof, it means you can't abuse the payment method. It doesn't mean people can't or won't still use multiple pools or move around to chase "luck".

If you are interested in more information, read Analysis of Bitcoin Pooled Mining Reward Systems.

Nice explanation, roy7. Someone should bookmark this post.
I guess also there needs to be clearly stated that anti-hop functionality is deployed on some pools - slush for example - that means you will not be paid your fair share if you disconnect from the pool either on purpose or due to an outage.

The expected loss is related to the amount of time you disconnect vs the % of the average block find time that the pool pays (whatever that may be)
If you simply just disconnected and don't come back, then it would be related to a % of the average expected loss.

You could compare that to a 'lock-in contract' where you lose out by leaving, except you don't gain much by staying.
There is possibly a very small gain when others lose out by leaving - the amount lost by anyone who disconnected may be distributed to everyone who didn't - so if the pool is 5PH and someone with 5TH lost 100% of their payout, everyone might gain about 0.1% more on that particular payout on their expected 100% less orphans less fees ... of course that depends on how the payout is calculated.

I'm not sure anyone has proven that to be the case, Kano. From your description you could as easily be talking about PPLNS with a very small N, which is still a fair reward method, and for any fair system expected value of a share should always be B/D.

The expected value of shares submitted over a period of time including disconnections should be the sum of the shares submitted.

legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
The title of this thread includes "Hop-Proof". While I expect that is really just hyperbole, I wondered what it would mean to be "Hop Proof"? Contracts with the pool members to never use another pool? Kind of like getting married?  Smiley

I don't think anyone ever properly answered your question so I'll give it a shot.

In this context, "hop" is a technical term related to ways you can exploit pools running certain older payment methods by changing between pools in a way that will increase your income at the expense of other miners on those pools who remain loyal to the pool and stay there all of the time. Eventually some alternate payment methods were developed which were hop-proof in the sense a miner can't enter/leave the pool in a systematic way in order to boost their earnings. The most popular is PPLNS, although to me the most interesting is DGM (by Meni Rosenfeld). (I ran a Terracoin pool for over a year that used DGM.)

So when p2pool says it is Hop-proof, it means you can't abuse the payment method. It doesn't mean people can't or won't still use multiple pools or move around to chase "luck".

If you are interested in more information, read Analysis of Bitcoin Pooled Mining Reward Systems.

Nice explanation, roy7. Someone should bookmark this post.
I guess also there needs to be clearly stated that anti-hop functionality is deployed on some pools - slush for example - that means you will not be paid your fair share if you disconnect from the pool either on purpose or due to an outage.

The expected loss is related to the amount of time you disconnect vs the % of the average block find time that the pool pays (whatever that may be)
If you simply just disconnected and don't come back, then it would be related to a % of the average expected loss.

You could compare that to a 'lock-in contract' where you lose out by leaving, except you don't gain much by staying.
There is possibly a very small gain when others lose out by leaving - the amount lost by anyone who disconnected may be distributed to everyone who didn't - so if the pool is 5PH and someone with 5TH lost 100% of their payout, everyone might gain about 0.1% more on that particular payout on their expected 100% less orphans less fees ... of course that depends on how the payout is calculated.
full member
Activity: 238
Merit: 100
Man.  Anyone have any idea what people are renting SHA256 rigs to mine.  Mining rental charges have gone through the roof and no matter what I change my rate to, they rent them.

New hot coin out there, or something?

I think there's been at least 2 SHA256 coins released over the past week.  Over the weekend my SPTech rentals reported over 3000 blocks found, so it sure as heck ain't Bitcoin...

Whatever it is, I'm fine if they keep it going, I'm getting double on my rentals and everything I have has been rented solid.

For sure... just curious, though.  If anyone finds out, post some info.


I think it is Californium
donator
Activity: 2058
Merit: 1007
Poor impulse control.
The title of this thread includes "Hop-Proof". While I expect that is really just hyperbole, I wondered what it would mean to be "Hop Proof"? Contracts with the pool members to never use another pool? Kind of like getting married?  Smiley

I don't think anyone ever properly answered your question so I'll give it a shot.

In this context, "hop" is a technical term related to ways you can exploit pools running certain older payment methods by changing between pools in a way that will increase your income at the expense of other miners on those pools who remain loyal to the pool and stay there all of the time. Eventually some alternate payment methods were developed which were hop-proof in the sense a miner can't enter/leave the pool in a systematic way in order to boost their earnings. The most popular is PPLNS, although to me the most interesting is DGM (by Meni Rosenfeld). (I ran a Terracoin pool for over a year that used DGM.)

So when p2pool says it is Hop-proof, it means you can't abuse the payment method. It doesn't mean people can't or won't still use multiple pools or move around to chase "luck".

If you are interested in more information, read Analysis of Bitcoin Pooled Mining Reward Systems.


Nice explanation, roy7. Someone should bookmark this post.
sr. member
Activity: 434
Merit: 250
The title of this thread includes "Hop-Proof". While I expect that is really just hyperbole, I wondered what it would mean to be "Hop Proof"? Contracts with the pool members to never use another pool? Kind of like getting married?  Smiley

I don't think anyone ever properly answered your question so I'll give it a shot.

In this context, "hop" is a technical term related to ways you can exploit pools running certain older payment methods by changing between pools in a way that will increase your income at the expense of other miners on those pools who remain loyal to the pool and stay there all of the time. Eventually some alternate payment methods were developed which were hop-proof in the sense a miner can't enter/leave the pool in a systematic way in order to boost their earnings. The most popular is PPLNS, although to me the most interesting is DGM (by Meni Rosenfeld). (I ran a Terracoin pool for over a year that used DGM.)

So when p2pool says it is Hop-proof, it means you can't abuse the payment method. It doesn't mean people can't or won't still use multiple pools or move around to chase "luck".

If you are interested in more information, read Analysis of Bitcoin Pooled Mining Reward Systems.
sr. member
Activity: 434
Merit: 250
That's why virtually every single p2pool UI out there doesn't show this block - they all base block finds off of accepted p2pool share chain shares.  The exception is windpath's node - he scrapes the blockchain data to get the p2pool blocks regardless of whether the share that solved it is on the p2pool share chain.

Ahhhhhhh. That finally solves the mystery of "missing" blocks on alt coins with rapid blockchains.
sr. member
Activity: 507
Merit: 253
I've been with P2Pool since April 5, and have had 8 payouts so far. Each successive payout has been more than the previous one. Is this what's meant that P2Pool is resistant to pool hopping, or is the fact that my payouts are monotonically increasing just a coincidence? (There have been decreasing numbers of P2Pool miners, thus making the payouts larger for those who stick with P2Pool.)
The more shares you have in the chain, the higher your payout.  That could be because of better luck, or the fact that hashrate has been leaving (or both), thus you are able to get more shares that way.

Nothing is "hop-proof".  But, in this case PPLNS is the next best thing.  By design there's a ramp-up period (3 days in this case) where you don't reach full block payout potential until after you've been mining for 3 days.  If you know you've got to put in that much work before you see your "proper" payouts, you're less likely to leave in-between to seek luck elsewhere, lest you've lost most of your past 3 days' worth of work.

From April 5th to April 8th you would have seen payouts increasing since that was your ramp-up period.  Then in the past 12 days we've gone from 304 miners (4/8) down to 267 miners with the most recent block today (woo, another one 2 hours ago!), so in theory you would see earnings increase there as well since there are less shares from the other miners competing with yours.
Yes, I figured the monotonic increase was due, mostly, to a coincidence.
Jump to: