Author

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 124. (Read 2591920 times)

legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
could this be the cause of the recent bad luck?

No.
His question was 'could'
The answer is 'Yes'

Any lost blocks will attribute to a lower 'pool luck'.
It's a lost block, it affects what is (incorrectly) referred to as 'pool luck'
So if there have been any (which no one will actually know) then it would have effected the 'p2pool luck'

Well, I 'could' manually solve the next two blocks in a row by typing in hashes.  However, I wouldn't go around saying that to people who might not understand the odds.
Any lost blocks affects 'pool luck'.
Mining with Bitmain firmware versions can lose blocks on p2pool.
Anyone mining with such hardware with this problem is risking losing blocks on p2pool.

Now, if instead you are implying that losing such blocks is rare ... you are mistaken.
95% of stale share blocks on p2pool are valid bitcoin network blocks.
That is the reason why forrestv changed p2pool to submit stale share blocks to bitcoind when we pointed out the problem to him.

How often do these stale share blocks occur?
At least as often as any p2pool blocks that don't appear in the p2pool share chain (Edit: these are the stale share blocks)
The actual number is higher, since as I stated, 95% of stale p2pool share blocks are valid bitcoin network blocks.
It would be even higher if people were mining with bitmain firmware versions throwing away these stale share blocks that are valid bitcoin blocks.
legendary
Activity: 1257
Merit: 1004
pool.sexy
Well, I 'could' manually solve the next two blocks in a row by typing in hashes.  However, I wouldn't go around saying that to people who might not understand the odds.

I know the odds, but here the problem is the antminer software and need to solve it.

could this be the cause of the recent bad luck?

No.

Can you explain please

By the way... what is your magic number?  You state, "...it would cost you a fortune to pay me...".  Define fortune Smiley.
Writing in c is very labour intensive (compared to python) so it's a lot of hours work. I charge less for free software than private software since I like to encourage free software use as much as possible. At current exchange rates I charge .29 btc/hr for free soft and .38 for private software. A rough estimate of the time required to rewrite p2pool in c (properly) would be 500 hours work. That's around 150 BTC.

EDIT: By the way this would be a total and utter massive waste of money. Even if all the scalability problems were addressed AND a solution was found for the variance problem, there are no miners left to attract. The sad truth is personal miners are an increasingly irrelevant entity and what little hashrate they have cannot save any small pool. Only farms are relevant today.

I am a small miner (3x s5) what should I do? abandon p2pool??!! I hope for a solution ... I will offer 0.29btc for one of your hours when and if it will be necessary, p2pool must not die..
donator
Activity: 4760
Merit: 4323
Leading Crypto Sports Betting & Casino Platform
could this be the cause of the recent bad luck?

No.
His question was 'could'
The answer is 'Yes'

Any lost blocks will attribute to a lower 'pool luck'.
It's a lost block, it affects what is (incorrectly) referred to as 'pool luck'
So if there have been any (which no one will actually know) then it would have effected the 'p2pool luck'

Well, I 'could' manually solve the next two blocks in a row by typing in hashes.  However, I wouldn't go around saying that to people who might not understand the odds.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
could this be the cause of the recent bad luck?

No.
His question was 'could'
The answer is 'Yes'

Any lost blocks will attribute to a lower 'pool luck'.
It's a lost block, it affects what is (incorrectly) referred to as 'pool luck'
So if there have been any (which no one will actually know) then it would have effected the 'p2pool luck'
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
By the way... what is your magic number?  You state, "...it would cost you a fortune to pay me...".  Define fortune Smiley.
Writing in c is very labour intensive (compared to python) so it's a lot of hours work. I charge less for free software than private software since I like to encourage free software use as much as possible. At current exchange rates I charge .29 btc/hr for free soft and .38 for private software. A rough estimate of the time required to rewrite p2pool in c (properly) would be 500 hours work. That's around 150 BTC.

EDIT: By the way this would be a total and utter massive waste of money. Even if all the scalability problems were addressed AND a solution was found for the variance problem, there are no miners left to attract. The sad truth is personal miners are an increasingly irrelevant entity and what little hashrate they have cannot save any small pool. Only farms are relevant today.
legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
I was kind of referring to you -ck when I made my statements above about simply translating from Python to C not solving the inherent problem of the pool.  Sure, a node will be faster and more scalable... heck you'd probably even do something like using multiple threads Tongue... but as you, me, and most everyone else has stated, the real problem is the variance.  Throw 100PH/s on p2pool and see how the share difficulty skyrockets and nobody with under 500TH/s mining is going to have a prayer of getting a share on the chain with any regularity.

Until that is solved, what's the point.  By the way... what is your magic number?  You state, "...it would cost you a fortune to pay me...".  Define fortune Smiley.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
Repeating this for the maybe 20th time:

I could rewrite it in c but it would cost you a fortune to pay me since I have no interest in doing it for free as I see it as a futile task and would only do it for paid work. The reason is - rewriting p2pool in c would fix its scalability issues but would do nothing for the design of the pool having variance opposite to that of other pools - the larger the pool the MORE the variance instead of what happens at other pools. Unless someone has a design solution for this it would be a waste of time - many many many people have talked and waved their hands lots about this but come up with nothing.
legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
...but I think the centralized nature of our payout system is a bit of a turn off for some folks.  I had hoped that my reputation and only having 1 week of earnings on the line would be enough to ease community concerns, but it appears that it is not.
I hear you.  And yet, people have no problems throwing their hash at the most centralized pools out there (Ant, Discus, etc).  It really strikes me as paradoxical.  We want decentralization, but we will gladly centralize all of our mining power to get it.  Oh well... I look forward to the stuff Nonnakip is doing.  Keep on fighting the good fight Smiley.
donator
Activity: 4760
Merit: 4323
Leading Crypto Sports Betting & Casino Platform
Nonnakip has done the closest thing to a solution with his NastyPoP method (ckpool combined with p2pool).  It's a decent workaround, but it's not ideal.  I've been mining on it for a year now, and have my thread comparing the results of it to standard p2pool nodes.  The biggest problem with their solution is that you lose the decentralized nature of p2pool.  Of course, if they expanded their operations from just their two current nodes, so that anyone could operate their own NastyPoP node, then we'd have something workable.

I definitely like what they've done, or I wouldn't have dedicated a year's worth of mining and time to them Smiley.

The ckpool implementation was actually turned off because it was unstable with our implementation (our use of it wasn't what it was intended for and required a LOT of customization to the code).  We will hopefully be able to stabilize it for our use and implement it again in the future.  There is just a LOT of things nonnakip is working on for NastyFans and the lack of community support on NastyPool has put that project a bit on the back burner.  If more people were interested in it and wanted to participate it might be a different story.  Perhaps if it ever is stable with the ckpool implementation we could open it up to anyone who wants to run a node.  I actually approached the Toomim bros to see if they were interested in hosting a node but I think the centralized nature of our payout system is a bit of a turn off for some folks.  I had hoped that my reputation and only having 1 week of earnings on the line would be enough to ease community concerns, but it appears that it is not.
donator
Activity: 4760
Merit: 4323
Leading Crypto Sports Betting & Casino Platform
could this be the cause of the recent bad luck?

No.
legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
I'm not sure, but why don't we raise funds for the development team that can do this (maybe it's a bad idea).
I think we should act more active rather than passively wait for volunteers.

This has also been discussed many times in the past, but finding a coder willing to step up (huge task) as well as people who are willing to donate has proven impossible unfortunately. I've read through this thread many times & it seems some users think that either there is no need for development because it's perfect already, or that it is some kind of crime to even think about it - so nothing happens & miners go elsewhere.

**waits for torrent of abuse**

I'm not sure this is the case.

There are qualified developers who I have spoken with that are willing to work on P2Pool, and Forrest is still around, if there was a viable proposed solution I would certainly contribute.

The real problem is solving P2Pool's scaleability challenges, and they are hard problems to solve that require inventing a solution.

I've been waiting for almost 2 years to bump my head in the shower and come up with it, but not there yet Wink

If 100 PH/s was pointed to P2Pool today most (if not all) of the existing miners would see a huge spike in variance, and a large reduction in payout.

Share difficulty would skyrocket, and keeping a share on the chain would become very difficult for most of us.

The share chain has a finite amount of space and there is a threshold to prevent dust payments that would not even constitute a transaction fee.

You can only split the 25BTC reward so many ways in a decentralized trust-less environment, and smaller miners will always get squeezed out.

I believe that the solution may lie with sidechains and micro payments where a miner could accrue shares in some type of trust-less sidechain that would be paid out in BTC once a given threshold amount is reached.

This would allow us to increase the size of the share-chain while eliminating dust payouts from the generation transaction.

This technology does not yet exist, but perhaps we are getting closer....


We actually had this discussion almost word for word previously Smiley.  I'm right there with you... all I've gotten is a few lumps on the head from bumping it so many times.  The idea of a side chain is interesting, but nobody's been able to come up with a way to implement it.  OgNasty and Nonnakip have done the closest thing to a solution with their NastyPoP method (ckpool combined with p2pool).  It's a decent workaround, but it's not ideal.  I've been mining on it for a year now, and have my thread comparing the results of it to standard p2pool nodes.  The biggest problem with their solution is that you lose the decentralized nature of p2pool.  Of course, if they expanded their operations from just their two current nodes, so that anyone could operate their own NastyPoP node, then we'd have something workable.

I definitely like what they've done, or I wouldn't have dedicated a year's worth of mining and time to them Smiley.
sr. member
Activity: 266
Merit: 250
It's a tough nut to crack.......
legendary
Activity: 1258
Merit: 1027
I'm not sure, but why don't we raise funds for the development team that can do this (maybe it's a bad idea).
I think we should act more active rather than passively wait for volunteers.

This has also been discussed many times in the past, but finding a coder willing to step up (huge task) as well as people who are willing to donate has proven impossible unfortunately. I've read through this thread many times & it seems some users think that either there is no need for development because it's perfect already, or that it is some kind of crime to even think about it - so nothing happens & miners go elsewhere.

**waits for torrent of abuse**

I'm not sure this is the case.

There are qualified developers who I have spoken with that are willing to work on P2Pool, and Forrest is still around, if there was a viable proposed solution I would certainly contribute.

The real problem is solving P2Pool's scaleability challenges, and they are hard problems to solve that require inventing a solution.

I've been waiting for almost 2 years to bump my head in the shower and come up with it, but not there yet Wink

If 100 PH/s was pointed to P2Pool today most (if not all) of the existing miners would see a huge spike in variance, and a large reduction in payout.

Share difficulty would skyrocket, and keeping a share on the chain would become very difficult for most of us.

The share chain has a finite amount of space and there is a threshold to prevent dust payments that would not even constitute a transaction fee.

You can only split the 25BTC reward so many ways in a decentralized trust-less environment, and smaller miners will always get squeezed out.

I believe that the solution may lie with sidechains and micro payments where a miner could accrue shares in some type of trust-less sidechain that would be paid out in BTC once a given threshold amount is reached.

This would allow us to increase the size of the share-chain while eliminating dust payouts from the generation transaction.

This technology does not yet exist, but perhaps we are getting closer....

legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
I think the vast majority of miners that leave p2pool do so because they can't deal with the variance.  It's also why pools like AntPool and f2pool continue to grow.  Less variance... regular payouts... regardless of how bad they are for the network.
sr. member
Activity: 266
Merit: 250
Yeah, I read about that. Unfortunately it seems it's easier/cheaper/less risky for miners just to up & leave instead of being prepared to "chip in" towards a long-term improvement - which at the rate the diff is rising, I can't say I blame them  Sad
legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
I'm not sure, but why don't we raise funds for the development team that can do this (maybe it's a bad idea).
I think we should act more active rather than passively wait for volunteers.

This has also been discussed many times in the past, but finding a coder willing to step up (huge task) as well as people who are willing to donate has proven impossible unfortunately. I've read through this thread many times & it seems some users think that either there is no need for development because it's perfect already, or that it is some kind of crime to even think about it - so nothing happens & miners go elsewhere.

**waits for torrent of abuse**
I don't think it's that people think it's perfect already... it's that people haven't come up with a better way to do it.  A number of ideas have been thrown around, but none solve the problems inherent in the system.  As much as we all want p2pool to represent a larger percentage of the network hash, if it were to do so, then the miners would experience more and more variance as share difficulty rose.  Nobody's been able to come up with a way to manage that, and nobody's willing to simply translate the pool code from python to C.  Heck, windpath even started a bounty months ago to get some work done.  Nobody took him up on the offer.
sr. member
Activity: 266
Merit: 250
I'm not sure, but why don't we raise funds for the development team that can do this (maybe it's a bad idea).
I think we should act more active rather than passively wait for volunteers.

This has also been discussed many times in the past, but finding a coder willing to step up (huge task) as well as people who are willing to donate has proven impossible unfortunately. I've read through this thread many times & it seems some users think that either there is no need for development because it's perfect already, or that it is some kind of crime to even think about it - so nothing happens & miners go elsewhere.

**waits for torrent of abuse**
legendary
Activity: 1257
Merit: 1004
pool.sexy

This version is not -ck's, but it is using the mainline cgminer.

ah ok! but fix the problem with p2pool, true?!
sr. member
Activity: 266
Merit: 250
ty but fan go to 100%....normal?

Only on startup, then it calms down  Wink otherwise re-do it.

solved with this
https://www.dropbox.com/s/a3qqhgh3awcrw1i/SD-S5-20150107_cgm_4_9_0-queue_1_submit.tar.gz?dl=0

thank you, and thank you -ck for this version

This version is not -ck's, but it is using the mainline cgminer.
legendary
Activity: 1308
Merit: 1011
Runs fine yes, but the bitmain driver throws away what could be potential block solves on p2pool...

the problem is only for s7? or even s5 - s3....?

There is a discussion that contains all modified antminer drivers for p2pool?
Except for Kano's firmware and my drivers for S1-3, all antminer S* cgminer forks from bitmain have the same driver design flaw. We mention it at regular intervals but it keeps getting forgotten or lost in the discussion by p2pool miners...

Maybe does not make sense to do firmware forks for S *, so we are constantly catching up and more and more losing hashrate.
Now it is so small that it can be said that p2pool is dead.
And many of those who get ASICs, can never replace firmware and getting out on p2pool they switch to a another type of pools.

Maybe it's time  for develpers to join and modify and adapt it for the new ASICs? Perhaps it's needed to rewrite it on C, as in Python it loads only one core and often on 100%
We need creative solutions, or this topic can be closed and we should stop our nodes and move to antpool, f2pool and similar.
We should absolutely not, in any way, shape or form, EVER move to AntPool or f2pool.  Not only are those pools way too big as it is, but they are also poorly written and take shortcuts that are bad for Bitcoin.  In case you don't remember, they caused a fork in BTC earlier this year precisely because of their shoddy software and mining practices.
I agree with you, but most people do not understand this.

As to your other points, we've discussed numerous times rewriting p2pool in a language that can handle multiple threads.  Thus far, nobody's stepped up to take on that challenge.  Even so, rewriting the pool code in C or some other language will not handle the underlying problem of crappy cgminer forks not submitting shares which could be solving blocks.
I'm not sure, but why don't we raise funds for the development team that can do this (maybe it's a bad idea).
I think we should act more active rather than passively wait for volunteers.
Jump to: