Author

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 353. (Read 2591964 times)

hero member
Activity: 798
Merit: 1000
Can someone go out back and kick the Luck Box again? I think the blocks are stuck...

Thanks Smiley
sr. member
Activity: 434
Merit: 250
new US node up - http://us.centralcavern.uk:9332 (West Coast - San Francisco) if anyone needs it.  Smiley
legendary
Activity: 2912
Merit: 1060
Hey guys, i have a technical question concerning p2pool.

While shares can quite easily be DOA because of the ~60times faster sharechain, found bitcoin blocks shouldn´t become dead/orphaned that quickly, as they are then broadcast on to the Bitcoin network.

So what happens, when my found share would be a Bitcoin block but the p2pool node considers it a "late" share? Is it still passed on to the Bitcoin network?



Yes, but it won't show up in the p2pool block list, either.... and you won't get the .5% bonus for it, just the "normal" amount earned.

... also why no-submit-stale is bad to use with p2pool.
You're correct about the no-submit-stale settings.  With p2pool you want to submit everything.  You're incorrect about not getting the 0.5%, though.  There's nothing in the generate_transaction code reliant upon whether or not the share is either stale or DOA.  It always divides up the rewards into 199/200 for miners on the share chain and 1/200 for the miner that found the block.

Hope that helps.

cgminer default used to be no submit now its yes submit right?

i still remember setting "submit-stale" : true,
legendary
Activity: 1344
Merit: 1024
Mine at Jonny's Pool
Hey guys, i have a technical question concerning p2pool.

While shares can quite easily be DOA because of the ~60times faster sharechain, found bitcoin blocks shouldn´t become dead/orphaned that quickly, as they are then broadcast on to the Bitcoin network.

So what happens, when my found share would be a Bitcoin block but the p2pool node considers it a "late" share? Is it still passed on to the Bitcoin network?



Yes, but it won't show up in the p2pool block list, either.... and you won't get the .5% bonus for it, just the "normal" amount earned.

... also why no-submit-stale is bad to use with p2pool.
You're correct about the no-submit-stale settings.  With p2pool you want to submit everything.  You're incorrect about not getting the 0.5%, though.  There's nothing in the generate_transaction code reliant upon whether or not the share is either stale or DOA.  It always divides up the rewards into 199/200 for miners on the share chain and 1/200 for the miner that found the block.

Hope that helps.
sr. member
Activity: 462
Merit: 250
you won't get the .5% bonus for it
I don't see how it is even possible that block finder bonus depends on whether the corresponding P2Pool share is stale or not.
hero member
Activity: 714
Merit: 500
So if you have 3 nodes on servers, with high bandwidth, manually peering them so every node has atleast two peers should give you a very good performance?

Is bandwidth the limiting factor on peers or is it ping time?
zvs
legendary
Activity: 1680
Merit: 1000
https://web.archive.org/web/*/nogleg.com
How does the peering system work and what do incoming and outgoing peers do?

Are the outgoing peers downloading the sharechain from you because they haven´t got the complete list?

What happens when your node has no ingoing peers?


Does your peernetwork automatically grow with your nodes´online time?

I read somewhere that you can also "force" your node to peer with a certain other node, is there a benefit in that?

Does a bigger peer network mean less orphaned shares? (as the found share should be distributed faster around the p2pool network)


Peers talk to each other Smiley 20 peers is enough, but 30 is better... Yes they download the chain if they havent got it up todate. You dont necessarily need incoming peers, but if You can open port 9333 it helps network - all cant do that.

Force connect if theres a node you think is working very well and might help with your dead or orphan shares..


These questions are relevant to me, thanks for posting Collider...

new p2pool node set up in Iceland http://is.centralcavern.uk:9332
currently downloading blockchain for a middle east node - to follow.

trying to set up nodes in regions of the world who are a little less served.
see - https://bitcointalksearch.org/topic/05-fee-pools-central-cavern-mining-co-pools-655493 for reference.

thanks Collider - i've asked the boss if we can set this all up for our nodes.



A single peer could easily be better than 50.  There are too many variables.

Upstream inparticular is limited for some people, so if you get connected to a peer downloading the blockchain (and don't have something like Netlimiter), it could saturate your upstream and put you up around 1/2 a second latency to your first hop.  Likewise, a lot of peers that you request the blockchain from will either not send it at all or (even worse) will send it very slowly.  The client isn't aggressive enough in switching peers it tries to download the blockchain from.  Get connected to 8 random nodes & sometimes you'll get crap.

I've said it before, but when I ran p2pool on any server other than nogleg.com, I ran a bitcoind with two connections, one went to nogleg.com, and one was for p2pool.  Probably a better idea is to have three, in case that one goes down or gets DoS'ed or DDoS'ed or something (happened occasionally).  The p2pool client sends any blocks discovered over the p2pool network which I assume is faster than bitcoind the vast majority of the time (compare your p2pool log to your bitcoin debug log & time it to the millisecond, see which one reports it first).  Bitcoind I don't believe is needed except to generate new blocks, getheaders & maybe transactions? Aren't the other blocks also relayed via the p2pool network?

Quote
I don't see how such a block wouldn't pay the .5% bonus. The blocktemplate must always include it from the start to make a valid block, it can't be added afterwards or the block wouldn't be valid.
I haven't gone over the source in a long time, maybe it got changed since mid-2012.
sr. member
Activity: 434
Merit: 250
How does the peering system work and what do incoming and outgoing peers do?

Are the outgoing peers downloading the sharechain from you because they haven´t got the complete list?

What happens when your node has no ingoing peers?


Does your peernetwork automatically grow with your nodes´online time?

I read somewhere that you can also "force" your node to peer with a certain other node, is there a benefit in that?

Does a bigger peer network mean less orphaned shares? (as the found share should be distributed faster around the p2pool network)


Peers talk to each other Smiley 20 peers is enough, but 30 is better... Yes they download the chain if they havent got it up todate. You dont necessarily need incoming peers, but if You can open port 9333 it helps network - all cant do that.

Force connect if theres a node you think is working very well and might help with your dead or orphan shares..


These questions are relevant to me, thanks for posting Collider...

new p2pool node set up in Iceland http://is.centralcavern.uk:9332
currently downloading blockchain for a middle east node - to follow.

trying to set up nodes in regions of the world who are a little less served.
see - https://bitcointalksearch.org/topic/05-fee-pools-central-cavern-mining-co-pools-655493 for reference.

thanks Collider - i've asked the boss if we can set this all up for our nodes.

full member
Activity: 932
Merit: 100
arcs-chain.com
How does the peering system work and what do incoming and outgoing peers do?

Are the outgoing peers downloading the sharechain from you because they haven´t got the complete list?

What happens when your node has no ingoing peers?


Does your peernetwork automatically grow with your nodes´online time?

I read somewhere that you can also "force" your node to peer with a certain other node, is there a benefit in that?

Does a bigger peer network mean less orphaned shares? (as the found share should be distributed faster around the p2pool network)


Peers talk to each other Smiley 20 peers is enough, but 30 is better... Yes they download the chain if they havent got it up todate. You dont necessarily need incoming peers, but if You can open port 9333 it helps network - all cant do that.

Force connect if theres a node you think is working very well and might help with your dead or orphan shares..
legendary
Activity: 1540
Merit: 1001
I already told you to do all that stuff a month or two ago, if it's the same project we're talking about, heh..  It won't have any impact on memory usage really, but you only need 2 connections on your bitcoind.

Using your settings I had virtually zero transactions.  Sad  That's why I asked...

M
legendary
Activity: 1258
Merit: 1027
...
I have built and managed large development teams in the past for specialized projects, and I have to say that none of them came close to the complexity and challenges offered by what we need in a p2pool dev team.
...
P2pool is not a massive difficult project ... forrestv is able to do it himself.

So I guess your comment above means you've only worked in small time projects Tongue

Hi Kano, nice to meet you???

Many applications start out as the brain child of a single dev (Bitcoin, Linux, Facebook, DOS, etc, etc...), but working alone inside a vacuum on a project that has already proved successful, like p2pool, can be detrimental to its overall success.

I'm hugely grateful for the substantial contribution of p2pool to the Bitcoin community by Forrest, it's a great accomplishment and I believe it can lead the mining world into a decentralized trust less future. But we need to fix the problems Forrest points out in his post first.

......
I have since been brainstorming and prototyping ways to fix the core problems with P2Pool - scalability and excessive variance for small miners. I prototyped a way to do trustless verification of blocks with O(1) storage so that we could free most P2Pool nodes from needing a local *coin node. I've spent a ton of time thinking of ways to implement parallel sharechains so that we can have shares more often, decreasing variance for small miners, while decreasing the strict latency requirements for hardware, sadly without any concrete results yet. Because this is a hard problem. I spent hundreds of hours in the summer of 2011 thinking about how P2Pool could possibly work, in a time before merged mining or pools with coinbase payouts even existed. Any further improvements that get us out of this local maxima that we're stuck in will require comparable amounts of thought.
.......

Like Forrest and others have said "this is a hard problem".

I'd like to see some really smart devs jump in and contribute to help solve the problems Forrest has been working on alone for some time now, fresh eyes and fresh ideas can go a long way and accelerate p2pools rise in popularity and use.

it is hard to work on this project when most people insist on cutting off the pool donations to me.

My node does not donate or charge a fee.

It is not because I think you are not entitled to compensation, its that I want to try and attract more miners to p2pool, and away from centralized pools.

Charging any fee above 0% right now will simply not get the job done.

I (and I think many others here) am willing to contribute to you Forrest, but I'd like to see it set up as some type of bounty system for work completed. As you said there are some real challenges, and throwing good money after bad does not seem like a solution to me...

So I guess the question is; what could the community do to motivate you to make p2pool a priority again other then the node donation %?


Thanks! I do plan to continue to develop it....

Does anyone know where i can get the p2pool luck over ~ 1 year?

Back to block 205238 is available on my pool stats page, all new blocks will be included. If anyone has the historical hash rate data for the pool the page could calculate luck back to the beginning....

Can Coin Cadence P2P Bitcoin Mining Pool frontend be dowloaded , it doesnt look so "beta" to me, I think it looks great and want to try it because it has many things in one place...

Not right now, it is still in beta, and there is a lot more I want to get done. To be candid, the full code may never be publicly available, there is a lot going on behind the scenes to generate the data, the db is already over 75MB, and I'm just not sure how to maintain all that data across different nodes running the front end. I've tried to make it platform independent, but some of the DB queries may be specific to MySQL...

That being said, when it's done, I will post a version of it on Github that will only require Bitcoind, p2pool, Highcharts, jquery, php, and Twitter Bootstrap. the addition of Highcharts, PHP, MySQL and Twitter Bootstrap



hero member
Activity: 714
Merit: 500
How does the peering system work and what do incoming and outgoing peers do?

Are the outgoing peers downloading the sharechain from you because they haven´t got the complete list?

What happens when your node has no ingoing peers?


Does your peernetwork automatically grow with your nodes´online time?

I read somewhere that you can also "force" your node to peer with a certain other node, is there a benefit in that?

Does a bigger peer network mean less orphaned shares? (as the found share should be distributed faster around the p2pool network)

hero member
Activity: 896
Merit: 1000
Hey guys, i have a technical question concerning p2pool.

While shares can quite easily be DOA because of the ~60times faster sharechain, found bitcoin blocks shouldn´t become dead/orphaned that quickly, as they are then broadcast on to the Bitcoin network.

So what happens, when my found share would be a Bitcoin block but the p2pool node considers it a "late" share? Is it still passed on to the Bitcoin network?



Yes, but it won't show up in the p2pool block list, either.... and you won't get the .5% bonus for it, just the "normal" amount earned.

I don't see how such a block wouldn't pay the .5% bonus. The blocktemplate must always include it from the start to make a valid block, it can't be added afterwards or the block wouldn't be valid.
hero member
Activity: 714
Merit: 500
Thanks, so if you had the no-submit-stale option turned on, it wouldn´t submit valid Bitcoin blocks that are not considered a valid share?

Isn´t there some mechanism every node immediatly recognizes a valid block as such, as every node runs the full Bitcoin client?

I could be wrong here since it's been so long since I've mined on a p2pool with block times (ed: *over* a minute that is) under a minute, but I think it only detects the stale shares after a block.... but all shares should be submitted, as some share that's detected as stale (say, 1ms to 500ms late) could easily win an orphan race, esp. vs some of the pools that aren't set up quite right (propagate their blocks slowly).

from personal experience, I've had a share that was detected as DOA solve a bitcoin block, back in 2012
Thanks again. Is the option to not send invalid shares in your miner or on the node?
It seems like it should be a setting in the miner...
zvs
legendary
Activity: 1680
Merit: 1000
https://web.archive.org/web/*/nogleg.com
Thanks, so if you had the no-submit-stale option turned on, it wouldn´t submit valid Bitcoin blocks that are not considered a valid share?

Isn´t there some mechanism every node immediatly recognizes a valid block as such, as every node runs the full Bitcoin client?

I could be wrong here since it's been so long since I've mined on a p2pool with block times (ed: *over* a minute that is) under a minute, but I think it only detects the stale shares after a block.... but all shares should be submitted, as some share that's detected as stale (say, 1ms to 500ms late) could easily win an orphan race, esp. vs some of the pools that aren't set up quite right (propagate their blocks slowly).

from personal experience, I've had a share that was detected as DOA solve a bitcoin block, back in 2012
hero member
Activity: 714
Merit: 500
Thanks, so if you had the no-submit-stale option turned on, it wouldn´t submit valid Bitcoin blocks that are not considered a valid share?

Isn´t there some mechanism every node immediatly recognizes a valid block as such, as every node runs the full Bitcoin client?
zvs
legendary
Activity: 1680
Merit: 1000
https://web.archive.org/web/*/nogleg.com
Hey guys, i have a technical question concerning p2pool.

While shares can quite easily be DOA because of the ~60times faster sharechain, found bitcoin blocks shouldn´t become dead/orphaned that quickly, as they are then broadcast on to the Bitcoin network.

So what happens, when my found share would be a Bitcoin block but the p2pool node considers it a "late" share? Is it still passed on to the Bitcoin network?



Yes, but it won't show up in the p2pool block list, either.... and you won't get the .5% bonus for it, just the "normal" amount earned.

... also why no-submit-stale is bad to use with p2pool.
hero member
Activity: 714
Merit: 500
Hey guys, i have a technical question concerning p2pool.

While shares can quite easily be DOA because of the ~60times faster sharechain, found bitcoin blocks shouldn´t become dead/orphaned that quickly, as they are then broadcast on to the Bitcoin network.

So what happens, when my found share would be a Bitcoin block but the p2pool node considers it a "late" share? Is it still passed on to the Bitcoin network?

zvs
legendary
Activity: 1680
Merit: 1000
https://web.archive.org/web/*/nogleg.com
"Just run one... there's no reason to run two instances of Bitcoin Core on same box."

Theres plenty of reasons, but it can be done with just one...

So... what are they?  If it can be done with just one, then it really isn't a reason to run two or more.

Modifications to the conf file (and/or source code) for p2pool purposes only, i.e. limitfreerelay, changes to mintxfee, etc.  If you have enough memory on your system, there are lots of things you could do to make it run faster.

Also, a p2pool bitcoind only needs 2 connections, one to your p2pool and the other to a single node.  I used to connect my p2pool bitcoind to 5.9.24.81 only.

Quote
Sun Jun 22 2014 11:59:24 GMT+0300 (EEST)   0.0489
Sun Jun 22 2014 11:56:36 GMT+0300 (EEST)   0.0662
Sun Jun 22 2014 11:53:48 GMT+0300 (EEST)   0.0334
Sun Jun 22 2014 11:51:00 GMT+0300 (EEST)   0.0242
Sun Jun 22 2014 11:48:12 GMT+0300 (EEST)   0.0162
Sun Jun 22 2014 11:45:24 GMT+0300 (EEST)   0.0175

This is getblocktemplate latency with 0.0001 fees...
Turn off all free transactions if they aren't already & increase fees to .00011.  Seems like a decent compromise to me.

Getblocktemplate latency isn't a huge deal, anyway... err, well, it is if those numbers are in seconds.  Shouldn't worry about it if it's 500ms or less, though.

ed:  Oh, and you do want minblocksize (or blockminsize?  been a while) set to 0, to get rid of all the free transactions.

Quote
I don't have control over the disk being used, as I'm using a VPS.  My maxconn is at 50, though, maybe that's my issue.  I'll tweak it and see what happens.

Thanks!

M
I already told you to do all that stuff a month or two ago, if it's the same project we're talking about, heh..  It won't have any impact on memory usage really, but you only need 2 connections on your bitcoind.
legendary
Activity: 1540
Merit: 1001
I updated the front end to my p2pool node to use Norgan's code.  Looks good, thank you Norgan!

M
Jump to: