[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 545.

furball

full member

Activity: 172

Merit: 100

Quote from: mdude77 on May 18, 2013, 05:13:24 AM

Quote from: Prattler on May 18, 2013, 04:50:09 AM

I can confirm the high getblocktemplate latency is bitcoind related.

1. Upgrade bitcoind to the latest possible version (https://bitcointalksearch.org/topic/082rc1-ready-for-testing-201124)

2. With p2pool we are real miners and make our own blocks, so edit the important settings in your bitcoin.conf!

Code:

blockmaxsize=250000 # default: 250000

blockprioritysize=27000 # default: 27000

mintxfee=0.0002 # default: 0.0001

minrelaytxfee=0.0002 # default: 0.0001

Setting mintxfee and minrelaytxfee higher than default will increase your getblocktemplate latency. It's a matter of opinion, but I think those defaults are too low for the network, so, by increasing them a bit, you are also benefiting the network. I would personally recommend setting both to 0.0005.

Until bitcoind gets optimized, getblocktemplate just gets too slow with the default minrelaytxfee=0.0001.

Maybe this is hardware or environment related? I'm still using default settings and latest p2pool. My stale rate is always 10-15% less than the pool's rate. Yes, according to p2pool my bitcoind latency is at 10s. So what?

M

EDIT: I restarted bitcoind, it did significantly reduce getblocktemplate latency. Don't see a big deal here, have to restart things regularly anyhow. Just had to restart cgminer because it went funny on me. Presumably hit the AMD driver time limit.

I've been running with the blockmaxsize and mintxfee setting per above for the last 12 hours and it didn't seem to help. I just added in the blockprioritysize and minrelaytxfee settings and it looks to have made a huge difference.

Went from 5s latency to sub 1s.

Thanks so much Prattler...awesome post!

mdude77

legendary

Activity: 1540

Merit: 1001

Quote from: Prattler on May 18, 2013, 04:50:09 AM

I can confirm the high getblocktemplate latency is bitcoind related.

1. Upgrade bitcoind to the latest possible version (https://bitcointalksearch.org/topic/082rc1-ready-for-testing-201124)

2. With p2pool we are real miners and make our own blocks, so edit the important settings in your bitcoin.conf!

Code:

blockmaxsize=250000 # default: 250000

blockprioritysize=27000 # default: 27000

mintxfee=0.0002 # default: 0.0001

minrelaytxfee=0.0002 # default: 0.0001

Setting mintxfee and minrelaytxfee higher than default will increase your getblocktemplate latency. It's a matter of opinion, but I think those defaults are too low for the network, so, by increasing them a bit, you are also benefiting the network. I would personally recommend setting both to 0.0005.

Until bitcoind gets optimized, getblocktemplate just gets too slow with the default minrelaytxfee=0.0001.

Maybe this is hardware or environment related? I'm still using default settings and latest p2pool. My stale rate is always 10-15% less than the pool's rate. Yes, according to p2pool my bitcoind latency is at 10s. So what?

M

EDIT: I restarted bitcoind, it did significantly reduce getblocktemplate latency. Don't see a big deal here, have to restart things regularly anyhow. Just had to restart cgminer because it went funny on me. Presumably hit the AMD driver time limit.

Prattler

full member

Activity: 192

Merit: 100

I can confirm the high getblocktemplate latency is bitcoind related.

1. Upgrade bitcoind to the latest possible version (https://bitcointalksearch.org/topic/082rc1-ready-for-testing-201124)

2. With p2pool we are real miners and make our own blocks, so edit the important settings in your bitcoin.conf!

Code:

blockmaxsize=250000 # default: 250000

blockprioritysize=27000 # default: 27000

mintxfee=0.0002 # default: 0.0001

minrelaytxfee=0.0002 # default: 0.0001

Setting mintxfee and minrelaytxfee higher than default will increase your getblocktemplate latency. It's a matter of opinion, but I think those defaults are too low for the network, so, by increasing them a bit, you are also benefiting the network. I would personally recommend setting both to 0.0005.

Edit: what happens is that lots of free and low fee transactions accumulate on the mempool, which increases getblocktemplate latency, due to an algorithmic flaw of bitcoind's getblocktemplate. It might not matter for most pools, but p2pool is more dependant on this. Setting mintxfee/minrelaytxfee above default 0.0001 reduces the accumulation of mempool garbage. Restarting bitcoind also clears the mempool and helps for a short time.

Xenotron

newbie

Activity: 31

Merit: 0

Quote from: azw409 on May 18, 2013, 04:02:05 AM

Maybe the issue with latency is nothing to do with the software but extremely large blocks :-

https://bitcointalksearch.org/topic/extremely-high-bitcoind-getblocktemplate-latency-207790

Nope. I have started to use blockmaxsize=5000 to see what will be changed. And so far there is no big difference.
bitcoin-qt continues to increase RAM consumption even when new block is found.
I can see on Graphs that when Bitcoin-qt releases occupied RAM then latency drops to its usual values.

azw409

member

Activity: 73

Merit: 10

Maybe the issue with latency is nothing to do with the software but extremely large blocks :-

https://bitcointalksearch.org/topic/extremely-high-bitcoind-getblocktemplate-latency-207790

azw409

member

Activity: 73

Merit: 10

The latency appears to be in bitcoin-qt because if I restart p2pool, the latency is still high > 10s on restart whereas if I restart both bitcoin-qt and p2pool the latency returns to around 0.5s.

So I'm not sure that using an earlier version of p2pool will have any bearing on this.

Xenotron

newbie

Activity: 31

Merit: 0

Is there any way to avoid high CPU load by bitcoin-qt? Also latency skyrocketing to 14s...
Tried 11.4 and it seems there is a memory leak. Now started 11.2 - will see.

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: K1773R on May 17, 2013, 07:12:26 PM

Quote from: zvs on May 17, 2013, 05:26:24 PM

Quote from: K1773R on May 17, 2013, 04:28:29 PM

Quote from: zvs on May 17, 2013, 04:25:40 PM

Quote from: Richy_T on May 17, 2013, 02:30:00 PM

I'm having an issue with p2pool and my router. It's not a show-stopper but I was wondering if someone might have an idea what's going on or be able to work with me to work out what is going on.

Basically, anytime I turn on QOS, I appear to stop doing any useful work. cgminer trundles along, fans spin, heat generated, Unfortunately, it appears that that's all wasted. the debug output of p2pool says that it's punishing shares for being stale. On the pools graph page, the incoming part of peers and the p2p in on traffic rate drop to zero. For whatever reason, it looks like peers are no longer supplying me with useful information (?)

I've even set things so that p2pool and bitcoind are QOSed to the highest speed category just in case they were getting squashed that way but it still gives the same result.

I can leave QOS off for now but I'd like to get this working properly. I'm using Tomato so the configuration is fairly straightforward and easy. I'll post some pictures in a little bit for illustration. I can perform packet captures and other good stuff but I'm not up on the p2pool networking.

the only thing i've ever used in windows that i've found did a proper job of managing bandwidth was netlimiter

the only thing that did almost a proper job is "format c: -y"

what, netlimiter?

why, because it installs some driver that you can disable on startup?

i don't get the big deal with it

i said on win only "format c: -y" does work almost proper Wink

oh, lots of netlimiter haters out there

but this is true

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: Richy_T on May 17, 2013, 06:41:44 PM

In case anyone isn't familiar with Tomato, it is a specialized Linux distribution that runs on certain hardware routers.

Other connections appear to occur with no issues (unless I goof the configuration) and if I look in the details, I can see connections between external hosts and my own on the bitcoind and p2pool ports. I can't think it would be anything about Python that would be causing the issue. Hard to see what about QOS would be an issue since it's basically just prioritizing packets. But there must be more to it.

Depends how Tomato handles QoS. You can block some traffic if you give higher priority and allow another to saturate your link. QoS isn't simple at all (you must define many compromises to make it work), if your tools make you think it is simple they probably do it wrong.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: zvs on May 17, 2013, 05:26:24 PM

Quote from: K1773R on May 17, 2013, 04:28:29 PM

Quote from: zvs on May 17, 2013, 04:25:40 PM

Quote from: Richy_T on May 17, 2013, 02:30:00 PM

I'm having an issue with p2pool and my router. It's not a show-stopper but I was wondering if someone might have an idea what's going on or be able to work with me to work out what is going on.

Basically, anytime I turn on QOS, I appear to stop doing any useful work. cgminer trundles along, fans spin, heat generated, Unfortunately, it appears that that's all wasted. the debug output of p2pool says that it's punishing shares for being stale. On the pools graph page, the incoming part of peers and the p2p in on traffic rate drop to zero. For whatever reason, it looks like peers are no longer supplying me with useful information (?)

I've even set things so that p2pool and bitcoind are QOSed to the highest speed category just in case they were getting squashed that way but it still gives the same result.

I can leave QOS off for now but I'd like to get this working properly. I'm using Tomato so the configuration is fairly straightforward and easy. I'll post some pictures in a little bit for illustration. I can perform packet captures and other good stuff but I'm not up on the p2pool networking.

the only thing i've ever used in windows that i've found did a proper job of managing bandwidth was netlimiter

the only thing that did almost a proper job is "format c: -y"

what, netlimiter?

why, because it installs some driver that you can disable on startup?

i don't get the big deal with it

i said on win only "format c: -y" does work almost proper Wink

Richy_T

legendary

Activity: 2576

Merit: 2267

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k

In case anyone isn't familiar with Tomato, it is a specialized Linux distribution that runs on certain hardware routers.

Other connections appear to occur with no issues (unless I goof the configuration) and if I look in the details, I can see connections between external hosts and my own on the bitcoind and p2pool ports. I can't think it would be anything about Python that would be causing the issue. Hard to see what about QOS would be an issue since it's basically just prioritizing packets. But there must be more to it.

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: K1773R on May 17, 2013, 04:28:29 PM

Quote from: zvs on May 17, 2013, 04:25:40 PM

Quote from: Richy_T on May 17, 2013, 02:30:00 PM

I'm having an issue with p2pool and my router. It's not a show-stopper but I was wondering if someone might have an idea what's going on or be able to work with me to work out what is going on.

Basically, anytime I turn on QOS, I appear to stop doing any useful work. cgminer trundles along, fans spin, heat generated, Unfortunately, it appears that that's all wasted. the debug output of p2pool says that it's punishing shares for being stale. On the pools graph page, the incoming part of peers and the p2p in on traffic rate drop to zero. For whatever reason, it looks like peers are no longer supplying me with useful information (?)

I've even set things so that p2pool and bitcoind are QOSed to the highest speed category just in case they were getting squashed that way but it still gives the same result.

I can leave QOS off for now but I'd like to get this working properly. I'm using Tomato so the configuration is fairly straightforward and easy. I'll post some pictures in a little bit for illustration. I can perform packet captures and other good stuff but I'm not up on the p2pool networking.

the only thing i've ever used in windows that i've found did a proper job of managing bandwidth was netlimiter

the only thing that did almost a proper job is "format c: -y"

what, netlimiter?

why, because it installs some driver that you can disable on startup?

i don't get the big deal with it

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: zvs on May 17, 2013, 04:25:40 PM

Quote from: Richy_T on May 17, 2013, 02:30:00 PM

I'm having an issue with p2pool and my router. It's not a show-stopper but I was wondering if someone might have an idea what's going on or be able to work with me to work out what is going on.

Basically, anytime I turn on QOS, I appear to stop doing any useful work. cgminer trundles along, fans spin, heat generated, Unfortunately, it appears that that's all wasted. the debug output of p2pool says that it's punishing shares for being stale. On the pools graph page, the incoming part of peers and the p2p in on traffic rate drop to zero. For whatever reason, it looks like peers are no longer supplying me with useful information (?)

I've even set things so that p2pool and bitcoind are QOSed to the highest speed category just in case they were getting squashed that way but it still gives the same result.

I can leave QOS off for now but I'd like to get this working properly. I'm using Tomato so the configuration is fairly straightforward and easy. I'll post some pictures in a little bit for illustration. I can perform packet captures and other good stuff but I'm not up on the p2pool networking.

the only thing i've ever used in windows that i've found did a proper job of managing bandwidth was netlimiter

the only thing that did almost a proper job is "format c: -y"

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: Richy_T on May 17, 2013, 02:30:00 PM

I'm having an issue with p2pool and my router. It's not a show-stopper but I was wondering if someone might have an idea what's going on or be able to work with me to work out what is going on.

Basically, anytime I turn on QOS, I appear to stop doing any useful work. cgminer trundles along, fans spin, heat generated, Unfortunately, it appears that that's all wasted. the debug output of p2pool says that it's punishing shares for being stale. On the pools graph page, the incoming part of peers and the p2p in on traffic rate drop to zero. For whatever reason, it looks like peers are no longer supplying me with useful information (?)

I've even set things so that p2pool and bitcoind are QOSed to the highest speed category just in case they were getting squashed that way but it still gives the same result.

I can leave QOS off for now but I'd like to get this working properly. I'm using Tomato so the configuration is fairly straightforward and easy. I'll post some pictures in a little bit for illustration. I can perform packet captures and other good stuff but I'm not up on the p2pool networking.

the only thing i've ever used in windows that i've found did a proper job of managing bandwidth was netlimiter

Richy_T

legendary

Activity: 2576

Merit: 2267

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k

I'm having an issue with p2pool and my router. It's not a show-stopper but I was wondering if someone might have an idea what's going on or be able to work with me to work out what is going on.

Basically, anytime I turn on QOS, I appear to stop doing any useful work. cgminer trundles along, fans spin, heat generated, Unfortunately, it appears that that's all wasted. the debug output of p2pool says that it's punishing shares for being stale. On the pools graph page, the incoming part of peers and the p2p in on traffic rate drop to zero. For whatever reason, it looks like peers are no longer supplying me with useful information (?)

I've even set things so that p2pool and bitcoind are QOSed to the highest speed category just in case they were getting squashed that way but it still gives the same result.

I can leave QOS off for now but I'd like to get this working properly. I'm using Tomato so the configuration is fairly straightforward and easy. I'll post some pictures in a little bit for illustration. I can perform packet captures and other good stuff but I'm not up on the p2pool networking.

Searinox

full member

Activity: 147

Merit: 100

Do you like fire? I'm full of it.

Quote from: azw409 on May 17, 2013, 01:40:09 PM

I've been mining on P2Pool for a week and I'm very impressed. I had a lot of problems with Phoenix originally but since switching to CGMiner all the stats are now good and I'm earning more than I ever did with 50BTC.

The only remaining issue is that the bitcoind getblocktemplate latency constantly increases over time in quite large jumps. It starts around 0.5s and creeps up slowly to around 2s then jumps to 10s and stays there.

Restarting everything: Bitcoin-qt, p2pool and CGMiner fixes it again but within 24 hours it'll be back up again.

Anybody have any ideas what could be going on ?

Thanks

Version 11.4 introduced a bug where the latency creeps up over time. The only fix I found was reverting to 11.3 which is located here http://u.forre.st/u/drxlzowj/p2pool_win32_11.3.zip

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Quote from: azw409 on May 17, 2013, 01:40:09 PM

I've been mining on P2Pool for a week and I'm very impressed. I had a lot of problems with Phoenix originally but since switching to CGMiner all the stats are now good and I'm earning more than I ever did with 50BTC.

The only remaining issue is that the bitcoind getblocktemplate latency constantly increases over time in quite large jumps. It starts around 0.5s and creeps up slowly to around 2s then jumps to 10s and stays there.

Restarting everything: Bitcoin-qt, p2pool and CGMiner fixes it again but within 24 hours it'll be back up again.

Anybody have any ideas what could be going on ?

Thanks

not sure about large jumps, but if your bitcoind is accepting transactions into memory that no pool will ever include then it'll gradually get slower

try bitcoind getmininginfo and see if it's much higher when it starts getting slow (or bitcoin-qt or whatever)

azw409

member

Activity: 73

Merit: 10

I've been mining on P2Pool for a week and I'm very impressed. I had a lot of problems with Phoenix originally but since switching to CGMiner all the stats are now good and I'm earning more than I ever did with 50BTC.

The only remaining issue is that the bitcoind getblocktemplate latency constantly increases over time in quite large jumps. It starts around 0.5s and creeps up slowly to around 2s then jumps to 10s and stays there.

Restarting everything: Bitcoin-qt, p2pool and CGMiner fixes it again but within 24 hours it'll be back up again.

Anybody have any ideas what could be going on ?

Thanks

PatMan

hero member

Activity: 924

Merit: 1000

Watch out for the "Neg-Rep-Dogie-Police".....

Quote from: Subo1977 on May 17, 2013, 11:01:00 AM

Quote from: PatMan on May 17, 2013, 10:06:38 AM

Quote from: -ck on May 16, 2013, 07:34:40 AM

As discussed in the previous few pages, there are clearly some stratum issues still to resolve, and scalability issues to cope with ASIC sized workloads. BFL are sending (?have sent) forrestv a 5GH unit which should help drive development of both of these.

Excellent. At last, it seems some progress is finally being made. Grin

Which? I Hope bugfixes for Stratum Grin

Exactly, that's the number one priority eh? Wink

Subo1977

sr. member

Activity: 344

Merit: 250

Flixxo - Watch, Share, Earn!

Quote from: PatMan on May 17, 2013, 10:06:38 AM

Quote from: -ck on May 16, 2013, 07:34:40 AM

As discussed in the previous few pages, there are clearly some stratum issues still to resolve, and scalability issues to cope with ASIC sized workloads. BFL are sending (?have sent) forrestv a 5GH unit which should help drive development of both of these.

Excellent. At last, it seems some progress is finally being made. Grin

Which? I Hope bugfixes for Stratum Grin

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 545. (Read 2591920 times)