[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 536.

mdude77

legendary

Activity: 1540

Merit: 1001

Quote from: lenny_ on May 29, 2013, 06:06:02 PM

Unfortunately, no success...
Blade just do not connect to stratum proxy on p2pool. There is not error message at all.
I would like really to have some developer in it, I can donate my Blade worktime to debug this.

You know, I've tried using the stratum proxy on p2pool with normal GPUs. It just doesn't work. I don't think this is a blade thing.

M

lenny_

legendary

Activity: 1036

Merit: 1000

DARKNETMARKETS.COM

Quote from: ok on May 29, 2013, 07:24:12 AM

Quote from: lenny_ on May 27, 2013, 04:26:36 AM

Quote from: forrestv on May 27, 2013, 03:31:45 AM

Quote from: lenny_ on May 27, 2013, 03:30:13 AM

Thanks, but this still doesnt' help us. How to merge stratum-forrestv with stratum-mining-proxy?

stratum-mining-proxy uses the stratum package, which is somewhere on your computer if you're running stratum-mining-proxy, and is what the patch needs to be applied to.

Thank you! It works.
Now I successfully installed new stratum package and proxy is working, just look:

Code:

$ python mining_proxy.py -gp 5001 -sp 5002 -o localhost -p 9332
2013-05-27 10:22:17,862 INFO proxy jobs. # Using C extension for midstate speedup. Good!
2013-05-27 10:22:17,871 ERROR proxy mining_proxy.main # Stratum host/port autodetection failed
Traceback (most recent call last):
  File "mining_proxy.py", line 178, in main
    new_host = (yield utils.detect_stratum(args.host, args.port))
  File "/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1070, in _inlineCallbacks
    result = g.send(result)
  File "/home/pioruns/stratum-mining-proxy/mining_libs/utils.py", line 69, in detect_stratum
    header = f.response_headers.get('x-stratum', None)[0]
TypeError: 'NoneType' object has no attribute '__getitem__'
2013-05-27 10:22:17,871 WARNING proxy mining_proxy.main # Stratum proxy version: 1.5.2
2013-05-27 10:22:17,873 WARNING proxy mining_proxy.test_update # Checking for updates...
2013-05-27 10:22:18,265 WARNING proxy mining_proxy.main # Trying to connect to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-05-27 10:22:18,303 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # PROXY IS LISTENING ON ALL IPs ON PORT 5002 (stratum) AND 5001 (getwork)
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:18,306 INFO proxy client_service.handle_event # New job 8850090419252557308580900352527982298 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:29,931 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:29,933 INFO proxy client_service.handle_event # New job 82294000856594409674845997521737547736 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:41,372 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:41,374 INFO proxy client_service.handle_event # New job 285117993302263092594898950693479724933 for prevhash 385766c3, clean_jobs=True

Quote from: bitpop on May 27, 2013, 03:36:08 AM

Why do you need a stratum proxy any way?

Tomorrow (hopefully today if time permits) I will be testing BE Blade on this proxy

Hi there lenny, please let me know if you were successful with setting up you BE Blade with the stratum proxy on p2pool. Thx

Unfortunately, no success...
Blade just do not connect to stratum proxy on p2pool. There is not error message at all.
I would like really to have some developer in it, I can donate my Blade worktime to debug this.

stepkrav

full member

Activity: 188

Merit: 100

can i use a remote bitcoind instead of running it in the same machine with the miners?

Or could i have bitcoind and p2pool on remote machine and miners to a local one ?

PrintMule

hero member

Activity: 980

Merit: 500

FREE $50 BONUS - STAKE - [click signature]

Is there a point of me trying out my 600mh/swhile having quite weak upload speeds?

Do not want to go through all the hassle for nothing.
What rates are expected?

FlappySocks

hero member

Activity: 546

Merit: 500

Anyone tried p2pool with shedskin or pypy to lighten the resources it uses?

ok

newbie

Activity: 26

Merit: 0

Quote from: lenny_ on May 27, 2013, 04:26:36 AM

Quote from: forrestv on May 27, 2013, 03:31:45 AM

Quote from: lenny_ on May 27, 2013, 03:30:13 AM

Thanks, but this still doesnt' help us. How to merge stratum-forrestv with stratum-mining-proxy?

stratum-mining-proxy uses the stratum package, which is somewhere on your computer if you're running stratum-mining-proxy, and is what the patch needs to be applied to.

Thank you! It works.
Now I successfully installed new stratum package and proxy is working, just look:

Code:

$ python mining_proxy.py -gp 5001 -sp 5002 -o localhost -p 9332
2013-05-27 10:22:17,862 INFO proxy jobs. # Using C extension for midstate speedup. Good!
2013-05-27 10:22:17,871 ERROR proxy mining_proxy.main # Stratum host/port autodetection failed
Traceback (most recent call last):
  File "mining_proxy.py", line 178, in main
    new_host = (yield utils.detect_stratum(args.host, args.port))
  File "/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1070, in _inlineCallbacks
    result = g.send(result)
  File "/home/pioruns/stratum-mining-proxy/mining_libs/utils.py", line 69, in detect_stratum
    header = f.response_headers.get('x-stratum', None)[0]
TypeError: 'NoneType' object has no attribute '__getitem__'
2013-05-27 10:22:17,871 WARNING proxy mining_proxy.main # Stratum proxy version: 1.5.2
2013-05-27 10:22:17,873 WARNING proxy mining_proxy.test_update # Checking for updates...
2013-05-27 10:22:18,265 WARNING proxy mining_proxy.main # Trying to connect to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at localhost:9332
2013-05-27 10:22:18,268 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-05-27 10:22:18,303 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # PROXY IS LISTENING ON ALL IPs ON PORT 5002 (stratum) AND 5001 (getwork)
2013-05-27 10:22:18,304 WARNING proxy mining_proxy.main # -----------------------------------------------------------------------
2013-05-27 10:22:18,304 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:18,306 INFO proxy client_service.handle_event # New job 8850090419252557308580900352527982298 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:29,931 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:29,933 INFO proxy client_service.handle_event # New job 82294000856594409674845997521737547736 for prevhash 385766c3, clean_jobs=True
2013-05-27 10:22:41,372 INFO proxy client_service.handle_event # Setting new difficulty: 0.999984741211
2013-05-27 10:22:41,374 INFO proxy client_service.handle_event # New job 285117993302263092594898950693479724933 for prevhash 385766c3, clean_jobs=True

Quote from: bitpop on May 27, 2013, 03:36:08 AM

Why do you need a stratum proxy any way?

Tomorrow (hopefully today if time permits) I will be testing BE Blade on this proxy

Hi there lenny, please let me know if you were successful with setting up you BE Blade with the stratum proxy on p2pool. Thx

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: furball on May 29, 2013, 05:39:24 AM

I've been testing some scenarios as well and found that for me, the txfees are very influential on the latency. I wanted to change to a bigger block size to hopefully make more on the pool but I found after that I had to tune the txfees to keep the latency low.

This is what I've settled on; this gives me circa 0.2s latency with an acceptable efficiency; maybe it will work for other people too.

blockprioritysize=27000
blockmaxsize=1000000
mintxfee=0.00007
minrelaytxfee=0.00007

You are right: if there are enough tx to fill a block raising the minimum txfees avoids maintaining some that can't fit in the block in the memory pool which lowers your latency.

I raised mined from 0.00001 to 0.00002 and although my latency went from 0.25s to ~0.1s my efficiency wasn't raised noticeably (still ~105%). I believe that more and more P2Pool nodes are being setup correctly which makes everybody's efficiency converge towards 100% (which is fair and good).

furball

full member

Activity: 172

Merit: 100

Quote from: gyverlb on May 27, 2013, 04:26:13 AM

Quote from: bitpop on May 26, 2013, 11:54:11 PM

Just to recap, is this now the recommended settings:

Code:

blockmaxsize=1000000
blockminsize=400000
mintxfee=0.00001
minrelaytxfee=0.00001

And that's for bitcoin.conf right? And solo miners should/nt use this too?

I'm guessing this only works if my node finds the block so I hope if this is good, most people add this because it effects me.

That's for bitcoin.conf, yes.

blockminsize is a personnal choice (I left it out in the guide).
This works if your node finds a block, the minrelaytxfee might have a direct impact as it helps propagate fees that could be paid to you when someone else finds a block.

I've been testing some scenarios as well and found that for me, the txfees are very influential on the latency. I wanted to change to a bigger block size to hopefully make more on the pool but I found after that I had to tune the txfees to keep the latency low.

This is what I've settled on; this gives me circa 0.2s latency with an acceptable efficiency; maybe it will work for other people too.

blockprioritysize=27000
blockmaxsize=1000000
mintxfee=0.00007
minrelaytxfee=0.00007

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

if you eliminated the DOA's from the ASIC miner that has my pool as a backup, i have 501 shares, 12 orphans, and ~25-30 DOA (ed: more like 460-470 shares, 10-11 orphans, and ~25-30 DOA. forgot to include the good ones too)

mining on my home connection nets me 0-1% DOA but 10%+ orphans. i think it's due to 1) not being able to open that many connections (bandwidth), and 2) most of the hashing power being in europe and asia

anyway, for someone out there looking for a pool to mine at, the lowest latency one isnt necessarily the best. i spent several days on my florida server (75ms latency) and ended up gonig back to germany, because the florida server's orphans >>> DOA on other server, even though it also had 50+ connections open. like i said, i think that's probably due to all the nodes in europe, russia, china, etc

i imagine the default outgoing limit was set to such a small number w/ ppl like myself in mind (crap upstream from home connection), but with 10Mbit connection you should at least up that to 15 or so. it looks like my average outgoing is around 200 KB/s with 70 connections, though on occasion it'll go > 500KB/s

i'm not that fond of incoming connections (due to all the messed up nodes that'll connect and disconnect + the nodes with 1000+ms latency), but if nobody accepted any incoming, then, well.... i set mine to 10 as a compromise.. it defaults to 40. the --p2pool-node you put as additions don't count as outgoing connections towards your limit. (i have 5 outgoing, 10 incoming, the rest are all from --p2pool-node)

daemondazz

sr. member

Activity: 448

Merit: 250

Quote from: walf_man on May 29, 2013, 01:43:26 AM

need continue update...

Of?

walf_man

member

Activity: 112

Merit: 10

need continue update...

Bitmong

newbie

Activity: 29

Merit: 0

Quote from: gyverlb on May 28, 2013, 09:29:34 PM

Quote from: Bitmong on May 28, 2013, 09:21:11 PM

When I browsed through my P2Pool log, I noticed that the P2Pool stale rate varied between 17% and 20%+ in the last couple of days. Therefore the efficiency figure fluctuates even if your own stale / DOA rate doesn't change at all. That is why I think one should watch one's own stale rate more than the efficiency figure.

Both are interesting: your own stale rate varying shows that something changed on your node and help you pinpoint local problems. But you don't want your efficiency to fall too much. Depending on your use of merged-mining (Namecoin essentially), if you fall below the 90-95% range, a centralized pool begins to makes sense if you can't lower your stale rate and wants to maximize your income.

Well, I am merge-mining Namecoin, Devcoin and Ixcoin, so there is some extra there.

And actually my efficiency is over 100% now that the incoming bitcoind connection issue was solved. Now I'm only interested to see how the 0.8.2rc3 update and increased maxblocksize affected the efficiency. We shall see about that this week.

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: Bitmong on May 28, 2013, 09:21:11 PM

When I browsed through my P2Pool log, I noticed that the P2Pool stale rate varied between 17% and 20%+ in the last couple of days. Therefore the efficiency figure fluctuates even if your own stale / DOA rate doesn't change at all. That is why I think one should watch one's own stale rate more than the efficiency figure.

Both are interesting: your own stale rate varying shows that something changed on your node and help you pinpoint local problems. But you don't want your efficiency to fall too much. Depending on your use of merged-mining (Namecoin essentially), if you fall below the 90-95% range, a centralized pool begins to makes sense if you can't lower your stale rate and wants to maximize your income.

Quote

One problem with reaching conclusions on the data is that one needs quite a high hashrate in order to get a narrow confidence interval on the stale rate in a timeframe of a day or so.

For example, my mining rate is about 4.5 GH/s, and the stale rate interval reported by P2Pool is -+5% of the real stale rate after mining with the pool for three days or so.

Yep, mining with less than 10GH/s makes things difficult. I'm at ~9GH/s and I don't fully trust my stats. I often double-check configuration changes when differences are low enough to be explained by variance in my share/stale rate.

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: Bitmong on May 28, 2013, 09:07:40 PM

Quote from: gyverlb on May 28, 2013, 08:18:40 PM

Quote from: Bitmong on May 28, 2013, 07:44:30 PM

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far (by nearly 3x, the worst I can rember was ~12s). With <0.8.2rc3 getblocktemplate depends heavily on you CPU speed and number of transactions in your memory pool. As you seem to have an adequate CPU, it probably doesn't explain the difference. Do you use non-default values in your bitcoin.conf?

I didn't change any block size / tx fee settings in bitcoin.conf, only rpc settings and server settings.

Actually the latency was around 30 seconds when I used the bitcoind on that computer or when I set P2Pool to use the bitcoin-qt on my Windows computer with Core i7-3820 processor. So, the latency was definitely not caused by the CPU.

Maybe I just upgraded soon enough to not see these kinds of latencies (it was just below 10s at the worst).

Quote from: Bitmong on May 28, 2013, 09:07:40 PM

Quote from: gyverlb

You have relatively high bandwidth, but what is your link latency? If you use traceroute/mtr or similar tools, what is the time-to-hop for your ISP main routers and addresses in North America/Europe/China (where most nodes and probably miners are according to http://blockchain.info/fr/nodes-globe).

I live in Finland btw.

MTR shows these results for some nodes:

First ISP router 2ms
Last ISP router 4ms
US (Sayreville) 111ms
Japan 294ms
Australia 430ms
Hong Kong 340ms
Netherlands 33ms
China 330ms
Switzerland 44ms
Ukraine 57ms

These looks good, I have better European and China results (most of the time <30ms in Europe and ~220ms with China) but I'm puzzled (I don't think it can explain our differences in efficiency).

Did you restart your p2pool node recently and let it run 24h to have a more accurate stat (efficiency isn't accurate with long runs : it compares your own rates of DOA/orphans since the p2pool process start with the average over 24h of the network).

Bitmong

newbie

Activity: 29

Merit: 0

Quote from: gyverlb on May 28, 2013, 08:43:02 PM

I just noticed that my own efficiency started to get down the last 24h (the first 24h with my settings where between 110 and 115% and now I'm at ~105%).
This may be because most P2Pool nodes are upgrading bitcoind and lowering their own getblocktemplate latency in the process. I'll have to test again to see if lowering my getblocktemplate latency helps my efficiency.
My current 0.25s might not be low enough now. One possibility is that this value actually influences efficiency when there's a noticeable difference between its value on a node and the average on all P2Pool nodes. Back to testing different bitcoin.conf values. Too bad: in its current configuration my node was including enough fees to make a 26.5BTC block! Fortunately with 0.8.2 I won't have to make much sacrifices in fees to reach very low latencies.

When I browsed through my P2Pool log, I noticed that the P2Pool stale rate varied between 17% and 20%+ in the last couple of days. Therefore the efficiency figure fluctuates even if your own stale / DOA rate doesn't change at all. That is why I think one should watch one's own stale rate more than the efficiency figure.

One problem with reaching conclusions on the data is that one needs quite a high hashrate in order to get a narrow confidence interval on the stale rate in a timeframe of a day or so.

For example, my mining rate is about 4.5 GH/s, and the stale rate interval reported by P2Pool is -+5% of the real stale rate after mining with the pool for three days or so.

Bitmong

newbie

Activity: 29

Merit: 0

Quote from: gyverlb on May 28, 2013, 08:18:40 PM

Quote from: Bitmong on May 28, 2013, 07:44:30 PM

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far (by nearly 3x, the worst I can rember was ~12s). With <0.8.2rc3 getblocktemplate depends heavily on you CPU speed and number of transactions in your memory pool. As you seem to have an adequate CPU, it probably doesn't explain the difference. Do you use non-default values in your bitcoin.conf?

I didn't change any block size / tx fee settings in bitcoin.conf, only rpc settings and server settings.

Actually the latency was around 30 seconds when I used the bitcoind on that computer or when I set P2Pool to use the bitcoin-qt on my Windows computer with Core i7-3820 processor. So, the latency was definitely not caused by the CPU.

Quote from: gyverlb

You have relatively high bandwidth, but what is your link latency? If you use traceroute/mtr or similar tools, what is the time-to-hop for your ISP main routers and addresses in North America/Europe/China (where most nodes and probably miners are according to http://blockchain.info/fr/nodes-globe).

I live in Finland btw.

MTR shows these results for some nodes:

First ISP router 2ms
Last ISP router 4ms
US (Sayreville) 111ms
Japan 294ms
Australia 430ms
Hong Kong 340ms
Netherlands 33ms
China 330ms
Switzerland 44ms
Ukraine 57ms

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: davidkassa on May 28, 2013, 08:33:41 PM

Quote from: gyverlb on May 28, 2013, 08:18:40 PM

Quote from: Bitmong on May 28, 2013, 07:44:30 PM

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far

My getblocktemplate latency hit about 30 seconds until I upgraded to the 0.8.2rc3.

Now I'm not sure if it's "bad luck" or something else but since I upgraded my latency is ~0.4 but my efficiency has been sitting in the low 80%s. I'm not quite sure how to debug this. My local DOA is ~2.5% but my orphans have been awful. I have both 8333 and 9333 traffic QOS'd. I guess I'm going to look closer at my network.

Check my guide if you see some low hanging fruits.

I just noticed that my own efficiency started to get down the last 24h (the first 24h with my settings where between 110 and 115% and now I'm at ~105%).
This may be because most P2Pool nodes are upgrading bitcoind and lowering their own getblocktemplate latency in the process. I'll have to test again to see if lowering my getblocktemplate latency helps my efficiency.
My current 0.25s might not be low enough now. One possibility is that this value actually influences efficiency when there's a noticeable difference between its value on a node and the average on all P2Pool nodes. Back to testing different bitcoin.conf values. Too bad: in its current configuration my node was including enough fees to make a 26.5BTC block! Fortunately with 0.8.2 I won't have to make much sacrifices in fees to reach very low latencies.

daemondazz

sr. member

Activity: 448

Merit: 250

Hwoever has started mining on cryptominer.org in the last little while, please check your username as it appears you've got an extra 'M' at the start:

Code:

darryl@server1:~$ bitcoind validateaddress M13iETYjuQ1Dnhz4vRPBRHJj6mztu4i86Fs
{
"isvalid" : false
}
darryl@server1:~$ bitcoind validateaddress 13iETYjuQ1Dnhz4vRPBRHJj6mztu4i86Fs
{
"isvalid" : true,
"address" : "13iETYjuQ1Dnhz4vRPBRHJj6mztu4i86Fs",
"ismine" : false
}

Unless of course you're donating your hashrate to me

davidkassa

newbie

Activity: 37

Merit: 0

Quote from: gyverlb on May 28, 2013, 08:18:40 PM

Quote from: Bitmong on May 28, 2013, 07:44:30 PM

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far

My getblocktemplate latency hit about 30 seconds until I upgraded to the 0.8.2rc3.

Now I'm not sure if it's "bad luck" or something else but since I upgraded my latency is ~0.4 but my efficiency has been sitting in the low 80%s. I'm not quite sure how to debug this. My local DOA is ~2.5% but my orphans have been awful. I have both 8333 and 9333 traffic QOS'd. I guess I'm going to look closer at my network.

gyverlb

hero member

Activity: 896

Merit: 1000

Quote from: Bitmong on May 28, 2013, 07:44:30 PM

When the getblocktemplate latency started to appear, my efficiency was still between 110-115%. My getblocktemplate latency was about 30 seconds at that time.

This is the worst latency I've seen reported so far (by nearly 3x, the worst I can rember was ~12s). With <0.8.2rc3 getblocktemplate depends heavily on you CPU speed and number of transactions in your memory pool. As you seem to have an adequate CPU, it probably doesn't explain the difference. Do you use non-default values in your bitcoin.conf?

You have relatively high bandwidth, but what is your link latency? If you use traceroute/mtr or similar tools, what is the time-to-hop for your ISP main routers and addresses in North America/Europe/China (where most nodes and probably miners are according to http://blockchain.info/fr/nodes-globe).

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 536. (Read 2591920 times)