Topic: A guide for mining efficiently on P2Pool, includes FUD repellent and FAQ - page 9. (Read 174901 times)

Well, in the link we have an example of a Jalapeno in the wild running on a p2pool instance with better 100% efficiency.  But thanks for ignoring it.

ckolivas and kano confirmed this. The instruction supposed to be implemented in the communication protocol by BFL to interrupt work doesn't work and BFL didn't answer when asked if/when it will be fixed.

I didn't read the link but a Jalapeno is supposed to have 2 chips for ~5GH/s a whole nonce range is 4GH (nonce is a 32 bit field), do the math...

Are you sure, or are you just going off the FUD on these forums?  Clearly the FPGA products have an issue, but these guys claim the ASICs blow through an entire nonce range in a few ms:
https://forums.butterflylabs.com/jalapeno-single-sc-support/3367-reducing-20%25-hash-rate-penalty-using-bfl-hardware-p2pool.html#post42224

It would be a pity if the p2pool guide was turning away major hashpower unnecessarily.  In fact, looking above the linked post, there is the exact quote from above used to justify not participating in p2pool.

that's the big question 

i didn't build off of 68f2aca3...

Both of you guys have been extremely helpful to me so I'm reluctant to get in the middle here but isn't that what happens when you submit work for an old block.I found this in my log...

[2013-06-19 19:14:18] Stratum from pool 0 detected new block

edit: I realize I could have nothing useful to add here you two obviously are in a different league than me.

edit2: I'm sorry I do see your point now what did happen to '68f2aca3'?

Just for the record, it did this again:

2013-06-19 19:14:17.235456  Shares: 117 (2 orphan, 3 dead) Stale rate: ~4.3% (1-10%) Efficiency: ~121.5% (114-125%) Current payout: 0.3549 BTC
2013-06-19 19:14:19.021332 Skipping from block a3c01ac1b7431fe4f3b141091650172a0ca3f147ad5472ebe8 to block 44635c8d256559a4bbd27e36c84c0b1349102cfc99fd5645c7!
2013-06-19 19:14:19.421899 Punishing share for 'Block-stale detected! a3c01ac1b7431fe4f3b141091650172a0ca3f147ad5472ebe8 < 44635c8d256559a4bbd27e36c84c0b1349102cfc99fd5645c7'! Jumping from 0ed16b1a to 68f2aca3!
2013-06-19 19:14:21.264227 GOT SHARE! Drizztztzzztztzzzzztzz b9803115 prev 0ed16b1a age 2.22s DEAD ON ARRIVAL
2013-06-19 19:14:27.240538  Shares: 118 (2 orphan, 3 dead) Stale rate: ~4.2% (1-10%) Efficiency: ~121.6% (114-125%) Current payout: 0.3576 BTC
2013-06-19 19:14:37.245425  Shares: 118 (2 orphan, 3 dead) Stale rate: ~4.2% (1-10%) Efficiency: ~121.3% (114-125%) Current payout: 0.3576 BTC


though that time it said DOA, but gave me credit for it.

so I guess maybe it's messed up sometimes when that's going on.  maybe in a few of those other cases the same thing happened

though I wonder what '68f2aca3' was. scratch.

You demonstrated that running a public node hurts efficiency by delaying work delivery to miners and you want me to do it?

The worker port is blocked by my firewall, I used to have a public node in 2012 but when I started studying how p2pool behaves in various conditions I blocked the port to avoid any unwanted perturbation (I was worried about bandwidth usage at the time and it seems I should have been worried about P2Pool process delays).

No need to take my word for it or doubt it, just test it yourself and publish your data here for everyone to see and try to reproduce. If your results don't match mine we'll have to find our why: your last test with 1MB blockmaxsize was enlightening and made me create a github issue.

My current configuration for BTC P2Pool is 10 connections (5+5). One month ago it was 6 (3+3). I see the current additional 4 as a safety margin in case P2Pool's behavior changes or my node connects to several "bad" nodes at the same time (6 has been enough and is the lowest I consider relatively safe, I'm advising to use slightly higher values if possible in the guide and follow my own advice and prefer to have a margin when I'm not actively collecting efficiency numbers).

I'm about to test a different setting: less outgoing and more incoming to help people behind firewalls who can't forward ports and see if it lowers my efficiency.

First off... no kidding, there usually isn't some 1 BTC fee transaction that's 250 bytes in size.  Sometimes there is.  Are you arguing that I drop that down to 1500?    1,000,000 obviously is detrimental to *my* income, so why would I use it?  

You need to do some more studying if you think 6 random connections will get "105+ efficiency constantly and most of the time 110+".  I could possibly see a modified source allowing 0 outgoing connections and then 6 chosen connections added via --p2pool-node getting a low # of orphans

i would like to see this mythical node of yours, it isnt listed on http://p2pool.hostv.pl/  for sure.

zvs, you only work by feeling, not stats. Feelings aren't good advisors...

1/
I've studied "6 random connections" for weeks and got 105+ efficiency constantly and most of the time 110+. It works, there's nothing to discuss if you didn't test it and then got bad efficiency repeatedly for days. Nothing is set in stones and things may change in the future, but right now there's no reason to advise to use large amount of connections unless you have hard data showing meaningful efficiency differences. There's one obvious reason to advise against it : it uses bandwidth that is limited: you use a dedicated server where it could be OK (additional CPU usage might make it bad) but many like me use an ADSL connection where it would simply kill efficiency (tested).

2/
your example of 25.915BTC with 3 transactions is just that, an example. Unless things changed dramatically these past days, there's no way you will get this amount of fees constantly with only 2500 block size. And even if it was the case, you'll have to compare both efficiency and fees to what you get with blockmaxsize=1_000_000 and other values to decide if it's advisable to use such a low block size.

I have tons of outgoing connections, I don't get many orphans.  That's the point of all the connections.

Right now I have 1 in 47, and that one occurred at the very end of that 1,000,000 blocksize test...  prior to that, I had the 3 orphaned out of 256...  

6 outgoing connections only certainly isnt optimal, especially if you aren't listening for incoming connections... but, then, I only have 11 so far after node being online for over 12hrs.   One of them connects within seconds (*cough* p2pool-node).  Relying on 6 random outgoing connections to not get orphans?   Some crap shoot


oh, here's a good example of why you should at least do like 2500 block size:

New work for worker! Difficulty: 5.000000 Share difficulty: 1218.324058 Total block value: 25.915000 BTC including 3 transactions

If you are writing about the P2Pool outgoing connections, so many can only hurt. P2Pool is event based in a single thread: by raising the number of connections you raise the rate of events and the time it spends processing them and being unavailable to provide work for miners. It's probably not much time, but if there's no good reasons to have more connections you definitely want to avoid raising them.

As said in my latest update of the guide, raising the total number of connections above 12 (4 outgoing, 8 incoming) didn't raise my efficiency.

I suspect zvs initial tuning results have been collected with older bitcoin and P2Pool versions (zvs latest tests with 1MB block have shown inefficiencies, but AFAIK from his reaction not to the extent zvs expected them).
Different versions behave differently and tuning recommendations must be updated (for example I did witness bad efficiencies with old P2Pool version when getblocktemplate was >0.2s and can't reproduce it anymore even with 0.5s).
I try to explain the process I follow in the guide so that people can try to find their own settings by themselves if my recommendations aren't optimal for them.

I'm still trying to wrap my head around why having all of the outgoing nodes helps. I made some changes zvs suggested dialled back a little with 30 outgoing vs. 60 like him because of bandwidth limitations but I don't get it. I'm not disagreeing because so far so good on my end (still too soon for any real conclusion) but I'm definitely confused.

Very interesting, we'll have to ask forrestv if there's something to be done.

I suspected CPU speed might be a factor, but not as much as that because I don't have a public node anymore: my node only generates work for a single payout address for all my miners (there's only one "New work for workers" line for each new share in my case). I don't know how much latency this is on my node but according to your log, if I were running it on your hardware it would be ~90ms (which isn't so bad: thats less than 1% of the average interval and less than other latencies).

So if we didn't miss something, for people hosting a public node P2Pool work generation is on average x slower than private nodes where is the number of payout addresses.

Not sure about why you are receiving more traffic than sending and how it could affect orphan shares.

I'll link your post in the main p2pool thread.

Well, it's doing better ATM, with 19 shares and just 1 dead.

But, it's also adding a lot of load to the system and I can tell that if you took a large enough sample it'd be worse...    simply because (if nothing else):

2013-06-17 06:45:57.826722 New work for worker! Difficulty: 1.586613 Share difficulty: 1237.801257 Total block value: 25.956635 BTC including 1204 transactions
2013-06-17 06:45:57.914912 New work for worker! Difficulty: 1.586613 Share difficulty: 1237.801257 Total block value: 25.956635 BTC including 1204 transactions
2013-06-17 06:45:58.003061 New work for worker! Difficulty: 1.586613 Share difficulty: 1237.801257 Total block value: 25.956635 BTC including 1204 transactions
2013-06-17 06:45:58.091624 New work for worker! Difficulty: 1.586613 Share difficulty: 1237.801257 Total block value: 25.956635 BTC including 1204 transactions
2013-06-17 06:45:58.182583 New work for worker! Difficulty: 5.000000 Share difficulty: 1237.801257 Total block value: 25.956635 BTC including 1204 transactions

vs

2013-06-17 08:06:45.156392 New work for worker! Difficulty: 1.170742 Share difficulty: 1063.111336 Total block value: 25.207522 BTC including 280 transactions
2013-06-17 08:06:45.180769 New work for worker! Difficulty: 1.170742 Share difficulty: 1063.111336 Total block value: 25.207522 BTC including 280 transactions
2013-06-17 08:06:45.204890 New work for worker! Difficulty: 1.170742 Share difficulty: 1063.111336 Total block value: 25.207522 BTC including 280 transactions
2013-06-17 08:06:45.229186 New work for worker! Difficulty: 1.170742 Share difficulty: 1063.111336 Total block value: 25.207522 BTC including 280 transactions
2013-06-17 08:06:45.256788 New work for worker! Difficulty: 5.000000 Share difficulty: 1063.111336 Total block value: 25.207522 BTC including 280 transactions

vs

2013-06-17 09:22:29.858692 New work for worker! Difficulty: 1.203949 Share difficulty: 1063.354280 Total block value: 25.000000 BTC including 0 transactions
2013-06-17 09:22:29.862099 New work for worker! Difficulty: 1.203949 Share difficulty: 1063.354280 Total block value: 25.000000 BTC including 0 transactions
2013-06-17 09:22:29.865353 New work for worker! Difficulty: 1.203949 Share difficulty: 1063.354280 Total block value: 25.000000 BTC including 0 transactions
2013-06-17 09:22:29.868686 New work for worker! Difficulty: 1.203949 Share difficulty: 1063.354280 Total block value: 25.000000 BTC including 0 transactions
2013-06-17 09:22:29.874997 New work for worker! Difficulty: 5.000000 Share difficulty: 1063.354280 Total block value: 25.000000 BTC including 0 transactions

...  I dunno if it goes by the order it says, but if so, it generates my new work last, and the other four people on my pool before me.  With 1204 transactions, it takes 0.355861 seconds between #1 and #5, with 280 transactions it takes 0.100396 seconds, with 0 transactions it takes 0.016305 seconds.  More people mining at your pool = more pronounced effect.  If my pool was private w/ just myself on it, it'd still generate work, what, about ~.05s slower with all those transactions vs having 0 (though the lowest I'd drop my maxblocksize to would probably have a dozen or so).

In the 1204 case, that's adding 355ms to a 10s cycle, which should result in 3.5% more DOA

ed: didn't check to see whether it was based on # of transactions or size of total transactions.  also, it's adding like *340ms, not 355ms.  machine load could have affected times a bit, but there's still the trend that's plain to see.  not doing anything else on it atm besides running bitcoind and p2pool

ed2: there's also the bizarre thing where now i'm suddenly receiving more data than sending.   logically (to me at least) that seems like it'd result in more orphans?    what's up w/ that, anyway?  anyway, based simply on the ^^ up there that indicates i'm receiving work probably about 150ms slower on average, i'm going to cut this short and reduce my maxblocksize again.  even if you go with an extreme best case scenario and say I find a block w/ 1BTC in transaction fees...  when I receive less than 1/100th of the total pie, that isn't worth 1.5% more DOA..  

though I suppose one could make a case for going with 10000, 20000, or 30000 maxblocksize...  (or setting the minimum relay fee much higher, and then setting maxblocksize to 1000000)

ed3:  oh, and while I was typing ed2, I did just pick up an orphan.  heh.

ed4:  6 transactions

2013-06-17 10:17:59.878132 New work for worker! Difficulty: 1.589736 Share difficulty: 1259.470588 Total block value: 25.004000 BTC including 6 transactions
2013-06-17 10:17:59.881803 New work for worker! Difficulty: 1.589736 Share difficulty: 1259.470588 Total block value: 25.004000 BTC including 6 transactions
2013-06-17 10:17:59.885549 New work for worker! Difficulty: 1.589736 Share difficulty: 1259.470588 Total block value: 25.004000 BTC including 6 transactions
2013-06-17 10:17:59.891466 New work for worker! Difficulty: 1.589736 Share difficulty: 1259.470588 Total block value: 25.004000 BTC including 6 transactions
2013-06-17 10:17:59.898578 New work for worker! Difficulty: 5.000000 Share difficulty: 1259.470588 Total block value: 25.004000 BTC including 6 transactions

0.020446 seconds

2013-06-17 10:23:14.937386 New work for worker! Difficulty: 1.336302 Share difficulty: 1245.582868 Total block value: 25.045000 BTC including 6 transactions
2013-06-17 10:23:14.940579 New work for worker! Difficulty: 1.336302 Share difficulty: 1245.582868 Total block value: 25.045000 BTC including 6 transactions
2013-06-17 10:23:14.943615 New work for worker! Difficulty: 1.336302 Share difficulty: 1245.582868 Total block value: 25.045000 BTC including 6 transactions
2013-06-17 10:23:14.946616 New work for worker! Difficulty: 1.336302 Share difficulty: 1245.582868 Total block value: 25.045000 BTC including 6 transactions
2013-06-17 10:23:14.952521 New work for worker! Difficulty: 5.000000 Share difficulty: 1245.582868 Total block value: 25.045000 BTC including 6 transactions

0.015135 seconds..   0 transactions probably comes with some base amount, i doubt the 6 adds much time at all

I'm extremely curious about the results. If it indeed kills your efficiency, we'll have to understand why it happens on your setup and not mine (I even lowered mintxfee and minrelaytxfee to make it easier for bitcoind to fill the blocks). I'm still at 110+ efficiency with getmininginfo confirming that the block templates generated by bitcoind are indeed ~1MB.

I only know of 2 reasons why it could kill your efficiency:

• it generates too much traffic, filling your pipe (reduce your bitcoind and p2pool number of connections). It can happen even with large pipes if your hoster have some peering limitation.
• your CPU is maxed out (you should have one core free for P2Pool to be safe)

Maybe we will find another one to document in the guide.

OK, so right now, my server is reading:

Node uptime: 3.124 days Peers: 73 out, 19 in

Shares: 256 total (3 orphaned, 14 dead) Efficiency: 116.2%

I started pinging the first hop (not counting my actual router) instead of the server, so the ping times are about 180ms lower than what I'd get to nogleg.com.  I set the packetloss scale to 10%, since local packetloss is rly, rly bad (a remote ping would show 10x as much packetloss).  Started doing this so I can give this info to ISP, which continues to deny bandwidth exhaust issue...

but, anyway:



Out of those 256 shares, about 85% are mine & the number of orphans/DOA from the other people wasn't disproportionate to my own.

So, now I'll set maxblocksize to 1000000 (!!), and restart the server.  Not a huge deal anymore since even with my 6.7c or so electricity, bitcoins are just marginal profit nowadays.  Don't mind a lot more DOA/orphans for the next day or two for the sake of showing that this setting kills.

I apologize to the other ppl that are pointed to my server atm.  Maybe come back in a few days?

Just updated the guide with a reference to zvs results with 300-350ms latency between node and miners.

Added more details about what/how to tune and why instead of blindly lowering getblocklatency without understanding the consequences.

It may not be enough: these CPUs are weak.
How many shares did you get when your efficiency was 75%? With your efficiency if you didn't have at least 50 shares, the value is probably not reliable yet.

Is a Intel Atom D525 CPU (1.8 GHz, dual-core) processor just not enough to run p2pool/bitcoind?

I have one of these: http://www.amazon.com/ZOTAC-Intel-Barebone-Mini-PC-ZBOX-ID41-U/dp/B004WO8O9Y/ref=sr_1_sc_1?ie=UTF8&qid=1371350342&sr=8-1-spell&keywords=zotac+zebox with an SSD.

Loads are around 0.8 constantly on Ubuntu 12.04 x64 (the computer isnt used for anything else). My efficiency is shit (like ~75%). Tried setting up QoS and tuning with your guide, but it didn't seem to help I'm thinking maybe the computer simply isn't powerful enough to handle p2pool/bitcoind.

I'm at 32 shares, about 25 being my own, w/ 1 dead share, thru the early morning hours-4PM.  Now is when my link gets really crappy, so will probably be around 15-20% DOA from here on out.

Anyway, ^^ is luck.  But it's 10s per cycle, so 180ms latency shouldn't cause a whole lot of DOA.  Packetloss will.