Pages:
Author

Topic: A guide for mining efficiently on P2Pool, includes FUD repellent and FAQ - page 5. (Read 174909 times)

legendary
Activity: 2912
Merit: 1060
Would this server cope well for a merged-mining P2Pool node - http://www.supermicro.com/products/system/1U/5018/SYS-5018A-FTN4.cfm - It's only Atom CPU based but it has eight cores.  So a core each for each of the five wallets, a core for P2Pool itself, a core for the OS and one spare core left over.

Sorry how do you plan to assign which CPU gets what core?



I know in Windows you can but he probably meant hypothetically
Try it, cpu affinity
hero member
Activity: 630
Merit: 501
Would this server cope well for a merged-mining P2Pool node - http://www.supermicro.com/products/system/1U/5018/SYS-5018A-FTN4.cfm - It's only Atom CPU based but it has eight cores.  So a core each for each of the five wallets, a core for P2Pool itself, a core for the OS and one spare core left over.

Sorry how do you plan to assign which CPU gets what core?

legendary
Activity: 2912
Merit: 1060
Would this server cope well for a merged-mining P2Pool node - http://www.supermicro.com/products/system/1U/5018/SYS-5018A-FTN4.cfm - It's only Atom CPU based but it has eight cores.  So a core each for each of the five wallets, a core for P2Pool itself, a core for the OS and one spare core left over.

That's a nice cpu
Not the old atom
legendary
Activity: 1372
Merit: 1003
Would this server cope well for a merged-mining P2Pool node - http://www.supermicro.com/products/system/1U/5018/SYS-5018A-FTN4.cfm - It's only Atom CPU based but it has eight cores.  So a core each for each of the five wallets, a core for P2Pool itself, a core for the OS and one spare core left over.
hero member
Activity: 798
Merit: 1000
That's the thing.  I'd love a getBlockTemplate of 0.2s.  Unfortunately my node lately is above 1.0s. That's what I need help with (if it does help any).

If you don't have low efficiency then getBlockTemplate being above 1.0s isn't a problem in itself.

The advice I gave above should lower your getBlockTemplate latency. The minimum fees are used to select which transactions are kept in your bitcoind's memory and which ones are included in the block template given to p2pool. If you raise these fees just a bit your node won't have to do as much work and your latency will lower. If you raise them to 0.0001 you can expect <0.2s according to the recent amount of txs.

Note that if you target a latency you'll have to change these values when more/less transactions are waiting to be included in a block (which is not doable as these changes happen unpredictably). This is why I advise to leave them alone if the efficiency is good: the target should be to get good income and good behavior on the network not reaching arbitrary latency targets.

Thanks. Efficiency has been bouncing around 100 but is 94 right now. Will try mintxfee and minrelaytxfee at 0.00006 and tweak from there.
hero member
Activity: 896
Merit: 1000
That's the thing.  I'd love a getBlockTemplate of 0.2s.  Unfortunately my node lately is above 1.0s. That's what I need help with (if it does help any).

If you don't have low efficiency then getBlockTemplate being above 1.0s isn't a problem in itself.

The advice I gave above should lower your getBlockTemplate latency. The minimum fees are used to select which transactions are kept in your bitcoind's memory and which ones are included in the block template given to p2pool. If you raise these fees just a bit your node won't have to do as much work and your latency will lower. If you raise them to 0.0001 you can expect <0.2s according to the recent amount of txs.

Note that if you target a latency you'll have to change these values when more/less transactions are waiting to be included in a block (which is not doable as these changes happen unpredictably). This is why I advise to leave them alone if the efficiency is good: the target should be to get good income and good behavior on the network not reaching arbitrary latency targets.
hero member
Activity: 798
Merit: 1000

What do you expect to get out of a dual Gbit connection when your internet is only 75/35Mbit/s? Your internet isn't anywhere near 1GbE so there no gain in what have have done. Smiley

The dual Link Aggregated Connection has nothing to do with my WAN bandwidth. It was to solve the following performance issues I was having:

1. Connection Starvation: My MacPro serves up multiple services on my LAN with p2pool, Bitcoin node, Namecoin Node, Stratum Proxy for Scrypt Mining, individual miners connecting to it, Apache reverse proxy and gateway to outside access of my network, Media serving (large HD video and FLAC music), etc. Even maxing out custom TCP/IP settings for connections didn't fully solve the contention and queuing I was seeing. LAG solved all of it.

2. Round Robin / Load Balancing across ethernet connections: I'm now able to support the above service connections AND transcode video to my SAN box with no/little constraints.  Previously whenever I transcoded video, it ground everything else to a halt, which put a damper on my p2pool mining node.  And I do transcode video often. No more issues.

3. Internal LAN QoS is much easier with more connections available. I can manage my LAN latency-defeating rules for certain traffic in a more flexible way now that I have more paths available to my main Mac Pro server. I can set priorities for various connections across my LAN and not have to worry about the traffic jam that I used to get to my Mac's IP.

Link Aggregation is almost never about pure bandwidth.  It's about freeing up queues and load balancing socket contention so you can support more parallel services on a single server, especially ones that are latency adverse.

I'd still like to see someone else answer my original questions above regarding dbcache setting.  Undecided

Thanks for the clarification, and good luck with that, but teaming as it's called shouldn't be required for you are doing if you have the right setup.


The "Right Setup" is exactly what this is.  Especially when dealing with multi stream uncompressed video to and from my SAN for edits. (#2 above)   Roll Eyes.

Nevermind. Obviously you didn't read anything I wrote.  
hero member
Activity: 798
Merit: 1000
I'd still like to see someone else answer my original questions above regarding dbcache setting.  Undecided

I don't think the dbcache setting is interesting. Even when bitcoind/qt was purely BerkeleyDB I didn't see any performance gains with large cache values. Now bitcoind/qt rely on LevelDB for a large part of it's data so if dbcache is only related to the BerkeleyDB database it is probably even less effective.

0.2s for getblocktemplate isn't bad at all especially with the changes of last summer which made p2pool even less dependent on getblocktemplate latency.

If your efficiency is ~100%, I wouldn't change anything. If it's below I'd advise raising the mintxfee and minrelaytxfee every 24h (maybe double it on each change) until you either reach the default value of 0.0001 or reach the 100% efficiency level.

That's the thing.  I'd love a getBlockTemplate of 0.2s.  Unfortunately my node lately is above 1.0s. That's what I need help with (if it does help any).
hero member
Activity: 896
Merit: 1000
I'd still like to see someone else answer my original questions above regarding dbcache setting.  Undecided

I don't think the dbcache setting is interesting. Even when bitcoind/qt was purely BerkeleyDB I didn't see any performance gains with large cache values. Now bitcoind/qt rely on LevelDB for a large part of it's data so if dbcache is only related to the BerkeleyDB database it is probably even less effective.

0.2s for getblocktemplate isn't bad at all especially with the changes of last summer which made p2pool even less dependent on getblocktemplate latency.

If your efficiency is ~100%, I wouldn't change anything. If it's below I'd advise raising the mintxfee and minrelaytxfee every 24h (maybe double it on each change) until you either reach the default value of 0.0001 or reach the 100% efficiency level.
hero member
Activity: 630
Merit: 501

What do you expect to get out of a dual Gbit connection when your internet is only 75/35Mbit/s? Your internet isn't anywhere near 1GbE so there no gain in what have have done. Smiley

The dual Link Aggregated Connection has nothing to do with my WAN bandwidth. It was to solve the following performance issues I was having:

1. Connection Starvation: My MacPro serves up multiple services on my LAN with p2pool, Bitcoin node, Namecoin Node, Stratum Proxy for Scrypt Mining, individual miners connecting to it, Apache reverse proxy and gateway to outside access of my network, Media serving (large HD video and FLAC music), etc. Even maxing out custom TCP/IP settings for connections didn't fully solve the contention and queuing I was seeing. LAG solved all of it.

2. Round Robin / Load Balancing across ethernet connections: I'm now able to support the above service connections AND transcode video to my SAN box with no/little constraints.  Previously whenever I transcoded video, it ground everything else to a halt, which put a damper on my p2pool mining node.  And I do transcode video often. No more issues.

3. Internal LAN QoS is much easier with more connections available. I can manage my LAN latency-defeating rules for certain traffic in a more flexible way now that I have more paths available to my main Mac Pro server. I can set priorities for various connections across my LAN and not have to worry about the traffic jam that I used to get to my Mac's IP.

Link Aggregation is almost never about pure bandwidth.  It's about freeing up queues and load balancing socket contention so you can support more parallel services on a single server, especially ones that are latency adverse.

I'd still like to see someone else answer my original questions above regarding dbcache setting.  Undecided

Thanks for the clarification, and good luck with that, but teaming as it's called shouldn't be required for you are doing if you have the right setup.
hero member
Activity: 798
Merit: 1000

What do you expect to get out of a dual Gbit connection when your internet is only 75/35Mbit/s? Your internet isn't anywhere near 1GbE so there no gain in what have have done. Smiley

The dual Link Aggregated Connection has nothing to do with my WAN bandwidth. It was to solve the following performance issues I was having:

1. Connection Starvation: My MacPro serves up multiple services on my LAN with p2pool, Bitcoin node, Namecoin Node, Stratum Proxy for Scrypt Mining, individual miners connecting to it, Apache reverse proxy and gateway to outside access of my network, Media serving (large HD video and FLAC music), etc. Even maxing out custom TCP/IP settings for connections didn't fully solve the contention and queuing I was seeing. LAG solved all of it.

2. Round Robin / Load Balancing across ethernet connections: I'm now able to support the above service connections AND transcode video to my SAN box with no/little constraints.  Previously whenever I transcoded video, it ground everything else to a halt, which put a damper on my p2pool mining node.  And I do transcode video often. No more issues.

3. Internal LAN QoS is much easier with more connections available. I can manage my LAN latency-defeating rules for certain traffic in a more flexible way now that I have more paths available to my main Mac Pro server. I can set priorities for various connections across my LAN and not have to worry about the traffic jam that I used to get to my Mac's IP.

Link Aggregation is almost never about pure bandwidth.  It's about freeing up queues and load balancing socket contention so you can support more parallel services on a single server, especially ones that are latency adverse.

I'd still like to see someone else answer my original questions above regarding dbcache setting.  Undecided
hero member
Activity: 630
Merit: 501
hero member
Activity: 798
Merit: 1000
Hi. Would like to ask about latency
Why going below 0.2s will hurt your income and everyone else's on P2Pool. ?
What is the reason of that?

It doesn't.

It does because lowering the getblocktemplate latency is a tradeoff.
As explained in the guide with current bitcoind version reducing this latency is done by reducing the number of transactions you include in the template. Doing so you reduce the fees included in the blocks you find : you reduce your own income and everyone else's when this happens.

Today this latency has only a negative effect when a new block is found by the whole network (every ~10 minutes) unless you have a very weak CPU (where CPU usage by bitcoind can slow down P2Pool).
This is the same effect for P2Pool and every other pool : until a getblocktemplate returns a result after a new block, the pool can only work on an empty template (no transaction). Going below 0.2s only tries to reduce this 0.2/600 = 0.03% negative impact on the fees income (the block reward isn't impacted) : including more transactions (which amounts to ~1% of the average block value) simply brings more income to everyone including yourself.

Continuing the discussion about the getBlockTemplate latency...

I see quite a few public nodes have getBlockTemplate latencies approaching that magic 0.2 mark... but I've always struggled.

My p2pool node used to be in the 1.5 to 2.0 range until about a month ago when I tweaked it further per your guide and other suggestions on the net.  That brought it down to the 0.5 to 0.7 range while trying to retain a high blockmaxsize for greater income.
Lately (or it seems right when the whole Bitcoin DDOS started happening in the last week), my latencies have bumped back up to the 1.0 range with occasional spikes much higher.  So let's discount that bump up for now.

My real question is the efficacy of the dbcache and datadir settings.  I've upped my dbcache from the default 25 to 1000.  I have quite a bit of free memory to use, but not quite enough to host the entire Blockchain on a ramdrive, so datadir is probably out.  Will I see any gains on getBlockTemplate Latency from upping dbcache further to lets say 2000, 3000, or even 5000 megabytes?

The Blockchain (as well as the OS and Apps) are all on an SSD already, running OSX Mavericks and using the Bitcoin-QT OMG10 0.8.5 client (waiting for the official 0.9 client to be out before I upgrade since OMG10 already has most of what went into 0.8.6 and I can't find a binary download for the Mac version of the 0.9 preview release) with wallet disabled.

Connection is over Verizon FiOS 75/35 and the Mac Pro is connected to my managed switch with dual gigabit ethernet with Link Aggregation.

Here's my bitcoin.conf (with certain parameters redacted of course):
Code:
rpcuser=XXXXREDACTEDXXXX
rpcpassword=XXXXREDACTEDXXXX
server=1
rpcport=8332
disablewallet=1
dbcache=1000
maxconnections=26
blockmaxsize=1000000
mintxfee=0.00001
minrelaytxfee=0.00001
addnode=67.186.224.85
addnode=88.198.58.172

hero member
Activity: 630
Merit: 501
Does anyone use cable internet 35/4Meg, or anything close to it? I just wonder what your maxconnections is set to?

Thanks,
hero member
Activity: 630
Merit: 501
FYI, QOS only works on OUTGOING traffic only your LAN, doesn't work on traffic coming back into your network from your WAN port.

So shaping the outgoing traffic is only good if you are running a porn site on the same network as your P2Pool! Smiley


Not true... Depends on the features of your router.

Mine allows detailed QoS priority settings for Outgoing and Incoming connections separately across the WAN <-> LAN and can also limit settings across various LAN IPs.

If your router only supports outgoing QoS, it's a shitty router/switch.

I think it depends on if you set it using a port or mac. Also, your router can't really control the next router.

This true, you can't control the ISP's router which sending you the traffic which might explain why the only routers I have seen that provide inbound WAN are enterprise level ones that ISP's would use. I doubt any bitcoin miner would want to become an ISP just so they can have QOS WAN inbound working properly.

Enterprise = products from RiverBed and Cisco.  

I have the somewhat crappy consumer Router that Verizon FioS customers get and like I said above, it supports Inbound and Outbound QoS settings tied to Port, IP, MAC, etc.   You can also get pretty granular with your priority tiers and how it manages them.


Note: this is not my setup...just using an example image from the 'net.  My settings are actually a bit more elaborate for my various development services, mining related stuff (bitciond, namecoind, p2pool, stratum proxy etc) and my media stuff.

And that's not even going into the settings for Traffic Shaping, DSCP, 802.1p, etc.

And this router is hardly Enterprise Class.

In short; there is a lot of delicate information on this topic.

You can't directly QoS traffic coming inbound. When the traffic reaches your router, there's little you can do about it. What you do, is slow the rate of TCP replies, or start dropping packets to limit a certain type, say HTTP traffic, inbound. So, those settings he sees is the router trying to QoS traffic inbound indirectly.

It may work, it may not. Ultimately you're at the mercy of whoever is sending you traffic, because once traffic has arrived at your router to QoS, there's nothing that you can do at that point; it's already gone over your internet connection to reach your router, taking up your bandwidth. And even then, the indirect QoS can only work on TCP traffic. UDP traffic is connectionless, and no amount of dropped packets will slow it down. anything will send UDP traffic as fast as it can.

You see, when applying QoS to traffic, you put certain types of traffic (HTTP, HTTPS, etc.) into different priority queues (High, medium, and low in this case). The router only applies those queues when sending traffic outbound, because it can't do anything directly to traffic inbound.

So, as an example:

HTTP traffic is to be put in a low priority queue. Coming inbound, it would traverse your internet connection, and arrive at your router. Then, your router would apply the QoS policy to put it in a low priority queue, which would send it out to your LAN after higher priority traffic, say your bitcoin miner and vice versa when going outbound. Your router can try to drop packets to stem the flow of traffic, hoping that TCP's algorithms take hold and slow down traffic once the other end realizes that traffic is being dropped. But then you end up with dropped packets, etc. in short, your router cannot QoS traffic directly that has not arrived at it yet.


   

   
hero member
Activity: 798
Merit: 1000
FYI, QOS only works on OUTGOING traffic only your LAN, doesn't work on traffic coming back into your network from your WAN port.

So shaping the outgoing traffic is only good if you are running a porn site on the same network as your P2Pool! Smiley


Not true... Depends on the features of your router.

Mine allows detailed QoS priority settings for Outgoing and Incoming connections separately across the WAN <-> LAN and can also limit settings across various LAN IPs.

If your router only supports outgoing QoS, it's a shitty router/switch.

I think it depends on if you set it using a port or mac. Also, your router can't really control the next router.

This true, you can't control the ISP's router which sending you the traffic which might explain why the only routers I have seen that provide inbound WAN are enterprise level ones that ISP's would use. I doubt any bitcoin miner would want to become an ISP just so they can have QOS WAN inbound working properly.

Enterprise = products from RiverBed and Cisco.  

I have the somewhat crappy consumer Router that Verizon FioS customers get and like I said above, it supports Inbound and Outbound QoS settings tied to Port, IP, MAC, etc.   You can also get pretty granular with your priority tiers and how it manages them.


Note: this is not my setup...just using an example image from the 'net.  My settings are actually a bit more elaborate for my various development services, mining related stuff (bitciond, namecoind, p2pool, stratum proxy etc) and my media stuff.

And that's not even going into the settings for Traffic Shaping, DSCP, 802.1p, etc.

And this router is hardly Enterprise Class.
hero member
Activity: 630
Merit: 501
FYI, QOS only works on OUTGOING traffic only your LAN, doesn't work on traffic coming back into your network from your WAN port.

So shaping the outgoing traffic is only good if you are running a porn site on the same network as your P2Pool! Smiley


Not true... Depends on the features of your router.

Mine allows detailed QoS priority settings for Outgoing and Incoming connections separately across the WAN <-> LAN and can also limit settings across various LAN IPs.

If your router only supports outgoing QoS, it's a shitty router/switch.

I think it depends on if you set it using a port or mac. Also, your router can't really control the next router.

This true, you can't control the ISP's router which sending you the traffic which might explain why the only routers I have seen that provide inbound WAN are enterprise level ones that ISP's would use. I doubt any bitcoin miner would want to become an ISP just so they can have QOS WAN inbound working properly.

Enterprise = products from RiverBed and Cisco. 
legendary
Activity: 2912
Merit: 1060
FYI, QOS only works on OUTGOING traffic only your LAN, doesn't work on traffic coming back into your network from your WAN port.

So shaping the outgoing traffic is only good if you are running a porn site on the same network as your P2Pool! Smiley


Not true... Depends on the features of your router.

Mine allows detailed QoS priority settings for Outgoing and Incoming connections separately across the WAN <-> LAN and can also limit settings across various LAN IPs.

If your router only supports outgoing QoS, it's a shitty router/switch.

I think it depends on if you set it using a port or mac. Also, your router can't really control the next router.
hero member
Activity: 798
Merit: 1000
FYI, QOS only works on OUTGOING traffic only your LAN, doesn't work on traffic coming back into your network from your WAN port.

So shaping the outgoing traffic is only good if you are running a porn site on the same network as your P2Pool! Smiley


Not true... Depends on the features of your router.

Mine allows detailed QoS priority settings for Outgoing and Incoming connections separately across the WAN <-> LAN and can also limit settings across various LAN IPs.

If your router only supports outgoing QoS, it's a shitty router/switch.
hero member
Activity: 630
Merit: 501
Figuring this is a bitcoin forum its more than likely the OP is talking about Bitcoin only.




Thanks for this guide,

I am wondering if the tuning perimeters below from your guide are applicable to litecoin and other script currencies as well?

I ready through all 8 pages and did not see any mention of this sorry if I missed it.

Thanks

blockmaxsize=1000000
mintxfee=0.00001
minrelaytxfee=0.00001

maxconnections=10 # 125 is the default, don't go below 8

--max-conns 8 --outgoing-conns 4


blockmaxsize=250000 #default is 500000
If you don't have this problem, you should raise the blockmaxsize value instead to get more income:
Code:
blockmaxsize=1000000 #default is 500000

# default is 500000, 1000000 is the maximum allowed and will fit more transactions (more fees)
blockmaxsize=1000000
#Fee-per-kilobyte amount (in BTC) considered the same as "free"
#Be careful setting this: if you set it to zero then
#a transaction spammer can cheaply fill blocks using
#1-satoshi-fee transactions. It should be set above the real
#cost to you of processing a transaction.
mintxfee=0.00001
# Same but for relaying the tx to our peers
minrelaytxfee=0.00001
Pages:
Jump to: