Python is inherently single-threaded. https://wiki.python.org/moin/GlobalInterpreterLock
It would be far easier and more effective to remove most of the transaction-handling code from p2pool, and make share objects contain only the merkle root plus the coinbase transaction (and the merkle path thereto). However, I have not had the time to do that, and nobody else seems to be interested in doing it. Even though it would be much easier, it's not super easy to do.
Note: Bitcoin Core is *also* mostly single-threaded, although not quite to the same extent as p2pool.
Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 10. (Read 2591626 times)
January 12, 2018, 11:47:52 PM
January 12, 2018, 06:07:54 PM
Is there any way to make p2pool mode modern so it can use multithreading and multi cores etc?
January 12, 2018, 02:48:05 PM
CPU load is under 20% per core.
"Per core" is not meaningful here. Python can only use one core at a time, and it's bouncing around between all four cores. If it's using about 20% of each core, that means that 80% of the time it is using at least one core, which is too much.Your getblocktemplate latency graph shows the same thing. It's taking about 2 seconds for p2pool/python to process the GBT results returned by bitcoind, which indicates that p2pool has a short backlog of tasks for it to perform that is delaying the GBT processing.
Reloading the webpage is sluggish (~2 second delay), which indicates the same thing.
You're running into CPU load issues. Try pypy and/or a CPU with faster single-threaded performance. Reducing your peer count can also help.
Thank you so much. Will try it and see if it helps.
January 12, 2018, 02:43:16 PM
CPU load is under 20% per core.
"Per core" is not meaningful here. Python can only use one core at a time, and it's bouncing around between all four cores. If it's using about 20% of each core, that means that 80% of the time it is using at least one core, which is too much.Your getblocktemplate latency graph shows the same thing. It's taking about 2 seconds for p2pool/python to process the GBT results returned by bitcoind, which indicates that p2pool has a short backlog of tasks for it to perform that is delaying the GBT processing.
Reloading the webpage is sluggish (~2 second delay), which indicates the same thing.
You're running into CPU load issues. Try pypy and/or a CPU with faster single-threaded performance. Pypy is likely to make a bigger difference than a CPU upgrade. Reducing your peer count can also help.
January 12, 2018, 02:33:07 PM
There are two likely causes: One, maybe your mining hardware is not compatible with the frequent work changes that p2pool requires. Two, maybe your node is slow.
1) What type of miner are you using? If you point it to someone else's node which is known to be working well (e.g. ml.toom.im:9332), what DOA rate do you get?
2) You'll have better results if you use pypy. That's easiest to do if you're running Linux, but you may also be able to get it to work using the Windows Subsystem for Linux:
https://bitcointalksearch.org/topic/m.21025074
You can also check your CPU usage to see if it ever gets pegged at 25% (i.e. 1 full core used).
P.S.: The Bitcoin Cash p2pool has different chain parameters that results in less work switching, so you will see better performance and efficiency there. However, it doesn't have much hashrate yet, so it might be a while before it finds a block.
P.P.S.: A higher dbcache setting would be a good idea, though it won't solve your problem. Also, blockmaxweight=4000000 is safe on 1mb_segwit.
I have 10 Antminer S9, but testing it with just one until get the node working. I tried different nodes and everything works fine there with DOA around 4-6%. CPU load is under 20% per core.1) What type of miner are you using? If you point it to someone else's node which is known to be working well (e.g. ml.toom.im:9332), what DOA rate do you get?
2) You'll have better results if you use pypy. That's easiest to do if you're running Linux, but you may also be able to get it to work using the Windows Subsystem for Linux:
https://bitcointalksearch.org/topic/m.21025074
You can also check your CPU usage to see if it ever gets pegged at 25% (i.e. 1 full core used).
P.S.: The Bitcoin Cash p2pool has different chain parameters that results in less work switching, so you will see better performance and efficiency there. However, it doesn't have much hashrate yet, so it might be a while before it finds a block.
P.P.S.: A higher dbcache setting would be a good idea, though it won't solve your problem. Also, blockmaxweight=4000000 is safe on 1mb_segwit.
Software is: Windows Server 2012 R2 64 bit, Python 2.7, Twisted 15.1.0, Zope.Interface 4.1.2, Python win32 api, Python win32 api wmi wrapper
Pool address is http://gminv.ge:9332/static/
Graphs are here: http://gminv.ge:9332/static/graphs.html?Day
January 12, 2018, 01:52:08 PM
There are two likely causes: One, maybe your mining hardware is not compatible with the frequent work changes that p2pool requires. Two, maybe your node is slow.
1) What type of miner are you using? If you point it to someone else's node which is known to be working well (e.g. ml.toom.im:9332), what DOA rate do you get?
2) You'll have better results if you use pypy. That's easiest to do if you're running Linux, but you may also be able to get it to work using the Windows Subsystem for Linux:
https://bitcointalksearch.org/topic/m.21025074
You can also check your CPU usage to see if it ever gets pegged at 25% (i.e. 1 full core used).
P.S.: The Bitcoin Cash p2pool has different chain parameters that results in less work switching, so you will see better performance and efficiency there. However, it doesn't have much hashrate yet, so it might be a while before it finds a block.
P.P.S.: A higher dbcache setting would be a good idea, though it won't solve your problem. Also, blockmaxweight=4000000 is safe on 1mb_segwit.
1) What type of miner are you using? If you point it to someone else's node which is known to be working well (e.g. ml.toom.im:9332), what DOA rate do you get?
2) You'll have better results if you use pypy. That's easiest to do if you're running Linux, but you may also be able to get it to work using the Windows Subsystem for Linux:
https://bitcointalksearch.org/topic/m.21025074
You can also check your CPU usage to see if it ever gets pegged at 25% (i.e. 1 full core used).
P.S.: The Bitcoin Cash p2pool has different chain parameters that results in less work switching, so you will see better performance and efficiency there. However, it doesn't have much hashrate yet, so it might be a while before it finds a block.
P.P.S.: A higher dbcache setting would be a good idea, though it won't solve your problem. Also, blockmaxweight=4000000 is safe on 1mb_segwit.
January 12, 2018, 09:39:00 AM
Hello. I’m trying to setup and make bitcoin node up and running (code was taken from here https://github.com/jtoomim/p2pool/tree/1mb_segwit), but it doesn’t work correctly since I’m getting high level of DOA (more than 20%). Miner and server located in the same network. Please help to make it working properly.
Server config: Xeon E3-1225, 8Gb RAM, SSD drive, OS Windows Server 2012 r2
bitcoin.conf:
I’m getting a lot of errors such as these ones:
I'll be thankful for any help.
Server config: Xeon E3-1225, 8Gb RAM, SSD drive, OS Windows Server 2012 r2
bitcoin.conf:
Code:
server=1
disablewallet=1
listen=1
daemon=1
prune=10240
rpcuser=*
rpcpassword=*
maxconnections=20
blockmaxweight=3950000
mintxfee=0.00001
minrelaytxfee=0.00001
dbcache=500
maxmempool=500
rpcthreads=50
I’m getting a lot of errors such as these ones:
Code:
2018-01-12 11:13:05.602000 Generating a share with 1108592 bytes, 3945962 WU (ne
w: 1114 B, 4456 WU) in 1431 tx (3 new), plus est gentx of 1702 bytes/6844 WU
2018-01-12 11:13:05.602000 Total block stripped size=947572 B, full size=1110374
B, weight: 3953046 WU
2018-01-12 11:13:06.055000 WARNING! Previous share's timestamp is 5 seconds in t
he future. This is not normal.
2018-01-12 11:13:06.055000 Make sure your system clock is accurate. Errors beyon
d 300 sec result in orphaned shares.
2018-01-12 11:12:11.836000 New work for 1FwAD8EinPKnW14LxQ7ug21Lw4yNMXNjfx! Diff
: 355190.15 Share diff: 1390473.73 Block value: 16.31 BTC (1321 tx, 1086 kB)
2018-01-12 11:12:15.836000 > Unhandled error in Deferred:
2018-01-12 11:12:15.836000 > Unhandled Error
2018-01-12 11:12:15.836000 > Traceback (most recent call last):
2018-01-12 11:12:15.836000 > Failure: twisted.internet.defer.TimeoutError: in Re
plyMatcher
2018-01-12 11:18:42.684000 > Peer referenced unknown transaction 34bd59d5478ebec
e779d6c84a2ab2eceb49fa7afe0293fb3bef781be9efa14d1, disconnecting
2018-01-12 11:18:42.684000 Peer sent entire transaction c5398526266124c129a276f3
cc0e388523207f9ad48289133604d28a90415de1 that was already received
2018-01-12 11:18:42.872000 Lost peer 178.63.18.3:9333 - Connection was aborted l
ocally, using.
I'll be thankful for any help.
January 10, 2018, 08:53:41 PM
Is a merge of the two networks a possibility? Main net has increased to 3.1Ph/s and Jtoomim to 7.6Ph/s. A combined 10.7Ph/s would be good to see and may help attract more to the pool.
The 3 PH/s spike on mainnet is over. It was probably just someone renting on Nicehash for an hour or so. Mainnet is back to ~1 PH/s.It is not possible to "merge" the two networks. They are separate share chains. What is possible is for people to switch chains so that everyone is on the same chain.
Also the mainnet reward is much more often in line with jtoomim these days. Presumably some development has taken place there...?
No, it's not. At this particular moment, I see:16.72565364 BTC/block on mainnet in 1681 transactions
17.48822478 BTC/block on jtoomimnet in 2026 transactions
You have to look at the actual shares being mined to see the difference. The "Current block value" metric on a node's front page shows the value before mainnet's transaction trimming takes place.
Something to keep in mind is that the bug/design flaw on mainnet that forces small blocks is intermittent. It shows up when a Bitcoin block has been recently published, and disappears a few minutes later. Just because they show the same thing at one moment doesn't mean it will always be the same.
Is is possible to run P2Pool on 64bit versions of Python, etc??
Both 32 bit and 64 bit versions of Python should work. I don't think I've tested with 32 bit versions before, though. January 10, 2018, 07:05:45 PM
The cover page to P2Pool doesn't say anything about needing to run 32bit versions of everything or if 64bit versions can be run. Is is possible to run P2Pool on 64bit versions of Python, etc??
Has anyone done a how-to?
Thanks
Has anyone done a how-to?
Thanks
January 10, 2018, 06:06:21 PM
Hey People,
i need an P2Pool Mining Server to connect my Miner. is there one without or low Fee, free for all ?
i used befor 178.63.18.3:9332 but it seams to denie my get works quests :-/
i need an P2Pool Mining Server to connect my Miner. is there one without or low Fee, free for all ?
i used befor 178.63.18.3:9332 but it seams to denie my get works quests :-/
Hi Xantus,
I have stratum+tcp://5.133.177.158:9332 avaliable on p2pool mainnet or stratum+tcp://5.133.177.158:9334 if you prefer to mine on p2pool\jtoomim (both with a low 0.5% fee and 0.5% donation to the developer). I'm in Sheffield, England (UK) so depending on your location it may or may not be quicker. See my site www.mine4.com for more info and current ping times from your location.
Jtoomim,
Is a merge of the two networks a possibility? Main net has increased to 3.1Ph/s and Jtoomim to 7.6Ph/s. A combined 10.7Ph/s would be good to see and may help attract more to the pool. Also the mainnet reward is much more often in line with jtoomim these days. Presumably some development has taken place there...?
All the best.
James.
January 10, 2018, 04:18:30 PM
i need an P2Pool Mining Server to connect my Miner. is there one without or low Fee, free for all ?
ml.toom.im:9332 and ml.toom.im:9334 are available for public use. Zero fee. January 10, 2018, 02:34:10 PM
Hey People,
i need an P2Pool Mining Server to connect my Miner. is there one without or low Fee, free for all ?
i used befor 178.63.18.3:9332 but it seams to denie my get works quests :-/
i need an P2Pool Mining Server to connect my Miner. is there one without or low Fee, free for all ?
i used befor 178.63.18.3:9332 but it seams to denie my get works quests :-/
January 09, 2018, 09:48:01 PM
Any news on P2Pool merge?
None yet. Still need to get altcoins working with my code, or verify that they work. I have not had any time to work on p2pool stuff recently, alas. January 09, 2018, 09:46:35 PM
Does anyone know how to setup a solo mining pool for scrypt machines?
Roughly, this: set PERSIST=False in p2pool/networks/(networkname).py or p2pool/bitcoin/networks/(networkname).py, then run it with the maximum peers set to 0.Let me know if you have trouble and I'll give more specific instructions.
January 08, 2018, 04:46:07 PM
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
It's allowed to be zero length.So how do I regard it when I calculate the coinbase tx? as "00000000" or simply as "" ?
January 08, 2018, 02:22:25 PM
I am sending the following stratum message to p2pool:
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
I opened the code of "p2pool-master\p2pool\bitcoin\stratum.py", and I found this. which kind of explained why xnonce1 is empty, regardless of the way I subscribe:
So, is Stratum not supported yet by p2pool?
It's allowed to be zero length.Code:
{"id":1,"method":"mining.subscribe","params":["myclient/0.1"]}
And I am getting back:Code:
{"error": null, "jsonrpc": "2.0", "id": 1, "result": [["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], "", 4]}
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
I opened the code of "p2pool-master\p2pool\bitcoin\stratum.py", and I found this. which kind of explained why xnonce1 is empty, regardless of the way I subscribe:
Code:
def rpc_subscribe(self, miner_version=None, session_id=None):
reactor.callLater(0, self._send_work)
return [
["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], # subscription details
"", # extranonce1
self.wb.COINBASE_NONCE_LENGTH, # extranonce2_size
]
reactor.callLater(0, self._send_work)
return [
["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], # subscription details
"", # extranonce1
self.wb.COINBASE_NONCE_LENGTH, # extranonce2_size
]
So, is Stratum not supported yet by p2pool?
So how do I regard it when I calculate the coinbase tx? as "00000000" or simply as "" ?
January 07, 2018, 05:36:45 PM
I am sending the following stratum message to p2pool:
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
I opened the code of "p2pool-master\p2pool\bitcoin\stratum.py", and I found this. which kind of explained why xnonce1 is empty, regardless of the way I subscribe:
So, is Stratum not supported yet by p2pool?
It's allowed to be zero length. Code:
{"id":1,"method":"mining.subscribe","params":["myclient/0.1"]}
And I am getting back:Code:
{"error": null, "jsonrpc": "2.0", "id": 1, "result": [["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], "", 4]}
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
I opened the code of "p2pool-master\p2pool\bitcoin\stratum.py", and I found this. which kind of explained why xnonce1 is empty, regardless of the way I subscribe:
Code:
def rpc_subscribe(self, miner_version=None, session_id=None):
reactor.callLater(0, self._send_work)
return [
["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], # subscription details
"", # extranonce1
self.wb.COINBASE_NONCE_LENGTH, # extranonce2_size
]
reactor.callLater(0, self._send_work)
return [
["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], # subscription details
"", # extranonce1
self.wb.COINBASE_NONCE_LENGTH, # extranonce2_size
]
So, is Stratum not supported yet by p2pool?
January 07, 2018, 04:39:28 PM
I am sending the following stratum message to p2pool:
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
I opened the code of "p2pool-master\p2pool\bitcoin\stratum.py", and I found this. which kind of explained why xnonce1 is empty, regardless of the way I subscribe:
So, is Stratum not supported yet by p2pool?
Code:
{"id":1,"method":"mining.subscribe","params":["myclient/0.1"]}
And I am getting back:Code:
{"error": null, "jsonrpc": "2.0", "id": 1, "result": [["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], "", 4]}
Note how the xnonce1 is returned as empty string, while xnonce2_size is 4, which is Ok.
I opened the code of "p2pool-master\p2pool\bitcoin\stratum.py", and I found this. which kind of explained why xnonce1 is empty, regardless of the way I subscribe:
Code:
def rpc_subscribe(self, miner_version=None, session_id=None):
reactor.callLater(0, self._send_work)
return [
["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], # subscription details
"", # extranonce1
self.wb.COINBASE_NONCE_LENGTH, # extranonce2_size
]
reactor.callLater(0, self._send_work)
return [
["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], # subscription details
"", # extranonce1
self.wb.COINBASE_NONCE_LENGTH, # extranonce2_size
]
So, is Stratum not supported yet by p2pool?
January 07, 2018, 01:43:02 PM
@jtoomim
Replaced the AMD 6128 processor with a AMD 6204 bursting at 3.3ghz which helped a bit.
BTC P2Pool running at ~82% Eff - i have several old S3 pointing at this node so probably holding it down, the Dash and LTC P2Pools on this same server are running 100%+ so
thanks for processor and pypy improvements - will probably upgrade at some point when the machine becomes unstable.
Any news on P2Pool merge?
thanks
Replaced the AMD 6128 processor with a AMD 6204 bursting at 3.3ghz which helped a bit.
BTC P2Pool running at ~82% Eff - i have several old S3 pointing at this node so probably holding it down, the Dash and LTC P2Pools on this same server are running 100%+ so
thanks for processor and pypy improvements - will probably upgrade at some point when the machine becomes unstable.
Any news on P2Pool merge?
thanks
January 06, 2018, 07:01:01 PM
Hi All,
I am new to P2P pool and I went though online documentations that explain how to mine with P2P pool, and I found these tutorials assume that the reader is expert with coding and network. In may opinion this is why there is less miners in the pool even it was established years ago.
It will be nice and helpful to boost mining hash rate of the pool by attracting more miners by the following:
1) making comprehensive tutorial from A to Z about how to mine with p2p pool and having in mind the reader is not expert.
2) developing the website to include every thing needed instead of searching all around the forums.
3) Having support team to answer question of miners.
I have an Antminers and I would like to create a node to join the pool, and I found it is difficult to follow the given tutorials.
Thank you
Moharni
I am new to P2P pool and I went though online documentations that explain how to mine with P2P pool, and I found these tutorials assume that the reader is expert with coding and network. In may opinion this is why there is less miners in the pool even it was established years ago.
It will be nice and helpful to boost mining hash rate of the pool by attracting more miners by the following:
1) making comprehensive tutorial from A to Z about how to mine with p2p pool and having in mind the reader is not expert.
2) developing the website to include every thing needed instead of searching all around the forums.
3) Having support team to answer question of miners.
I have an Antminers and I would like to create a node to join the pool, and I found it is difficult to follow the given tutorials.
Thank you
Moharni
Jump to: