[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 9.

Cryptonomist

newbie

Activity: 27

Merit: 0

Hello

Quote from: -ck on January 12, 2018, 11:48:52 PM

Quote from: crypto_curious on January 12, 2018, 06:07:54 PM

Is there any way to make p2pool mode modern so it can use multithreading and multi cores etc?

Rewrite it from scratch in another programming language.

Quote from: jtoomim on January 12, 2018, 11:47:52 PM

Python is inherently single-threaded. https://wiki.python.org/moin/GlobalInterpreterLock

It would be far easier and more effective to remove most of the transaction-handling code from p2pool, and make share objects contain only the merkle root plus the coinbase transaction (and the merkle path thereto). However, I have not had the time to do that, and nobody else seems to be interested in doing it. Even though it would be much easier, it's not super easy to do.

Note: Bitcoin Core is *also* mostly single-threaded, although not quite to the same extent as p2pool.

Wouldn't it be possible to rewrite the code and compile it with cython? This avoids the GIL. It also easily allows to mix python with C/C++ code, which might be useful for some parts of the code, and it would avoid rewriting everything in another language. However, I don't think twisted can be used by cython (I might be wrong on this), so this would need to be replaced with something different to handle the asynchronous networking (which might be quite a lot of work).

I've been running the 1mb_segwit code for quite some time now. It seems that the optimalisations done by jtoomim and the use of pypy has reduced the single core load quite considerably compared to the main code. So, although some people still have problems, it alleviated in my opinion the single thread bottleneck considerably.

@jtoomim: what parts of the transaction handling can be removed in your opinion?

jtoomim

hero member

Activity: 818

Merit: 1006

Assuming you're using my code (1mb_segwit), run p2pool with '--net bitcoincash' as one of the arguments, and make sure a Bitcoin Cash-compatible version of bitcoind is running (e.g. Bitcoin ABC).

Keep in mind that Bitcoin Cash support is still experimental and p2pool's BCH hashrate is very low. No blocks have been mined yet on p2pool BCH.

I intend to switch a substantial portion of my hashrate (> 1 PH/s) to Bitcoin Cash shortly after the next Bitcoin block is found. That will likely not happen for a couple weeks.

smoothrunnings

hero member

Activity: 616

Merit: 500

Can P2POOL be setup for BCH?

Where do I get the info for configuring it?

Thanks,

jtoomim

hero member

Activity: 818

Merit: 1006

Quote from: smoothrunnings on January 14, 2018, 08:57:37 PM

Can I mine bitcoin case using a SHA256 ASIC? I haven't been mining in quite a long time. lol

Yes.

jtoomim

hero member

Activity: 818

Merit: 1006

Bitcoin Cash had a spam event 1/13-1/15 (yesterday). Many blocks were mined at 8 MB by other pools. Unfortunately, I had my BCH p2pool node using the default 2 MB limit, so I didn't see any 8 MB shares being mined by my node. But I got this:

Code:

2018-01-15 10:19:19.521629 Generating a share with 1996157 bytes, 7984628 WU (new: 6549 B, 26196 WU) in 362 tx (10 new), plus estimated gentx of 528 bytes/2112 WU
2018-01-15 10:19:19.521795 Total block size, weight: 1996765 bytes, 7986980 WU

My node (pypy, on a i7 4790k) handled the load just fine, maintaining GBT latencies of around 0.1 sec. (I did not have benchmarking enabled, so I don't have more detailed performance stats. Alas.) However, I notice that several other p2pool nodes appear to have collapsed under the load: peer counts were all over the place during the spam event, which indicates that several peers were getting CPU locked, then timing out on network replies, and eventually reconnecting and starting the cycle over again.

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: jtoomim on January 14, 2018, 08:41:27 PM

Strange. It worked for me a few months ago. Maybe you need to do "sudo apt-get update" first?

Bitcoin Core works for bitcoind. You can also use Bitcoin ABC or Bitcoin Unlimited if you want to mine Bitcoin Cash.

Another line needs to be added before twisted is installed.

sudo apt-get install pypy-setuptools

smoothrunnings

hero member

Activity: 616

Merit: 500

Ran into another problem. Please advise. Thanks!

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: jtoomim on January 14, 2018, 08:41:27 PM

Strange. It worked for me a few months ago. Maybe you need to do "sudo apt-get update" first?

Bitcoin Core works for bitcoind. You can also use Bitcoin ABC or Bitcoin Unlimited if you want to mine Bitcoin Cash.

Can I mine bitcoin case using a SHA256 ASIC? I haven't been mining in quite a long time. lol

The 'sudo apt-get update' did the trick. You might want to update your how-to info to include that so that others who aren't familiar with *nix can get ahead without having to ask the same question I did.

Thanks,

jtoomim

hero member

Activity: 818

Merit: 1006

Strange. It worked for me a few months ago. Maybe you need to do "sudo apt-get update" first?

Bitcoin Core works for bitcoind. You can also use Bitcoin ABC or Bitcoin Unlimited if you want to mine Bitcoin Cash.

smoothrunnings

hero member

Activity: 616

Merit: 500

And what do you recommend I use to install bitcoind?

I found this site, but do I need to install Python for Bitcoind?

https://degreesofzero.com/article/installing-bitcoind-on-ubuntu.html

Thanks,

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: jtoomim on January 14, 2018, 06:19:20 PM

The how-to doc for Windows is the link I posted.

I typed in the first line in your install and got this, please advise.

I am using Windows 10 x64 Enterprise v1709 (Fall Creators Edition), with the Spectre patch.

jtoomim

hero member

Activity: 818

Merit: 1006

No, not that post. That post is msg21143072, but the one with instructions for Pypy on WSL/Windows is msg21025074. Since you're having trouble finding it, I'll just quote the whole thing.

One of the dependencies for p2pool on Windows is pywin32. That's available on CPython, but it is not available on PyPy. The only way to get p2pool running on PyPy on Windows is to use WSL or a VM to avoid the pywin32 dependency. PyPy is about 3x as fast as CPython due to the wonders of JIT compilation, whereas the WSL layer only slows it down by about 1-5%.

Instructions below.

Quote from: jtoomim on August 19, 2017, 06:39:50 PM

I tried installing pypy/p2pool on Windows myself, and while I got a bit farther than you did by installing twisted from source, I eventually ran into a roadblock because twisted requires pywin32 on Windows and pypy does not support pywin32.

However, I had better luck when I tried to run p2pool with pypy on the Windows Subsystem for Linux (Ubuntu for Windows). If you have Windows 10, I suggest you give that a try:

https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

Once you have Ubuntu for Windows installed, run bash and then copy-paste these lines into your bash command line:

Code:

sudo apt-get update
sudo apt-get install pypy pypy-dev pypy-setuptools gcc build-essential git

wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo pypy
sudo rm setuptools-*.zip

wget https://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.1.3.tar.gz#md5=9ae3d24c0c7415deb249dd1a132f0f79
tar zxf zope.interface-4.1.3.tar.gz
cd zope.interface-4.1.3/
sudo pypy setup.py install
cd ..
sudo rm -r zope.interface-4.1.3*

wget https://pypi.python.org/packages/source/T/Twisted/Twisted-15.4.0.tar.bz2
tar jxf Twisted-15.4.0.tar.bz2
cd Twisted-15.4.0
sudo pypy setup.py install
cd ..
sudo rm -r Twisted-15.4.0*

git clone https://github.com/jtoomim/p2pool.git
cd p2pool
git checkout 1mb_segwit

Those commands should also work if you're on native Ubuntu or Debian Linux

You'll also need to install and run your bitcoind of choice ~~(I recommend btc1, because I think segwit2x is the future of Bitcoin)~~ and edit ~/.bitcoin/bitcoin.conf with your bitcoind's RPC username and password (or pass it to p2pool from the pypy run_p2pool.py command line), then from your p2pool directory, run

Code:

pypy run_p2pool.py

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: jtoomim on August 23, 2017, 04:45:04 PM

39.2% so far, 6.4 hours remaining.

I'm going to go ahead and make a bunch of min-diff shares to get some headroom.

Your saying the code in the window is all of how to get p2pool going?

Does the linux for Windows blow away what's currently installed or does it run in a virtual session like they did for Windows 7 with the XP emulator? My currently machine is joined to my home network domain, am I going to loose that too?

Thanks,

jtoomim

hero member

Activity: 818

Merit: 1006

The how-to doc for Windows is the link I posted.

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: jtoomim on January 14, 2018, 03:27:01 PM

smoothrunnings, if you're planning on mining Bitcoin with p2pool, I strongly advise using pypy instead of regular Python. Mining with regular Python will usually result in CPU overload, which will reduce your revenue by 20-40%. This is true both for jtoomimnet and for mainnet.

Instructions for installing pypy on WSL on Windows can be found here: https://bitcointalksearch.org/topic/m.21025074

I am not really a linux guy. Is there a how-to doc to setup P2Pool using PYPY?

Thanks,

jtoomim

hero member

Activity: 818

Merit: 1006

smoothrunnings, if you're planning on mining Bitcoin with p2pool, I strongly advise using pypy instead of regular Python. Mining with regular Python will usually result in CPU overload, which will reduce your revenue by 20-40%. This is true both for jtoomimnet and for mainnet.

Instructions for installing pypy on WSL on Windows can be found here: https://bitcointalksearch.org/topic/m.21025074

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: bitlock1 on January 14, 2018, 12:38:51 PM

Disregard the vps.forre.st it is a known issue, probably resolved when jtoomin gets the main code updated with forrest permission.... right now there are two versions on p2pool running main and jtoomin

regards

I see more errors too, this one and these ones when I connect a miner to my p2pool.

See this when I connect my Neptune to the pool.

bitlock1

newbie

Activity: 12

Merit: 0

Disregard the vps.forre.st it is a known issue, probably resolved when jtoomin gets the main code updated with forrest permission.... right now there are two versions on p2pool running main and jtoomin

regards

smoothrunnings

hero member

Activity: 616

Merit: 500

Quote from: jtoomim on January 10, 2018, 08:53:41 PM

Quote from: smoothrunnings on January 10, 2018, 07:05:45 PM

Is is possible to run P2Pool on 64bit versions of Python, etc??

Both 32 bit and 64 bit versions of Python should work. I don't think I've tested with 32 bit versions before, though.

I am running Windows 10 x64 v1709 spectre patched. When I try to install the wraper it doesn't see Python installed because the app is looking for the 32bit installation of Python and it's location. Do I still need the wraper?

-- On a different note.

I started over, this time I installed the 32bit versions of everything on Windows 10 x64 v1709 Spectre patched. I getting an error with the twisted. I am using Python 17.0. It can't find vps.forre.st when I check it against Google, L3, and open DNS public DNS server they too can't find it. Safe to ignore? There is no mention or update on the first post...

(Moderator's note: This post was edited by frodocooper to fix broken quote formatting.)

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: crypto_curious on January 12, 2018, 06:07:54 PM

Is there any way to make p2pool mode modern so it can use multithreading and multi cores etc?

Rewrite it from scratch in another programming language.

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 9. (Read 2591626 times)