Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 129. (Read 2591920 times)

I am having trouble installing p2pool.

I still consider myself to be a beginner with Linux.
I am trying to do this on a Dell Percision 390 desktop with 8GB memory and plenty big drive.
I have more memory and SSD on order but hope this will do for now.
I have a T1 connection to the Internet.
I am running Linux Mint 64 bit version 17.2 and KDE desktop.

I am currently running Bitcoin Core on this same machine.

I believe I did all the preparations:

apt-get install python-twisted python-argparse # should work on Ubuntu
apt-get update
apt-get upgrade

I am looking at Linux instructions on http://p2pool.in/

-------------------------------------------
sudo apt-get install git # if git is not installed
git clone https://github.com/forrestv/p2pool.git
cd p2pool
make
----------------------------------------------

First attempt did not work with exit error message, so I did "rm -rf p2pool" to try again.

I read in earlier post that git clone https://github.com/forrestv/p2pool.git   

should now be changed to:  git clone https://github.com/p2pool/p2pool.git

So I tried again with following:
-------------------------------------------
sudo apt-get install git # if git is not installed
git clone https://github.com/p2pool/p2pool.git
cd p2pool
make
----------------------------------------------

Both ways, I get same output:

ttravis@ttravis-Precision-WorkStation-390:~ > git clone https://github.com/p2pool/p2pool.git 
Cloning into 'p2pool'...
remote: Counting objects: 9407, done.
remote: Total 9407 (delta 0), reused 0 (delta 0), pack-reused 9407
Receiving objects: 100% (9407/9407), 3.72 MiB | 135.00 KiB/s, done.
Resolving deltas: 100% (5927/5927), done.
Checking connectivity... done.
ttravis@ttravis-Precision-WorkStation-390:~ > cd p2pool
ttravis@ttravis-Precision-WorkStation-390:~/p2pool > make
mkdir -p "/home/ttravis/p2pool/.cache"/pyenv
curl -L 'https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz' >'/home/ttravis/p2pool/.cache/pyenv/virtualenv-1.11.6.tar.gz' || { rm -f '/home/ttravis/p2pool/.cache/pyenv/virtualenv-1.11.6.tar.gz'; exit 1; }
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1572k  100 1572k    0     0   136k      0  0:00:11  0:00:11 --:--:--  149k
rm -rf "/home/ttravis/p2pool/.pyenv"
mkdir -p "/home/ttravis/p2pool/.pyenv"
# virtualenv is used to create a separate Python installation
# for this project in /home/ttravis/p2pool/.pyenv.
tar \
            -C "/home/ttravis/p2pool/.cache"/pyenv --gzip \
            -xf "/home/ttravis/p2pool/.cache"/pyenv/virtualenv-1.11.6.tar.gz
python "/home/ttravis/p2pool/.cache"/pyenv/virtualenv-1.11.6/virtualenv.py \
            --clear \
            --distribute \
            --never-download \                                                                                                                                                     
            --prompt="(p2pool) " \                                                                                                                                                 
            "/home/ttravis/p2pool/.pyenv"                                                                                                                                           
Not deleting /home/ttravis/p2pool/.pyenv/bin                                                                                                                                       
New python executable in /home/ttravis/p2pool/.pyenv/bin/python                                                                                                                     
Installing setuptools, pip...done.                                                                                                                                                 
rm -rf "/home/ttravis/p2pool/.cache"/pyenv/virtualenv-1.11.6                                                                                                                       
# Snapshot the Python environment                                                                                                                                                   
tar -C "/home/ttravis/p2pool/.pyenv" --gzip -cf "/home/ttravis/p2pool/.cache/pyenv/pyenv-1.11.6-base.tar.gz" .                                                                     
rm -rf "/home/ttravis/p2pool/.pyenv"                                                                                                                                               
rm -rf "/home/ttravis/p2pool/.pyenv"                                                                                                                                               
mkdir -p "/home/ttravis/p2pool/.pyenv"                                                                                                                                             
mkdir -p "/home/ttravis/p2pool/.cache"/pypi                                                                                                                                         
# Uncompress saved Python environment                                                                                                                                               
tar -C "/home/ttravis/p2pool/.pyenv" --gzip -xf "/home/ttravis/p2pool/.cache"/pyenv/pyenv-1.11.6-base.tar.gz                                                                       
find "/home/ttravis/p2pool/.pyenv" -not -type d -print0 >"/home/ttravis/p2pool"/.pkglist                                                                                           
# readline is installed here to get around a bug on Mac OS X                                                                                                                       
# which is causing readline to not build properly if installed                                                                                                                     
# from pip, and the fact that a different package must be used                                                                                                                     
# to support it on Windows/Cygwin.                                                                                                                                                 
if [ "x`uname -s`" = "xCygwin" ]; then \                                                                                                                                           
            "/home/ttravis/p2pool/.pyenv"/bin/pip install pyreadline; \                                                                                                             
        else \                                                                                                                                                                     
            "/home/ttravis/p2pool/.pyenv"/bin/easy_install readline; \                                                                                                             
        fi                                                                                                                                                                         
Searching for readline                                                                                                                                                             
Reading https://pypi.python.org/simple/readline/                                                                                                                                   
Best match: readline 6.2.4.1                                                                                                                                                       
Downloading https://pypi.python.org/packages/source/r/readline/readline-6.2.4.1.tar.gz#md5=578237939c81fdbc2c8334d168b17907                                                         
Processing readline-6.2.4.1.tar.gz                                                                                                                                                 
Writing /tmp/easy_install-eO9jSr/readline-6.2.4.1/setup.cfg                                                                                                                         
Running readline-6.2.4.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-eO9jSr/readline-6.2.4.1/egg-dist-tmp-nxOplh                                                             
/usr/bin/ld: cannot find -lncurses                                                                                                                                                 
collect2: error: ld returned 1 exit status                                                                                                                                         
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1                                                                                     
make: *** [/home/ttravis/p2pool/.cache/pyenv/pyenv-1.11.6-extras.tar.gz] Error 1
ttravis@ttravis-Precision-WorkStation-390:~/p2pool >
_______________________________

Where am I going wrong?

Thanks in advance.  If I get this working, I hope to put 150TH on p2pool and see how it does.

Tom Travis

I'm not sure I can explain this properly to you but, the higher you set your tx fee and the more little spam transactions you reject the bigger the blocks (in terms of BTC) your miner mines will be.

An analogy:

You've got an unlimited supply of wax beads. And the beads come in two sizes 10mm and 1cm. Lets say the standard for a wax ball is 100 beads squished together and the wax ball vendor who buys standard wax balls pays for them by the gram.

If you just grab a handful of wax beads at random and count out 100 of them you'll get a mix of big and small ones and you'll end up with a wax ball that's a lot lighter than if you just picked out the large beads and squished 100 of them together.

Now would you rather sell the wax ball vendor a light ball or a heavy one? remember the vendor pays by the gram.

I see that the block includes transaction, or does it mean something else:
Rejecting these transactions I do not receive a share containing the transaction.

The tx fees currently add up to not much however, the higher you set the tx fee the more money miners get when the pool finds a block i.e. a block full of 10 satoshi spam transactions vs. a block full of 0.0005 BTC transactions. Since the fees add up to so little it currently matters not if the block is empty or full.

Also, the tx fee has nothing to do with blocks going to other pools, your understanding of the Bitcoin protocol is incorrect in that regards.

I'm here still thinking maybe we have cutting low-fee transactions on nodes for nothing?
mintxfee=0.00001
minrelaytxfee=0.00001
Someone cuts more.
Maybe we should not reject the blocks with a low-fee transactions with these cuts?
But everyone turned these parameters on because afraid of spam.
Do I understand correctly? Transactions go with the block and if we have limited low-fee transactions - the block goes to other pool which is not limited these sums.
You can refuse these limitations when used pypy and reduce the ammount of connections due to the relay network.

you thinks like that ... but i think in other way : network need to transmit transactions.
we have plenty of bitcoin core that they officially filtered (non-relay) the transactions ... i don't love this.
i want a fairplay network open to others.

spam filter need to activate ONLY because we have spam ... and upper the fee strategic with FIXED value is a non-intelligent solution for me.

i prefer purge my mempool (because mempool is shared between each nodes) when it requiert more CPU (every day, i change the limit from 1200 to 800 from the past CPU monitoring time) and not implemente fee filter.

long chain win ... don't loose this goal.

Bear in mind also that several devs have taken several months to come to a consensus about the design approach for the new 0.12 mempool limiter. It's a fairly refined solution, which is why I say that everyone will be able to return to min relay and min fee settings similar to pre-flooding values once 0.12 is available.

No, for mining it's better to use minrelaytxfee=0.00005 or so and not use Gavin's random eviction code too heavily. If you do, especially with a threshold of 1200 txns, you'll lose a lot of fees, and you'll include a lot of those nasty 14790 byte 0.00015 fee spam transactions that were broadcast at the beginning of October.

The new evictRandomBytewise code is much better for mining (https://github.com/bitcoinxt/bitcoinxt/pull/89/), since it mostly evicts large low-fee transactions, but it isn't in any releases yet (or even merged into master -- still waiting for testing results from people other than me). I've found maxmempoolbytes=5000000 (5 MB of mempool usage, or about 1.8 MB of blocksize) to be pretty decent for now, and I use that on http://74.82.233.205:9332.

Keep in mind that the use of random eviction code may mess up the Relay Network's effectiveness.

i want this !

24 hours on with these settings and things look fantastic.. RSS usage is staying around the 1Gb mark (total memory ~5Gb) and I have not had any of the issues (RPC delays etc).

Just need to get that getwork/getblocktemplate latency down.; but that's a 'nice to have'.

Bear in mind also that 0.12 will include a dynamic mempool adjuster, node operators can return to similar minrelay and minfee values to when tx flooding was not happening.

Interesting... I'll run this setting on my laptop and compare the results.

Hey All,

Sorry for posting not a very relevant question to the topic but I am quite new with this stuff.

I restarted my p2pool recently and got 16 shares and been paid twice but my current payout says 000000 BTC and I was curious why that is.

Thanks

oh come on ... we don't have a mempool problem actually.


I use this (XT only for the first parameter).

Firstly, thankyou very much for the assist....

I had minrelaytxfee set to 0.00001 ; this probably made me wide open to the spam transactions.




I had already employed MALLOC_ARENA_MAX which didn't seem to help much (again, I think the spam hit me) ---  however I've switched to preloading libjemalloc WITHOUT the MALLOC_ARENA_MAX setting.

Early indications are that these 2 changes are drastically reducing my memory usage.

Am looking forward to the getblocktemplate enhancements...

# I grabbed latest from git and built that with tests

build log with tests here: http://dpaste.com/1PJNZV7 http://dpaste.com/31GA0PG

Ran the bitcoind with
started the p2pool and testing now. Cheers for sharing that man!.

Ed : Wed Nov  4 21:17:15 ACDT 2015
err, I may of balls this up.

Back to the docs now ...

Ed : Thu Nov  5 21:16:33 ACDT 2015

First: in ~/.bitcoin/bitcoin.conf, add


That will reject any transactions that don't include at least 0.05 mBTC/kb in fees. That should drop your mempool size from about 500 MB down to about 5 MB. This is necessary due to about 1 GB of spam that was sent about a month ago, plus the fact that 1 MB blocks just isn't enough any longer.

Second: before you start up bitcoind, run


in the same terminal (or bash session) that you use to start bitcoind. This is a workaround for an issue described here:

https://github.com/bitcoin/bitcoin/issues/6876

BitcoinXT will probably be switching over to jemalloc soon. Instead of the MALLOC_ARENA_MAX=1 fix, you can also try using jemalloc if you prefer.


Then, when you want to run bitcoind,


jemalloc does a better job of deallocating memory that's no longer in use than glibc, at least without the MALLOC_ARENA_MAX=1 fix:

http://toom.im/files/bitcoinxt_jemalloc_vs_glibc_16_threads.png

I haven't yet directly compared jemalloc to glibc with MALLOC_ARENA_MAX=1, but the testing I have done suggests that jemalloc is qualitatively better.

(Btw, folks, I'm working on some enhancements to BitcoinXT's getblocktemplate. I think I'll be able to get a 5x reduction in getblocktemplate latency. Stay tuned. I think jtimon is working on something similar for Core too. Edit: oh, and it will likely improve memory usage too, at least as it currently stands.)

Hoping someone else has experienced this and can advise me...

My setup consists of 16Gb memory, 8 cores...

I'm finding that p2pool is reporting loss of connection to bitcoind (actually bitcoinxtd) at least once a day.

CPU is at least 80% idle all day

Memory utilisation is high --- bitcoin is using ~12 Gb and p2pool is using ~3Gb (I was running under pypy but switched back to python to reduce memory usage).

netstat -an shows *hundreds* of abandoned/idle connections in CLOSE_WAIT from the p2pool process to the bitcoin RPC port.

I've tried everything I can think of, I've turned the TCP timeouts down from 2hr to 5min but have only shifted the problem from TIME_WAIT state to CLOSE_WAIT state connections.

Has anyone else experienced this situation; and what (if anything) can be done to get around it ?

IFTTT uses the chain.com PI to monitor the blockchain, does not matter how you interact with the address. I think it sends on 0 confirmations, but not 100% on that.

Would love to find 500 blocks in a month and run up your phone bill

OK, I think I get 500 free sms p/m so I should be fine cos I hardly ever use it. Do you get the sms when the payment is sent to the address - or when it arrives in the wallet? I ask because my receiving wallet is not on 24/7, so it would be handy if the sms was sent when the payment is sent.....