Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 55. (Read 2591916 times)

I've been working on a new scanner for p2pool.org, it's very close to ready. Tomorrow evening or maybe Monday it will be live, heres a preview:

http://p2pool.co/ Domain Suspension
http://poolnode.info/ all blank

I got round to sorting my ICMP ping today on the firewall but it seems the p2Pool node scanners are not working.
Well at least the 2 I tried.
Anyone check them and see if it's a problem with my PC please?

Here is a rough version of my step-by-step guide for p2pool on Debian. I have attempted to improve the readability of what I originally intended to be notes to myself. Fair warning, it may still be hard to read. I did leave out a section I had on using the official linearize tool for creating blockchain bootstrap files as it is not directly related to the setup process.

Installing Debian is fairly straight forward and there are plenty of youtube videos for specific help so I will only detail a few notes. When you get to the "Software selection" menu, you can safely deselect all packages except "standard system utilities" as it includes utilities required to complete this guide. I recommend that you also select "SSH server" as it provides remote login capability. If you absolutely must have a GUI, you can select "Debian desktop environment", but remember to select at least one of the window managers such as "MATE".

This guide works primarily from the users home directory, but it can be changed at the users discretion. Additionally, for speed and security, this guide does not use a wallet on bitcoind and requires you to provide addresses from an external wallet.

I can't wait to exploit this! P2Pool doesn't validate the block reward because of the significant resource usage of querying the UTXO set so it relies on miner honesty. I guess while hardcoding the new txns/share size limit to 1MB I could also hardcode the reward to 10M BTC! Let the free market decide!

I started out life on DOS and 3+Share many moons ago and then spent some time with Mac and Windows.
We moved all our back office to Linux about 14 years ago and the first server came with Centos.
I've never fired a Debian disto up in my life but you know what I might take a look.
Just I've learned so much about Centos the last few years sometimes better the devil you know.

Also like Debian, but I could not adjust. Where can I see your instructions to try again.

I already have a simple Debian based step-by-step that can be included if you want. I use it whenever I do a fresh install for my p2pool node. It includes installing all required packages, compiling bitcoind to include options for --disable-wallet and --without-gui, creating a bootstrap file the official way using linearize, creating a simple script for starting p2pool with options such as a manually specified payout address, and configuring some basic options for bitcoin.conf.

With regard to OS consideration, I have the following to say: CentOS... Meh, Ubuntu... Yuck, Windows... Crazy, and Debian... Winner!

With Ubuntu, in addition to what is stated in the p2pool GitHub readme, the following needed to be installed: python-dev lib32ncurses5-dev libffi-dev libssl-dev. I actually installed many more packages on previous installations, but on my last one I documented what I installed and whether it helped with errors during make.

This was very difficult to figure out how to install and run p2pool, especially since I have had no Linux exposure before. It shouldn't have been that difficult, and it presents a serious barrier for p2pool.

  Block! Smiley
  https://blockchain.info/block/000000000000000001d23d06a74cfc8f136bbfa4efc568b108a78a8f520493a3
 !!    

Yes agree.
I followed the on line linux guide for Ubuntu and gave up.
Went back to Centos and did it in a few hours via command prompt.
I've used Centos for years and don't get on with Ubuntu.

I would have to re work everything but I thought of writing a bash script to download and do everything.
Load Centos, run the script away you go.
Then some might moan and say it's not trusted etc etc so others would need to verify the script and vouch for it.
Could be a new project to keep me out of trouble?

I wrote one specifically for vertcoin (for a bounty) so it could be adapted: https://gist.github.com/veqtrus/e2b7c34d9c8b3ad285209ae1b13ca2b2

If anyone feels inclined to write such a guide I'd be happy to host it on p2pool.org...

What ?
Which version of python to install or
the entire procedure 

And the install guide should be translated into Chinese.

Yes
It needed a reliable source with the packages required for a correct installation on windows.
Without broken links but updated links

p2pool Windows binaries
Python >=2.6
Twisted >=10.0.0
python-argparse (for Python =2.6) What a big mess ( http://p2pool.in/#download ) 

I completely agree, for example, it took me more than a month to configure the node.

IMHO to help p2pool scale there should be one or several simple but useful HOWTOs on p2pool node deployment, and these HOWTOs should be constantly kept up-to-date by their maintainers. So every newcomer will be able to deploy a p2pool node completely by himself.

Just my two cents. 

Currently, p2pool weights each share by its difficulty alone. The expected revenue for the pool from each share equals the difficulty times the block reward (with fees). This means that shares with low fees get overcompensated, and shares with high fees get undercompensated, which means that the optimal strategy for most p2pool miners is to make small shares/blocks and hope that the rest of the pool irrationally mines big blocks.

This misaligned incentive can be fixed by weighting shares by their difficulty times the block reward they would have generated. Pretty simple. As it affects payment calculations, which need to be precise down to the satoshi, t's a completely backwards-incompatible change, but that's something I want for this fork anyway. Yay!

I'm trying to figure out the implementation now. These revenue calculations are done with a SkipList implementation so that p2pool doesn't have to add up every single share's weight every time it assigns new work, and it's taking me a while to understand the code well enough to change it without breaking everything. But I think it will be working soon. Once I've got the calculations correct, I'll deploy them along with a share version change to v64 (just to give the legacy chain room for plenty of upgrades if it continues to be used), and then I think the code will be ready for everyone else to use.

After that, I might take a break from coding on p2pool, or I might continue for another week or two. If I continue, I have some ideas for optimizations that could improve p2pool's performance and orphan rates. They fall into two categories: CPU performance, and network performance.

For CPU performance, almost all of p2pool's CPU time is spent packing and unpacking shares and transactions between Python objects and serialized bytestreams. I can probably accelerate this process either by rewriting the Python code into faster Python code, simplifying the Python data structures for transactions, and/or adding Cython support. If I go the Cython route, I can make it so that p2pool can run without Cython by stripping the static typing information from the code, but if you have Cython and a C compiler installed (or if you downloaded a binary package), you will use C compiled versions of a couple of the most performance-critical modules. This should give us about the same CPU performance benefit of pypy (or better) without any of pypy's 2GB memory overhead. Improving CPU performance could improve share propagation speed and new work generation, thereby reducing orphan/DOA rates and making revenue allocation more fair. Also, it will make it cheaper to run a p2pool node, and help p2pool scale.

For network performance, it seems that the current code has a lot of room for improvement. I'd like to add a new network message that sends a list of transaction hashes to a p2pool peer and asks them to download the corresponding transactions from their bitcoind if possible. This would allow p2pool peers to get most of their transactions via localhost tcp instead of using the p2pool network, which should save a lot of bandwidth. I'd think it would reduce traffic by more than half, since p2pool is a lot more aggressive about forgetting transactions than bitcoind is. Another thing that can be done is to add a shorthash implementation like the one in xthin and compact blocks for share transmission. Once these two optimizations are in place, it should be practical for nodes to send the template of the share they're working on mining *before* it gets mined, and send only the nonce values and miner address once a share is found, at least to one's immediate peers. If I get that done, then I could also teach nodes to encode a message to each peer using the diff between the share being sent and the recipient's current work-in-progress, which would allow 1 MB shares with 95% new transactions to potentially be sent with only a dozen kilobytes or so of one-way traffic in the latency-critical path. After that, maybe I could play with bloom filters or IBLTs and and and...

Anyway, first things first. Gonna try to get the properly weighted share rewards done in the next couple of days, and then things should be ready for others to join in on the fork fun. (By the way, my fork is over 1 PH/s of non-rented miners now.)