Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 20. (Read 2591916 times)

How many set virtual memory with 16gb of ram in windows 10 ? .

Hello,

There are a few websites that give the results from a node scanner. For example http://p2pool.jir.dk/bitcoin/ gives publicly reachable nodes in the P2pool main network and in the jtoomim network. Nodes with a version number 17 or 16 are nodes of the main network. Nodes with a version number of 15.0-33 are nodes from the jtoomim network.
Another useful site is https://p2pool.coinpool.pw/. This site also gives the publicly reachable nodes for both networks.
P2pool.org has also a node scanner, but this one only shows results for the p2pool main network. All the information shown on the p2pool.org website is currently that of the p2pool main network and not of the jtoomim network.
If you want to know the ip address of nodes that are not publicly reachable, the only way to do that is run your own node and look to which nodes it makes a connection.

I hope this answers your question.

Hey big Programmer jtoomim,

i dont belive in these CPU overload thinks you talking about. first time you sayed quad core ist with 25% overloaded, now it is 12,5% an i use it with python2,7 an see load peaks of this singel task by 48% without any connection lost message. there is an other big problem a displayed but no one is intereted.

this linux think hangs and freeze for a few sec everey round about 30sec. and the tcp-connections in these linux box in an problem, perhaps because timeouts between linux box and system network interface.

i run now p2pool on phyton 2.7 an waste a day after start up until it reduce his mining difficult and hoping ther will be a good running windows version with Segwit in the future.

@davidadrake
i have no list, but for now these is a node : "212.43.81.114:9332"
p2pool.org i the "non segwit" version, jtoomim goes his own "segwit way" for now. - korregt me if i talking wrong things!

I haven't been on p2pool-bitcointalk for a while, but I remember reading through the posts every day, and jtoomin, I believe had the version of p2pool to run. On p2pool.org, it shows it has been 83 days since the last block, so are they not running jtoomin's fork?

Is there a list somewhere of pools that are running the fork? Sorry if this has been answered somewhere, but I looked through a bunch of pages without finding any info.

Thanks,

Dave

That's what happens when your CPU gets overloaded and stalled. Let's say you have enough incoming traffic in your TCP buffers to keep p2pool's CPU busy for 1.1 minutes. P2pool first sees that it has a queued need to get a new block template, so p2pool sends a getblocktemplate request to bitcoind. Bitcoind may respond immediately, but the response goes to the end of that 1.1 minute queue. P2pool then works through its queue of stuff, then gets to the bitcoind request, and process it, but notices that it took 1.1 minutes and so complains about it.

Same thing for web UI requests. When you point your browser to localhost:9332, that request goes into p2pool's queue of jobs to work on, and p2pool doesn't get around to it until e.g. 1.1 minutes has passed.

The 15% CPU is the problem. 15% is above the 12.5% threshold for fully utilizing a single core on your machine, which means that the work in p2pool's to-do queue is snowballing.

The --bench output is somewhat helpful, but it seems that the bulk of the load is happening in a function that I didn't put benchmarking code into. In your log files, I frequently see line pairs like this one (note the timestamps):

The "Lost contact" message means that your CPU was stalled for 1.6 minutes. 1.6 minutes before the "Lost contact" message at 12:52:59 would be 12:51:20 or something like that. That means that your CPU was busy between 21:52:37 and 21:52:58. However, we don't know what was happening at that time, since the code that was running was something that I didn't add benchmarking code to.

Can you add this to your p2pool startup command line and send me the resulting profile1.log file? Try to run it for about an hour. If you run it for too much less than that, and the profile data will be dominated by the start-up share loading time.


If anyone is curious, you can analyze those cProfile log files with a python script that looks like this:

well, that is strange,
i tested that. its true, i get more connection ...

but i also see ofen this :
"Warning: LOST CONTACT WITH BITCOIND for" is displayed ... perhaps these linux bash box dont runs cleanly.
it fells like it freeze every few minutes:

and after restart now in bench mode (i also put in new tcp rule for in- and outgoing tcp connection on port 9332 and 9333) :
without miner connected : www.c-xantus.de/log4
with miner connected : www.c-xantus.de/log5

by the way, P2Pool is using at linux bash with pypy 1,3Gb Memory an betwen 0 and 15% cpu

Xantus, maybe what's happening is that there just aren't any exceptions in Windows Firewall for this. Native Windows apps usually trigger a request to open ports in the firewall when needed, but I maybe WSL pypy p2pool isn't doing that. Try disabling Windows Firewall for an hour to see if that helps. If it does, then you can manually add a firewall rule to allow incoming tcp connections on port 9332 and 9333 (or whatever you're using).

Edit: Pages like this suggest that you do indeed need to open up ports on the Windows Firewall if you want to allow WSL processes to serve as servers.

no, system look fine, Memory is 50%free, CPU usage is now with pypy somethink around 15% (5%CPU for pypy avergage, and 20%peek)
but other strange thing happend, since i Wrote this morning, i esay let it run and go to work. now i am here again, take a look and p2pool is a bit online 
i cant connect the web interface from an other computer, and the miner cant connect to p2pool node. also i got no p2pool incomming connection. but it got 6 outgoing connection.
does this linux box have an own software firewall ? - i think that is an network problem ...


Log file from p2pool on pypy in an ubuntu bash box on windows 10 :
www.c-xantus.de/log3

System Preformance (using ubuntu bash and p2pool on pypy):
http://www.c-xantus.de/Zwischenablage0.jpg

That sounds like you might be out of RAM and have swapping to disk. Can you check? Task manager -> Performance -> Memory. In Use, Available, Committed, Paged Pool.

pypy cpu usage is lower than 5%, but it seams as anything is verry slow. it look like p2pool on pypy lags. and also get now connection.
but i did now "sudo apt-get update and "upgrade" and gave pypy another changse this day. loocks better for this moment.
But i have to go to work now and look this evening again does pypy work or not. - i will edit this post ....

thanks agein so far jtoomin

Interesting, it seems that time.time() on Windows only has 15 or 16 ms resolution, so all of the short function calls are reported as taking either 0 ms, 15 ms, or 16 ms, even though they're actually probably taking about 4 ms each.

I also see a few lines like this one:
453 ms for handle_remember_tx when no transactions are being added seems strange and significant. I'll have to reread the code and see if I can figure out what could be taking so long.

There's also this:
That makes about 1.7 seconds of latency just in those two functions for your node switching work. (IIRC, handle_shares() calls get_work, so the 750 ms is already included in the 1734 ms.) There may be a few other functions that I didn't add the benchmarking code to that also play a role, but that latency alone should be responsible for about 5.8% of your DOA+orphan rates.

The 1188 ms (unpacking) is something I think I know how to fix via a CPU/RAM tradeoff. That will probably be my next task. However, that will only reduce the CPU usage in processing new getblocktemplate responses, which I think is not in the critical code path for share propagation, so I don't expect it to help with DOA+orphan rates much. It's a pretty easy and obvious optimization, though. The node I do my testing on (4.4 GHz Core i7, pypy) only shows about 40 ms for unpacking instead of 1188 ms, so I wasn't sure it would be worthwhile. But if it's stalling your CPU for over a second, that sounds like it could be interfering with things.

Not sure what's the issue with pypy on your machine. Does your CPU show activity when you run it? How much?

hey there,
here is the log file from p2pool on phyton2.7 on windows 10 with --bench option:
www.c-xantus.de/log2
by the way, there is a bug on the p2pool mining difficult. it dont will be set down if smaler miner are working, or it takes days !

i also tried to install these Linux box and followed the intall steps in the ubuntu bash box to test it.
installation runs fine without anny errors. i copied the bitcoin.conf file to /home/user/.bitcoin/bitcoin.conf
and was able to start P2Pool with --bench option on pypy. But seams that twisted dont work ? - i get no connetion to other nodes.
alwas show the twistet errors :

jtoomimnet found another block two days ago.

https://blockchain.info/block/00000000000000000096cef4bfd159307030923b0111acd39bfb6561b277a375

1014 kB, 3998 kWU, 13.858 BTC.

We also added a few hundred TH/s yesterday, and are now up to 2.2 PH/s.

That assertion is inconsistent with most of the benchmarks I have seen, in which WSL is often faster than native Linux and rarely more than 5% slower.
https://www.phoronix.com/scan.php?page=article&item=windows-10-lxcore&num=2

However, for some reason, code compilation appears to be the exception to that rule. For some strange reason, compiling code on WSL is extremely slow:
https://www.phoronix.com/scan.php?page=article&item=windows-10-lxcore&num=4

Since running p2pool is not code compilation, performance on WSL should be quite good. P2pool is mostly just a CPU-bound task with very little disk IO and a small amount of network IO, so the poor results from code compiling should not manifest here.

This "linux subsystem" is like 5-10x slower than you make a Vitrualbox VM and run Ubuntu in it.
Tested on cross-compiling coin daemons.
It would be way better choice to install Ubuntu desktop in VM, run it in seamless mode and use for whatever you want.

A 2.0 GHz A10 is not what i would call a medium-speed CPU, unfortunately. Sure, it might have 4 cores, but Python is effectively single-threaded, so that doesn't help at all. Pre-Ryzen AMD chips have poor single-threaded performance, so a 2 GHz A10 is equivalent in performance to a roughly 1.3 GHz Core i3 (if such a thing existed). I wish I could get p2pool to run nicely on such a CPU, but I'm afraid that's out of reach at the moment.

You've got plenty of RAM, so I think you can get it to work decently if you go through the steps I described to get Pypy running on Windows.

Also, if you could run p2pool with the --bench option, that would make the log file much more useful to me.

Ok great. Thank you for the clarifications. I'll try the methodology you suggest and see if I can get the information I need.

okey, first night run is done. Synchronising and getting down the share dificulty to test run with 4Th was take somethink about 4 hours. This morning i switched the
Miner Hash rate down to somethink about 1Th.
CPU last is now down to 10-15% avergade, put there are peaks to 30% :
http://www.c-xantus.de/Zwischenablage03.jpg

my Miner dont runs 24Hr a day but p2pool node is alwas on an reachabel at:
http://212.43.81.114:9332

config is Windows 10 64Bit
http://www.c-xantus.de/Zwischenablage02.jpg

here is the Log File (i dont Start the Debug mode ..):
www.c-xantus.de/log

i will take a look for a few more days. Efficenity is better now, but i could be better 

P2pool has a share chain that is very much like the Bitcoin blockchain, with a few differences. The tip of a chain (a block/share with no children) is known as a head. The root of a chain is known as the genesis block in Bitcoin (the block with no parent), but in p2pool it's known as a tail. P2pool's tails *do* have parents, unlike bitcoin, and consequently have a height greater than 1; however, they're called tails in p2pool because their parents have been pruned off from the node's copy of the share chain, so as far as your node is concerned, they have no living parents.

The best share is always a head. It's usually the head with the most work behind it, but not always. The rules for determining the head are a little complicated, and are different in different versions of p2pool.

Not necessarily, but usually. Just because a share is recent doesn't mean that it is placed at the tip of the chain.

Yes, precisely.

I'm not sure. I think unverified heads or tails are shares that have not yet been connected to the share chain, such as for a share that is was just downloaded during initial sync but whose ancestors have not yet been downloaded. However, I haven't verified that hypothesis, as I have not yet needed to work on that part of the code.

Completely downloaded and mostly deserialized.

Yes, but it's a sendShares or share_reply message.

Correct. In the current p2pool master, the timestamp is required to be between 1 second after the previous share's timestamp and 59 seconds after the previous share's timestamp. Since the timestamps are used for minimum share difficulty adjustments, this can result in a situation where the share difficulty is very slow to adjust downwards after a rapid decrease in the network hashrate. In the jtoomimnet fork, I got rid of this timestamp clipping, and replaced it with a rule that rejects shares that are timestamped to come from the future in order to prevent share difficulty manipulation.

Keep in mind that the timestamp is the time at which the mining job was sent to the hardware, not the time at which the hardware returned the nonce value.

Nope, it's not close. It will usually be off by around 5-50 seconds. But what you can do is use the "Peer first received from" line to browse to the info page for that share on the peer's address (you may have to guess the port number), and repeat, until eventually you get to a peer that says "Peer first received from: Null" for that share, and then you can use the "Time first seen" shown by that ndoe (or just the node that you got the share from, if you're lazy).

When I've done this measurement in the past, it's usually around 1 second per hop for nodes running CPython with 100 ms network latency, and around half that for nodes running pypy.

If you run my code (e.g. 1mb_segwit), you can install the rfoo package for python, then run run_p2pool.py with the --rconsole command line option, and then run the rconsole command in a separate terminal. This will give you a python interpreter that effectively runs inside the p2pool process, and allows you access to all of the variables and functions of p2pool while it's running. The share objects can be found in the gnode.tracker.verified object.
1mb_segwit also now has a --bench command-line option, which I think you would find interesting.

You could also maybe parse the log in data/bitcoin/log. That's just the stdout output of p2pool for (usually) the last few days.

I was able to get pypy running on Windows 10 by using the Linux Subsystem for Windows:

https://bitcointalksearch.org/topic/m.21025074

However, the reason I made these CPU improvement changes was in the hope that it would allow people to run CPython on medium-speed CPUs without crazy orphan rates. I'm very much interested in hearing whether it works.