I'm starting to set up p2pool on my little spare machine. Bitcoin-QT is using 98% CPU usage while downloading the blockchain.
It is normal. 
Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 503. (Read 2591928 times)
I'm starting to set up p2pool on my little spare machine. Bitcoin-QT is using 98% CPU usage while downloading the blockchain. Is this a bad sign for the future? Do I abandon the idea of running p2pool on this machine, and run it off something with a bit more go? Drive is a Samsung 840 250GB SSD, and it has 4GB RAM.
Quote
...Bad CPU performance isn't necessarily a show stopper...
upgrade from Pentium D 915 2.8Ghz (dual core) to i7 3.2Ghz reduced GBT latency by 1s on unchanged OS (linux), p2pool, and bitcoind config.
That's higher than I would have expected, are you using bitcoind 0.8.3? If not, you should upgrade.
GBT latency is latency in bitcoind not p2pool and it doesn't matter much as forrestv pointed out months ago and tests confirmed.
That said, on a dual core which is used for some other CPU intensive tasks, you may experience lower efficiency (you need to have free CPU time for p2pool *and* bitcoind). A quad-core will probably handle this without lowering efficiency (count one core for bitcoind, one for p2pool and other ones as needed for the CPU intensive tasks you want to run in parallel).
... the previous block was 86 seconds before it ...
So that was a fast block, probably expected to get 86 / 600 as much transaction fees as an average block.
It's not empty, so got nothing to do with it ... as you can easily have seen by looking at the 3 blocks mined in the last 24 hours without having to guess which is the block I found ... even though my block is listed there in my post 
Probably but I'm lazy...
Quote
You can check what your bitcoind includes in its coinbase with : currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).
Code:
bitcoind getmininginfo
Code:
# ./bitcoind getmininginfo
{
"blocks" : 248451,
"currentblocksize" : 77889,
"currentblocktx" : 133,
"difficulty" : 31256960.72776893,
"errors" : "",
"generate" : false,
"genproclimit" : -1,
"hashespersec" : 0,
"pooledtx" : 1759,
"testnet" : false
}
#
Not really evil, txs that aren't included with default bitcoind settings pay below-minimum fees. Some pools obviously have more permissive settings and it probably pays off.
... looks like I need to decrease those 2 'min' values to at least stop bitcoind from screwing over the BTC network ...
Your bitcoind isn't really screwing anyone as it includes txs paying the minimum fees. TXs with lower fees are eventually included (older inputs get higher priority and there's provision for txs with no fees in a block), not giving a standard fee gets the expected outcome: slower processing by the Bitcoin network.
Someone else I was discussing this with suggested that p2pool decides to drops transactions itself based on performance, it wont use all the txns given to it by bitcoind ... is that correct?
I'll bet not. I didn't check the code but:
- p2pool logs match what you would expect when modifying bitcoind settings
- it's easier (and probably faster) to let bitcoind compute the coinbase instead of fiddling with it
- my node has the settings given in my previous mail which includes more txs than average and it is sending more data to other p2pool nodes than it receives, which matches the expectation that it processes more transaction and need to forward them to other nodes so that they can verify its shares
Interestingly related to that is that python sucks badly.
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes
python doesn't suck badly, it just isn't designed for HPC. Bad CPU performance isn't necessarily a show stopper: as long as p2pool finishes to process data before it's needed or not delay the process waiting too much (almost everything happens asynchronously in p2pool) it's fine. Recent changes have helped in this regard: p2pool can prepare work before it's needed with the new protocol. So it could be at 100% CPU for a whole second preparing work (that's assuming a slow CPU with multiple miners using a different payout address) and only delay the workbase communication to miners by ~3%.
That said I actually don't like the language itself but that's a matter of taste.
... the previous block was 86 seconds before it ...
So that was a fast block, probably expected to get 86 / 600 as much transaction fees as an average block.
It's not empty, so got nothing to do with it ... as you can easily have seen by looking at the 3 blocks mined in the last 24 hours without having to guess which is the block I found ... even though my block is listed there in my post
Probably but I'm lazy...
Quote
You can check what your bitcoind includes in its coinbase with : currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).
Code:
bitcoind getmininginfo
Code:
# ./bitcoind getmininginfo
{
"blocks" : 248451,
"currentblocksize" : 77889,
"currentblocktx" : 133,
"difficulty" : 31256960.72776893,
"errors" : "",
"generate" : false,
"genproclimit" : -1,
"hashespersec" : 0,
"pooledtx" : 1759,
"testnet" : false
}
#
Not really evil, txs that aren't included with default bitcoind settings pay below-minimum fees. Some pools obviously have more permissive settings and it probably pays off.
... looks like I need to decrease those 2 'min' values to at least stop bitcoind from screwing over the BTC network ...
Your bitcoind isn't really screwing anyone as it includes txs paying the minimum fees. TXs with lower fees are eventually included (older inputs get higher priority and there's provision for txs with no fees in a block), not giving a standard fee gets the expected outcome: slower processing by the Bitcoin network.
Someone else I was discussing this with suggested that p2pool decides to drops transactions itself based on performance, it wont use all the txns given to it by bitcoind ... is that correct?
I'll bet not. I didn't check the code but:
- p2pool logs match what you would expect when modifying bitcoind settings
- it's easier (and probably faster) to let bitcoind compute the coinbase instead of fiddling with it
- my node has the settings given in my previous mail which includes more txs than average and it is sending more data to other p2pool nodes than it receives, which matches the expectation that it processes more transaction and need to forward them to other nodes so that they can verify its shares
Interestingly related to that is that python sucks badly.
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes
python doesn't suck badly, it just isn't designed for HPC. Bad CPU performance isn't necessarily a show stopper: as long as p2pool finishes to process data before it's needed or not delay the process waiting too much (almost everything happens asynchronously in p2pool) it's fine. Recent changes have helped in this regard: p2pool can prepare work before it's needed with the new protocol. So it could be at 100% CPU for a whole second preparing work (that's assuming a slow CPU with multiple miners using a different payout address) and only delay the workbase communication to miners by ~3%.
That said I actually don't like the language itself but that's a matter of taste.
...
... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?
... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?
Just after a block is broadcasted on the bitcoin network there is less juicy txs on the network to include: if a block is found just after another one it may not have much tx fees.
Code:
2013-07-25 16:24:17 CTxMemPool::accept() : accepted 529cf3fc055a54a2c73dd04c9fddd948c92e955ebd4383faf3d71562a5164d18 (poolsz 1860)
2013-07-25 16:24:17 received block 000000000000006de7e7d91c59bbcbd7578b09a81a2241192c8accd98d076f6e
2013-07-25 16:24:17 Committing 2522 changed transactions to coin database...
2013-07-25 16:24:18 SetBestChain: new best=000000000000006de7e7d91c59bbcbd7578b09a81a2241192c8accd98d076f6e height=248393 log2_work=70.874776 tx=21101455 date=2013-07-25 16:23:19 progress=0.999996
2013-07-25 16:24:18 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:24:18 CTxMemPool::accept() : accepted e2d20ab0608ec065214f8c24ff2505ac4d6a2cd1a5e3904d61aff1bc80e2f39a (poolsz 1776)
Quote
Another rare cause for low income blocks : pools (not only p2pool) use a trick to avoid miners working on stale coinbases: when they detect a new block they immediately ask for (or maybe compute themselves) an empty coinbase which is really fast to compute and makes it possible for miners to do useful work as soon as possible. Then they ask for a "normal" coinbase which is slower.
It's not empty, so got nothing to do with it ... as you can easily have seen by looking at the 3 blocks mined in the last 24 hours without having to guess which is the block I found ... even though my block is listed there in my post Quote
You can check what your bitcoind includes in its coinbase with : currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).
Code:
bitcoind getmininginfo
Code:
# ./bitcoind getmininginfo
{
"blocks" : 248451,
"currentblocksize" : 77889,
"currentblocktx" : 133,
"difficulty" : 31256960.72776893,
"errors" : "",
"generate" : false,
"genproclimit" : -1,
"hashespersec" : 0,
"pooledtx" : 1759,
"testnet" : false
}
#
Quote
paytxfee only affects what your node includes in its own transactions.
Yes I knowQuote
To make juicier blocks (to maximize profit and speed up transaction confirmations everyone on p2pool should probably use this if they don't have any out-of-the-ordinary setup):
Code:
mintxfee=0.00001
minrelaytxfee=0.00001
blockmaxsize=1000000
The defaults (which are my settings since I haven't changed them - I only upgraded from 0.7.0 to 0.8.3 on Jul-8
) are:mintxfee=10000 = 0.00010000
minrelaytxfee=10000 = 0.00010000
blockmaxsize=250000
... looks like I need to decrease those 2 'min' values to at least stop bitcoind from screwing over the BTC network ...
Imagine how bad it must be to the BTC network for people who INCREASE those values ...
Someone else I was discussing this with suggested that p2pool decides to drops transactions itself based on performance, it wont use all the txns given to it by bitcoind ... is that correct?
Interestingly related to that is that python sucks badly.
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes
Nice, 2 more blocks last night while I was asleep ... I got one of them 
(my first block in MANY months)
(my first block in MANY months)
Edit: ... and I've spent some time now (since that post) working out why my block sux.
It is only 16k and only had 28 txn in it.
My bitcoind (default 0.8.3 settings for txns etc except: paytxfee=0.0005) on the other hand at the time had 1854 in it's mempool:
Code:
2013-07-25 16:25:43 CTxMemPool::accept() : accepted 7b1f0fd72e281abdb9b6724aa839b872550a3e7ff8de395dfbe20cd92777d4ae (poolsz 1854)
2013-07-25 16:25:43 ThreadRPCServer method=submitblock
.
.
2013-07-25 16:25:44 SetBestChain: new best=000000000000006e312482364ffc008b8fabad506fefde343a1acfc1f64ae982 height=248394 log2_work=70.874866 tx=21101483 date=2013-07-25 16:24:32 progress=0.999995
2013-07-25 16:25:44 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:25:44 CTxMemPool::accept() : accepted c75dd649944be1e262c3c82fda612e29c1631b74b7563f0142e28486623c2217 (poolsz 1828)
... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?
Just after a block is broadcasted on the bitcoin network there is less juicy txs on the network to include: if a block is found just after another one it may not have much tx fees.
Another rare cause for low income blocks : pools (not only p2pool) use a trick to avoid miners working on stale coinbases: when they detect a new block they immediately ask for (or maybe compute themselves) an empty coinbase which is really fast to compute and makes it possible for miners to do useful work as soon as possible. Then they ask for a "normal" coinbase which is slower.
You can check what your bitcoind includes in its coinbase with
Code:
bitcoind getmininginfo
paytxfee only affects what your node includes in its own transactions. To make juicier blocks (to maximize profit and speed up transaction confirmations everyone on p2pool should probably use this if they don't have any out-of-the-ordinary setup):
Code:
mintxfee=0.00001
minrelaytxfee=0.00001
blockmaxsize=1000000
Nice, 2 more blocks last night while I was asleep ... I got one of them
(my first block in MANY months)
(my first block in MANY months)
Edit: ... and I've spent some time now (since that post) working out why my block sux.
It is only 16k and only had 28 txn in it.
My bitcoind (default 0.8.3 settings for txns etc except: paytxfee=0.0005) on the other hand at the time had 1854 in it's mempool:
Code:
2013-07-25 16:25:43 CTxMemPool::accept() : accepted 7b1f0fd72e281abdb9b6724aa839b872550a3e7ff8de395dfbe20cd92777d4ae (poolsz 1854)
2013-07-25 16:25:43 ThreadRPCServer method=submitblock
.
.
2013-07-25 16:25:44 SetBestChain: new best=000000000000006e312482364ffc008b8fabad506fefde343a1acfc1f64ae982 height=248394 log2_work=70.874866 tx=21101483 date=2013-07-25 16:24:32 progress=0.999995
2013-07-25 16:25:44 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:25:44 CTxMemPool::accept() : accepted c75dd649944be1e262c3c82fda612e29c1631b74b7563f0142e28486623c2217 (poolsz 1828)
... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?
Ok, I can look for the javascript errors.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it
I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.
... and will have a zero balance until:best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it
I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.
1) You get a share
2) P2Pool gets a block after 1)
3) The payment switches to generate (about 16 hours)
Thanks for the heads up!
But it is an altcoin, 1 minute blocks, 60 confirms. The pool has pulled around 40 blocks in the last 3 hours. Checking the transactions on the client, they are not showing up as 'immature', and via the block explorer, the payments were sent to that strange address. The split was 95.5% going to that address, .5% going to the only miner on the pool.
I'll check it again in an hour, just to be sure. Also, if this is the wrong thread to post this, let me know where it should be posted.
Ok, I can look for the javascript errors.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it
I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.
... and will have a zero balance until:best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it
I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.
1) You get a share
2) P2Pool gets a block after 1)
3) The payment switches to generate (about 16 hours)
Ok, I can look for the javascript errors.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it
I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it
I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.
You're using non-standard UI?
can you share node's address?
can you share node's address?
Yes, it the ExtendedUI. I cloned p2pool from github git://github.com/forrestv/p2pool.git
then got the extended UI. The issue exists with both frontends.
Here's the link: http://sexcoin.lavajumper.com:9699
Over the last couple of hours some of the stats have started filling in.
There's another issue I'm seeing. Coins are not being dropped into the mining wallet. My miner is getting .5% even when its the only miner in the pool. I've chased down the address that they are going to, and I've never seen it before. Its never been on the pool.
If I need to clarify anything, let me know. I'm a bit frustrated at the moment.
Thanks for the help!
Data is not loading because there are javascript errors on the frontend. In particular, the frontend is trying to load /web/best_share_hash and is getting a 404 error. That is then cascading into several other javascript errors as a result. I have no idea why /web/best_share_hash is erroring out.
For the payout issue, did you specify a payout address on the command line? If not, p2pool will allocate a new address from the wallet it is connected to for things like any configured fee, etc.
You're using non-standard UI?
can you share node's address?
can you share node's address?
Yes, it the ExtendedUI. I cloned p2pool from github git://github.com/forrestv/p2pool.git
then got the extended UI. The issue exists with both frontends.
Here's the link: http://sexcoin.lavajumper.com:9699
Over the last couple of hours some of the stats have started filling in.
There's another issue I'm seeing. Coins are not being dropped into the mining wallet. My miner is getting .5% even when its the only miner in the pool. I've chased down the address that they are going to, and I've never seen it before. Its never been on the pool.
If I need to clarify anything, let me know. I'm a bit frustrated at the moment.
Thanks for the help!
You're using non-standard UI?
can you share node's address?
can you share node's address?
Javascript is on.
I'm getting this:
And this:
The only data filled in is the graphs.
If I got to /web/tails or /web/head in my browser I get '[]' for both.
Is there an idiots guide to setting this up for another coin. I got the .math entries, but I'm thinking I'm missing something big.
I'm getting this:
And this:
The only data filled in is the graphs.
If I got to /web/tails or /web/head in my browser I get '[]' for both.
Is there an idiots guide to setting this up for another coin. I got the .math entries, but I'm thinking I'm missing something big.
I'm setting up a p2pool for an altcoin. Mining appears to work well, but I can't get the stat page to display any stats. The stat headings show up, but not the data. The graphs are working. I'm not a python guy, and I'm having a hard time figuring out where to start. There's no errors in the logs. Could someone give a suggestion where to start looking?
You have to wait a while for the chain to get long enough for some of the stats to work, so mine for a few hours first. A screenshot would be helpful.
I'm setting up a p2pool for an altcoin. Mining appears to work well, but I can't get the stat page to display any stats. The stat headings show up, but not the data. The graphs are working. I'm not a python guy, and I'm having a hard time figuring out where to start. There's no errors in the logs. Could someone give a suggestion where to start looking?
javascript enabled?
I'm setting up a p2pool for an altcoin. Mining appears to work well, but I can't get the stat page to display any stats. The stat headings show up, but not the data. The graphs are working. I'm not a python guy, and I'm having a hard time figuring out where to start. There's no errors in the logs. Could someone give a suggestion where to start looking?
Idk the celerons, atoms and i5s I've used are trash
I'm running p2pool/bitcoind/altcoinds/cgminer on an Intel Next Unit of Computing with a Celeron 847 @ 1.10GHz, 4GB RAM with 4x AsicMiner USB Eruptors. I had to move the blockchains (bitcoin, devcoin, namecoin, ixcoin) to an external USB hard drive (system drive is 32GB SSD). It took some tweaking, but my get work latency is ~0.2-0.3 and I see DOA of 1.5% and 101.5% efficiency. CPU load seems pretty low (~80-90% idle mostly). Seems fine to me. Biggest problem is that my share luck seems to be inversely proportional to p2pool's block luck.
So looking at the two highest hash rate public nodes,
http://195.72.200.193:9332/ at ~230 GH/s and ~15 (!!) second latency has an efficiency of >105%
http://178.27.235.42:9332/ at ~150 GH/s and ~.25 second latency has an efficiency of ~100%
Even looking at ~3 GH/s nodes,
http://1.202.128.218:9332/ has ~2.5 second latency
http://88.190.31.3:9332/ has ~.1 second latency
Both are around ~101% efficient
http://195.72.200.193:9332/ at ~230 GH/s and ~15 (!!) second latency has an efficiency of >105%
http://178.27.235.42:9332/ at ~150 GH/s and ~.25 second latency has an efficiency of ~100%
Even looking at ~3 GH/s nodes,
http://1.202.128.218:9332/ has ~2.5 second latency
http://88.190.31.3:9332/ has ~.1 second latency
Both are around ~101% efficient
Jump to: