Author

Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 754. (Read 2591916 times)

donator
Activity: 1218
Merit: 1079
Gerald Davis
I'm pretty sure I'm running bitcoin 0.6.0rc2. But i guess rc1 broke my bitcoin db somehow.
Im currently testing on another machine, where I switched directly from 0.5.2 to 0.6.0rc2 at block 167000. But still have to download 1000 blocks. If this works ill just copy the database to the other machine.

1) You can use 0.5 bitcoind.

2) If you used the windows installer you almost certainly are not running 0.6.0 rc2.  It will say it installed but it doesn't overwrite everything from 0.6.0 rc1.

3) For p2pool I recommend using a bitcoind instance w/ a "blank wallet" and manually setting the payment address.  That way a bad wallet simply means delete wallet.dat and relaunch bitcoind to create a new one.

4) For custom windows installs the zip file is preferable.  You can drop it in a custom folder, delete the GUI bitcoin, and set data directory to the same folder.  Upgrading or replacing simple means deleting the folder.

hero member
Activity: 737
Merit: 500
Does p2pool still work with 0.5.2 or is 0.6.0rc2 absolutely necessary?

Yes, you can still use 0.5.x
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...
Does p2pool still work with 0.5.2 or is 0.6.0rc2 absolutely necessary?
I knew I shouldn't have upgraded to a unstable version on a productive system.
Yeah they are called 'rcX' for a reason.
Nothing should be depending on them.
newbie
Activity: 55
Merit: 0
tucenaber and broker, you both need to upgrade to bitcoin 0.6.0rc2.

If you're on windows, look at my most recent post in this thread.  The 0.6.0rc2 installer doesn't work right.  You have to use the zip file.
I was sure I'd upgraded to rc2 but because the installer didn't work I was still on rc1 and had the problem you had.

I'm pretty sure I'm running bitcoin 0.6.0rc2. But i guess rc1 broke my bitcoin db somehow.
Im currently testing on another machine, where I switched directly from 0.5.2 to 0.6.0rc2 at block 167000. But still have to download 1000 blocks. If this works ill just copy the database to the other machine.

edit:

Does p2pool still work with 0.5.2 or is 0.6.0rc2 absolutely necessary?
I knew I shouldn't have upgraded to a unstable version on a productive system.
legendary
Activity: 916
Merit: 1003
tucenaber and broker, you both need to upgrade to bitcoin 0.6.0rc2.

If you're on windows, look at my most recent post in this thread.  The 0.6.0rc2 installer doesn't work right.  You have to use the zip file.
I was sure I'd upgraded to rc2 but because the installer didn't work I was still on rc1 and had the problem you had.
sr. member
Activity: 337
Merit: 252
I have the same issue. I started getting
Quote
WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nodes may need to upgrade.
with every RCP query and tried to upgrade to rc2 without success.

bitcoind sits out database error once in a while:

Quote
************************
EXCEPTION: 11DbException       
Db::get: Cannot allocate memory       
bitcoin in ProcessMessage()       

Now 0.5.2 gives the same error.

The only difference from the above post is that my transaction count is 170061

I have tried to delete the database log.0000* but that doesn't change anything.

(I am running Arch linux which use a different version of BDB, could that be an issue? I haven't had any issues with that in the past)
newbie
Activity: 55
Merit: 0
It cant manage to upgrade to bitcoin-0.6.0-rc2. It always seems it still starts rc1.

# The files are apperently different

broken@rig02:~/bitcoin$ ls -la bitcoin-0.6.0rc1-linux/bin/64/ | grep bitcoin
-rwxr-xr-x 1 broken broken 4247696 2012-02-07 19:46 bitcoind
-rwxr-xr-x 1 broken broken 8891013 2012-02-07 19:46 bitcoin-qt

broken@rig02:~/bitcoin$ ls -la bitcoin-0.6.0rc2-linux/bin/64/ | grep bitcoin
-rwxr-xr-x 1 broken broken 4251824 2012-02-29 21:59 bitcoind
-rwxr-xr-x 1 broken broken 9053450 2012-02-29 21:59 bitcoin-qt

# Bitcoin is stopped
broken@rig02:~/bitcoin$ ps -ef | grep coin
broken     2165     1  0 Feb14 ?        02:40:52 ./namecoind -daemon

# Starting bitcoin
broken@rig02:~/bitcoin$ cd bitcoin-0.6.0rc2-linux/bin/64/
broken@rig02:~/bitcoin/bitcoin-0.6.0rc2-linux/bin/64$ ./bitcoind -daemon
bitcoin server starting
************************
EXCEPTION: 11DbException      
Db::put: Cannot allocate memory      
bitcoin in ProcessMessage()

# Bitcoin is now running
broken@rig02:~/bitcoin$ ps -ef | grep coin
broken     2165     1  0 Feb14 ?        02:40:53 ./namecoind -daemon
broken    15663     1 10 09:45 ?        00:00:12 ./bitcoind -daemon

# Its the correct binary
broken@rig02:~/bitcoin$ readlink -f /proc/15663/exe
/home/broken/bitcoin/bitcoin-0.6.0rc2-linux/bin/64/bitcoind

# Bitcoin gives error and is stuck at 170059
broken@rig02:~/bitcoin/bitcoin-0.6.0rc2-linux/bin/64$ ./bitcoind getinfo
{
    "version" : 60000,
    "protocolversion" : 60000,
    "balance" : 0.00000000,
    "blocks" : 170059,
    "connections" : 8,
    "proxy" : "",
    "difficulty" : 1496978.59502557,
    "testnet" : false,
    "keypoololdest" : 1322670639,
    "keypoolsize" : 103,
    "paytxfee" : 0.00000000,
    "errors" : "WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nodes may need to upgrade."
}

Anyone have a idea how to upgrade?
full member
Activity: 125
Merit: 100
So, I might be wrong, but I just did a git pull earlier and now I keep seeing this every now and then:

Code:




  Pool: 267GH/s Stale rate: 8.4% Expected time to block: 6.7 hours
2012-03-08 02:52:25.538244 Incoming connection to peer 77.91.201.77:33987 established. p2pool version: 2 '5ad7c89'
2012-03-08 02:52:25.539173 Got share hash, requesting! Hash: 86a8a3f4
2012-03-08 02:52:25.724369 Sending 1 shares to 77.91.201.77:33987
2012-03-08 02:52:25.725745 RECV shares 0100fd2801010000008c428aa19c5e7994dbb352b430a4e506a526cecf7fe47958f304000000000000e84453d66984eb29f05f6b94dfce3fd4144b3b8944d9545a621a8cd2e32e7b965263584f0c350b1ad480175e99b93670dcb09552a12756c4d581e2...
2012-03-08 02:52:25.727150 > Error handling message: (see RECV line)
2012-03-08 02:52:25.727829 > Traceback (most recent call last):
2012-03-08 02:52:25.728164 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
2012-03-08 02:52:25.728636 >     why = getattr(selectable, method)()
2012-03-08 02:52:25.729543 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-03-08 02:52:25.730067 >     rval = self.protocol.dataReceived(data)
2012-03-08 02:52:25.730413 >   File "/home/miner/p2pool/p2pool/p2p.py", line 55, in new_dataReceived
2012-03-08 02:52:25.730968 >     old_dataReceived(data)
2012-03-08 02:52:25.731593 >   File "/home/miner/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-03-08 02:52:25.731945 >     wants = receiver.send(buf.get(wants))
2012-03-08 02:52:25.732330 > --- ---
2012-03-08 02:52:25.732657 >   File "/home/miner/p2pool/p2pool/bitcoin/p2p.py", line 53, in dataReceiver
2012-03-08 02:52:25.733014 >     self.packetReceived(command, type_.unpack(payload))
2012-03-08 02:52:25.733376 >   File "/home/miner/p2pool/p2pool/p2p.py", line 68, in packetReceived
2012-03-08 02:52:25.733734 >     bitcoin_p2p.BaseProtocol.packetReceived(self, command, payload2)
2012-03-08 02:52:25.734063 >   File "/home/miner/p2pool/p2pool/bitcoin/p2p.py", line 66, in packetReceived
2012-03-08 02:52:25.734418 >     handler(**payload2)
2012-03-08 02:52:25.734744 >   File "/home/miner/p2pool/p2pool/p2p.py", line 199, in handle_shares
2012-03-08 02:52:25.735096 >     self.node.handle_shares([p2pool_data.Share.from_share(share, self.node.net, self) for share in shares], self)
2012-03-08 02:52:25.735476 >   File "/home/miner/p2pool/p2pool/data.py", line 187, in from_share
2012-03-08 02:52:25.735829 >     raise ValueError('unknown share type: %r' % (share['type'],))
2012-03-08 02:52:25.736208 > exceptions.ValueError: unknown share type: 0
2012-03-08 02:52:25.737452 Lost peer 77.91.201.77:33987 - Connection was closed cleanly.

:19.499847  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0413 BTC
2012-03-08 03:25:19.499969  Pool: 272GH/s Stale rate: 9.5% Expected time to block: 6.5 hours
2012-03-08 03:25:20.160618 Incoming connection to peer 67.183.247.222:60716 established. p2pool version: 2 '053d7ca'
2012-03-08 03:25:20.161605 Got share hash, requesting! Hash: 1a6c8e71
2012-03-08 03:25:20.266125 Sending 1 shares to 67.183.247.222:60716
2012-03-08 03:25:20.372467 RECV shares 0100fd88010100000067a2a2950ec6bad48b6c37088f5e9531f725fa730b3a49059e0500000000000004a3f8996d8b4b95e03f5d57f68625e0bef73a8edbefba292612bc7feb5b9305fe4d584f0c350b1ab73d4a27dd55c17161a4cb5868763ed263aaad...
2012-03-08 03:25:20.374046 > Error handling message: (see RECV line)
2012-03-08 03:25:20.374423 > Traceback (most recent call last):
2012-03-08 03:25:20.374775 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
2012-03-08 03:25:20.375150 >     why = getattr(selectable, method)()
2012-03-08 03:25:20.375504 >   File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-03-08 03:25:20.375845 >     rval = self.protocol.dataReceived(data)
2012-03-08 03:25:20.376244 >   File "/home/miner/p2pool/p2pool/p2p.py", line 55, in new_dataReceived
2012-03-08 03:25:20.376582 >     old_dataReceived(data)
2012-03-08 03:25:20.376904 >   File "/home/miner/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-03-08 03:25:20.378011 >     wants = receiver.send(buf.get(wants))
2012-03-08 03:25:20.378355 > --- ---
2012-03-08 03:25:20.378689 >   File "/home/miner/p2pool/p2pool/bitcoin/p2p.py", line 53, in dataReceiver
2012-03-08 03:25:20.379071 >     self.packetReceived(command, type_.unpack(payload))
2012-03-08 03:25:20.379417 >   File "/home/miner/p2pool/p2pool/p2p.py", line 68, in packetReceived
2012-03-08 03:25:20.379792 >     bitcoin_p2p.BaseProtocol.packetReceived(self, command, payload2)
2012-03-08 03:25:20.380138 >   File "/home/miner/p2pool/p2pool/bitcoin/p2p.py", line 66, in packetReceived
2012-03-08 03:25:20.380540 >     handler(**payload2)
2012-03-08 03:25:20.380842 >   File "/home/miner/p2pool/p2pool/p2p.py", line 199, in handle_shares
2012-03-08 03:25:20.381192 >     self.node.handle_shares([p2pool_data.Share.from_share(share, self.node.net, self) for share in shares], self)
2012-03-08 03:25:20.381564 >   File "/home/miner/p2pool/p2pool/data.py", line 187, in from_share
2012-03-08 03:25:20.381911 >     raise ValueError('unknown share type: %r' % (share['type'],))
2012-03-08 03:25:20.382259 > exceptions.ValueError: unknown share type: 0
2012-03-08 03:25:20.383555 Lost peer 67.183.247.222:60716 - Connection was closed cleanly.


Is this a share that I am submitting that is somehow returning an error message?

Edited:

Just got this again, looks like its everytime I am finding a share, any ideas?  Is the bitcoin 6rc now required, because I am still using 0.5.2

Well, after digging in the logs a bit it looks like this happens whether or not I'm mining.
full member
Activity: 125
Merit: 100
I suppose, but we are talking about a 40 dollar processor, so its not a huge loss if something goes wrong =)
c_k
donator
Activity: 242
Merit: 100
Bad idea unlocking core imo
full member
Activity: 125
Merit: 100
I am running bitcoind and p2pool on a sempron 145 right now, but will soon have another rig up and running with a dual core processor.  I will probably move my p2pool over to that rig then (or run it on both as a backup) then I'll be able to give people a real world example of processor powers impact on p2pool stales. 

It might be worth checking if your single core can be unlocked, I know some of the semprons can be, mine can't unfortunately, but worth a look.
donator
Activity: 1218
Merit: 1079
Gerald Davis
using BAMT on usb sticks.  I can install it in a parallel directory I suppose.  Probably a good idea to keep closer to the latest release anyhow

Here is why I usually do.  install new version in /opt/miners folder under a new directory.  test it on one rig by stopping gpumon and running "new" cgminer manually from that directory.  If everything looks good rename the cgminer folder to cgminer-2.2.7 and rename new cgminer folder (i.e. cgminer-2.3.1 to cgminer.  start from gpumon.  Makes sure everything looks good for a day and then upgrade all other rigs.

It has never failed me.  BAMT has no knowledge of what version of cgminer it is running it simply executes the cgminer in the /opt/miners/cgminer folder.

Quote
bitcoind is running on what appears to be an underpowered server (see earlier response to Forrest).  P2pool software is piggy-backing on a miner with 2GB and seems to be not taxing at all (that miner is a BAMT USB stick FWIW).

Stop the troubleshooting.  This is it then.  bitcoind needs to be responsive.  I mean fraction of a second responsive.  Once you get it running fast somewhere put p2pool there too then point all your rig(s) at that p2pool instance.  Having bitcoind and p2pool on the same "server" means less latency.

I think you will find one bitcoind is fast and responsive your stales will plummet.
member
Activity: 65
Merit: 10
Not sure why your stales are so high.  First thing it always starts at 0% that is pretty much meaningless.  It spiking to 112% in a short period of time is equally meaningless.  When you have a small number of shares the "luck" of a single stale can affect that stat by a lot.  Say hypothetically you have 10 shares w/ 1 stale (10% stale or ~100% efficiency)  The 11th share is coming in.  If it is stale your efficiency will drop to 91% if it is valid then your efficiency will rise to 102%.  What matters if the value after some time (say 12 to 24 hours @ 3 GH/s).   Once you have hundreds of shares the value of each one is less material and the number will stabilize.

makes sense, thanks.

Quote
A couple of ideas:
1) Use latest version of cgminer.  2.2.7 may be fine but some versions are less p2pool friendly.

using BAMT on usb sticks.  I can install it in a parallel directory I suppose.  Probably a good idea to keep closer to the latest release anyhow

Quote
2) Are your stales mostly dead or orphaned?  Some dead is unavoidable (similar to any other pool) but a high dead may indicate you are pushing the cards too hard and causing invalid hashes.

I've got 7 orphaned, 4 dead since earlier today (is that statistically significant?).  I'm not seeing any hardware errors reported by cgminer

Quote
3) Make sure your system running bitcoind & p2pool isn't overloaded.  Preferably this would be a non-mining system which is mostly idle.  bitcoind & p2pool use some memory and their operations are very atency sensitive.  If they are slow to issue LP you are going to have excessive stales.


bitcoind is running on what appears to be an underpowered server (see earlier response to Forrest).  P2pool software is piggy-backing on a miner with 2GB and seems to be not taxing at all (that miner is a BAMT USB stick FWIW).

Quote
4) Make sure p2pool and bitcoind have a high number of connections (this means port forwarding)

check - iptables rules forwarding inbound 9333 to p2pool server and verified - 21 peers, 11 incoming

Quote
5) Make sure your internet connection is solid w/ low latency and good uptime.

Verizon FIOS - 20/20...I'm very lucky to have it in my area.

Quote
As a test try using public p2pool node and see if your stale rates improve.  If it does then it is something in your internet, LAN, p2pool machine which is being inefficient.  If it doesn't improve then the issue lies with the miner.

That's a great idea...going to do this tomorrow morning, leave it for a day or so.  Thank you!
hero member
Activity: 682
Merit: 500
Hello Wachtwoord, I was using the one day window as an example, I've been observing this for over a week now, restarting p2pool and sometimes bitcoind when it zombies out.  I should have mentioned that bit before, I've got bitcoind 0.6rc2 running on a separate machine where it has enough ram and disk yet a slower processor.  I have observed p2pool reporting that bitcoind is not responding for some number of minutes and I have to kill -9 bitcoind and restart.  This happens some of the time and often self-corrects (i.e. p2pool eventually reconnects with bitcoind).

P2Pool's wiki page has some suggestions for increasing your efficiency: https://en.bitcoin.it/wiki/P2Pool#Frequently_Asked_Questions

EDIT: Any idea what's going wrong with bitcoind? If bitcoind is dead and you continue to mine, you'll get orphans because your shares aren't up-to-date (any block solutions would be invalid).

I think I know what's going on here.  The "server" I'm running bitcoind on is an older AMD Geode running at 500MHz at a whopping 996 bogomips and 1GB of memory.  I thought that would be enough, but it appears it's not.  I just checked on the p2pool server (a miner doing double duty) and saw in the logs:

2012-03-03 01:27:15.510957 > Failure: twisted.internet.defer.TimeoutError: Getting http://192.168.1.105:8332/ took longer than 5 seconds.

so how pervasive is this timeout?

user@btc2:~$ grep "took longer" p2pool/data/bitcoin/log |wc
   9067  108804 1251239
user@btc2:~$

over 9K since last Saturday, clearly that's sub-optimal, so I need to find a different system to host bitcoind.

It's over nine-thousand!!!

With that out of the way, I got my mining rig updated to 0.6.0 rc2. To anyone having issues with the installer, the fix is simple, use the .zip. I like to personally overwrite files anyway. Just about anything can go wrong when the installer overwrites (and that is painfully obvious in this case).
legendary
Activity: 916
Merit: 1003
Ahh.. this happened to me too.. I thought rc1 had corrupted my database.. but I guess it just never updated. The installer is definitely broken.
Is there a thread to report this on?  My guess is we're not the only ones having this problem.

A side note: you can still use rc1 but you have to put a speshul switch on it to work.  I found this in this thread:
https://bitcointalksearch.org/topic/m.789043

Apparently the only difference between rc1 and rc2 is the internal date where it stops working and starts complaining to upgrade due to the BIP16 switchover.  The "-paytoscripthashtime=1333238400" switch delays it until 20120401.
member
Activity: 65
Merit: 10
Hello Wachtwoord, I was using the one day window as an example, I've been observing this for over a week now, restarting p2pool and sometimes bitcoind when it zombies out.  I should have mentioned that bit before, I've got bitcoind 0.6rc2 running on a separate machine where it has enough ram and disk yet a slower processor.  I have observed p2pool reporting that bitcoind is not responding for some number of minutes and I have to kill -9 bitcoind and restart.  This happens some of the time and often self-corrects (i.e. p2pool eventually reconnects with bitcoind).

P2Pool's wiki page has some suggestions for increasing your efficiency: https://en.bitcoin.it/wiki/P2Pool#Frequently_Asked_Questions

EDIT: Any idea what's going wrong with bitcoind? If bitcoind is dead and you continue to mine, you'll get orphans because your shares aren't up-to-date (any block solutions would be invalid).

I think I know what's going on here.  The "server" I'm running bitcoind on is an older AMD Geode running at 500MHz at a whopping 996 bogomips and 1GB of memory.  I thought that would be enough, but it appears it's not.  I just checked on the p2pool server (a miner doing double duty) and saw in the logs:

2012-03-03 01:27:15.510957 > Failure: twisted.internet.defer.TimeoutError: Getting http://192.168.1.105:8332/ took longer than 5 seconds.

so how pervasive is this timeout?

user@btc2:~$ grep "took longer" p2pool/data/bitcoin/log |wc
   9067  108804 1251239
user@btc2:~$

over 9K since last Saturday, clearly that's sub-optimal, so I need to find a different system to host bitcoind.
sr. member
Activity: 604
Merit: 250
Ahh.. this happened to me too.. I thought rc1 had corrupted my database.. but I guess it just never updated. The installer is definitely broken.

It seems like the bitcoin client itself is unhappy.  I'm using bitcoin 0.6.0rc2 which is the latest AFAIK.  It reports the error "WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nodes may need to upgrade."

Based on your comments I'm quite confident that you're actually running 0.6.0rc1. What you are seeing is exactly symptomatic of rc1. You can get rc2 from http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.0/test/   if you really did install rc2, are you sure you restarted after doing so?

I went to that URL and ran the windows installer .exe and it turns out it didn't overwrite the previous version's executables for some reason.  So I downloaded the corresponding zip file and extracted that, pointed my shortcuts to it and now it's fine.  Weird.

I'm sure I did this upgrade a while back but clearly there's some kind of installer problem with the rc2 windows installer.
legendary
Activity: 916
Merit: 1003
It seems like the bitcoin client itself is unhappy.  I'm using bitcoin 0.6.0rc2 which is the latest AFAIK.  It reports the error "WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nodes may need to upgrade."

Based on your comments I'm quite confident that you're actually running 0.6.0rc1. What you are seeing is exactly symptomatic of rc1. You can get rc2 from http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.0/test/   if you really did install rc2, are you sure you restarted after doing so?

I went to that URL and ran the windows installer .exe and it turns out it didn't overwrite the previous version's executables for some reason.  So I downloaded the corresponding zip file and extracted that, pointed my shortcuts to it and now it's fine.  Weird.

I'm sure I did this upgrade a while back but clearly there's some kind of installer problem with the rc2 windows installer.
staff
Activity: 4284
Merit: 8808
It seems like the bitcoin client itself is unhappy.  I'm using bitcoin 0.6.0rc2 which is the latest AFAIK.  It reports the error "WARNING: Displayed transactions may not be correct!  You may need to upgrade, or other nodes may need to upgrade."

Based on your comments I'm quite confident that you're actually running 0.6.0rc1. What you are seeing is exactly symptomatic of rc1. You can get rc2 from http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.0/test/   if you really did install rc2, are you sure you restarted after doing so?
hero member
Activity: 737
Merit: 500
Just thinking about how I would detect these...  I could look for sendmany transactions that have known addresses in them.  Maybe the donation address is enough? 

The donation address appears not to be included in subsidy payments.  So I don't have a consistent way to identify subsidy payments unless someone can provide me one or more addresses that have consistently been mining with a reasonably high hashrate (and therefore has received every subsidy sent in the past) and will continue to be consistently mining with a high hashrate (and will therefore receive every subsidy sent in the future).

I can identify the subsidy payments that I have received, but I know that some payments happened before I switched to p2pool.  And my hashrate is just above the cutoff where if the pool grows much more, I may not receive every payment in the future (because my share would be below 0.01 of the total payment).

The only other thing I can do is maintain a list by hand.  It could still appear on the website, but someone would have to tell me every time a subsidy was sent and I'd have to manually add it to the list.

So...
Jump to: