Memory leak found. p2pool process eating +600 MB of memory (+1GB virtual). Node uptime was about 12 hours?
From /proc:
$ cat status
Name: python
State: D (disk sleep)
Tgid: 9064
Pid: 9064
PPid: 9063
TracerPid: 0
Uid: 1000 1000 1000 1000
Gid: 1000 1000 1000 1000
FDSize: 128
Groups: 24 25 29 30 44 46 1000
VmPeak: 1105160 kB
VmSize: 1105156 kB
VmLck: 0 kB
VmHWM: 852568 kB
VmRSS: 643744 kB
VmData: 1038240 kB
VmStk: 204 kB
VmExe: 2164 kB
VmLib: 4976 kB
VmPTE: 2048 kB
Threads: 4
SigQ: 0/16382
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001001000
SigCgt: 0000000180016202
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed: f
Cpus_allowed_list: 0-3
Mems_allowed: 00000000,00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 803930
nonvoluntary_ctxt_switches: 272976
From htop:
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
9064 pioruns 20 0 1079M 620M 616 D 1.0 62.6 8h25:41 python /home/pioruns/p2pool/run_p2pool.py --fee=0.1 --give-author=0.1 --disable-upnp --outgoing-conns 5 --merged=xxx
EDIT: I just updated p2pool to latest version from git. Maybe will be better this time.
Newest version from git and first errors founded:
2012-11-26 10:31:28.835087 Punishing share for 'not all txs present'! Jumping from ecbce9f3 to 7b2cc82d!
2012-11-26 10:31:29.650609 Punishing share for 'not all txs present'! Jumping from ecbce9f3 to 7b2cc82d!
2012-11-26 10:31:59.581719 Processing 501 shares from 83.169.39.90:9333...
2012-11-26 10:32:00.210566 Punishing share for 'not all txs present'! Jumping from ecbce9f3 to 7b2cc82d!
2012-11-26 10:32:00.211285 ... done processing 501 shares. New: 501 Have: 19226/~17280
2012-11-26 10:32:00.211971 Requesting parent share 69de4908 from 24.211.170.224:9333
2012-11-26 10:32:00.215959 > Watchdog timer went off at:
2012-11-26 10:32:00.216313 > File "/home/pioruns/p2pool/run_p2pool.py", line 5, in
2012-11-26 10:32:00.216520 > main.run()
2012-11-26 10:32:00.216704 > File "/home/pioruns/p2pool/p2pool/main.py", line 561, in run
2012-11-26 10:32:00.216889 > reactor.run()
2012-11-26 10:32:00.217071 > File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1165, in run
2012-11-26 10:32:00.217257 > self.mainLoop()
2012-11-26 10:32:00.217434 > File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1177, in mainLoop
2012-11-26 10:32:00.217615 > self.doIteration(t)
2012-11-26 10:32:00.217789 > File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 140, in doSelect
2012-11-26 10:32:00.217972 > _logrun(selectable, _drdw, selectable, method, dict)
2012-11-26 10:32:00.218150 > File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 84, in callWithLogger
2012-11-26 10:32:00.218330 > return callWithContext({"system": lp}, func, *args, **kw)
2012-11-26 10:32:00.218505 > File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 69, in callWithContext
2012-11-26 10:32:00.218682 > return context.call({ILogContext: newCtx}, func, *args, **kw)
2012-11-26 10:32:00.218858 > File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 59, in callWithContext
2012-11-26 10:32:00.219210 > return self.currentContext().callWithContext(ctx, func, *args, **kw)
2012-11-26 10:32:00.219488 > File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 37, in callWithContext
2012-11-26 10:32:00.219753 > return func(*args,**kw)
2012-11-26 10:32:00.220006 > File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite
2012-11-26 10:32:00.220273 > why = getattr(selectable, method)()
2012-11-26 10:32:00.220537 > File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 460, in doRead
2012-11-26 10:32:00.220827 > return self.protocol.dataReceived(data)
2012-11-26 10:32:00.221105 > File "/home/pioruns/p2pool/p2pool/p2p.py", line 146, in new_dataReceived
2012-11-26 10:32:00.221366 > old_dataReceived(data)
2012-11-26 10:32:00.221618 > File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 39, in dataReceived
2012-11-26 10:32:00.221893 > self.dataReceived2(data)
2012-11-26 10:32:00.222179 > File "/home/pioruns/p2pool/p2pool/util/datachunker.py", line 40, in _DataChunker
2012-11-26 10:32:00.222467 > wants = receiver.send(buf.get(wants))
2012-11-26 10:32:00.222760 > File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 66, in dataReceiver
2012-11-26 10:32:00.223102 > self.packetReceived(command, type_.unpack(payload))
2012-11-26 10:32:00.223384 > File "/home/pioruns/p2pool/p2pool/p2p.py", line 91, in packetReceived
2012-11-26 10:32:00.223616 > p2protocol.Protocol.packetReceived(self, command, payload2)
2012-11-26 10:32:00.223809 > File "/home/pioruns/p2pool/p2pool/util/p2protocol.py", line 80, in packetReceived
2012-11-26 10:32:00.223989 > handler(**payload2)
2012-11-26 10:32:00.224166 > File "/home/pioruns/p2pool/p2pool/p2p.py", line 316, in handle_sharereply
2012-11-26 10:32:00.224348 > self.get_shares.got_response(id, res)
2012-11-26 10:32:00.224525 > File "/home/pioruns/p2pool/p2pool/util/deferral.py", line 124, in got_response
2012-11-26 10:32:00.224704 > df.callback(resp)
2012-11-26 10:32:00.224880 > File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 318, in callback
2012-11-26 10:32:00.225058 > self._startRunCallbacks(result)
2012-11-26 10:32:00.225285 > File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 424, in _startRunCallbacks
2012-11-26 10:32:00.225476 > self._runCallbacks()
2012-11-26 10:32:00.225655 > File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 441, in _runCallbacks
2012-11-26 10:32:00.225834 > self.result = callback(self.result, *args, **kw)
2012-11-26 10:32:00.226008 > File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 949, in gotResult
2012-11-26 10:32:00.226185 > _inlineCallbacks(r, g, deferred)
2012-11-26 10:32:00.226361 > File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 893, in _inlineCallbacks
2012-11-26 10:32:00.226539 > result = g.send(result)
2012-11-26 10:32:00.226713 > File "/home/pioruns/p2pool/p2pool/node.py", line 124, in download_shares
2012-11-26 10:32:00.226892 > self.handle_shares(shares, peer)
2012-11-26 10:32:00.227306 > File "/home/pioruns/p2pool/p2pool/node.py", line 40, in handle_shares
2012-11-26 10:32:00.227593 > self.node.set_best_share()
2012-11-26 10:32:00.227851 > File "/home/pioruns/p2pool/p2pool/node.py", line 284, in set_best_share
2012-11-26 10:32:00.228106 > best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2012-11-26 10:32:00.228406 > File "/home/pioruns/p2pool/p2pool/data.py", line 1030, in think
2012-11-26 10:32:00.228670 > ), h) for h in self.verified.tails.get(best_tail, []))
2012-11-26 10:32:00.228924 > File "/home/pioruns/p2pool/p2pool/data.py", line 1030, in
2012-11-26 10:32:00.229177 > ), h) for h in self.verified.tails.get(best_tail, []))
2012-11-26 10:32:00.229460 > File "/home/pioruns/p2pool/p2pool/data.py", line 336, in should_punish_reason
2012-11-26 10:32:00.229750 > other_txs = self._get_other_txs(tracker, known_txs)
2012-11-26 10:32:00.230013 > File "/home/pioruns/p2pool/p2pool/data.py", line 320, in _get_other_txs
2012-11-26 10:32:00.230266 > other_tx_hashes = self.get_other_tx_hashes(tracker)
2012-11-26 10:32:00.230516 > File "/home/pioruns/p2pool/p2pool/data.py", line 317, in get_other_tx_hashes
2012-11-26 10:32:00.230779 > return [tracker.items[tracker.get_nth_parent_hash(self.hash, x['share_count'])].share_info['new_transaction_hashes'][x['tx_count']] for x in self.share_info['transaction_hash_refs']]
2012-11-26 10:32:00.231127 > File "/home/pioruns/p2pool/p2pool/util/memoize.py", line 30, in b
2012-11-26 10:32:00.231403 > backing[(f, args)] = res
2012-11-26 10:32:00.231708 > File "/home/pioruns/p2pool/p2pool/util/memoize.py", line 16, in __setitem__
2012-11-26 10:32:00.231989 > while len(self.inner) > self.n:
2012-11-26 10:32:00.232262 > File "/home/pioruns/p2pool/p2pool/main.py", line 242, in
2012-11-26 10:32:00.232547 > sys.stderr.write, 'Watchdog timer went off at:\n' + ''.join(traceback.format_stack())
2012-11-26 10:32:00.361499 P2Pool: 18719 shares in chain (18724 verified/19226 total) Peers: 3 (0 incoming)
EDIT:
After 2 hours, it doesn't look nice. Actual memory usage 457MB, virtual 607MB. Pool is hanging for 2-3 minutes when processing packs of "501 shares" form different peers. Sometime processing takes 2 seconds, sometime up to 7 minutes. During processing, p2pool is completely unresponsible, mining doesn't work and webpage does not responding. Also p2pool reporting that it lost communication with bitcoind, while it is working fine all times.
Looks like with this new features, p2pool become completely unavailable for laptops and other low-end machines. You can check my daily stats here:
http://lenny.dnsd.me:9332/static/graphs.html?Day looks horrible