Just a thought, but another possibility might be that the
Linux out-of-memory killer was tripped. Many VPS providers don't create a swap file by default, which might be a cause of this (but be careful, creating a swap file might cost you $ depending on the provider and on where the swap file is created).
I am familiar with that killer from before, that's why I upgraded RAM on the server. However, it's constantly been high, and I was also running testnet, which didn't make it any better. Bitcoind debug.log did not give any indication of memory failure however, and I couldn't see that in any other logfiles either, but the first time, before RAM upgrade, logfiles clearly showed memory was an issue.
I'm just surprised I did not get an e-mail from the host when the VPS did shut down.
As for top, it currently shows:
KiB Mem: 2097152 total, 2026652 used, 70500 free, 0 buffers
KiB Swap: 2097152 total, 48504 used, 2048648 free, 1216420 cached
The host says there's 2GB Vswap, and from what I found about this:
What is VSwap?
The new CentOS 6 OpenVZ kernel has a new memory management model, which supersedes User beancounters. It is called VSwap.
When the Guaranteed Ram limit is reached, memory pages belonging to the container are pushed out to so called virtual swap (vswap). The difference between normal swap and vswap is that with vswap no actual disk I/O usually occurs. Instead, a container is artificially slowed down, to emulate the effect of the real swapping. Actual swap out occurs only if there is a global memory shortage on the system.
So perhaps this is the problem?
ps -e -o pid,vsz,comm= | sort -n -k 2
previously gave.
1632 124348 rsyslogd
7784 434028 bitcoind (testnet i assume)
7816 1246896 bitcoind
I've shutdown the testnet node, and limiting connections on the bitcoind, and will try to clean up other processes I do not need.
I will see if the problem persists after the upgrade, if it does, I will contact the hosting company. 