So this might be related to the USB3 - SATA - SSD dongle I'm using...
When I reboot the computer any command that tries to access the SSD freezes the SSH terminal.
But if I pull the USB3 plug and push it back in everything works fine!?!
Any clues?!
Is there a command to reset the USB3 ports without manually reconnecting them?
echo 0 > /sys/bus/usb/drivers/usb/X-X.X/authorized also freezes the prompt!
Ok, seems to be something bad with the USB3 - SATA stuff:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a9c54caa456dccba938005f6479892b589975e6a
But the most recent kernel is 3.10 for my hardware... how could I apply a fix without recompiling the kernel?
Found this fix, but I'm missing mkinitcpio!?
https://bbs.archlinux.org/viewtopic.php?pid=1428782#p1428782
Topic: Bitcoind 0.12 pruned locks file system (Read 1278 times)
Ok, seems it works fine if you start the bitcoind normally (= it downloads the whole chain on disk) and then enable pruning.
Might be the amount of pruning that solved it, I used -prune=650 first time around, now I used -prune=10000.
Seems to run fine... better hurry and boot/prune those bitcoinds before the chain grows beyond cheap SSD size!
Ok, yes it's a SSD, but it wasn't slow; it was frozen!
And if linux doesn't handle SSDs properly by default on recent versions I don't know what to say...
It is connected via a USB3 -> SATA adapter though, maybe that's the problem?
What is the syntax for discard?
Might be the amount of pruning that solved it, I used -prune=650 first time around, now I used -prune=10000.
Seems to run fine... better hurry and boot/prune those bitcoinds before the chain grows beyond cheap SSD size!
Just a thought; you're using an SSD. Are you using discard on the mount options, if supported by your distribution? If not, are you using trim on the file system at regular intervals?
Ok, yes it's a SSD, but it wasn't slow; it was frozen!
And if linux doesn't handle SSDs properly by default on recent versions I don't know what to say...
It is connected via a USB3 -> SATA adapter though, maybe that's the problem?
Code:
/dev/sda on /mnt/ssd type ext4 (rw,relatime,data=ordered)
What is the syntax for discard?
Code:
mount -o discard /dev/sda /mnt/ssd
Just a thought; you're using an SSD. Are you using discard on the mount options, if supported by your distribution? If not, are you using trim on the file system at regular intervals?
@knightdk There was nothing in the debug.log at the failure time I checked.
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
use ubuntu 14.04 because 15 not recommend
@knightdk There was nothing in the debug.log at the failure time I checked.
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
I'm using 15.10 as well and I cannot reproduce this. Can you provide specific steps for getting this problem? Also what are the specs of your system? @cr1776 Ubuntu 15.10, locally hosted? I guess yes!
@knightdk There was nothing in the debug.log at the failure time I checked.
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
use ubuntu 14.04 because 15 not recommend
@knightdk There was nothing in the debug.log at the failure time I checked.
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
@cr1776 Ubuntu 15.10, locally hosted? I guess yes!
...
How do you debug a process that freezes linux like that?!
How do you debug a process that freezes linux like that?!
What distro? Version? Locally hosted? Etc
Can you post the debug log?
Ok, pruned is broken on 0.12. Nobody cares?... well when the blockchain reaches 100GB and can't fit on my SSD I'll throw a hissyfit like Mike Hern!
To reproduce launch a new bitcoind with -prune=600 and let it sync, then reboot the machine and the bitcoind will lockup disk and the bitcoind process while verifying the block headers.
It will take you 4 days to reproduce every time, enjoy!
I'll copy the blockchain and prune it from this state to see if it works then... maybe this bug is due to the "prune from scratch" premise?
How do you debug a process that freezes linux like that?!
To reproduce launch a new bitcoind with -prune=600 and let it sync, then reboot the machine and the bitcoind will lockup disk and the bitcoind process while verifying the block headers.
It will take you 4 days to reproduce every time, enjoy!
I'll copy the blockchain and prune it from this state to see if it works then... maybe this bug is due to the "prune from scratch" premise?
How do you debug a process that freezes linux like that?!
I'm trying to run bitcoind pruned, the blockchain sync worked fine, and also once the sync was done bitcoind was running smoothly.
But then after a reboot, when I start bitcoind it boots up ok and does it's thing verifying some stuff but then it blocks 3 cores at 100% io-wait and the disk is frozen (if I try to cat any file bitcoind is holding the cat process freezes) and I can't kill bitcoind?!
Is anyone actually using pruned?
How should pruned be used? What is prune=? MB of what, how, where?
I'm trying to reindex and run unpruned to see if pruned is the problem. Just have to wait 4 days!
So bitcoind goes from taking 3 cores to 100% io-wait and locking these files:
to having only 1-2 cores 100% io-wait and locking this when I try kill -9:
But then after a reboot, when I start bitcoind it boots up ok and does it's thing verifying some stuff but then it blocks 3 cores at 100% io-wait and the disk is frozen (if I try to cat any file bitcoind is holding the cat process freezes) and I can't kill bitcoind?!
Is anyone actually using pruned?
How should pruned be used? What is prune=
I'm trying to reindex and run unpruned to see if pruned is the problem. Just have to wait 4 days!
So bitcoind goes from taking 3 cores to 100% io-wait and locking these files:
Code:
COMMAND PID TYPE SIZE MODE M START END PATH
cron 653 FLOCK 0B WRITE 0 0 0 /run
whoopsie 640 FLOCK 0B WRITE 0 0 0 /run/lock
master 864 FLOCK 0B WRITE 0 0 0 /
master 864 FLOCK 0B WRITE 0 0 0 /
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd/bitcoin/.lock
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd/bitcoin/blocks/index/LOCK
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd/bitcoin/chainstate/LOCK
cron 653 FLOCK 0B WRITE 0 0 0 /run
whoopsie 640 FLOCK 0B WRITE 0 0 0 /run/lock
master 864 FLOCK 0B WRITE 0 0 0 /
master 864 FLOCK 0B WRITE 0 0 0 /
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd/bitcoin/.lock
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd/bitcoin/blocks/index/LOCK
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd/bitcoin/chainstate/LOCK
to having only 1-2 cores 100% io-wait and locking this when I try kill -9:
Code:
COMMAND PID TYPE SIZE MODE M START END PATH
cron 653 FLOCK 0B WRITE 0 0 0 /run
whoopsie 640 FLOCK 0B WRITE 0 0 0 /run/lock
master 864 FLOCK 0B WRITE 0 0 0 /
master 864 FLOCK 0B WRITE 0 0 0 /
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd
cron 653 FLOCK 0B WRITE 0 0 0 /run
whoopsie 640 FLOCK 0B WRITE 0 0 0 /run/lock
master 864 FLOCK 0B WRITE 0 0 0 /
master 864 FLOCK 0B WRITE 0 0 0 /
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd
bitcoind 2014 POSIX 0B WRITE 0 0 0 /mnt/ssd
Jump to: