Author

Topic: How to deal with kernel updates (Read 305 times)

hero member
Activity: 924
Merit: 5943
not your keys, not your coins!
March 22, 2022, 04:15:43 PM
#16
Now the only process that I have to start manually is JS-Node and the coin core. But all the other services are configured full auto now.
Cool! I don't know what JS-Node is, but for Bitcoin Core if you need assistance, there's a section on it in my full node guide. Should be possible to copy the service file with minor modifications (username, path).
legendary
Activity: 3388
Merit: 3154
March 22, 2022, 11:10:16 AM
#15
...
Honestly, no matter if you're a developer or not, if a reboot is such a big issue, your server isn't setup correctly. In my opinion, you should setup your stuff in a way that you can reboot (or boot e.g. after a power loss) at any time without issues.

I assume you have multiple tmux or screen sessions in the background, each running some important scripts. All of this type of stuff should be set up as a service so it's restarted automatically after booting, but also after a dropped internet connection or a power loss. Anything else will be pretty risky and unreliable in the long run.

You are totally right, the reboot isn't a big deal if our server is configured the right way.

A couple of days ago I configured my services, that way they auto-start when I reboot the machine, and now the only process that I have to start manually is JS-Node and the coin core. But all the other services are configured full auto now.
hero member
Activity: 924
Merit: 5943
not your keys, not your coins!
March 11, 2022, 10:42:48 AM
#14
Any reason you are worried about it?

Sometimes a reboot is a complex task, is not about stopping and starting bitcoin core, as a developer I'm running other services like SQL, Apache, JSNode, and Other services, and if you think about the fact that we have a kernel update really often, it would be nice to avoid restarting the workstation.
Honestly, no matter if you're a developer or not, if a reboot is such a big issue, your server isn't setup correctly. In my opinion, you should setup your stuff in a way that you can reboot (or boot e.g. after a power loss) at any time without issues.

I assume you have multiple tmux or screen sessions in the background, each running some important scripts. All of this type of stuff should be set up as a service so it's restarted automatically after booting, but also after a dropped internet connection or a power loss. Anything else will be pretty risky and unreliable in the long run.
copper member
Activity: 1666
Merit: 1901
Amazon Prime Member #7
February 27, 2022, 05:03:31 PM
#13
From the things you listed that runs on your server you can consider a cluster network where one system will reboot without affecting a running program on the other systems. You can also consider using a Virtual IP to create a firewall or cluster router to make sure that they are no single point of failure. Also check this thread on stack

https://unix.stackexchange.com/questions/345561/how-linux-servers-update-their-kernel-without-rebooting

This solution is like having a clon of the server this way we can direct the IP to the second server while we update the other one. And this is a good solution if we have a public service, but in my case, I run the server for development, and isn't public at all. So, I don't think this is the best solution for me.

I was reading more about the topic and rebooting the system is the best way to deal with updates. The problem with the live patch for updates is not only the kernel, if I update any of the services I'm running (like SQL or Apache) I will have to restart the service to use the new version. That's why is important to restart after the updates.


If your server is only used for development, as others have mentioned, your best bet is to just exit bitcoin core cleanly and restart. You can create a script to do this after hours at a time when the lowest number of developers are going to be using your server, and provide those who use your server advance notification of the downtime.
legendary
Activity: 3388
Merit: 3154
February 27, 2022, 03:31:45 PM
#12
From the things you listed that runs on your server you can consider a cluster network where one system will reboot without affecting a running program on the other systems. You can also consider using a Virtual IP to create a firewall or cluster router to make sure that they are no single point of failure. Also check this thread on stack

https://unix.stackexchange.com/questions/345561/how-linux-servers-update-their-kernel-without-rebooting

This solution is like having a clon of the server this way we can direct the IP to the second server while we update the other one. And this is a good solution if we have a public service, but in my case, I run the server for development, and isn't public at all. So, I don't think this is the best solution for me.

I was reading more about the topic and rebooting the system is the best way to deal with updates. The problem with the live patch for updates is not only the kernel, if I update any of the services I'm running (like SQL or Apache) I will have to restart the service to use the new version. That's why is important to restart after the updates.

hero member
Activity: 1302
Merit: 561
Leading Crypto Sports Betting & Casino Platform
February 27, 2022, 01:30:36 PM
#11
From the things you listed that runs on your server you can consider a cluster network where one system will reboot without affecting a running program on the other systems. You can also consider using a Virtual IP to create a firewall or cluster router to make sure that they are no single point of failure. Also check this thread on stack

https://unix.stackexchange.com/questions/345561/how-linux-servers-update-their-kernel-without-rebooting
legendary
Activity: 2212
Merit: 7064
February 23, 2022, 12:51:49 PM
#10
This is a question I had since a long time ago, if I'm running a Linux distro with my Bitcoin Node each time I update de system and it has a Kernel update (which is really common) I have to reboot the system. Which means to stop the Bitcoin Core.
You don't need to update Linux OS all the time, except it the case of some important security update, so you can disable it and do manual updates.
There is nothing wrong in restarting your computer and node sometimes, do some cleanup if needed and than quickly sync up your node after that.
I don't know what Linux OS you are suing but I prefer stable version that is rock solid and just works, unlike rolling release system that are often updated with latest software releases.
legendary
Activity: 3388
Merit: 3154
February 23, 2022, 12:46:06 PM
#9
Any reason you are worried about it?

Sometimes a reboot is a complex task, is not about stopping and starting bitcoin core, as a developer I'm running other services like SQL, Apache, JSNode, and Other services, and if you think about the fact that we have a kernel update really often, it would be nice to avoid restarting the workstation.

...
Code:
yum install kpatch
yum kpatch auto

My server is running Fedora, i will try kpatch, that should do the magic. And is important to mention that kpatch works in the next linux distros:

    Fedora, RHEL, CentOS
    Oracle Linux 7
    Ubuntu
    Debian 9 (Stretch)
    Debian 8 (Jessie)
    Debian 7 (Lenny)
    Gentoo

Source: https://github.com/dynup/kpatch

legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
February 23, 2022, 09:16:39 AM
#8
For centos there is kernelcare package free & paid both available suitable for apply patch without reboot.

Just reboot it, but make sure Bitcoin Core exit cleanly. Init system such as systemd kill any process if it took longer than 90 second.

Red Hat distros have a particular Live Patch program that installs the kernel updates on the running kernel so that a reboot is not required.

IIRC it meant to fix critical CVE without rebooting and Red Hat recommend you to make a reboot during planned downtime.

I do not believe it's available for other distros though, since Red Hat builds their own kernels.

Code:
yum install kpatch
yum kpatch auto

Any distro based on Red Hat should have such feature. If you're Arch user, you could install and configure kpatch manually.

I do not believe it's available for other distros though, since Red Hat builds their own kernels.
There are other programs for almost all other distros too. Some of them may require a license tough.
For more details:
https://blog.kernelcare.com/avoid-death-taxes-and-linux-server-reboots-kernel-updates-3-different-ways

kpatch & friends should be sufficient if you're just concerned about applying kernel security updates without reboots. And generally, if you're leaving a node and supporting software running for an extended amount of time, chances are, you don't want feature updates since they could break your workflow.

newbie
Activity: 2
Merit: 3
February 23, 2022, 09:08:49 AM
#7
I do not believe it's available for other distros though, since Red Hat builds their own kernels.
There are other programs for almost all other distros too. Some of them may require a license tough.
For more details:
https://blog.kernelcare.com/avoid-death-taxes-and-linux-server-reboots-kernel-updates-3-different-ways
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
February 23, 2022, 06:29:11 AM
#6
Just reboot it, but make sure Bitcoin Core exit cleanly. Init system such as systemd kill any process if it took longer than 90 second.

Red Hat distros have a particular Live Patch program that installs the kernel updates on the running kernel so that a reboot is not required.

IIRC it meant to fix critical CVE without rebooting and Red Hat recommend you to make a reboot during planned downtime.

I do not believe it's available for other distros though, since Red Hat builds their own kernels.

Code:
yum install kpatch
yum kpatch auto

Any distro based on Red Hat should have such feature. If you're Arch user, you could install and configure kpatch manually.
member
Activity: 70
Merit: 53
February 23, 2022, 06:25:19 AM
#5
For centos there is kernelcare package free & paid both available suitable for apply patch without reboot.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
February 23, 2022, 05:56:23 AM
#4
Red Hat distros have a particular Live Patch program that installs the kernel updates on the running kernel so that a reboot is not required. I do not believe it's available for other distros though, since Red Hat builds their own kernels.

Code:
yum install kpatch
yum kpatch auto
legendary
Activity: 4551
Merit: 3445
Vile Vixen and Miss Bitcointalk 2021-2023
February 23, 2022, 12:17:54 AM
#3
Which means to stop the Bitcoin Core.
Then restart it, either manually or with a script to start it automatically on boot. Bitcoin Core will automatically resynchronise with the network and download all blocks and transactions it missed during the time it was offline; you do not need to be online continuously in order to receive transactions.

If for any reason you do need to be online continuously, the solution is the same as any other network service: run multiple servers with automatic failover (and don't upgrade them all at once).
legendary
Activity: 3500
Merit: 6320
Crypto Swap Exchange
February 22, 2022, 09:48:09 PM
#2
You reboot, not a big deal. Unless you are running a mining pool a few minutes of downtime is not a big deal.
Even if you are running a lightning node a few minutes of being down here or there is not a big deal.

Any reason you are worried about it?

-Dave
legendary
Activity: 3388
Merit: 3154
February 22, 2022, 09:13:56 PM
#1
This is a question I had since a long time ago, if I'm running a Linux distro with my Bitcoin Node each time I update de system and it has a Kernel update (which is really common) I have to reboot the system. Which means to stop the Bitcoin Core.

I would like to know how the community deal with kernel updates.
Jump to: