time to re-sync a full node after power-outage

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: LoyceV on May 19, 2024, 01:53:59 AM

Quote from: nc50lc on May 09, 2024, 12:26:08 AM

Otherwise, most of the time, you'll only need --reindex-chainstate or --reindex depending on the error in your "debug.log" file.

I'd expect the reindex chainstate to take about as long as restarting the IBD, but without the bandwidth consumption.

You also forget without verifying the blocks and creating block/index undo files, which should be noticeable faster than IBD on certain hardware.

DaveF

legendary

Activity: 3500

Merit: 6320

Crypto Swap Exchange

Quote from: snerd on May 18, 2024, 04:11:55 PM

I agree with a UPS solution. Mine is set to automatically shut down the computer after 5 minutes on battery power. Plus, I backup the blockchain once a day. If it gets corrupted, I'll have just one day to resync.

Quote from: MysteryMiner on May 18, 2024, 07:39:53 AM

I think UPS or replacement battery is cheaper than 2TB SSD and also a better solution. Losing power can lead to data loss and silent corruption all over system, not only Bitcoin.

A good UPS not a cheap I got it at WalMart UPS.

If you want a cheap one and are in NY I have a pile of them that have to go to the recycling center to be disposed of properly. They just don't last, and don't work well and have really poor transfer times (from when a drop is detected to when they take over we are talking fractions of a second) and hosts of other issues. I could save time and just take them from the box and drop them into the recycling pile bypassing the time and effort to install them.

There is a reason why this one is $ 345 for a 750va
https://www.amazon.com/CyberPower-PR750LCD3C-Sinewave-Outlets-Mini-Tower/dp/B0BY3NDG3Y

And this one is $ 100 for a 750va
https://www.amazon.com/CyberPower-EC750G-Ecologic-Outlets-Compact/dp/B00DBAA33K

-Dave

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: nc50lc on May 09, 2024, 12:26:08 AM

Otherwise, most of the time, you'll only need --reindex-chainstate or --reindex depending on the error in your "debug.log" file.

I'd expect the reindex chainstate to take about as long as restarting the IBD, but without the bandwidth consumption.

The one time I messed up and deleted the latest block file from disk, Bitcoin Core started a full fresh download. That was a few years ago.

Quote from: satscraper on May 09, 2024, 01:37:06 AM

I have backup 2TB SSD disk (which I synch on regular monthly base) for such cases. Thus should the outrage happens I will copy the content of this disk to SSD which is attached to my bitcoin node.

If I'd do this, I would use rsync:

Code:

--link-dest=DIR hardlink to files in DIR when unchanged

This allows to create several snapshots of older full blockchains, without needing much additional disk space. If it's ever needed, you can easily roll back to another month.

If you're worried about power outages: a laptop is much easier than a UPS.

snerd

member

Activity: 77

Merit: 10

I agree with a UPS solution. Mine is set to automatically shut down the computer after 5 minutes on battery power. Plus, I backup the blockchain once a day. If it gets corrupted, I'll have just one day to resync.

MysteryMiner

legendary

Activity: 1512

Merit: 1049

Death to enemies!

Quote from: satscraper on May 09, 2024, 01:37:06 AM

Quote from: joe1234 on May 08, 2024, 08:59:41 AM

Hello,

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

Thank you!

Most commonly a power outage results in irreversible corruption of data bases relevant to blockchain copy on you disk thus IBD will be inevitable fact of life if you don't have the backup.

I have backup 2TB SSD disk (which I synch on regular monthly base) for such cases. Thus should the outrage happens I will copy the content of this disk to SSD which is attached to my bitcoin node.

I think UPS or replacement battery is cheaper than 2TB SSD and also a better solution. Losing power can lead to data loss and silent corruption all over system, not only Bitcoin.

satscraper

hero member

Activity: 714

Merit: 1298

Quote from: joe1234 on May 08, 2024, 08:59:41 AM

Hello,

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

Thank you!

Most commonly a power outage results in irreversible corruption of data bases relevant to blockchain copy on you disk thus IBD will be inevitable fact of life if you don't have the backup.

I have backup 2TB SSD disk (which I synch on regular monthly base) for such cases. Thus should the outrage happens I will copy the content of this disk to SSD which is attached to my bitcoin node.

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

Quote from: joe1234 on May 08, 2024, 08:59:41 AM

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

This only always applies in case of corruption if your blockchain is "pruned". (check your settings and bitcoin.conf file)
That's because whether the corruption is only in your chainstate or block index, the pruned blocks needs to be re-downloaded and verified to rebuild those data.

Otherwise, most of the time, you'll only need --reindex-chainstate or --reindex depending on the error in your "debug.log" file.
Those wont download the blockchain but build the target database from your existing blockchain.
Don't blindly restart IBD or --reindex without consulting your logs first since it could save you a lot of time and/or bandwidth.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: Faisal2202 on May 08, 2024, 12:04:20 PM

Good question, and the answer is simple, No, you don't have to download the entire blockchain again as you used the word again which means you have already downloaded let's say at some X block you have downloaded and synchronized all. And after that your machines shut down by power outage or any other reason and don't opens for like 1 or 2 days even for 1 hours let's say.

The blocks made after those X blocks will be the ones, downloaded and synchrinze in order to start a full node again. Just like when a new person plans to start a node he have to download all of the previous blockchain data.

That is provided that there is no database corruption, which can be quite common for Bitcoin Core which are in the midst of writing to disk.

Faisal2202

hero member

Activity: 1232

Merit: 475

Payment Gateway Allows Recurring Payments

Good question, and the answer is simple, No, you don't have to download the entire blockchain again as you used the word again which means you have already downloaded let's say at some X block you have downloaded and synchronized all. And after that your machines shut down by power outage or any other reason and don't opens for like 1 or 2 days even for 1 hours let's say.

The blocks made after those X blocks will be the ones, downloaded and synchrinze in order to start a full node again. Just like when a new person plans to start a node he have to download all of the previous blockchain data.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Bitcoin Core is quite sensitive to database corruption if you're met with an unclean shutdown especially during IBD. However, there are instances where the corruption lies in the chainstate rather than the blocks. In this case, it would be faster than reindexing the blocks from the start.

A lot of the reports were from the earlier versions of Bitcoin Core where the architecture of the leveldb was different and it relied on memory-mapped files for UTXOs. You probably would get a corruption warning if you're actively writing to the database during the point of shutdown. If you only need to reindex the chainstate, you can run --reindex-chainstate which is faster than reindexing all the blocks (and chainstate) with -reindex.

Findingnemo

hero member

Activity: 2366

Merit: 793

Bitcoin = Financial freedom

Quote from: joe1234 on May 08, 2024, 08:59:41 AM

Hello,

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

Thank you!

It can go either way, there are people who said they just started from where it left even after the hard shutdown and in many cases they might needed reindexing which means you have to download the entire blockchain again to get things going.

It's always better to have a backup power or atleast the mini PC inverter which can give you few minutes time to save and shutdown properly, it's necessary to have one not just for nodes but anyone who use PC's to avoid data corruption.

Mia Chloe

sr. member

Activity: 448

Merit: 560

Crypto Casino and Sportsbook

Quote from: joe1234 on May 08, 2024, 08:59:41 AM

Hello,

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

Thank you!

Synchronization of a node with the block chain mainly entails downloading the block chain data and storing the necessary files to your computer's drive. And just like DaveF mentioned, there should be no problem as long as none of the data got corrupt during the process of sudden shut down. To avoid this you can make use of a backup power plan like a UPS during the sync period. I also believe that your device processing power also has a nice role to play here.

For a majority of devices you should be able to continue from your point during the sync. I suggest that if the syncronisation hasn't gone far before the outage you could just restart the entire process to avoid a situation where you get an error after the sync is complete.

DaveF

legendary

Activity: 3500

Merit: 6320

Crypto Swap Exchange

Quote from: joe1234 on May 08, 2024, 08:59:41 AM

Hello,

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

Thank you!

Loaded question. IF there was no corruption it *should* pick up where it left off more or less.
BUT, if things were being written to the drive and corruption happened then you may need to start again.

Sadly, the best answer is 'it depends' you can search the forum and see a lot of people who have had problems due to power blips.
BUT, what you don't see are the countless people who had power issues and just kept on going with no problems when the power came back.
We only see the problems.

Kind of like my job, I only get calls when there is a problem. I don't get calls with people just wanting to say "Hi, the network is fine just wanted to let you know"

-Dave

joe1234

member

Activity: 82

Merit: 33

Hello,

for the case there should be a power outage, will the entire blockchain be downloaded again or only the changes (since the outage)?

Thank you!

Topic: time to re-sync a full node after power-outage (Read 198 times)