Topic: Bitcoin core reindexing (Read 258 times)

It looks as if the SSD is overloaded. The Netbook went to standby, as I wasn't using it for anything other than sync'ing Bitcoin. When I restarted it, the sync report indicated that it had dropped back over a year, so obviously the disk is unhappy. It's a decent Samsung one, and it hasn't been used all that much, but it is reporting only 47Gb of free space, and it is going to use a lot more than that to get much further. I think I'll buy a new drive for the blockchain.

I like Samsung, and there is quite a lot of choice in 2Tb SSDs. I need to weigh performance, cost and reliability, and my initial decision is to get an 860 QVO internal drive, and insert it into a case for USB connection. As the drive will just be used for the Bitcoin node, price was a consideration. The QVO seems to be slightly inferior to the EVO, but this relates to disk activity. As the blockchain is incremental rather than constantly being moved and replaced, I thought that the cost saving was worth the marginal loss in performance and endurance. Models other than the QVO and EVO are more expensive, and I don't think the improvements are worth the extra costs. I decided to case the drive myself for two reasons. I may want to move it into a notebook at some stage in the future, and there is a cost saving. Transfer speeds may be faster with a factory built external drive, but I don't believe that this will be significant on low to mid spec note/net books. At the moment the netbook is running under Ubuntu, but I may replace this if the machine is going to be used mainly for crypto projects. I'll need to research that.

Any comments or advice will be received gratefully.

That will teach me to avoid doing my homework. The last time I used this SSD the blockchain was about 128Gb. Given that I believe you should leave at least a third of an SSD free for processing files. I probably need at least 1Tb. I've got a 2Tb HDD, but it is slower, and it needs more power, and that defeats the purpose of using the Netbook. I've also got a 1Tb SSD that I was going to use to replace the HDD in the Notebook, so that I could get rid of Windows at last. Probably the sensible thing to do is to copy the 256Gb data onto the 1Tb SSD, and carry on with that. I can then buy a 2Tb SSD for the Notebook.

Note on the current synchronisation status. I went through the correct sequence when shutting down yesterday - close core and wait for completion, eject the SSD, and then shut down Ubuntu. I've just restarted the reindexing, and it started again at 6 years and 42 weeks, that's about a year earlier than the previous day's closing point,  but it has skipped quite quickly to 5 years and 23 weeks. Thinking about the post by HCP, it looks as if the disk is short of space, and I need to switch to the 1Tb drive before going much further. I'll leave it to run for another hour or so, as I've got some other stuff to sort out. I'll probably swop drives tomorrow, and let you guys know how it goes. I've got Mint installed on the !Tb SSD, but I might reformat that before copying the Bitcoin files.

Are you pruning? If not, you're on a bit of a fool's errand trying to sync everything to a 250Gb drive!!?!

Given that the current size of Blockchain on it's own is currently larger than that (nevermind the chainstate and other assorted data files), you're going to run out of room before it finishes syncing.

If the disk cam with a drive utility to check it, I'd suggest running that - it'll mark the bad cells.

Ssds with a io driver (most have them nowadays) should have a nice amount of ram to check the block they write to the drive. It's especially important to eject an ssd safely rather than pulling it out. And if it says drive in use, I'd shutdown the computer to avoid further issue. As the others have suggested an initial sync takes a long time when you get to the later years... And for a few months in 2017 it looks as if it gets stuck.

If it has to reindex again after this one then the corrupted file may need renaming (the last blk file it uses) afaik bitcoin core checks the last 7 on startup so you can already work out where the error may arise but it may not cause a problem anyway as core can usually overwrite it.

The first couple of years are always quick, trust me that the last 2 or 3 years will seem to take forever

What version of Bitcoin Core are you using?

It running faster than I expected. It has already caught up about a year of data, so I'm now in the high 6 years recovery band. I think that is good for McDonalds WiFi using a Celeron netbook.

It's a shame I need to leave and sort out the van. The resync' is rattling along, and I'm under 6 years backlog now.

The bandwidth you required depends on your blockchain data. If the corrupted files aren't that big then it won't take long. Your computer specs will also affects it. CMIIW. Further reading:

https://bitcointalksearch.org/topic/how-do-i-reindex-582554
https://coinguides.org/rescan-reindex-wallet/
https://bitcoin.stackexchange.com/questions/60709/when-should-i-use-reindex-chainstate-and-when-reindex

I've got 3 computers with Bitcoin core installed, and I decided to do this as a form of backup, as well as an experiment as part of my digital nomad life. I have Ubuntu installed on a Netbook with a smallish hard drive, so my wallet and the blockchain are kept on an external 250Gb Samsung SSD. It is a while since I synchronised this, and I decided to bring it up to date. It appeared to start correctly, but then told me it was reindexing the blockchain from about 7 years ago. This probably means there is a corrupted record on the disk. I could have copied the blockchain, and associated files such as chainstate from a backup drive, but I decided to run the reindexing to see how long it took. The wallet data seems to be correct, and I am able to see some early transactions. I stated this thread to remind new Bitcoiners to backup their files regularly if they are running their own node.

Whilst reindexing does not download the complete blockchain, it does try to reuse the data you have stored, and it seems to discard the  blockchain and chainstate indices, so it will still consume a large quantity of ISP bandwidth. I'm running this over public WiFi, so I expect it will take a couple of weeks to rebuild the files.

Please feel free to comment if I have misunderstood this process, or if you have any suggestions or comments about recovery from minor disk errors. I would also remind users of external SSD drives that they should "eject" the drive before disconnecting it - this ensures that write buffers are cleared to maintain the integrity of the data structure.