If you do it on a full desktop system, mind the file permissions. 99% of the time there is no issue that 1% will drive you nuts.
And I know it's been said but I'll say it again, get a SSD
And, with all the disk I/O and everything else make sure the RPi is not throttling for either temperature because the CPU being hammered or power because of the constant pull from the drive.
-Dave
Topic: Extremely slow reindexing (Read 181 times)
Bitcoin Client Software and Version Number: Bitcoin Core 26.0
Operating System: Raspbian Lite OS (64-bit)
System Hardware Specs: 4 GB memory, 2 TB external HDD.
Description of Problem:
TL;DR: I switched disks, and somehow either the blocks or the chainstate got corrupted, and I had to reindex. I'm one week now, at progress=0.313028. It seems like it goes much slower now. Yesterday it was in 0.30. At this rate, it will take more than a month to reach the tip. I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish... this month. It should be possible to do, right? The disk is loaded with Raspbian, so I could theoretically load it during boot in any other machine.
(Asking just in case I don't mess things up)
Operating System: Raspbian Lite OS (64-bit)
System Hardware Specs: 4 GB memory, 2 TB external HDD.
Description of Problem:
TL;DR: I switched disks, and somehow either the blocks or the chainstate got corrupted, and I had to reindex. I'm one week now, at progress=0.313028. It seems like it goes much slower now. Yesterday it was in 0.30. At this rate, it will take more than a month to reach the tip. I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish... this month. It should be possible to do, right? The disk is loaded with Raspbian, so I could theoretically load it during boot in any other machine.
(Asking just in case I don't mess things up)
I had the same experience some time ago. After deterioration of chainstate data on my external SSD I have fallen for reindex. This process was as slow as it was less than effective. Thus I had to start the sync from scratch.
Hence, to not waste time, I would advocate IBD rather than reindex in the case of any chainstate corruption.
I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish...
That should work, although I'd prefer to use it as a secondary disk in another system, and not boot from it.Your problem is the 4 GB RAM in combination with 11 GB chainstate. Add HDD, and it's a very bad combination.
As far as I know, reindexing chainstate requires almost as much writing to disk as a full fresh sync. It only saves writing 580 GB blocks. I wrote 4.58 GB to disk with 8 GB RAM during IBD. Remove the blocks, and you're left with 4 TB. And that was with twice the ram, so you'll need to write more than 4 TB. That's 4+ TB reading and writing more or less random access on a HDD. If it takes a month, that's 1.5+ MB/s on average, which could very well be the best the HDD can do with random reads and writes. This is why I prefer to put chainstate on SSD.
Progress will get worse once you're further ahead, see my graph.
Yes you can. Format of data directories between versions are the exact same and there is no compatibility issues whatsoever. Of course, remember to shut it down gracefully.
In fact, this was how I encouraged people to do IBD on a proper Desktop instead of their RPi. Resync on RPi was really long for me, and using the Desktop was much less frustrating in comparison.
In fact, this was how I encouraged people to do IBD on a proper Desktop instead of their RPi. Resync on RPi was really long for me, and using the Desktop was much less frustrating in comparison.
What I have tried in the past, and it worked, was to download the blockchain on my SSD using Windows 11 OS.
I've considered doing that, but I don't have a terabyte of SSD, and I don't want to buy one just for that purpose. HDD satisfies my needs. Sure, Bitcoin Core would be less troublesome, but if it works, even slowly, then I don't care. Then I downloaded Core again and set it to look in the SSD again. It worked flawlessly. I guess it's a relevant situation, isn't it ?
Hmm. Maybe if I used Bitcoin Core from my main computer without booting into Raspbian, and pointed to the Bitcoin data directory in the plugged HDD, then maybe it would reindex more rapidly. There are quite a lot of permission issues this way, though. (The datadir is read only, and I'll have to run Bitcoin Core as root) 
I don't see a problem with doing what you suggest. I have never tried it though. To be honest, I don't understand how the directories were corrupted, but reindexing can be painful, indeed.
What I have tried in the past, and it worked, was to download the blockchain on my SSD using Windows 11 OS. Then I deleted Bitcoin Core, because I performed a format. Then I downloaded Core again and set it to look in the SSD again. It worked flawlessly. I guess it's a relevant situation, isn't it ?
What I have tried in the past, and it worked, was to download the blockchain on my SSD using Windows 11 OS. Then I deleted Bitcoin Core, because I performed a format. Then I downloaded Core again and set it to look in the SSD again. It worked flawlessly. I guess it's a relevant situation, isn't it ?
Bitcoin Client Software and Version Number: Bitcoin Core 26.0
Operating System: Raspbian Lite OS (64-bit)
System Hardware Specs: 4 GB memory, 2 TB external HDD.
Description of Problem:
TL;DR: I switched disks, and somehow either the blocks or the chainstate got corrupted, and I had to reindex. I'm one week now, at progress=0.313028. It seems like it goes much slower now. Yesterday it was in 0.30. At this rate, it will take more than a month to reach the tip. I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish... this month. It should be possible to do, right? The disk is loaded with Raspbian, so I could theoretically load it during boot in any other machine.
(Asking just in case I don't mess things up)
Operating System: Raspbian Lite OS (64-bit)
System Hardware Specs: 4 GB memory, 2 TB external HDD.
Description of Problem:
TL;DR: I switched disks, and somehow either the blocks or the chainstate got corrupted, and I had to reindex. I'm one week now, at progress=0.313028. It seems like it goes much slower now. Yesterday it was in 0.30. At this rate, it will take more than a month to reach the tip. I was thinking if it's possible to eject the HDD, plug it a computer with better specs, so it could finish... this month. It should be possible to do, right? The disk is loaded with Raspbian, so I could theoretically load it during boot in any other machine.
(Asking just in case I don't mess things up)
Jump to: