Database corrupt question | Bitcointalksearch.org

gabrio81

member

Activity: 94

Merit: 56

Quote from: bob123 on October 21, 2020, 06:50:03 AM

Quote from: gabrio81 on October 21, 2020, 06:36:33 AM

[...] yeah but all things considered it's not a bad hdd since it's usb3 and peaks 100-120 mb/sec or 800-950 mbit.

If you are fine with the speed, no reason to change something.

But an external drive will always be slower than an internal (HDD) one. The protocol overhead alone already makes a huge difference.
In addition to that, you'll only achieve best results if you only have your drive attached to the USB root as other devices will definitely influence the speed.

After all, the main issue with external drives is the stability. Core requires a constant connection which can not be guaranteed using external drives.

But if you only very rarely encounter an error and are fine with the (relatively acceptable) consequences, there is no reason to change your setup.

heheh yep, i was contemplating adding another HDD inside this laptop since there's a bay available, but it will add to the weight a bit ;-) but yeah for now is ok, in the end, as long as there's a backup there's hope hehehehe

bob123

legendary

Activity: 1624

Merit: 2481

Quote from: gabrio81 on October 21, 2020, 06:36:33 AM

[...] yeah but all things considered it's not a bad hdd since it's usb3 and peaks 100-120 mb/sec or 800-950 mbit.

If you are fine with the speed, no reason to change something.

But an external drive will always be slower than an internal (HDD) one. The protocol overhead alone already makes a huge difference.
In addition to that, you'll only achieve best results if you only have your drive attached to the USB root as other devices will definitely influence the speed.

After all, the main issue with external drives is the stability. Core requires a constant connection which can not be guaranteed using external drives.

But if you only very rarely encounter an error and are fine with the (relatively acceptable) consequences, there is no reason to change your setup.

gabrio81

member

Activity: 94

Merit: 56

Quote from: nc50lc on October 20, 2020, 10:21:59 PM

Quote from: gabrio81 on October 20, 2020, 10:19:43 AM

Quote from: nc50lc on October 19, 2020, 09:20:46 PM

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

Set a value depending on how much is left when your system's RAM usage is at peak, just make sure to give room for some spare space, half of 16 is a good amount.
As I've said, set that after finishing the -reindex process since it should be done by now.

But do not expect a lot of boost since you're using an external drive, it's the bottleneck.

ok thanks i will try yep it's finished. yeah but all things considered it's not a bad hdd since it's usb3 and peaks 100-120 mb/sec or 800-950 mbit.

actually on a normal day - when it doesn't crash, for 7 days to catch up new blocks, it takes around 30-40 mins max only. From 1 or 2 latest releases, core is much much faster to catch up. Downloads really well blocks first maxxing out my connection usually and then does the grunt work.

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

Quote from: gabrio81 on October 20, 2020, 10:19:43 AM

Quote from: nc50lc on October 19, 2020, 09:20:46 PM

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

Set a value depending on how much is left when your system's RAM usage is at peak, just make sure to give room for some spare space, half of 16 is a good amount.
As I've said, set that after finishing the -reindex process since it should be done by now.

But do not expect a lot of boost since you're using an external drive, it's the bottleneck.

HCP

legendary

Activity: 2086

Merit: 4361

Quote from: gabrio81 on October 20, 2020, 10:19:43 AM

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

Yes, try setting that value to 8192. It'll allow Bitcoin Core to cache more data in RAM and help alleviate some of the read/write load on the HDD and should theoretically help increase the sync speed.

gabrio81

member

Activity: 94

Merit: 56

Quote from: nc50lc on October 19, 2020, 09:20:46 PM

Quote from: gabrio81 on October 19, 2020, 12:25:09 PM

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

BTW dbcache size, under Options > Main is 450 MiB right now. i have 16gb of ram on this machine. is that the one you mentioned to change?

gabrio81

member

Activity: 94

Merit: 56

Quote from: nc50lc on October 19, 2020, 09:20:46 PM

Quote from: gabrio81 on October 19, 2020, 12:25:09 PM

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

Interesting, finished sync and catchup new blocks, so it's finished. 173 blocks to go. i will check that. thanks

Jet Cash

legendary

Activity: 2814

Merit: 2472

https://JetCash.com

I would have copied the bad drive onto another disk, and used that to resync. Drives are so cheap now that it isn't worth the risk of hitting the same problem again. If you have a bad spot on the disk, then you need to flag it so that it win't be used again. I'm running a couple of nodes using external USB drives, and Core seems pretty good at coping with problems, as long as they aren't hardware errors.

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

Quote from: gabrio81 on October 19, 2020, 12:25:09 PM

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

How low is your dbcache size?
If it's on default, increasing it to at least half your RAM will help a lot.

But do that after your current task since that "61%" would be a waste if you stop it now.

gabrio81

member

Activity: 94

Merit: 56

just as a quick update, 24 hours after, just 61% reprocessed lol, maybe it will really take a week.

yes of course internal would be best but i don't have 300GB on my SSD unfortunately.

bob123

legendary

Activity: 1624

Merit: 2481

This happens relatively often with external drives.
Core needs a steady connection. Therefore an internal drive would be definitely better in terms of stability and speed.

If you want to circumvent this in the future, your best bet would be to use an internal drive.

gabrio81

member

Activity: 94

Merit: 56

Quote from: BitMaxz on October 18, 2020, 12:29:54 PM

Quote from: gabrio81 on October 18, 2020, 11:59:21 AM

got it, if i do "reindex" option on command prompt, is he gonna RE DOWNLOAD (which takes 1 week) or just re-index in fact, which i think is like 12 hours? sorry but want to be sure.

thanks!

No, according to this it will only rebuilds the two leveldb databases for the block index and the chainstate, using the block files saved on disk.

Which means it doesn't need to redownload the whole blockchain and it will only rebuild the blocks that you already downloaded.

Quote from: gabrio81 on October 18, 2020, 12:00:02 PM

if i just delete that bad block he pointed out, that won't work either?

No you still need to use reindex command if you want to fix this issue.

thanks, it's re-indexing now, see you in 1 week 😁

BitMaxz

legendary

Activity: 3374

Merit: 3095

Playbet.io - Crypto Casino and Sportsbook

Quote from: gabrio81 on October 18, 2020, 11:59:21 AM

got it, if i do "reindex" option on command prompt, is he gonna RE DOWNLOAD (which takes 1 week) or just re-index in fact, which i think is like 12 hours? sorry but want to be sure.

thanks!

No, according to this it will only rebuilds the two leveldb databases for the block index and the chainstate, using the block files saved on disk.

Which means it doesn't need to redownload the whole blockchain and it will only rebuild the blocks that you already downloaded.

Quote from: gabrio81 on October 18, 2020, 12:00:02 PM

if i just delete that bad block he pointed out, that won't work either?

No you still need to use reindex command if you want to fix this issue.

gabrio81

member

Activity: 94

Merit: 56

if i just delete that bad block he pointed out, that won't work either?

gabrio81

member

Activity: 94

Merit: 56

got it, if i do "reindex" option on command prompt, is he gonna RE DOWNLOAD (which takes 1 week) or just re-index in fact, which i think is like 12 hours? sorry but want to be sure.

thanks!

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

If you've got enough space on the drive. Try placing core and the backup (potentially in a new partition of the drive if you're used to tja) and see if it will be able to use that.. Make sure you run it with the -datadir extension from a command line linking to where you've placed the backup (don't work directly from the backup).

If not you'll have to reindex the whole chain at least once as it'll work through the full 300 GB.

gabrio81

member

Activity: 94

Merit: 56

i checked log :

2020-10-18T16:51:39Z Fatal LevelDB error: Corruption: block checksum mismatch: X:\XXXX\chainstate/513482.ldb
2020-10-18T16:51:39Z You can use -debug=leveldb to get more complete diagnostic messages
2020-10-18T16:51:39Z Fatal LevelDB error: Corruption: block checksum mismatch: X:\XXXX\chainstate/513482.ldb
2020-10-18T16:51:39Z : Error opening block database.
Please restart with -reindex or -reindex-chainstate to recover.
2020-10-18T16:54:58Z Aborted block database rebuild. Exiting.

gabrio81

member

Activity: 94

Merit: 56

hi guys,

this morning i was doing the "usual" sync, then the hard disk started clicking, most likely dust or bad cables (it's external), anyway the hdd is fine, did a good check on it and now i wanted to resync the blockchain.

Classic, I get "Error opening block database" Do you want to rebuild the database now?

Earlier on i had to kill the hard disk and unplug it as it was stuck so most likely the last block didn't write up correctly a closing.

If i click OK is it gonna resync 300+ gig worth of stuff or is he so smart that he can simply roll back the bad blocks say OK let's resume from here and be relatively short?

Or otherwise is there another command prompt thingie or anything to launch?

i have a backup so not end of the world if i have to resync etc but not ideal

thanks so much,
Gab

Topic: Database corrupt question (Read 240 times)