bitcoin daemon database error in a block

Amirofski

newbie

Activity: 9

Merit: 0

Quote from: HCP on August 12, 2020, 09:04:55 AM

Quote from: Amirofski on August 12, 2020, 08:31:56 AM

it's normal?

I don't see any obvious error or problem with that output...

It is simply showing that it is updating with each new block that is received and parsed. Note how the "height" number is incrementing. Looks like it's a little over 26.5% synced... the "progress" value is the percentage (divided by 100)... so 0.265078 * 100 = 26.5078%.

thanks for your useful reply Kiss

HCP

legendary

Activity: 2086

Merit: 4361

Quote from: Amirofski on August 12, 2020, 08:31:56 AM

it's normal?

I don't see any obvious error or problem with that output...

It is simply showing that it is updating with each new block that is received and parsed. Note how the "height" number is incrementing. Looks like it's a little over 26.5% synced... the "progress" value is the percentage (divided by 100)... so 0.265078 * 100 = 26.5078%.

Amirofski

newbie

Activity: 9

Merit: 0

i srart new node and processing somthing like this

Code:

20-08-12T13:29:56Z UpdateTip: new best=000000000000000002c8bb221485c0f******5c8bb904fde06fbeebd0153 height=423773 version=0x20000000 log2_work=85.087637 tx=147146353 date='2016-08-05T09:49:25Z' progress=0.265073 cache=110.2MiB(430827txo)
2020-08-12T13:30:01Z UpdateTip: new best=000000000000000002cb3e3d652bc2da****c9abfb81788b8189db101b height=423774 version=0x20000000 log2_work=85.087668 tx=147146929 date='2016-08-05T09:52:35Z' progress=0.265074 cache=111.0MiB(436662txo)
2020-08-12T13:30:08Z UpdateTip: new best=0000000000000000008c9e1b5134e4365f7fb1347beec90aafd*****05eaa2a28 height=423775 version=0x20000000 log2_work=85.087698 tx=147147295 date='2016-08-05T09:54:14Z' progress=0.265074 cache=111.7MiB(443114txo)
2020-08-12T13:30:08Z UpdateTip: new best=0000000000000000053b04f519d1983ab88932600e682de78********03b3976 height=423776 version=0x20000000 log2_work=85.087729 tx=147147501 date='2016-08-05T09:55:37Z' progress=0.265075 cache=111.8MiB(443354txo)
2020-08-12T13:30:09Z Pre-allocating up to position 0xb00000 in rev00589.dat
2020-08-12T13:30:09Z UpdateTip: new best=0000000000000000051c2f0cbbec5db8858426f65fa***********7974f3 height=423777 version=0x20000000 log2_work=85.087759 tx=147149633 date='2016-08-05T10:10:36Z' progress=0.265078 cache=112.1MiB(446128txo)

it's normal?

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Quote from: LoyceV on June 10, 2020, 05:00:25 AM

Quote from: darkv0rt3x on June 10, 2020, 04:45:52 AM

Is this already know by devs? Anyone opened an issue on Github? (I'm sure you did, obviously)

If achow101 knows, I'm sure it's known at the right places. I didn't open an issue myself.
I want to test what happens if I re-download the entire blockchain with the same watch-only wallet instead of just doing a rescan, but as always I don't have enough time for everything I want to do.

I know the feeling, for sure.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: darkv0rt3x on June 10, 2020, 04:45:52 AM

Is this already know by devs? Anyone opened an issue on Github? (I'm sure you did, obviously)

If achow101 knows, I'm sure it's known at the right places. I didn't open an issue myself.
I want to test what happens if I re-download the entire blockchain with the same watch-only wallet instead of just doing a rescan, but as always I don't have enough time for everything I want to do.

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

That seems to be a bit of an issue indeed.
Is this already know by devs? Anyone opened an issue on Github? (I'm sure you did, obviously)

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: darkv0rt3x on June 09, 2020, 01:57:38 PM

Really?

Really!

Quote

I don't want to insult you because I don't know your level of knowledge, but you did the rescan, right?

Yes. Don't worry about insulting me, if only the problem would be that simple Tongue

It's worse:
From Bitcointalk Party - Discord server (05/25/2020):

Quote from: LoyceV

Quick question... I have Bitcoin Core, import 145 addresses read-only, type "rescanblockchain" in Console, wait 2 hours, and none of the old transactions show up. Only a transaction received today shows up. Am I missing something?

Quote from: Csmiami

how did you import them in first place?

Quote from: achow101

there are soem ... issues with imports and rescans
issues that are difficult to reproduce and nail down

Quote from: LoyceV

I imported them through Console: copy/paste all at once.
The "issues" I guess I ran into :stuck_out_tongue: I've never had this problem before, but I'm not using the latest version of Bitcoin Core, so the "issues" might be fixed already in later versions.

Quote from: achow101

might not be fixed. I've ran into import and rescan issues before in various versions of the master branch and haven't been able to figure out what was wrong

Quote from: Csmiami

it is not; I tried imported one when Loyce posted (to try some new stuff out) and after the whole rescan (debug log shows it's completed), the balance shows 0

I trust achow101's knowledge on this. So Bitcoin Core has "issues with imports and rescans" Sad

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Quote from: LoyceV on June 09, 2020, 11:45:27 AM

Quote from: darkv0rt3x on June 08, 2020, 03:05:44 PM

I imported an address that had 2 inputs and 2 outputs and I didn't thought about that. So, there were no unspent utxo's to show. Address was empty and I was expecting to see the transactions there. The 2 inputs and the 2 outputs. My bad... What can I say. Sad

I recently found out that importing addresses into Bitcoin Core has some bugs: the balance doesn't always show up. I tested this with 145 old addresses, and only one very recent transaction showed up, while it didn't show the old balances.

Really? I don't want to insult you because I don't know your level of knowledge, but you did the rescan, right?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: darkv0rt3x on June 08, 2020, 03:05:44 PM

I imported an address that had 2 inputs and 2 outputs and I didn't thought about that. So, there were no unspent utxo's to show. Address was empty and I was expecting to see the transactions there. The 2 inputs and the 2 outputs. My bad... What can I say. Sad

I recently found out that importing addresses into Bitcoin Core has some bugs: the balance doesn't always show up. I tested this with 145 old addresses, and only one very recent transaction showed up, while it didn't show the old balances.

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Quote from: HCP on June 07, 2020, 04:03:06 PM

Quote from: darkv0rt3x on June 07, 2020, 12:47:34 PM

Nevermind.

I think it's fixed.

Did the balance just show up? or did you do something else after the -rescan? Huh

It would be helpful to know for anyone else that stumbles across this thread when trying to diagnose a similar problem to yours Wink

Sure I'm sorry.

I was the problem (as usual).

I imported an address that had 2 inputs and 2 outputs and I didn't thought about that. So, there were no unspent utxo's to show. Address was empty and I was expecting to see the transactions there. The 2 inputs and the 2 outputs. My bad... What can I say. Sad

HCP

legendary

Activity: 2086

Merit: 4361

Quote from: darkv0rt3x on June 07, 2020, 12:47:34 PM

Nevermind.

I think it's fixed.

Did the balance just show up? or did you do something else after the -rescan? Huh

It would be helpful to know for anyone else that stumbles across this thread when trying to diagnose a similar problem to yours Wink

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Nevermind.

I think it's fixed.

I'll use this thread if I need to keep asking for help.

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Quote from: HCP on June 07, 2020, 06:19:50 AM

-rescan usually takes a few minutes to complete... possibly as long as 30mins (depending on your computer etc) but it shouldn't take days or hours etc.

Theoretically, -rescanblockchain should do the job, but it's not working for you... whether that is caused by issues you've had with your block data and reindexing, I don't know. I suggested -rescan as it will simply start at the beginning and run through everything making sure that everything is checked.

Combined with the fact that the "big" rescanblockchain timed out on you, I'm not 100% convinced that your node is running without issues Undecided

Rescan is now complete but still no output from listunspent command. Rescan took probably 2h. I have an i7-3740QM with 16Gb DDR3 of RAM.

I've just been told that:

Quote

a timeout means it's still going. rescans take potentially hours.
it holds the main thread so your peers will disconnect, the RPC call will fail.

I don't know what else can I do.

When I run

Code:

bitcoin-cli getwalletinfo

I get

Code:

{
"walletname": "",
"walletversion": 169900,
"balance": 0.00000000,
"unconfirmed_balance": 0.00000000,
"immature_balance": 0.00000000,
"txcount": 2,
"keypoololdest": 1591223818,
"keypoolsize": 1000,
"hdseedid": "8e........................................",
"keypoolsize_hd_internal": 1000,
"paytxfee": 0.00000000,
"private_keys_enabled": true,
"avoid_reuse": false,
"scanning": false,
"descriptors": false
}

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: HCP on June 07, 2020, 06:19:50 AM

-rescan usually takes a few minutes to complete... possibly as long as 30mins (depending on your computer etc) but it shouldn't take days or hours etc.

I did a rescan recently, and it took my (old) laptop several hours (reading from HDD).

HCP

legendary

Activity: 2086

Merit: 4361

-rescan usually takes a few minutes to complete... possibly as long as 30mins (depending on your computer etc) but it shouldn't take days or hours etc.

Theoretically, -rescanblockchain should do the job, but it's not working for you... whether that is caused by issues you've had with your block data and reindexing, I don't know. I suggested -rescan as it will simply start at the beginning and run through everything making sure that everything is checked.

Combined with the fact that the "big" rescanblockchain timed out on you, I'm not 100% convinced that your node is running without issues Undecided

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Quote from: HCP on June 06, 2020, 11:35:59 PM

If you stop Bitcoin Core... and then restart bitcoind using:

Code:

bitcoind -rescan

It should do a full rescan of all transactions relating to addresses in your wallet... WITHOUT needing to redownload the blocks etc.

Also, what output do you get if you use:

Code:

bitcoin-cli getaddressinfo "YOUR_BTC_ADDRESS"

Would that -rescan take how much time?
And isn't the '-rescanblockchain' option the same as that but with the possibility of limiting the range of that scan?

'getaddressinfo' shows me:
(I hide some of the chars. Not sure if they contain any sensitive info)

Code:

{
"address": "bc1...............................",
"scriptPubKey": "0014................................................................",
"ismine": true,
"solvable": true,
"desc": "wpkh([0006fafb]030f587a24dc88d12909415....................................................................)#ay5ng3rp",
"iswatchonly": false,
"isscript": false,
"iswitness": true,
"witness_version": 0,
"witness_program": "0006f...............................................",
"pubkey": "030........................................................................................",
"ischange": false,
"timestamp": 1,
"labels": [
"TugaRev"
]
}

HCP

legendary

Activity: 2086

Merit: 4361

If you stop Bitcoin Core... and then restart bitcoind using:

Code:

bitcoind -rescan

It should do a full rescan of all transactions relating to addresses in your wallet... WITHOUT needing to redownload the blocks etc.

Also, what output do you get if you use:

Code:

bitcoin-cli getaddressinfo "YOUR_BTC_ADDRESS"

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Quote from: HCP on June 06, 2020, 06:29:13 PM

Those first messages are just showing that it was syncing new blocks... you can ignore the "unknown version" messages... while it can be alarming to see, they're 'normal'.

It's a bit tricky to diagnose when you're giving snippets of debug.log... so you got that "Error: Out of memory" AFTER you executed bitcoin-cli stop and the node was shutting down? Did you get the normal shutting down debug like this:

Code:

2020-06-05T06:31:18Z tor: Thread interrupt
2020-06-05T06:31:18Z opencon thread exit
2020-06-05T06:31:18Z addcon thread exit
2020-06-05T06:31:18Z Shutdown: In progress...
2020-06-05T06:31:18Z torcontrol thread exit
2020-06-05T06:31:18Z net thread exit
2020-06-05T06:31:18Z msghand thread exit
2020-06-05T06:31:44Z scheduler thread exit
2020-06-05T06:31:44Z Dumped mempool: 0.000997s to copy, 0.152187s to dump
2020-06-05T06:31:44Z FlushStateToDisk: write coins cache to disk (677775 coins, 94304kB) started

And then it died at this point with the OOM? Huh

Sorry to miss critical information. My debug.log file is set to be split every 300Mb, so it would be silly of me to paste here the entire file.
That's why I was just pasting here the error messages plus a few lines before.

The error was happening when I issued 'bitcoin-cli stop'.
I was not getting the same output as you because the error happened. Otherwise I would get similar output as yours.

So, I think it was due to the fact that I tried to use a bigger 'dbcache' value of 12000 MB. And when it tried to dump the cache to the disk, it was running out of memory. I 'only' have 16Gb of RAM. Not sure this was being an issue.

Anyway, the process has finished, and I think I already have my full node data back.

And now that I have my full data back (I hope), I tried to import an address where I have a small amount of BTC. I imported the address using the PK. Then I tried to rescan the blockchain to update the wallet with this address transactions. I used 'bitcoin-cli rescanblockchain 597000 633472' but I got this error:

Code:

$ bitcoin-cli rescanblockchain 590000 633465
error: Could not connect to the server 127.0.0.1:8332 (error code 0 - "timeout reached")
Make sure the bitcoind server is running and that you are connecting to the correct RPC port.

So I tried smaller ranges. With smaller ranges it is working and I get no errors. I tried between 597000 and 600000 and no errors. I tried other ranges and no errors either. But I still can't make that 'listunspent' command shows the utxos in this address I just imported.

What else can I do to make listunspent show my utxos?

HCP

legendary

Activity: 2086

Merit: 4361

Those first messages are just showing that it was syncing new blocks... you can ignore the "unknown version" messages... while it can be alarming to see, they're 'normal'.

It's a bit tricky to diagnose when you're giving snippets of debug.log... so you got that "Error: Out of memory" AFTER you executed bitcoin-cli stop and the node was shutting down? Did you get the normal shutting down debug like this:

Code:

2020-06-05T06:31:18Z tor: Thread interrupt
2020-06-05T06:31:18Z opencon thread exit
2020-06-05T06:31:18Z addcon thread exit
2020-06-05T06:31:18Z Shutdown: In progress...
2020-06-05T06:31:18Z torcontrol thread exit
2020-06-05T06:31:18Z net thread exit
2020-06-05T06:31:18Z msghand thread exit
2020-06-05T06:31:44Z scheduler thread exit
2020-06-05T06:31:44Z Dumped mempool: 0.000997s to copy, 0.152187s to dump
2020-06-05T06:31:44Z FlushStateToDisk: write coins cache to disk (677775 coins, 94304kB) started

And then it died at this point with the OOM? Huh

darkv0rt3x

hero member

Activity: 1274

Merit: 681

I rather die on my feet than to live on my knees

Well, this is in the end but I started getting this messages:

Code:

2020-06-06T16:58:03Z Pre-allocating up to position 0x500000 in rev02102.dat
2020-06-06T16:58:03Z UpdateTip: new best=0000000000000000000386f1b8776307e3e54a92e0a8edfd8dbd97d0d8f253d9 height=633218 version=0x20000000 log2_work=92.006175 tx=536228356 date='2020-06-05T16:02:34Z' progress=0.999447 cache=9803.2MiB(72194509txo)
2020-06-06T16:58:03Z UpdateTip: new best=00000000000000000009013052d1b341bf00db0740e14cf9e97fe5700d4c5c4a height=633219 version=0x20000000 log2_work=92.006192 tx=536230681 date='2020-06-05T16:19:44Z' progress=0.999453 cache=9803.4MiB(72196232txo)
2020-06-06T16:58:04Z UpdateTip: new best=00000000000000000000b078867de04ff83dd8a5d77c30ef92505cbb0d544d05 height=633220 version=0x3fffe000 log2_work=92.006209 tx=536233017 date='2020-06-05T16:21:52Z' progress=0.999454 cache=9803.3MiB(72195112txo)
2020-06-06T16:58:04Z UpdateTip: new best=0000000000000000000455ee424e84e2d2cdd811b35396b5ef1a2cbbf0e7662b height=633221 version=0x20400000 log2_work=92.006226 tx=536235909 date='2020-06-05T16:32:02Z' progress=0.999458 cache=9803.5MiB(72197006txo)
2020-06-06T16:58:05Z UpdateTip: new best=000000000000000000015f4b7af01c7b64b1d0ca47776bad6d4af03fbc361101 height=633222 version=0x20400000 log2_work=92.006243 tx=536238667 date='2020-06-05T16:49:24Z' progress=0.999464 cache=9803.8MiB(72199590txo)
2020-06-06T16:58:05Z UpdateTip: new best=000000000000000000024cfd85e688590ecd8318c296fcb54025700c283c5d42 height=633223 version=0x20400000 log2_work=92.00626 tx=536241630 date='2020-06-05T16:52:13Z' progress=0.999465 cache=9803.8MiB(72199791txo)
2020-06-06T16:58:06Z Leaving InitialBlockDownload (latching to false)
2020-06-06T16:58:07Z UpdateTip: new best=0000000000000000001315d3b171b3bbfb604d2ae84b92ce0271d0613d2bf3ca height=633224 version=0x20000000 log2_work=92.006277 tx=536244335 date='2020-06-05T16:59:28Z' progress=0.999468 cache=9803.9MiB(72200219txo) warning='62 of last 100 blocks have unexpected version'
2020-06-06T16:58:08Z UpdateTip: new best=000000000000000000045a44c0a3193a78726d49e7ceb15330d97452f7b38058 height=633225 version=0x37ffe000 log2_work=92.006294 tx=536246822 date='2020-06-05T17:03:42Z' progress=0.999470 cache=9803.8MiB(72199978txo) warning='63 of last 100 blocks have unexpected version'

Edited:

This is doing the same whole shit again.
My chainstate folder is only about 1000 items.
I stopped the daemon again an I got the following output:

Code:

2020-06-06T17:13:04Z FlushStateToDisk: write coins cache to disk (72150244 coins, 10273727kB) started
2020-06-06T17:15:32Z Error: Out of memory. Terminating.
Aborted (core dumped)

Topic: bitcoin daemon database error in a block (Read 987 times)