Bitcoin Core IBD slow - page 2. | Bitcointalksearch.org

apogio

sr. member

Activity: 406

Merit: 896

Quote from: ABCbits on November 30, 2023, 07:36:26 AM

It leads to same result, although i find it's unusual to see configuration on multiple places. And since your device has 8GB RAM, you could set higher value for dbcache (assuming you don't run other memory-intensive software on that device).

Like how much? Perhaps all of it? 8192? I don't run anything else on this device. My plan, in general, is to run Bitcoin & Monero node on the same machine.

EDIT:
I set it to 7000 and let's see...

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: apogio on November 30, 2023, 07:32:30 AM

Quote from: ABCbits on November 30, 2023, 07:27:18 AM

Is it your full configuration file or you removed line dbcache?

I have set my dbcache on the commandline running

Code:

bitcoind -dbcache=4096 -conf=

The reason is that I want to run with default dbcache when IBD is finished.

Doesn't it have the same result as adding dbcache=4096 on the conf file and then run:

Code:

bitcoind -conf=

It leads to same result, although i find it's unusual to see configuration on multiple places. And since your device has 8GB RAM, you could set higher value for dbcache (assuming you don't run other memory-intensive software on that device).

apogio

sr. member

Activity: 406

Merit: 896

Quote from: ABCbits on November 30, 2023, 07:27:18 AM

Is it your full configuration file or you removed line dbcache?

I have set my dbcache on the commandline running

Code:

bitcoind -dbcache=4096 -conf=

The reason is that I want to run with default dbcache when IBD is finished.

Doesn't it have the same result as adding dbcache=4096 on the conf file and then run:

Code:

bitcoind -conf=

Quote from: ABCbits on November 30, 2023, 07:27:18 AM

Pruned node need full UTXO in order to verify new TX/block, so i doubt it's possible. At best, we'll see slightly more efficient UTXO representation.

Yeap, you must be right.

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: apogio on November 30, 2023, 06:25:01 AM

My conf is:

Code:

datadir=...
server=1
daemon=1
txindex=1

Is it your full configuration file or you removed line dbcache?

Quote from: apogio on November 30, 2023, 07:22:27 AM

Quote from: LoyceV on November 30, 2023, 07:15:23 AM

Yep, it sucks Sad

And the Bitcoin dust those Ordinal spammers create will remain unspent taking up space in chainstate forever.

Unfortunately we have to live with it. Pruned nodes in the future will avoid the issue, I suppose.

Pruned node need full UTXO in order to verify new TX/block, so i doubt it's possible. At best, we'll see slightly more efficient UTXO representation.

apogio

sr. member

Activity: 406

Merit: 896

Quote from: nc50lc on November 30, 2023, 07:15:23 AM

Must be your RPi's CPU struggling on script verifications after v25.0's default "assumevalid" blocks which is before block height 784000.
Here's for reference: github.com/bitcoin/bitcoin/blob/v25.0/src/kernel/chainparams.cpp#L107C128-L107C128

It's not recommended but if you think it's necessary to speed up your node's IBD, you can set a custom assumevalid block by setting -assumevalid=.

Thanks, I will see the reference because it is something I ignored until today.

Wouldn't it be bad for security though? Is this why you say it is not recommended?

Quote from: LoyceV on November 30, 2023, 07:15:23 AM

I'd copy everything in ~/.bitcoin/ (after shutting down Bitcoin Core).

Definetely gonna try it after the sync is finished.

Quote from: LoyceV on November 30, 2023, 07:15:23 AM

Yep, it sucks Sad

And the Bitcoin dust those Ordinal spammers create will remain unspent taking up space in chainstate forever.

Unfortunately we have to live with it. Pruned nodes in the future will avoid the issue, I suppose.

nc50lc

legendary

Activity: 2394

Merit: 5531

Self-proclaimed Genius

Quote from: apogio on November 30, 2023, 06:25:01 AM

Software:
Raspbian OS
Bitcoin Core 25.0
-snip-
After reaching block 790,000 the process slowed down a lot.

Must be your RPi's CPU struggling on script verifications after v25.0's default "assumevalid" blocks which is before block height 784000.
Here's for reference: github.com/bitcoin/bitcoin/blob/v25.0/src/kernel/chainparams.cpp#L107C128-L107C128

It's not recommended but if you think it's necessary to speed up your node's IBD, you can set a custom assumevalid block by setting -assumevalid=.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: apogio on November 30, 2023, 07:07:09 AM

supposing I wanted to do it, would I need to copy the chainstate and blocks directories?

I'd copy everything in ~/.bitcoin/ (after shutting down Bitcoin Core).

Quote

Makes sense, but 16 GB is a lot for a cheap computer. RPi doesn't support 16GB anyway.

Yep, it sucks Sad

And the Bitcoin dust those Ordinal spammers create will remain unspent taking up space in chainstate forever.

apogio

sr. member

Activity: 406

Merit: 896

Quote from: LoyceV on November 30, 2023, 06:59:47 AM

Why don't you just copy the blockchain from your existing nodes to the new node?

Ok you made me lough out loud. How can I be so silly?! It didn't even cross my mind. However, supposing I wanted to do it, would I need to copy the chainstate and blocks directories? Or all the items from the Bitcoin Core directory (.conf, .log etc) ?

I won't stop the process now, since I am almost done, I guess.

Quote from: LoyceV on November 30, 2023, 06:59:47 AM

I guess that's around the time the Ordinal spam largely increased the number of transactions, and the size of chainstate. It currently takes 8.5 GB on disk, which won't fit your 8 GB RAM anymore. It looks like 16 GB is the new minimum for a fast IBD.

Makes sense, but 16 GB is a lot for a cheap computer. RPi doesn't support 16GB anyway.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: apogio on November 30, 2023, 06:25:01 AM

I have 2 nodes and I am currently setting up my 3rd node.

Why don't you just copy the blockchain from your existing nodes to the new node?

Quote

After reaching block 790,000 the process slowed down a lot.
I have spent 5 days until block 790,000 and 3 more days until 810,000 and there are still approximately 10,000 blocks left, which will take even more Tongue

I guess that's around the time the Ordinal spam largely increased the number of transactions, and the size of chainstate. It currently takes 8.5 GB on disk, which won't fit your 8 GB RAM anymore. It looks like 16 GB is the new minimum for a fast IBD.

apogio

sr. member

Activity: 406

Merit: 896

Quote from: NotATether on November 30, 2023, 06:35:38 AM

txindex does slow down the IBD, but it shouldn't make it that slow. So I want to ask you, where did you configure the datadir? Because if you're going to make I/O go through the SATA connection to USB3, that's going to be quite slower than just writing on the SSD directly. USB3 standard is 10GB/s maximum, and the disk is probably doing much less.

Not sure I understand this. The datadir is on the SSD. The SSD is external to the Raspberry. So essentially I run:

Quote

bitcoind --datadir=/media/.../...

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

txindex does slow down the IBD, but it shouldn't make it that slow. So I want to ask you, where did you configure the datadir? Because if you're going to make I/O go through the SATA connection to USB3, that's going to be quite slower than just writing on the SSD directly. USB3 standard is 10GB/s maximum, and the disk is probably doing much less.

apogio

sr. member

Activity: 406

Merit: 896

There are too many topics about it, but I haven't found the answer I want.
I would post on older topics but I got the warning that the topics were older than 120 days, so I started a new one.

I have 2 nodes and I am currently setting up my 3rd node.

The first 2 nodes were constructed between blocks 700,000 and 750,000 and the process went okay-ish.

On the 3rd node, my IBD is very slow. Let me share my setup.

Hardware:
1. Raspberry Pi 4 Model B with 8GB RAM.
2. Cat6 ethernet cable.
3. 2TB SSD 2.5''.
4. SATA (on the disk) to USB 3.0 (on the Rpi)

Internet speed:
Download: 200 Mbps
Upload: 5 Mbps
Ping: 4 ms

Software:
Raspbian OS
Bitcoin Core 25.0

My conf is:

Code:

datadir=...
server=1
daemon=1
txindex=1

After reaching block 790,000 the process slowed down a lot.
I have spent 5 days until block 790,000 and 3 more days until 810,000 and there are still approximately 10,000 blocks left, which will take even more Tongue

It's normal, I know! Because it scans and validates recursively.

Does anyone know if -txindex=1 slows up the Initial Blockchain Download ?

If so, is there a reason?

Topic: Bitcoin Core IBD slow - page 2. (Read 176 times)