Can I add nodes to speed up Bitcoin Core syncing?

SilverCryptoBullet

member

Activity: 97

Merit: 43

Quote from: LoyceV on January 10, 2024, 07:16:29 AM

You can of course use prunednode.today, and download a pruned node. I just tried: it took my server 75 seconds.
But you'll never be absolutely sure it's completely legit. And if enough people do it, someone will find a way to scam them.

This guy knows a thing or 2 about the subject:

Quote from: gmaxwell on May 31, 2019, 04:32:06 PM

FWIW, I just removed a nearly year old post with a .bitcoin directory download that would have stolen all your bitcoins.

Case in point as to why these "download the blockchain" links are a disaster. God knows how many people got robbed due to it.

Quote from: gmaxwell on January 23, 2021, 10:57:35 PM

But the chainstate database/etc. aren't intended as external interfaces and I would be totally unsurprised if there was a way for malicious data in them to result in code execution.

Thank you.

Quote from: ABCbits on January 11, 2024, 05:23:54 AM

Downloading whole blockchain is necessary to verify that all transaction and block follow Bitcoin protocol/consensus.

And if OP want more recent data, he can check https://bitcoin-snapshots.jaonoct.us/. I haven't tried it, but the owner also provide assumeutxo file which should be better than copy of Bitcoin Core folder.

Thank you too.

Quote from: ranochigo on January 11, 2024, 07:56:50 AM

The reason for IBD is to ensure that the chainstate is built correctly, hence all of the unspent outputs are correctly indicated. It would then be impossible for a rogue node to falsify transactions with inputs that doesn't exist. Risk is relatively low, but it doesn't mean that it is impossible. If you are unable to conduct IBD for your Bitcoin Core node, I would recommend using an SPV wallet instead.

Thank you too.

From your replies, I understand like chainstate is important to check status (spent or unspent of UTXOs from previous blocks) and it will be stored on my Bitcoin Core prune node. Other data will be deleted after IBD.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: SilverCryptoBullet on January 10, 2024, 06:42:48 AM

Thank you.

I have another question.

IBD, why every node has to do it?
If I only run a prune node, I can not use my node to check too old block details like genesis block, because it exceeds my prune node limit. I feel it is like unnecessary to sync it from genesis block.

It is more convenient if I set my prune node is 50 GB and I start directly at a past block that accounts for 50GB data till the current newest block.

You don't need to query for data of a block, in fact, Bitcoin Core also doesn't look at the earlier blocks after the initial synchronization. That is other than the startup which checks for the last XXX blocks, can't remember how many off the top of my head.

The reason for IBD is to ensure that the chainstate is built correctly, hence all of the unspent outputs are correctly indicated. It would then be impossible for a rogue node to falsify transactions with inputs that doesn't exist. Risk is relatively low, but it doesn't mean that it is impossible. If you are unable to conduct IBD for your Bitcoin Core node, I would recommend using an SPV wallet instead.

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: SilverCryptoBullet on January 10, 2024, 06:42:48 AM

IBD, why every node has to do it?
If I only run a prune node, I can not use my node to check too old block details like genesis block, because it exceeds my prune node limit. I feel it is like unnecessary to sync it from genesis block.

Downloading whole blockchain is necessary to verify that all transaction and block follow Bitcoin protocol/consensus.

Quote from: LoyceV on January 10, 2024, 07:16:29 AM

Quote from: SilverCryptoBullet on January 10, 2024, 06:42:48 AM

IBD, why every node has to do it?

You can of course use prunednode.today, and download a pruned node. I just tried: it took my server 75 seconds.
But you'll never be absolutely sure it's completely legit. And if enough people do it, someone will find a way to scam them.

This guy knows a thing or 2 about the subject:

Quote from: gmaxwell on May 31, 2019, 04:32:06 PM

FWIW, I just removed a nearly year old post with a .bitcoin directory download that would have stolen all your bitcoins.

Case in point as to why these "download the blockchain" links are a disaster. God knows how many people got robbed due to it.

Quote from: gmaxwell on January 23, 2021, 10:57:35 PM

But the chainstate database/etc. aren't intended as external interfaces and I would be totally unsurprised if there was a way for malicious data in them to result in code execution.

And if OP want more recent data, he can check https://bitcoin-snapshots.jaonoct.us/. I haven't tried it, but the owner also provide assumeutxo file which should be better than copy of Bitcoin Core folder.

o_e_l_e_o

legendary

Activity: 2268

Merit: 18771

Quote from: ranochigo on January 10, 2024, 03:23:27 AM

Besides, you can only connect to a maximum of 10 other nodes.

You can actually have 18 outbound connections in total - 8 automatic full relay connections, 2 automatic blocks only connections, and up to 8 manual outbound connections using the addnode command.

I do agree that adding more nodes will make no difference to the speed of the IBD, though.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: SilverCryptoBullet on January 10, 2024, 06:42:48 AM

IBD, why every node has to do it?

You can of course use prunednode.today, and download a pruned node. I just tried: it took my server 75 seconds.
But you'll never be absolutely sure it's completely legit. And if enough people do it, someone will find a way to scam them.

This guy knows a thing or 2 about the subject:

Quote from: gmaxwell on May 31, 2019, 04:32:06 PM

FWIW, I just removed a nearly year old post with a .bitcoin directory download that would have stolen all your bitcoins.

Case in point as to why these "download the blockchain" links are a disaster. God knows how many people got robbed due to it.

Quote from: gmaxwell on January 23, 2021, 10:57:35 PM

But the chainstate database/etc. aren't intended as external interfaces and I would be totally unsurprised if there was a way for malicious data in them to result in code execution.

satscraper

hero member

Activity: 714

Merit: 1298

Quote from: SilverCryptoBullet on January 10, 2024, 06:42:48 AM

It is more convenient if I set my prune node is 50 GB and I start directly at a past block that accounts for 50GB data till the current newest block.

Convenient - YES

Possible - NO

Bitcoin Core deletes previous history data after their downloading and verification. That’s how it works, and there’s nothing you can do about it. I think this due to the implemented procedure design to index both the chainstate and blocks on disk that keep copy of pruned blockchain.

SilverCryptoBullet

member

Activity: 97

Merit: 43

Thank you.

I have another question.

IBD, why every node has to do it?
If I only run a prune node, I can not use my node to check too old block details like genesis block, because it exceeds my prune node limit. I feel it is like unnecessary to sync it from genesis block.

It is more convenient if I set my prune node is 50 GB and I start directly at a past block that accounts for 50GB data till the current newest block.

Block 0
https://blockchair.com/bitcoin/block/0

hash
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f

I get this

Code:

Block not available (pruned data) (code -1)

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: SilverCryptoBullet on January 09, 2024, 05:37:08 AM

Can I add extra nodes that are more close to me, to speed up the syncing?

As stated by other user, usually there are other factor which limit sync speed. But FYI if you use Bitcoin Core, you can't connect to more node since Bitcoin Core only let you connect up to only 10 nodes (outgoing connection). The only way to increase that is by editing Bitcoin Core source code and then compile it.

Quote from: SilverCryptoBullet on January 09, 2024, 05:37:08 AM

Quote from: ABCbits on January 06, 2024, 04:40:46 AM

2. Bitcoin Core happen connect to node with slow connection speed or physically very far from where you live (which cause slower download speed).

You can try close and reopen Bitcoin Core in order connect to different full nodes, but don't expect much difference.

Where to find the node list?

There are many website such as bitnodes.io/ which provide such list, but you can't know which of those have fast connection.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Block downloads are highly parallelized, and it would be difficult for you to obtain only the slow nodes unless you were intentionally doing so. Misbehaving or slow nodes would be disconnected by your client after a certain threshold. Hence, it would be likely that you are experiencing slow validation because of hardware limitations. Besides, you can only connect to a maximum of 10 other nodes.

You can take a look at your task manager for more statistics and check if there are other bottlenecks in your system; CPU, memory or I/O speeds. They are the likely culprit.

nc50lc

legendary

Activity: 2618

Merit: 6452

Self-proclaimed Genius

Quote from: SilverCryptoBullet on January 09, 2024, 05:37:08 AM

Can I add extra nodes that are more close to me, to speed up the syncing?

Check your peers' ping first if it's really the cause of your slow IBD:
Click the network icon below and select "Show peers tab", or click "Window->Peers".
Or via command: getpeerinfo

It'll only be slow if you're exclusively connected to very slow nodes which rarely happens;
Most of the case, it's the hardware and node settings.

satscraper

hero member

Activity: 714

Merit: 1298

Quote from: o_e_l_e_o on January 09, 2024, 09:43:54 AM

Quote from: satscraper on January 09, 2024, 06:04:51 AM

And, I would add blocksonly=1 entry in bitcoin.conf file

It's been a long time since I last performed an IBD, but I thought transactions were not downloaded or added to your mempool until your node was full synced (since your node will be unable to verify if a transaction is valid until it is fully synced). If that's the case, then blocksonly=1 should make no difference to the speed of the IBD, since you aren't syncing transactions anyway.

Yeah, you are correct. blocksonly=1 helps to speed up a little the initial block download and at node being fully sync the entry should be commented/deleted.

Another case when this entry is helpful is the resuming sync after long nod's deadtime . I accustomed myself to turn off my node at going for two-tree weeks vacation and be back usually activate this entry in bitcoin.conf file until sync is completed.

o_e_l_e_o

legendary

Activity: 2268

Merit: 18771

Quote from: satscraper on January 09, 2024, 06:04:51 AM

And, I would add blocksonly=1 entry in bitcoin.conf file

It's been a long time since I last performed an IBD, but I thought transactions were not downloaded or added to your mempool until your node was full synced (since your node will be unable to verify if a transaction is valid until it is fully synced). If that's the case, then blocksonly=1 should make no difference to the speed of the IBD, since you aren't syncing transactions anyway.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: satscraper on January 09, 2024, 06:04:51 AM

I accustomed myself to VPN service when connecting Bitcoin Core to peers and sometime, being connecting via servers with the certain IPs, experienced the slow sync.

If your VPN can't handle the bandwidth, obviously changing can help.

Quote

I would add blocksonly=1 entry in bitcoin.conf file and set the correct value of dbcache in it. In general situation the rule of thumb dbcache=1/4 RAM, i.e. for 8 Gb of RAM dbcache=2048

I prefer to have dbcache > chainstate, but that only works if you can allocate 10 GB to it. I wouldn't recommend a quarter of your RAM, that depends on what else you're doing on the system.

apogio

hero member

Activity: 560

Merit: 1060

Quote from: satscraper on January 09, 2024, 06:04:51 AM

And, I would add blocksonly=1 entry in bitcoin.conf file and set the correct value of dbcache in it. In general situation the rule of thumb dbcache=1/4 RAM, i.e. for 8 Gb of RAM dbcache=2048

Good suggestion. Recently I downloaded the whole blockchain in 72hrs using a Raspberry Pi 4b with 8 GB RAM and a 2TB SSD.

I used dbcache=5000 in terms of configuration.

Just make sure to change the conf and restart bitcoin core after the IBD is finished.

satscraper

hero member

Activity: 714

Merit: 1298

Quote from: LoyceV on January 09, 2024, 05:40:22 AM

Quote from: SilverCryptoBullet on January 09, 2024, 05:37:08 AM

Can I add extra nodes that are more close to me, to speed up the syncing?

In my experience, it's not needed. If your ISP isn't throttling you, Bitcoin Core should find fast enough nodes on it's own. Odds are the rest of your hardware is more limiting though.

I accustomed myself to VPN service when connecting Bitcoin Core to peers and sometime, being connecting via servers with the certain IPs, experienced the slow sync.

Changing VPN server (by simply choosing different country for it) and relaunching Bitcoin Core after that usually helped to to speed up the sync, in which cases, the connected nodes were modified.

Quote from: apogio on January 09, 2024, 06:00:40 AM

Exactly what Loyce said above.

The only thing, in my experience, that will significantly increase your speed is the dbcache option. Set it high enough. Of course, provided that the hardware is proper:
1. high RAM
2. SSD instead of HDD
3. USB 3 connection if the disk is external
4. high internet speed
5. good ethernet connection cable

And, I would add blocksonly=1 entry in bitcoin.conf file and set the correct value of dbcache in it. In general situation the rule of thumb dbcache=1/4 RAM, i.e. for 8 Gb of RAM dbcache=2048

apogio

hero member

Activity: 560

Merit: 1060

Exactly what Loyce said above.

The only thing, in my experience, that will significantly increase your speed is the dbcache option. Set it high enough. Of course, provided that the hardware is proper:
1. high RAM
2. SSD instead of HDD
3. USB 3 connection if the disk is external
4. high internet speed
5. good ethernet connection cable

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: SilverCryptoBullet on January 09, 2024, 05:37:08 AM

Can I add extra nodes that are more close to me, to speed up the syncing?

In my experience, it's not needed. If your ISP isn't throttling you, Bitcoin Core should find fast enough nodes on it's own. Odds are the rest of your hardware is more limiting though.

SilverCryptoBullet

member

Activity: 97

Merit: 43

I am running a prune node.

I read that Bitcoin Core will connect my node to most remote node.

Can I add extra nodes that are more close to me, to speed up the syncing?

Quote from: ABCbits on January 06, 2024, 04:40:46 AM

2. Bitcoin Core happen connect to node with slow connection speed or physically very far from where you live (which cause slower download speed).

You can try close and reopen Bitcoin Core in order connect to different full nodes, but don't expect much difference.

Where to find the node list?

Topic: Can I add nodes to speed up Bitcoin Core syncing? (Read 220 times)