Topic: What is the bottleneck in synchronizing? (Read 297 times)

It's actually ram and HDD

Made 2 test
8 core 3.2ghz 8gb ddr3 ram normal HDD sync time 4days

Seconde running as proxmox container 4 cores 2.2ghz 72gb DDR 3 ram and ssd sync time 14 hours

This was really helpful. Thank you.
pebwindkraft, your post also gave me a clue how the process works. Thank you too.

Summary: there is no generic rule - one would need to understand the way, how the initial load of the blockchain works :-)
There are too many variables. One would have to setup a weighted matrix with all dependencies... hence you find only general info, like this:

With my bitcoin core node (a fully validating node), it fetches all transactions from it peers, and verifies the transactions for the correctness (see here: https://en.bitcoin.it/wiki/Protocol_rules#.22tx.22_messages).
This involves many, many lookups of previous transactions in the local blockchain, hence the dependency to the hard disk. I have had situations, where an SSD provided a times five speedup.
The calculations with a Quad-Core CPU is usually not a limiting factor, I had it running on a 4 years old Macintosh, and it never got limited. CPU calculations was always faster than data delivered from other peers.
On the bandwidth: I had a 10mbit/s connection, and did never run into congestion. Au contraire: I didn't have "enough" peers to send data continuously.
Then I did the opposite: I took a RasPI2+, attached it to a 2mbit line, and let it run 3 weeks, before he was nearly sync'd with the blockchain. And then it came into the time of full blocks (near the end 2017), and suddenly the RasPi would never completly sync. The CPU was probably too slow to verify the tx data, and the attached external disk (no SSD!) via USB could not cope with the amount of requests. I set up a monitor, and measured CPU, processes (load averages), disk I/O, network I/O... I could see that the disk I/O was not going down to zero a single time, and the load average never went under 1.0, it merely stayed between 3 and 4. So there were many processes in the working queue, that could not be worked on by the CPU, cause other tasks occupied it.

Summary: you have to have a setup, and measure all details.
I believe that core team does a similiar thing, and I recall they had a process setup, which (re-) loads the blockchain on a daily basis (to ensure correctness of the system). But I haven't seen any statistics.

• 1st bottleneck is internet connection speed
• 2nd bottleneck is CPU speed
• 3rd bottleneck is hard disk speed

Improve your setup in that order, or all 3 simultaneously. Improvements to Bitcoin Core can also increase blockchain sync performance with any hardware. RAM is not so much of a bottleneck; even basic computers have 1GB+ RAM these days which is good enough (if you look at Settings -> Options menu in Bitcoin Core, you can improve the syncing performance a little if you increase database cache size to 1GB from the default value)

That's the best summary I can provide you.

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?

Bitcoin 0.17.0 will speed up a little if you have fast (>10 Mbit/s) internet, fast/modern CPU & SSD disk

• Changes to the way leveldb handles blockchain I/O will speed things up 5-10%
• Validation will improve <5% for a range of CPUs with SSE4 and AVX2 instructions (maybe including ARM)

Most of the big gains have already been leveraged, but the developers often bounce new ideas around on the bitcoin dev mailing list.

Internet bandwith is not a problem nowadays (assuming you live in a country with somewhat developed technology).
The biggets bottleneck are the CPU processing and the disk usage.

If you are running a somewhat modern computer (e.g. quad core), the device which slows down the synchronisation process is the hard drive.
SSD's decrease the time it takes to sync the blockchain noticeable, compared to a HDD.

Depends on your hardware and internet connection. On a PC with a DSL (or slower) connection, it is internet bandwidth. For a Raspi Pi, it is CPU power. HW write speed will never be an issue.

What is the bottleneck in synchronizing the blockchain? Computational Power (CPU), Internet bandwidth or hard disk speed?