Author

Topic: Speed up syncing of bitcoin core when the blockchain becomes colossal (Read 322 times)

copper member
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
I don't think the hardware is the problem. Given that my netbook is a low spec "celery", and it seems to be quite happy sync'ing, I suspect the problem lies with network connectivity. Give things another few years, and babies will be fitted with blockchain implants shortly after birth.

This is not necessarily true. It's possible to have a hardware configuration that is the bottleneck. This is especially true when running Raspberry Pi or similar. Also, slow USB drives can become the bottleneck as well. Most modern desktops and laptops won't run into this though.

EDIT: To answer OPs original question. Yes, I'm pretty sure dedicated chips could speed things up but as another poster said, the initial investment would be large. Also, they would only be useful on initial sync, they wouldn't really be necessary after that.

It's $3 a chip. I probably eat that in 12 hours ($3), not something completely rediculous.

I tried running core on a USB external hard drive and it was definitely a lot slower than running it straight off the machine. (although we need an ecdsa chip for stuff to get really ramped up Cheesy).
newbie
Activity: 3
Merit: 2
I don't think the hardware is the problem. Given that my netbook is a low spec "celery", and it seems to be quite happy sync'ing, I suspect the problem lies with network connectivity. Give things another few years, and babies will be fitted with blockchain implants shortly after birth.

This is not necessarily true. It's possible to have a hardware configuration that is the bottleneck. This is especially true when running Raspberry Pi or similar. Also, slow USB drives can become the bottleneck as well. Most modern desktops and laptops won't run into this though.

EDIT: To answer OPs original question. Yes, I'm pretty sure dedicated chips could speed things up but as another poster said, the initial investment would be large. Also, they would only be useful on initial sync, they wouldn't really be necessary after that.
jr. member
Activity: 39
Merit: 25
Chips are cheap for one.

Quote

Or we can offer our Clarke ASIC chip 90 GH/s, 75mJ/GH at USD 3.50 per chip.


I made an enquiry as to the price of a bit fury chip. A while ago.



That's mining chip. ASIC (Application Specific Integrated Circuit). It's only for mining. Doing it for validation needs alot more capital. Some estimated about 100mUSD.
copper member
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
Chips are cheap for one.

Quote

Or we can offer our Clarke ASIC chip 90 GH/s, 75mJ/GH at USD 3.50 per chip.


I made an enquiry as to the price of a bit fury chip. A while ago.

So 90GHs is more than enough.
A block has about 4000 transactions when full.
2 hashes per transaction is 8000.
Then 8000+4000+2000+1000+500+250+125...= <16000
If we don't do any validation of signatures, it becomes 5.6M blocks per second.
jr. member
Activity: 39
Merit: 25

I agree implementing GPU validation is the cheapest and simplest way to increase validation speed. I have limited experience on writing code for GPU. But I'm interested.

When I run a program on Windows it spossible to run the executable on a gpu. Given Linux has its talents of multitasking I'd have thought it already pushes that sort of process onto a gpu?

@carlton, isn't there something on the Intel instruction set that increases the efficiency of hashing algorithms also. I'm sure someone mentioned that a while back.

Baking ASICs for validating bitcoin would give the highest efficiency. There are two reason for not doing it.

1. ASIC cannot be changed once it's out there.
2. Cost of manufacturing an ASIC is not realistic at the moment. Also it may raise a security issue.

Instead using small compute units is the perfect solution to scale. It improves efficiency against a CPU core. And it's general purpose computing unit.
Look at the thread i did open. https://bitcointalksearch.org/topic/concurrent-validation-engine-for-bitcoin-core-5103538
jr. member
Activity: 39
Merit: 25

I agree implementing GPU validation is the cheapest and simplest way to increase validation speed. I have limited experience on writing code for GPU. But I'm interested.

When I run a program on Windows it spossible to run the executable on a gpu. Given Linux has its talents of multitasking I'd have thought it already pushes that sort of process onto a gpu?

@carlton, isn't there something on the Intel instruction set that increases the efficiency of hashing algorithms also. I'm sure someone mentioned that a while back.

bitcoin scripting language should get ported to opencl. For it to be able to run on GPU. I'm sure we can find shortcuts to automate most of the process of porting.
legendary
Activity: 3430
Merit: 3080
isn't there something on the Intel instruction set that increases the efficiency of hashing algorithms also

right, SSE4 (or is it 4.1/2 ?). Bitcoin's been using those instructions for SHA256 on x86 since 0.15.0
copper member
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory

I agree implementing GPU validation is the cheapest and simplest way to increase validation speed. I have limited experience on writing code for GPU. But I'm interested.

When I run a program on Windows it spossible to run the executable on a gpu. Given Linux has its talents of multitasking I'd have thought it already pushes that sort of process onto a gpu?

@carlton, isn't there something on the Intel instruction set that increases the efficiency of hashing algorithms also. I'm sure someone mentioned that a while back.
legendary
Activity: 3430
Merit: 3080
Ah. I think it is more the signature s that take a while to validate then in comparison to the hashes.

SHA256 is already accelerated on x86 platforms using SSE4 instructions (since Bitcoin 0.15.0)

Signature verification performance can be improved with batch validation (part of the proposed Schnorr sigs soft fork, hopefully coming this year)
jr. member
Activity: 39
Merit: 25
Storage I/O speed and network connectivity usually are the main bottleneck on most cases. Computational power bottleneck usually only happen on barebone PC (which have few cores and .5-2GB) such as Raspberry Pi, at least on initial sync, not when on full nodes.

Bitcoin core already use multi-core/thread for signature verification anyway & ASIC for hashing or/and signature verification only useful for initial sync as CPU should be good enough. IMO GPGPU / GPU acceleration is better and few core developer suggest it, but no one contribute to do it until now since it was suggested on 2013 (https://bitcointalksearch.org/topic/m.1451582)

I agree implementing GPU validation is the cheapest and simplest way to increase validation speed. I have limited experience on writing code for GPU. But I'm interested.
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
Storage I/O speed and network connectivity usually are the main bottleneck on most cases. Computational power bottleneck usually only happen on barebone PC (which have few cores and .5-2GB) such as Raspberry Pi, at least on initial sync, not when on full nodes.

Bitcoin core already use multi-core/thread for signature verification anyway & ASIC for hashing or/and signature verification only useful for initial sync as CPU should be good enough. IMO GPGPU / GPU acceleration is better and few core developer suggest it, but no one contribute to do it until now since it was suggested on 2013 (https://bitcointalksearch.org/topic/m.1451582)
copper member
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
Ah. I think it is more the signature s that take a while to validate then in comparison to the hashes.

I don't think the current main bottle neck jet cash states as the network is the true bottleneck for a private network.
qwk
donator
Activity: 3542
Merit: 3413
Shitcoin Minimalist
Signature validation is only one of a few bottlenecks, and it's unlikely that hardware optimization for that would result in a completely different "user experience".
For a simple user, running a full node and thence requiring both the download and processing of somewhere on the order of 150GB of data ist simply not the best option.
A lightweight SPV-client will usually serve the needs of a regular user best.

Just for reference, a number of bottlenecks to consider:
https://bitcoincore.org/en/2017/03/13/performance-optimizations-1/
legendary
Activity: 2828
Merit: 2472
https://JetCash.com
I don't think the hardware is the problem. Given that my netbook is a low spec "celery", and it seems to be quite happy sync'ing, I suspect the problem lies with network connectivity. Give things another few years, and babies will be fitted with blockchain implants shortly after birth.
copper member
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
Something occurred to me a few days ago while I was daydreaming in a lecture, I focused on a slide that said that hardwired hardware is much faster than modifible hardware inside a cpu.

Does this also work with bitcoin?

If there was for example: an asic chip for sha256 and an asic or a cpu core dedicated to the signature validation, a 256Gbyte ram (and any variant upwards) and a hard drive to push everything back onto once the sync was done would this be a possible way for speeding up the initial sync so that only the network becomes the bottleneck or have I missed something?
Jump to: