Topic: High speed Blockchain Download (Read 2742 times)

Because the blockchain is full of hashes that are in effect random numbers it does not compress well.
You only get 20% compression, so does not seem worth it for all the extra effort.

So a potential attack might go like this:
- Create corrupt blockchain, showing that you have 100,000 BTC at a particular address from a transaction back in block 100,000.
- Get more than 50% of miners to download said blockchain (impossible, given that major pools make up far more than 50% of hashing power, and they already have the blockchain)
- Send those 100,000 BTC to another real address.  The miners with the corrupted blockchain would accept the transaction.
- Spend, spend away!

What other concerns might this present?

When the Bitcoin client loads up, it only checks that the last 2500 blocks in the chain that it has are valid. Beyond that, you can change anything you want in the previous blocks without effecting what the client thinks the block hash is. Remember, the data on disk is assumed to have already been checked by your client, so there is no reason to check it on every startup. If it did, it would take hours to start your client.

That said, I have to thank you. I just discovered an undocumented command line option: -checkblocks. If you use that, you will be safe if you download an untrusted blockchain and put it in your data directory.

Line 474 in src/db.cpp

This is not true, at all.

I still disagree.

Looking at the source when a new block message comes in it is passed to ProcessBlock (in main.cpp).
After checking that the block is valid (which it is as came from p2p) it checks we have the previous block (using the hash of that block).
In the case of a "fake chain" it would not be able to find the previous block (as it is impossible to create a fake block with the correct hash) so would mark the new block as an Orphan. What is even smarter is that it then requests from the p2p the predecessor.
And this continues till the chain is correct (the real chain will become the best chain as soon as the last block (block 1) links to the Genesis Block.

So there is no security risk of downloading a pre-created chain. Only risk is that if you did download a fake chain your client would end up downloading the real chain backwards (and I assume rather slowly).

Only security risks are if download fake chain and you only connect to "fake nodes" (or there is a bug in the above code). As such if you do download a chain seems sensible to delete the addr.dat file as that could cause you to connect to fake nodes.

But I do agree with you, if you have 100's of coins and want to be sure you should let the client download all from the p2p, but if you are a newbie and just want to have a play sending a few cents about, then downloading the precreated chain from http://bitcoin.bluematt.me/bitcoin-nightly/blockchain-nightly/ is a speedy and secure route.

Maged - I am not trying to start a fight, just want to be sure that I really understand how the client works, as am working on BitcoinJ and do not want to introduce bugs there.

I don't know if the p2p snippet download of the blockchain is compressed on the fly snippet by snippet or not. Even if each snippet is downloaded compressed, you would have an overhead because you would download a huge amount of single compressed snippets and not one huge compressed file.

So the regular block chain isn't compressed at all? Why not? Isn't regular RLE pretty trivial to implement these days? Maybe some other reason.

For that you would need to know that there is an addnode switch, how to use it and you would need know IPs too add. Besides that, if you download it via http you can download a compressed archive of the blockchain, if you download it through the client you will get the blockchain snippet by snippet (uncompressed, if i'm not wrong). Also if you are behind Router that uses NAT, you might only get ~8 connections in the client which will cause the download to slow down eventually.

Because it's not part of the Bitcoin project, the http source is provided by a third party. The external download, outside of the client and from a third party source, comes with the risk of downloading a manipulated block chain. That's why it's not mentioned in the official readme (i guess).

It's not mine, all thanks goes to Matt Corallo. :) As Maged said, he's a trusted developer on this board.
https://bitcointalksearch.org/topic/nightly-builds-4530

Arh.. someone has already uploaded it.

Great, I will delete mine and just use yours!
Also just discovered the blockchain in sourceforge.

I think this should be in the README as a suggestion, as it take 12-24 hours on some clients.

How is having one http link that much faster than having your client download it?

I mean, what if you connect to specific, trusted nodes with the addnode switch... shouldn't some bitcoin nodes be just as fast to download from as an http link?

Or, almost as fast?

because it is not just an hour in severe cases..
if ppl get to know the idea and want to instantly try some transaction or even buy sth with the backed up wallet.dat using another computer.. I think one could make up much more cases weher it could be handy to setup the env very fast.. until there is a trustworthy bank online and/or apps dealing with all of that "nerdy" hurdles

Why not just wait the hour or so it takes to download it? Surprising how often people want to sacrifice security in the context of a currency that is supposed to be secure...

Why aint that Link to fastly get the blockchain in the readmy of the client?!
I think that would be a very good idea. Am I somehow wrong?

it wasn't a trust issue. it was a concern that they might not be able to get the client from a us based depository due to crypto export rules.

Someone just a few days ago mentioned a reason not to trust sourceforge.... *searches forum*

Why not just use the one that the client builders provide?

http://sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/

If your trusting the client, no reason not to trust their blockchain download.

The problem is that the client won't check that the block hash is correct, nor will it check the merkle tree. As such, you could modify a block and any client who uses your blockchain download will just assume that it's correct. That's why you should only download the blockchain files from the P2P network, where it is verified, or a HIGHLY trusted developer (which is the case for the bluematt.me site).

This sentence makes no sense at all. You were sick of waiting for the download, so you uploaded it???? So you uploaded it before you even downloaded it? I think nibor just wants to collect points for his ul.to upload....his "hard work".

If you want to download a snapshot of the recent blockchain, use this:
http://bitcoin.bluematt.me/bitcoin-nightly/blockchain-nightly/

The blockchain snapshot gets updated frequently and the download is directly and more high speed than through stupid ul.to
thread: https://bitcointalksearch.org/topic/nightly-builds-4530

I disagree.

When the next block comes in from the p2p network, it contains in it the HASH of the previous block.

It is impossible to create a block that has a given HASH (this is the whole principle on which Bitcoin is built).

As such it would be impossible for me to create a fake blockchain that included a block at its head that had the correct HASH.

So if I was a scammer and created a fake blockchain you would never get any new blocks in the client as it would reject them all as they would have an incorrect HASH in them.

Only way I could scam you would be to ensure that you got a fake blockchain and only connected to my fake servers, and I had enough resources to create a fake blockchain the same length as the current live one.


Explain to me why you think I am wrong.


Thanks

Robin

PS - If I am wrong will happily remove file from site as do not want to reduce the security.

No, a rescan isn't thorough enough.