Any idea how long it takes to perform the validation of bootstrap.dat once it's downloaded (on typical hardware)?
Sadly it varies wildly depending on your hard drive configuration. Import time here is under an hour.
Topic: [BETA] Bitcoin blockchain torrent - page 6. (Read 57756 times)
Sadly it varies wildly depending on your hard drive configuration. Import time here is under an hour.
We want to do exactly the same amount of validation as the client does with network blocks... break no additional link in the trust chain 
Oh, ic…I guess that's what you meant by "The client will verify this data during import."
Any idea how long it takes to perform the validation of bootstrap.dat once it's downloaded (on typical hardware)?
This is similar to that in the sense that you are downloading a block chain that you trust to be validated. It happens to match the 193,000 block baked into the client and signed by the core developers (by virtue of their signature on the hash of the software download). Since this download.dat has block 193,000 whose hash matches the one baked into the client, you can be sure of the fact that you've got a block chain that matches that which was signed by the core developers up to block 193,000. If all the merkle hashes and previous block pointers check out, you're good to go.
Not quite... this data is the precisely the same data you see on the P2P network.
The client validates bootstrap.dat data to the same level it validates data downloaded from the P2P network.
If all checkpointing code is disabled, this data remains fully valid and useful.
Quote
But, I always thought the bulk of the time to sync up the block chain was due to transaction validation and not actually due to the download time.
A lot of the time is due to Berkeley DB slowness (fixed by ultraprune). Another yet-unfixed cause of slowness is poor bitcoin P2P network peer selection, for block download.
Quote
When it comes to the matters of trust, could you not achieve the equivalent result by starting up the client in a mode that doesn't do full validation for blocks prior to the most recent, baked in block (i.e. 193,000)? On startup (if it's GUI), the user could be prompted to decide whether to perform a full validation, or to trust the baked in checkpoint. Maybe downloading through Bittorrent still has advantages (i.e. not burdening the bitcoin network with download traffic), but I wonder how the performance of this would compare with downloading via Bittorrent.
We want to do exactly the same amount of validation as the client does with network blocks... break no additional link in the trust chain
Cool. It got me thinking and questioning a few things. I've often thought that a good alternative to downloading & validating the full block chain when bootstrapping a new client is to find a node you trust and simply copy it block chain and transaction data. I think of this kind of like cloning a trusted node. You can bypass a all the validation work.
This is similar to that in the sense that you are downloading a block chain that you trust to be validated. It happens to match the 193,000 block baked into the client and signed by the core developers (by virtue of their signature on the hash of the software download). Since this download.dat has block 193,000 whose hash matches the one baked into the client, you can be sure of the fact that you've got a block chain that matches that which was signed by the core developers up to block 193,000. If all the merkle hashes and previous block pointers check out, you're good to go.
But, I always thought the bulk of the time to sync up the block chain was due to transaction validation and not actually due to the download time. When it comes to the matters of trust, could you not achieve the equivalent result by starting up the client in a mode that doesn't do full validation for blocks prior to the most recent, baked in block (i.e. 193,000)? On startup (if it's GUI), the user could be prompted to decide whether to perform a full validation, or to trust the baked in checkpoint. Maybe downloading through Bittorrent still has advantages (i.e. not burdening the bitcoin network with download traffic), but I wonder how the performance of this would compare with downloading via Bittorrent.
This is similar to that in the sense that you are downloading a block chain that you trust to be validated. It happens to match the 193,000 block baked into the client and signed by the core developers (by virtue of their signature on the hash of the software download). Since this download.dat has block 193,000 whose hash matches the one baked into the client, you can be sure of the fact that you've got a block chain that matches that which was signed by the core developers up to block 193,000. If all the merkle hashes and previous block pointers check out, you're good to go.
But, I always thought the bulk of the time to sync up the block chain was due to transaction validation and not actually due to the download time. When it comes to the matters of trust, could you not achieve the equivalent result by starting up the client in a mode that doesn't do full validation for blocks prior to the most recent, baked in block (i.e. 193,000)? On startup (if it's GUI), the user could be prompted to decide whether to perform a full validation, or to trust the baked in checkpoint. Maybe downloading through Bittorrent still has advantages (i.e. not burdening the bitcoin network with download traffic), but I wonder how the performance of this would compare with downloading via Bittorrent.
UPDATE Feb 2013: This thread is obsolete. See the non-beta Bitcoin blockchain data torrent thread for further torrents and updates.
This is a beta test of a blockchain torrent project. Interested participants are invited to try, and comment.
Version 0.7.1, which just entered testing, includes a new feature: If the file "bootstrap.dat" is found in the bitcoin data directory, it will validate and import all blockchain data found in that file. The following torrent presents a bootstrap.dat file for that feature.
Here is the PGP-signed torrent information. Details follow below the signature.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Torrent info hash: 0bb0521942f586ed96203c6f4d136324756f8a9a
Torrent magnet link:
magnet:?xt=urn:btih:0bb0521942f586ed96203c6f4d136324756f8a9a&dn=bootstrap.dat
Filename: bootstrap.dat
Byte size: 2491771562
SHA1: e70ca90775dfdb13fd0014425805a0bdf4a31677
SHA256: a3f258e7af030165360596e4cb0b9beb24b4ce97352c22e65349b89ad5fc5d3e
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iQIVAwUBUHeHSdodwg8tvwyoAQJ2DA/8CcINxuD7DzLv96fE+R6GVLvRASLcy0ig
8d2YyDbJ9r9dPVODDPGC7I/ooDVJAcQsqdrLYHST2DTCt6c4zZ/7iXzFEahRopsH
PmxYOgkHie7438nqpmH9uC+d5t0pPHUFS6dBSBgdSVPaLfS86fToXrV/bx30rHBi
60FJz5A99aXrrrUny0cGIjafqVv8XjqNoA1lzjsRjeiu3EgTm8Ibcr8ZI6DLp/80
Siv3potlOArTI6sxjc/vKUa6nZILnW8mKdwc/d8LUdRaBPoo71c6Q4YOSQh/OVht
B1rZ8NEX/2SlS3PbhhMELcY/2wvgPGovkIqOgiw6dDIkhsD8iJAD1DqCkZbsq9S1
kKobGmceuQBcyRUCavkafdHJpZzyCAKLnQLV3FvZ3O4QZQVmYGPYn1H8a5UFRDQq
LGKcQmwm4Cet7162woEiNAaR9p7HvTJ4LV2uEvY56m2GfZbToCk2aiycO+v6Fm8K
ZY8cX+cbEOW3AM5rYVa6Rks029LIrFFzIutlo5MJ7uc7oMqciWrcsPtEh59QY+yJ
SENk6cLyWCAHl4jsvUTBvdNGy3fHKSEyQOjG5cmAMXiJTX/iiB/DJf7koTZAj4ub
wez3/HwuenMYjHgjsVXJAXprcEpTjpEcicg4W0MIDw07dPjol0dnRRNpwfJ1HMRc
dlTqLwlXqgs=
=uwRB
-----END PGP SIGNATURE-----
What is bootstrap.dat?
It is a flat, binary file containing bitcoin blockchain data, from the genesis block through height 193,000. Height 193,000 is the height of the current checkpoint baked into the reference client.
Version 0.7.1 (when released) will automatically validate and import a file in the data directory named "bootstrap.dat". Version 0.7 or later will also import this file by passing the command line argument "-loadblock=/path/to/bootstrap.dat" to bitcoin-qt or bitcoind.
Who wants bootstrap.dat?
Anyone bringing up a new node using the reference client. This is one method of accelerating the initial blockchain download process, while helping the bitcoin P2P network by offloading data download traffic from public P2P nodes.
This download is not for those who are already running the bitcoin client.
NOTE: This torrent requires DHT torrent capability
This torrent is a so-called "trackerless torrent", to avoid making any of the open torrent trackers targets of any bitcoin antipathy. Peers for this torrent are discovered via DHT, and early results seem to indicate that some bittorrent clients take a while to find their initial peers. We have also discovered clients (rtorrent) that disable DHT by default; you will need to turn it on.
Please report results, good or bad, on using this trackerless torrent.
How often will this torrent be updated?
Assuming this project is deemed useful and worth continuing... the torrent will be updated once every few months, when the checkpoints are updated in the reference client source code.
Why not update the torrent more often?
A torrent works best when it is a large, static dataset that changes infrequently. That maximizes the ability to seed the data, enabling even part-timer seeders to contribute meaningfully. Less frequent changes also minimizes the risk that a malicious torrent will appear, with a long, malicious side chain. The current policy only updates the torrent after blocks are buried many thousands deep in the chain.
Why should I trust you?
You don't have to: This data is raw block chain data. The client will verify this data during import.
Independent third parties may generate their own bootstrap.dat, up to height 193000, and verify that the sha256sum matches that posted above. The file format is simple and publicly known:
<4-byte pchMessageStart><32-bit length>
Torrent file download?
If the magnet link does not work, download http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent
Comments welcome
Post any comments or experiences in this thread. I'll update the OP as needed.
Maybe trackerless will be a #fail, but let's see how it goes.
Jump to: