Topic: [ANN] Bitcoin blockchain data torrent - page 20. (Read 211194 times)

So I am in the process of downloading the latest version of this torrent now, but still slightly confused as to how I use it...

I am using Bitcoin Qt 0.8.3 Beta, so it should support auto-import of this file, right?

Just a little unsure of where to put it, and if I need to delete the folders "blocks" and/or "Chainstate" and the files "db.log", "debug.log" or "peers.dat"?

I am on OSX so presume that it is going somewhere inside the /Users/will/Library/Application Support/ folder?

Thanks

Offer an RSS feed as well, so seeders would just need to clear old torrents from time to time... [/list]

If you keep updating the file it will break people seeding it and so you'll never have a big distribution network.

Downloading now...  started off with 11 peers and consistently over 12mbits/sec and peaking at 20mbits/sec, and that is on my 20mbit cable connection.  Yep, bitcoiners know how to make any kind of P2P kick ass.

This is iconic.  I'll seed this file

I was just wondering if people would find it usefull if there was a website that did the following:

• Provide one big button to download a torrent of a recent version of the blockchain
• Offer an alternative download (zipped)
• Show the number of active seeders
• Post a new torrent on a weekly basis (or some other viable interval) automaticly based on a bootstrap.dat maintained by a node running on the server.
• Provide information to verify the integrity of the download

That's when I saw this thread . Just downloaded the bootstap.dat in 18 minutes!

What do you think, will there be enough users of a service like this? Or will a solution like this thread in combination with a 6 month interval suffice?

1. Cloud server - no. To have even marginal performance, blockchain files would need to be on a local network drive. The blockchain database works much better on SSD than rotational hard drive. You would be using something much slower than a standard hard drive and sharing it with 50 computers.

2. Rasberry Pi, no. So much slower than a used $25 craigslist computer.

3. The blockchain torrent can be imported into one system's bitcoin-qt. The complete data directory (except wallet.dat) could be copied to other machines so you don't have import multiple times.

4. Then SOME of the blockchain files that never change could be moved to a network drive and replaced with symbolic links on the client filesystems. The only files that won't be changing are past BLK00001.dat-BLK00064.dat files in the blocks sub-directory. The most recent BLK000xx.dat file must not be copied to network, it will still be added to by the individual client bitcoins and must be local and unique to that bitcoin.

Your question appears to have nothing to do with the bootstrap torrent.

I want to host the blockchain on a cloud server and then point about 50 raspberry pi's to it. Is there a way to do this?

OK, updated torrent file at SourceForge and http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent to add trackers mentioned upthread.

Same info hash (thus, same bootstrap.dat file data).

Once enough peers have this new torrent, the magnet link should pick it up for the rest.

The new torrent is working great with rtorrent using DHT for me.

2-3 of those seem to be giving people timeouts.

Well, if it is truly a problem, I would prefer to regenerate the .torrent file.

What are everybody's favorite trackers?  IRC also reports a couple trackers listed in the magnet link are down.

Ideally the DHT should find things, but trackers seem to be necessary for some clients.

Edit: rtorrent users should turn on the DHT.

Updated torrent.  See OP for full information.

The torrent is directly available from http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent but ideally the magnet should find it.

As with other updates, simply replace (delete) the old .torrent with this new .torrent.  Any existing bootstrap.dat will be appended, avoiding a lengthy download process for the first ~200,000 blocks.

Please seed this new torrent (and drop the old)!

Seeding on 30mbit, added tcp (http) trackers as requested.

Yes, the blockchain torrent will be updated this week (probably today).

As others have noted, if you compress with 7zip, that eliminates a useful property from the current torrent:  The first X gigabytes are the same in the old bootstrap.dat torrent, and any new bootstrap.dat torrent.  Using the same file, and replacing the .torrent file, will cause your torrent client to download only the new blocks, not the entire torrent.

This one is a split IPv4/IPv6 tracker. IPv4 reachability seems to be determined by geolocation or some similar mechanism. IPv6 seems to be very reliable and long-term stable.

https://github.com/bitcoin/bitcoin/blob/master/src/checkpoints.cpp
Latest checkpoint: 225430

There is no need for multipart other than the 4 GB/file limit on FAT32 maybe. Most modern torrent clients already support HTTP sources too and would download pieces from these only when there are not enough seeders around in the swarm. As you already said, once a new torrent gets released, all previous data in case of 7zip is invalidated so it is not a case of "I ass another 200 MB piece and the previous ones are still valid" as it is with an uncompressed chain file.

In other news, I announced to these other trackers too, though switching on DHT should already give enough peers to connect to the rest of the swarm via peer discovery - trackers are rarely used these days any more other than for bootstrapping and maybe connecting swarms that were split. Once there is even a single peer connected to both parts of a swarm, via PEX they will be soon reunited anyways. The desync.com one is offline for me too by the way.

@jgarzik:
Could we (until you decide on whether to compress or not) at least have an "official" bootstrap.dat torrent until 225430 (or ideally: ~240k)? I mean it's not that hard to create one ourselves, it would just lead to more fragmented swarms and just having a single "official" one makes it easier to seed.

I would suggest the following scheme by the way:
Every 4032 blocks (= 2 difficulty changes or ~every 4 weeks) release a new torrent link for current block - 2016 (so the block of the last difficulty change). This should be by far deep down enough to not be part of any chain splits any more and updating a link every 4 weeks (or just getting it from an RSS feed) should be not too much hassle for seeders either.

If you set the part size in the torrent to a certain fixed size, you could even let people come up with the info hash programmatically just by processing their own chain files!

A file compressed with 7zip is a solid archive with a very large dictionary. Extracting any part of the file requires the data that came before it; essentially you must have the complete 7zip archive to extract data. A multipart 7zip archive is not independent files, the parts are just a binary split of the whole large archive file, and downloading later parts without the first part is useless. You must have all 7zipped compressed data in order to extract a verifiable bootstrap.dat.

A future torrent created with multi-part 7z files instead of one big uncompressed bootstrap.dat means that you could supplement your download or seeding by getting some of the torrent multi-part files off different websites where they might be hosted, and manually adding them to the torrent.

I am disappointed that 0.8.2 was released without a new checkpoint. Current block is 240709, I could force the issue by creating a block 0-240,000 bootstrap.dat torrent, if many would choose to adopt and seed such a torrent as a new "official" one. I'll see if Gavin is interested in picking a new checkpoint block yet with a code pull.

block 216116, it is 240000+ today

I am assuming the compressed bootstrap.dat differs in the headers (1st MB) and whatever is appended at the end.
So I guess you need to byte compare the first chunk and the last chunk to check my assumption?

great learn people it's ok to mix torrents with financial data ...

On side note I'l be adding better compressed .exe file to the list pretty soon. Auto extracts to correct location and all.