[ANN] Bitcoin blockchain data torrent - page 22.

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: jgarzik on October 12, 2012, 12:01:57 PM

This is special to our use case: bootstrap.dat is essentially an append-only file. Blocks are simply concatenated onto the end.

((...snip...))

The next torrent, a few months from now, will have the same first 2,491,771,562 bytes.

Thus, to bittorrent, the next torrent will simply appear to be a truncated / not fully downloaded bootstrap.dat. Bittorrent is built to fill in the missing pieces of a file, so that is what it does here

I just copied that over from the beta thread

... wanted to point out that it DID work for me

(I was able to use the older/smaller/earlier bootstrap.dat, and then my bittorrent client was able to smartly "fix" the new torrent... like...
the smaller file is basically the same & I only needed to download the missing end. This procedure results in the same updated bootstrap.dat)

... it should continue to work for the next new/larger/updated torrent just the same.

thanks jgarzik, you rock

jonathan

member

Activity: 84

Merit: 14

same story on my box. This is the torrent that never sleeps lately.

streblo

full member

Activity: 165

Merit: 100

Quote from: cypherdoc on March 27, 2013, 04:16:17 PM

i'll just say that i've been maintaining a few torrent sites since Jeff started this thread. activity was initially zilch.

in the last couple of weeks the upload bandwidth has skyrocketed.

Link?

cypherdoc

legendary

Activity: 1764

Merit: 1002

i'll just say that i've been maintaining a few torrent sites since Jeff started this thread. activity was initially zilch.

in the last couple of weeks the upload bandwidth has skyrocketed.

streblo

full member

Activity: 165

Merit: 100

I'm sure this has been discussed before, but I couldn't find it: Why isn't a (moderately up-to-date) bootstrap.dat (optionally) included with the offical bitcoind/bitcoin-qt binaries? The trust model isn't invalidated because (a) you're downloading binaries and (b) the bootstrap's data are verified. Sourceforge already hosts the blockchain. It seems it would remove a lot of redundant data transfer from the bitcoin network, no?

jgarzik

legendary

Activity: 1596

Merit: 1100

Torrent file now also available via bitcoin's SourceForge page:

https://sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/

mrlithium

newbie

Activity: 46

Merit: 0

Can this be added to the sourceforge blockchain project sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/ or update that somehow?
It took me forever to find this thread.

neofutur

full member

Activity: 146

Merit: 100

Quote from: Scrat Acorns on February 20, 2013, 11:00:17 AM

Direct download mirror: https://bitfetch.com/static/bootstrap.7z
(compressed using 7z/LZMA2 - 2.7 vs. 4.7 GiB)

I also added a download on :

http://fh.gd/blockchain/

you ll find the torrent, the bootstrap.dat and the md5sum, and the same bootstrap.dat gzipped

I ll try to update it whenever a new torrent and checkpoint comes out
( also seeding the torrent )

OP: perhaps you should setup a mailing list for mirrors, so you can ping us all when a new version of the torrent have to be downloaded ?

Quote

If you can seed, use the torrent instead.

same ! prefer the torrent !

mirror sponsored by http://bitcoin.gw.gd/

jomplox

member

Activity: 90

Merit: 10

Thanks for your work OP. I'm seeding the new torrent and will be doing so for at least a year at the very least.

deepceleron

legendary

Activity: 1512

Merit: 1036

Quote from: coblee on February 21, 2013, 03:21:14 AM

I'm running this on a Mac if that makes a difference. Seems like 0.8 is buggy.

Seems like your computer is sick.

Run Linpack for a few hours, your computer will likely give inconsistent results, crash, hit 99 degrees C, throttle cores.

http://software.intel.com/en-us/articles/intel-math-kernel-library-linpack-download

A forum for it

Or run this Windows software: IntelBurnTest

coblee

donator

Activity: 1654

Merit: 1351

Creator of Litecoin. Cryptocurrency enthusiast.

I've tried a few times and couldn't get this to work. First time, I just put the bootstrap.dat file there and tried to run it against my current blocks. It didn't like that and got stuck.

The next time, I deleted all my block data and tried it again. This time it got stuck at block 211551 (iirc) and wouldn't load any block after that. Looking at the debug.log, I see that my peers are trying to give me blocks higher than that but my client keeps rejecting them for some reason.

The third time, I tried the same thing again. This time, when it got to block 155088, it gave me a database corrupt message and the client quit. Restarting the client does the same thing:
init message: Importing blocks from block database...
LevelDB read failure: Corruption: block checksum mismatch

I'm running this on a Mac if that makes a difference. Seems like 0.8 is buggy.

dooglus

legendary

Activity: 2940

Merit: 1333

Quote from: jgarzik on February 21, 2013, 12:12:05 AM

In theory, someone could write a script that processes bitcoind's $DATADIR/blocks directory into bootstrap.dat, but no one has written that yet.

They have now...

https://gist.github.com/dooglus/5002764

It uses the Armory code to do all the work, and creates a bootstrap.dat with the correct sha256 hash:

$ sha256sum bootstrap.dat
bf658c7055b733bfc15ea167f298c5599b89d220b14dbe7c8ef20b18e468c451 bootstrap.dat

jgarzik

legendary

Activity: 1596

Merit: 1100

Quote from: dooglus on February 21, 2013, 12:03:36 AM

I have an up-to-date copy of the blockchain. Is there an easy way to generate bootstrap.dat from it so I can seed it without having to download the whole thing again?

This script is used to generate bootstrap.dat:
https://github.com/jgarzik/pynode/blob/master/mkbootstrap.py

It requires an up-to-date pynode chain database.

In theory, someone could write a script that processes bitcoind's $DATADIR/blocks directory into bootstrap.dat, but no one has written that yet.

dooglus

legendary

Activity: 2940

Merit: 1333

I have an up-to-date copy of the blockchain. Is there an easy way to generate bootstrap.dat from it so I can seed it without having to download the whole thing again?

I'm thinking it might be relatively easy to modify the armory source code to dump out a bootstrap.dat file, but won't bother if something already exists.

deepceleron

legendary

Activity: 1512

Merit: 1036

Quote from: jgarzik on February 20, 2013, 02:53:52 PM

Quote from: coblee on February 20, 2013, 12:57:56 PM

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

This is not a replacement for block data you already have. This is a separate file, that is only used once, during data import.

After data import, the file is renamed to bootstrap.dat.old, and never touched again. Most will want to manually delete it, after use.

More specifically answering the question, if your Bitcoin client is already close to block 216116 (January 11) or even halfway completed (block 190000), the bootstrap.dat likely won't be of much use to you. It may take more time to download and then import the torrent than to complete the blockchain synchronization normally. This torrent is also only for those using the v0.8.0 release or after, as 0.7.1-0.7.2 are not able to import more than 2.0GiB of the 4.7GiB file.

You can use the bootstrap.dat torrent with no ill effects regardless of your current block. When you restart and Bitcoin sees the bootstrap.dat that you have dropped in it's data directory, any block hashes you already have will be skipped quickly, and the import will pick up at the blocks you don't have.

jgarzik

legendary

Activity: 1596

Merit: 1100

Quote from: coblee on February 20, 2013, 12:57:56 PM

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

This is not a replacement for block data you already have. This is a separate file, that is only used once, during data import.

After data import, the file is renamed to bootstrap.dat.old, and never touched again. Most will want to manually delete it, after use.

Sukrim

legendary

Activity: 2618

Merit: 1007

How to find the data directory (where this file belongs to):
https://en.bitcoin.it/wiki/Data_directory (<-- this also needs a bit of updating for the different files/folders with 0.8)

Some more hash values of this bootstrap.dat file (up to 216116):
Adler32: FA0EEF4C
CRC32: DDF59E6B
MD5: 1B437D44213B7D98C974546B55834D10
SHA-1: 985380032B618281EC5B9B49AFDCC99A4BCD69EB
SHA-256: BF658C7055B733BFC15EA167F298C5599B89D220B14DBE7C8EF20B18E468C451

Unfortunately it's not possible (yet?) in Bittorrent to share HTTP seeds in a swarm (DDoS potential!), but still - if you have a web server to seed this file but cannot run Bittorrent, you might still post a link here, so you can be added to the Magnet URI/torrent file.
HTTP seeds are in most clients actually quite rarely used, so unless people start to direct download from these you might not even have heavy traffic going on.

Also if you cannot use Bittorrent, there are services around that download torrents for BTC for you and offer a HTTPS download link. I won't link these here as I don't want to make it look like I endorse any of these.

mc_lovin

legendary

Activity: 1190

Merit: 1000

www.bitcointrading.com

This is a great idea, although I am a bit confused as to the future as we won't all need to keep a full node anymore, but I'm sure that will come into understanding in the coming weeks.

Seeding this as much as possible!

BitcoinBug

full member

Activity: 196

Merit: 100

I was seeding the previous version (2.3 GB, share ratio 15) and when I added this one, it picked up and verified the existing version and continued downloading from 51%. Very cool!

coblee

donator

Activity: 1654

Merit: 1351

Creator of Litecoin. Cryptocurrency enthusiast.

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

Topic: [ANN] Bitcoin blockchain data torrent - page 22. (Read 211194 times)