How to automatically create a bootstrap.dat file on a node

CoinFascination

newbie

Activity: 74

Merit: 0

Quote from: fronti on March 12, 2015, 04:37:32 PM

Quote from: gmaxwell on March 12, 2015, 02:27:08 PM

Though with 0.10+ the marginal utility of a separate bootstrap.dat is low-- if you include the time it takes to download one it often slows down your synchronization time now.

there are other bitcoin daemons then bitcoin-core out there where a bootstrap.dat still might be usefull

Does this script have to be run when the daemon is off?

When I use the lastest linearize code from bitcoin on a dash fork, it complains about the magic number:
Invalid magic: 00000000

This is after blk2, there is no blk3

When I use the dash version:
"Premature end of block data" (after it lists blk3)

I can't figure out the problem.

I am running it with the daemon running.

fronti

legendary

Activity: 2912

Merit: 1309

Quote from: gmaxwell on March 12, 2015, 02:27:08 PM

Though with 0.10+ the marginal utility of a separate bootstrap.dat is low-- if you include the time it takes to download one it often slows down your synchronization time now.

there are other bitcoin daemons then bitcoin-core out there where a bootstrap.dat still might be usefull

EcuaMobi

legendary

Activity: 1876

Merit: 1475

Quote from: siameze on March 12, 2015, 02:48:27 PM

Quote from: gmaxwell on March 12, 2015, 02:27:08 PM

This will not (reliably) create a valid bootstrap.dat.

A valid bootstrap.dat contains no orphans (block files contain orphans) and will have all the blocks in order (blocks in block files may be out of order).

There is a script included with bitcoin core called linearize.py that is used to create bootstrap.dats.

Hmmm, I was unaware of this one (linearize.py). I have seeded a updated bootstrap.dat torrent as often as possible though I agree the speed increase of using it is negligible.

I wasn't aware of this either. I will definitely check it, thanks gmaxwell.

Originally I wrote this script for an altcoin without the latest features for bitcoin-qt, I changed it for Bitcoin to make it more general,
but you're right probably that's not required for Bitcoin 0.10.0 anymore.

siameze

legendary

Activity: 1064

Merit: 1000

Quote from: gmaxwell on March 12, 2015, 02:27:08 PM

This will not (reliably) create a valid bootstrap.dat.

A valid bootstrap.dat contains no orphans (block files contain orphans) and will have all the blocks in order (blocks in block files may be out of order).

There is a script included with bitcoin core called linearize.py that is used to create bootstrap.dats.

Hmmm, I was unaware of this one (linearize.py). I have seeded a updated bootstrap.dat torrent as often as possible though I agree the speed increase of using it is negligible.

gmaxwell

staff

Activity: 4284

Merit: 8808

This will not (reliably) create a valid bootstrap.dat.

A valid bootstrap.dat contains no orphans (block files contain orphans) and will have all the blocks in order (blocks in block files may be out of order).

There is a script included with bitcoin core called linearize.py that is used to create bootstrap.dats.

Also, blocks themselves are self authenticating, there is no need to 'verify' as you've described.

Though with 0.10+ the marginal utility of a separate bootstrap.dat is low-- if you include the time it takes to download one it often slows down your synchronization time now.

EcuaMobi

legendary

Activity: 1876

Merit: 1475

If you are running a full node of Bitcoin (or any other crypto-currency) you can easily provide a bootstrap.dat file and help anyone to sync faster the wallet.

I've written a simple bash script that does that:

Quote

# Go to the coin’s data directory
cd ~/.Bitcoin
# Find all the block files with the command "ls -1 blk[0-9]*.dat"
# Use the sort command to make sure they’re in order
# Use xargs to prevent an "Argument list too long" error
ls -1 blk[0-9]*.dat | sort | xargs cat > bootstrap.dat
# Create a file including the current date and the checksum of the file
echo "bootstrap.dat file, created at `date`" > bootstrap-sum.txt
echo "sha256sum: `sha256sum bootstrap.dat | awk ‘{print $1}’`" >> bootstrap-sum.txt
echo "md5sum: `md5sum bootstrap.dat | awk ‘{print $1}’`" >> bootstrap-sum.txt
echo "Verify before using it!" >> bootstrap-sum.txt
# Optionally, compress the file. You need to install zip for this
zip bootstrap.zip bootstrap.dat
# Remove the original file
rm -f bootstrap.dat
# Finally, move the resulting files to a public directory
# It’s recommended to move bootstrap-sum.txt to another server for security
mv bootstrap.zip /var/www/captcoin.com
sftp … bootstrap-sum.txt

Here's the full post:
https://ecua.mobi/how-to-create-bootstrap-dat/

I hope this will be useful to somebody.

Topic: How to automatically create a bootstrap.dat file on a node (Read 1768 times)