You don't have to download the pre-fork blockchain again for each Bitcoin fork!

If you are annoyed by having to download 150 GB of the blockchain data over and over again for each Bitcoin fork despite the fact that all the pre-fork blocks are naturally identical to each fork as well as to the original Bitcoin blockchain, there might be a solution for you. It might be possible to import the pre-fork blocks from the original Bitcoin block files or even from another fork's block files.

The original idea comes from here (which in turn is basically just a duplicated LoadExternalBlockFile() with a few tweaks), I merely made it universal and added command-line options. Unfortunately since most forks logically didn't use the exactly same revision of the original Bitcoin source code as their starting point there is no way how to make one single universal patch for all of them. From my experience of patching nearly 15 different forks I determined 2 different patch sets that (one or the other) after some little adjusting fit to all forks I have tried so far. If you compare the patches and look at your particular source code you will easily see which patch you should use for your particular fork, but either way it will most probably still need some (very little and very trivial) manual work. I will not help you with that but you can ask for a particular fork patch and if I already patched that fork I will just publish it.

Patchset 1:

Code:

--- a/src/init.cpp
+++ b/src/init.cpp
@@ -361,7 +361,16 @@ std::string HelpMessage(HelpMessageMode mode)
   strUsage += HelpMessageOpt("-dbcache=", strprintf(_("Set database cache size in megabytes (%d to %d, default: %d)"), nMinDbCache, nMaxDbCache, nDefaultDbCache));
   if (showDebug)
   strUsage += HelpMessageOpt("-feefilter", strprintf("Tell other nodes to filter invs to us by our mempool min fee (default: %u)", DEFAULT_FEEFILTER));
- strUsage += HelpMessageOpt("-loadblock=", _("Imports blocks from external blk000??.dat file on startup"));
+ strUsage += HelpMessageOpt("-loadblock=", _("Imports blocks from external blk000??.dat file(s) from the specified directory on startup"));
+ strUsage += HelpMessageOpt("-loadblockfirst=", _("The number of the first blk000??.dat file to import blocks from (default: 0)"));
+ strUsage += HelpMessageOpt("-loadblocklast=", _("The number of the last blk000??.dat file to import blocks from (default: 0)"));
+ strUsage += HelpMessageOpt("-importblocksbelow=", _("Import only blocks below from external file(s)"));
+ strUsage += HelpMessageOpt("-ignorecheckqueuefailed", _("Ignore 'ConnectBlock(): CheckQueue failed' error; use with extreme caution!"));
+ strUsage += HelpMessageOpt("-messagestart0=", strprintf("Magic byte 0 for external file(s) import (default: 0xf9)"));
+ strUsage += HelpMessageOpt("-messagestart1=", strprintf("Magic byte 1 for external file(s) import (default: 0xbe)"));
+ strUsage += HelpMessageOpt("-messagestart2=", strprintf("Magic byte 2 for external file(s) import (default: 0xb4)"));
+ strUsage += HelpMessageOpt("-messagestart3=", strprintf("Magic byte 3 for external file(s) import (default: 0xd9)"));
+ strUsage += HelpMessageOpt("-logwriteblockok", _("Log each successful block write"));
   strUsage += HelpMessageOpt("-maxorphantx=", strprintf(_("Keep at most unconnectable transactions in memory (default: %u)"), DEFAULT_MAX_ORPHAN_TRANSACTIONS));
   strUsage += HelpMessageOpt("-maxmempool=", strprintf(_("Keep the transaction memory pool below megabytes (default: %u)"), DEFAULT_MAX_MEMPOOL_SIZE));
   strUsage += HelpMessageOpt("-mempoolexpiry=", strprintf(_("Do not keep transactions in the mempool longer than hours (default: %u)"), DEFAULT_MEMPOOL_EXPIRY));
@@ -628,6 +637,7 @@ void CleanupBlockRevFiles()
   }
}

+extern bool LoadExternalBlockFileX(const CChainParams& chainparams, FILE* fileIn, CDiskBlockPos *dbp);
void ThreadImport(std::vector vImportFiles)
{
   const CChainParams& chainparams = Params();
@@ -672,13 +682,26 @@ void ThreadImport(std::vector vImportFiles)
   }

   // -loadblock=
+ unsigned char btcMagic[4];
+ btcMagic[0] = 0xf9; const std::string magic0 = gArgs.GetArg("-messagestart0", ""); if (IsHexNumber(magic0)) btcMagic[0] = (unsigned char)strtoul(magic0.c_str(), NULL, 16);
+ btcMagic[1] = 0xbe; const std::string magic1 = gArgs.GetArg("-messagestart1", ""); if (IsHexNumber(magic1)) btcMagic[1] = (unsigned char)strtoul(magic1.c_str(), NULL, 16);
+ btcMagic[2] = 0xb4; const std::string magic2 = gArgs.GetArg("-messagestart2", ""); if (IsHexNumber(magic2)) btcMagic[2] = (unsigned char)strtoul(magic2.c_str(), NULL, 16);
+ btcMagic[3] = 0xd9; const std::string magic3 = gArgs.GetArg("-messagestart3", ""); if (IsHexNumber(magic3)) btcMagic[3] = (unsigned char)strtoul(magic3.c_str(), NULL, 16);
   for (const fs::path& path : vImportFiles) {
- FILE *file = fsbridge::fopen(path, "rb");
- if (file) {
- LogPrintf("Importing blocks file %s...\n", path.string());
- LoadExternalBlockFile(chainparams, file);
- } else {
- LogPrintf("Warning: Could not open blocks file %s\n", path.string());
+ int nFile = gArgs.GetArg("-loadblockfirst", 0), nFilelast = gArgs.GetArg("-loadblocklast", 0);
+ while (true) {
+ if (nFile > nFilelast)
+ break;
+ std::string pathstr = strprintf("%s/blk%05u.dat", path.string(), nFile);
+ FILE *file = fsbridge::fopen(pathstr, "rb");
+ if (file) {
+ LogPrintf("Importing magic %#x %#x %#x %#x blocks from file %s...\n", btcMagic[0], btcMagic[1], btcMagic[2], btcMagic[3], pathstr);
+ LoadExternalBlockFileX(chainparams, file, NULL);
+ } else {
+ LogPrintf("Warning: Could not open blocks file %s\n", pathstr);
+ }
+
+ nFile++;
   }
   }

--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -1716,6 +1716,7 @@ static bool ConnectBlock(const CBlock& block, CValidationState& state, CBlockInd
   assert((pindex->phashBlock == nullptr) ||
   (*pindex->phashBlock == block.GetHash()));
   int64_t nTimeStart = GetTimeMicros();
+ bool ignorecheckqueuefailed = gArgs.GetBoolArg("-ignorecheckqueuefailed", false);

   // Check it again in case a previous version let a bad block in
   if (!CheckBlock(block, state, chainparams.GetConsensus(), !fJustCheck, !fJustCheck))
@@ -1892,8 +1893,12 @@ static bool ConnectBlock(const CBlock& block, CValidationState& state, CBlockInd
   block.vtx[0]->GetValueOut(), blockReward),
   REJECT_INVALID, "bad-cb-amount");

- if (!control.Wait())
- return state.DoS(100, error("%s: CheckQueue failed", __func__), REJECT_INVALID, "block-validation-failed");
+ if (!control.Wait()) {
+ if (ignorecheckqueuefailed)
+ LogPrintf("%s: CheckQueue failed\n", __func__);
+ else
+ return state.DoS(100, error("%s: CheckQueue failed", __func__), REJECT_INVALID, "block-validation-failed");
+ }
   int64_t nTime4 = GetTimeMicros(); nTimeVerify += nTime4 - nTime2;
   LogPrint(BCLog::BENCH, " - Verify %u txins: %.2fms (%.3fms/txin) [%.2fs]\n", nInputs - 1, 0.001 * (nTime4 - nTime2), nInputs <= 1 ? 0 : 0.001 * (nTime4 - nTime2) / (nInputs-1), nTimeVerify * 0.000001);

@@ -3232,6 +3237,7 @@ static bool AcceptBlock(const std::shared_ptr& pblock, CValidation

   CBlockIndex *pindexDummy = nullptr;
   CBlockIndex *&pindex = ppindex ? *ppindex : pindexDummy;
+ bool logwriteblockok = gArgs.GetBoolArg("-logwriteblockok", false);

   if (!AcceptBlockHeader(block, state, chainparams, &pindex))
   return false;
@@ -3293,9 +3299,12 @@ static bool AcceptBlock(const std::shared_ptr& pblock, CValidation
   blockPos = *dbp;
   if (!FindBlockPos(state, blockPos, nBlockSize+8, nHeight, block.GetBlockTime(), dbp != nullptr))
   return error("AcceptBlock(): FindBlockPos failed");
- if (dbp == nullptr)
+ if (dbp == nullptr) {
   if (!WriteBlockToDisk(block, blockPos, chainparams.MessageStart()))
   AbortNode(state, "Failed to write block");
+ else if (logwriteblockok)
+ LogPrintf("Write block ok\n");
+ }
   if (!ReceivedBlockTransactions(block, state, pindex, blockPos, chainparams.GetConsensus()))
   return error("AcceptBlock(): ReceivedBlockTransactions failed");
   } catch (const std::runtime_error& e) {
@@ -4079,6 +4088,127 @@ bool LoadGenesisBlock(const CChainParams& chainparams)
   return true;
}

+
+bool LoadExternalBlockFileX(const CChainParams& chainparams, FILE* fileIn, CDiskBlockPos *dbp)
+{
+ // Map of disk positions for blocks with unknown parent (only used for reindex)
+ static std::multimap mapBlocksUnknownParent;
+ int64_t nStart = GetTimeMillis();
+
+ unsigned char btcMagic[4];
+ btcMagic[0] = 0xf9; const std::string magic0 = gArgs.GetArg("-messagestart0", ""); if (IsHexNumber(magic0)) btcMagic[0] = (unsigned char)strtoul(magic0.c_str(), NULL, 16);
+ btcMagic[1] = 0xbe; const std::string magic1 = gArgs.GetArg("-messagestart1", ""); if (IsHexNumber(magic1)) btcMagic[1] = (unsigned char)strtoul(magic1.c_str(), NULL, 16);
+ btcMagic[2] = 0xb4; const std::string magic2 = gArgs.GetArg("-messagestart2", ""); if (IsHexNumber(magic2)) btcMagic[2] = (unsigned char)strtoul(magic2.c_str(), NULL, 16);
+ btcMagic[3] = 0xd9; const std::string magic3 = gArgs.GetArg("-messagestart3", ""); if (IsHexNumber(magic3)) btcMagic[3] = (unsigned char)strtoul(magic3.c_str(), NULL, 16);
+ int nLoaded = 0, blocknheight = 0, importblocksbelow = gArgs.GetArg("-importblocksbelow", 0);
+ try {
+ // This takes over fileIn and calls fclose() on it in the CBufferedFile destructor
+ CBufferedFile blkdat(fileIn, 2*MAX_BLOCK_SERIALIZED_SIZE, MAX_BLOCK_SERIALIZED_SIZE+8, SER_DISK, PROTOCOL_VERSION);
+ uint64_t nRewind = blkdat.GetPos();
+ while (!blkdat.eof()) {
+ boost::this_thread::interruption_point();
+
+ blkdat.SetPos(nRewind);
+ nRewind++; // start one byte further next time, in case of failure
+ blkdat.SetLimit(); // remove former limit
+ unsigned int nSize = 0;
+ try {
+ // locate a header
+ unsigned char buf[CMessageHeader::MESSAGE_START_SIZE];
+ blkdat.FindByte(btcMagic[0]);
+ nRewind = blkdat.GetPos()+1;
+ blkdat >> FLATDATA(buf);
+
+ if (memcmp(buf, btcMagic, CMessageHeader::MESSAGE_START_SIZE))
+ continue;
+
+ // read size
+ blkdat >> nSize;
+ if (nSize < 80 || nSize > MAX_BLOCK_SERIALIZED_SIZE)
+ continue;
+ } catch (const std::exception&) {
+ // no valid block header found; don't complain
+ LogPrintf("Import exception\n");
+ break;
+ }
+ try {
+ // read block
+ uint64_t nBlockPos = blkdat.GetPos();
+ if (dbp)
+ dbp->nPos = nBlockPos;
+ blkdat.SetLimit(nBlockPos + nSize);
+ blkdat.SetPos(nBlockPos);
+ std::shared_ptr pblock = std::make_shared();
+ CBlock& block = *pblock;
+ //block.new_format = false;
+ blkdat >> block;
+ //block >> blkdat;
+ nRewind = blkdat.GetPos();
+
+ LogPrint(BCLog::REINDEX, "version: %d hash: %s prev: %s\n",
+ block.nVersion,
+ block.GetHash().ToString(),
+ block.hashPrevBlock.ToString());
+
+ // detect out of order blocks, and store them for later
+ uint256 hash = block.GetHash();
+ if (hash != chainparams.GetConsensus().hashGenesisBlock && mapBlockIndex.find(block.hashPrevBlock) == mapBlockIndex.end()) {
+ LogPrint(BCLog::REINDEX, "%s: Out of order block %s, parent %s not known\n", __func__, hash.ToString(),
+ block.hashPrevBlock.ToString());
+ if (dbp)
+ mapBlocksUnknownParent.insert(std::make_pair(block.hashPrevBlock, *dbp));
+ continue;
+ }
+
+ if (mapBlockIndex.count(hash) == 0 || (mapBlockIndex[hash]->nStatus & BLOCK_HAVE_DATA) == 0) {
+ LOCK(cs_main);
+ CValidationState state;
+ LogPrint(BCLog::REINDEX, "Found prev hash in index\n");
+ BlockMap::iterator mi = mapBlockIndex.find(block.hashPrevBlock);
+
+ if (mi != mapBlockIndex.end()) {
+ assert(mi != mapBlockIndex.end());
+ const CBlockIndex *pindex = mi->second;
+ LogPrint(BCLog::REINDEX, "prev height %d\n", pindex->nHeight);
+ blocknheight = pindex->nHeight +1;
+ }
+
+ if (importblocksbelow > 0 && blocknheight >= importblocksbelow)
+ continue;
+ if (AcceptBlock(pblock, state, chainparams, nullptr, true, dbp, nullptr)) {
+ LogPrint(BCLog::REINDEX, "BLOCK ACCEPTEED\n");
+ nLoaded++;
+ } else
+ LogPrint(BCLog::REINDEX, "BLOCK REJECTED\n");
+ if (state.IsError())
+ break;
+ } else if (hash != chainparams.GetConsensus().hashGenesisBlock && mapBlockIndex[hash]->nHeight % 1000 == 0) {
+ LogPrint(BCLog::REINDEX, "Block Import: already had block %s at height %d\n", hash.ToString(), mapBlockIndex[hash]->nHeight);
+ }
+
+ // Activate the genesis block so normal node progress can continue
+ if (hash == chainparams.GetConsensus().hashGenesisBlock) {
+ LogPrint(BCLog::REINDEX, "Found Genesis Block\n" );
+ CValidationState state;
+ if (!ActivateBestChain(state, chainparams)) {
+ break;
+ }
+ }
+
+ NotifyHeaderTip();
+
+ } catch (const std::exception& e) {
+ LogPrintf("%s: Deserialize or I/O error - %s\n", __func__, e.what());
+ }
+ }
+ } catch (const std::runtime_error& e) {
+ AbortNode(std::string("System error: ") + e.what());
+ }
+ if (nLoaded > 0)
+ LogPrintf("Loaded %i blocks from external file in %dms\n", nLoaded, GetTimeMillis() - nStart);
+ return nLoaded > 0;
+}
+
bool LoadExternalBlockFile(const CChainParams& chainparams, FILE* fileIn, CDiskBlockPos *dbp)
{
   // Map of disk positions for blocks with unknown parent (only used for reindex)

Patchset 2:

Code:

--- a/src/init.cpp
+++ b/src/init.cpp
@@ -357,9 +357,16 @@ std::string HelpMessage(HelpMessageMode mode) {
   "-feefilter", strprintf("Tell other nodes to filter invs to us by "
   "our mempool min fee (default: %d)",
   DEFAULT_FEEFILTER));
- strUsage += HelpMessageOpt(
- "-loadblock=",
- _("Imports blocks from external blk000??.dat file on startup"));
+ strUsage += HelpMessageOpt("-loadblock=", _("Imports blocks from external blk000??.dat file(s) from the specified directory on startup"));
+ strUsage += HelpMessageOpt("-loadblockfirst=", _("The number of the first blk000??.dat file to import blocks from (default: 0)"));
+ strUsage += HelpMessageOpt("-loadblocklast=", _("The number of the last blk000??.dat file to import blocks from (default: 0)"));
+ strUsage += HelpMessageOpt("-importblocksbelow=", _("Import only blocks below from external file(s)"));
+ strUsage += HelpMessageOpt("-ignorecheckqueuefailed", _("Ignore 'ConnectBlock(): CheckQueue failed' error; use with extreme caution!"));
+ strUsage += HelpMessageOpt("-messagestart0=", strprintf("Magic byte 0 for external file(s) import (default: 0xf9)"));
+ strUsage += HelpMessageOpt("-messagestart1=", strprintf("Magic byte 1 for external file(s) import (default: 0xbe)"));
+ strUsage += HelpMessageOpt("-messagestart2=", strprintf("Magic byte 2 for external file(s) import (default: 0xb4)"));
+ strUsage += HelpMessageOpt("-messagestart3=", strprintf("Magic byte 3 for external file(s) import (default: 0xd9)"));
+ strUsage += HelpMessageOpt("-logwriteblockok", _("Log each successful block write"));
   strUsage += HelpMessageOpt(
   "-maxorphantx=", strprintf(_("Keep at most unconnectable "
   "transactions in memory (default: %u)"),
@@ -971,6 +978,7 @@ void CleanupBlockRevFiles() {
   }
}

+extern bool LoadExternalBlockFileX(const Config &config, FILE* fileIn, CDiskBlockPos *dbp);
void ThreadImport(const Config &config,
   std::vector vImportFiles) {
   RenameThread("bitcoin-loadblk");
@@ -1017,14 +1025,26 @@ void ThreadImport(const Config &config,
   }

   // -loadblock=
+ unsigned char btcMagic[4];
+ btcMagic[0] = 0xf9; const std::string magic0 = GetArg("-messagestart0", ""); if (IsHexNumber(magic0)) btcMagic[0] = (unsigned char)strtoul(magic0.c_str(), NULL, 16);
+ btcMagic[1] = 0xbe; const std::string magic1 = GetArg("-messagestart1", ""); if (IsHexNumber(magic1)) btcMagic[1] = (unsigned char)strtoul(magic1.c_str(), NULL, 16);
+ btcMagic[2] = 0xb4; const std::string magic2 = GetArg("-messagestart2", ""); if (IsHexNumber(magic2)) btcMagic[2] = (unsigned char)strtoul(magic2.c_str(), NULL, 16);
+ btcMagic[3] = 0xd9; const std::string magic3 = GetArg("-messagestart3", ""); if (IsHexNumber(magic3)) btcMagic[3] = (unsigned char)strtoul(magic3.c_str(), NULL, 16);
   for (const boost::filesystem::path &path : vImportFiles) {
- FILE *file = fopen(path.string().c_str(), "rb");
- if (file) {
- LogPrintf("Importing blocks file %s...\n", path.string());
- LoadExternalBlockFile(config, file);
- } else {
- LogPrintf("Warning: Could not open blocks file %s\n",
- path.string());
+ int nFile = GetArg("-loadblockfirst", 0), nFilelast = GetArg("-loadblocklast", 0);
+ while (true) {
+ if (nFile > nFilelast)
+ break;
+ std::string pathstr = strprintf("%s/blk%05u.dat", path.string(), nFile);
+ FILE *file = fopen(pathstr.c_str(), "rb");
+ if (file) {
+ LogPrintf("Importing magic %#x %#x %#x %#x blocks from file %s...\n", btcMagic[0], btcMagic[1], btcMagic[2], btcMagic[3], pathstr);
+ LoadExternalBlockFileX(config, file, NULL);
+ } else {
+ LogPrintf("Warning: Could not open blocks file %s\n", pathstr);
+ }
+
+ nFile++;
   }
   }

--- a/src/utilstrencodings.cpp
+++ b/src/utilstrencodings.cpp
@@ -61,6 +61,19 @@ bool IsHex(const std::string &str) {
   return (str.size() > 0) && (str.size() % 2 == 0);
}

+bool IsHexNumber(const std::string& str)
+{
+ size_t starting_location = 0;
+ if (str.size() > 2 && *str.begin() == '0' && *(str.begin()+1) == 'x') {
+ starting_location = 2;
+ }
+ for (auto c : str.substr(starting_location)) {
+ if (HexDigit(c) < 0) return false;
+ }
+ // Return false for empty string or "0x".
+ return (str.size() > starting_location);
+}
+
std::vector ParseHex(const char *psz) {
   // convert hex dump to vector
   std::vector vch;
--- a/src/utilstrencodings.h
+++ b/src/utilstrencodings.h
@@ -41,6 +41,7 @@ std::vector ParseHex(const char *psz);
std::vector ParseHex(const std::string &str);
signed char HexDigit(char c);
bool IsHex(const std::string &str);
+bool IsHexNumber(const std::string& str);
std::vector DecodeBase64(const char *p, bool *pfInvalid = nullptr);
std::string DecodeBase64(const std::string &str);
std::string EncodeBase64(const uint8_t *pch, size_t len);
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -1872,6 +1872,7 @@ static bool ConnectBlock(const Config &config, const CBlock &block,
   AssertLockHeld(cs_main);

   int64_t nTimeStart = GetTimeMicros();
+ bool ignorecheckqueuefailed = GetBoolArg("-ignorecheckqueuefailed", false);

   // Check it again in case a previous version let a bad block in
   if (!CheckBlock(config, block, state, !fJustCheck, !fJustCheck)) {
@@ -2118,8 +2119,10 @@ static bool ConnectBlock(const Config &config, const CBlock &block,
   }

   if (!control.Wait()) {
- return state.DoS(100, false, REJECT_INVALID, "blk-bad-inputs", false,
- "parallel script check failed");
+ if (ignorecheckqueuefailed)
+ LogPrintf("%s: CheckQueue failed\n", __func__);
+ else
+ return state.DoS(100, error("%s: CheckQueue failed", __func__), REJECT_INVALID, "block-validation-failed");
   }

   int64_t nTime4 = GetTimeMicros();
@@ -3622,6 +3625,7 @@ static bool AcceptBlock(const Config &config,

   CBlockIndex *pindexDummy = nullptr;
   CBlockIndex *&pindex = ppindex ? *ppindex : pindexDummy;
+ bool logwriteblockok = GetBoolArg("-logwriteblockok", false);

   if (!AcceptBlockHeader(config, block, state, &pindex)) {
   return false;
@@ -3710,9 +3714,10 @@ static bool AcceptBlock(const Config &config,
   return error("AcceptBlock(): FindBlockPos failed");
   }
   if (dbp == nullptr) {
- if (!WriteBlockToDisk(block, blockPos, chainparams.DiskMagic())) {
+ if (!WriteBlockToDisk(block, blockPos, chainparams.DiskMagic()))
   AbortNode(state, "Failed to write block");
- }
+ else if (logwriteblockok)
+ LogPrintf("Write block ok\n");
   }
   if (!ReceivedBlockTransactions(block, state, pindex, blockPos)) {
   return error("AcceptBlock(): ReceivedBlockTransactions failed");
@@ -4454,6 +4459,127 @@ bool InitBlockIndex(const Config &config) {
   return true;
}

+bool LoadExternalBlockFileX(const Config &config, FILE* fileIn, CDiskBlockPos *dbp)
+{
+ // Map of disk positions for blocks with unknown parent (only used for reindex)
+ static std::multimap mapBlocksUnknownParent;
+ int64_t nStart = GetTimeMillis();
+ const CChainParams &chainparams = config.GetChainParams();
+
+ unsigned char btcMagic[4];
+ btcMagic[0] = 0xf9; const std::string magic0 = GetArg("-messagestart0", ""); if (IsHexNumber(magic0)) btcMagic[0] = (unsigned char)strtoul(magic0.c_str(), NULL, 16);
+ btcMagic[1] = 0xbe; const std::string magic1 = GetArg("-messagestart1", ""); if (IsHexNumber(magic1)) btcMagic[1] = (unsigned char)strtoul(magic1.c_str(), NULL, 16);
+ btcMagic[2] = 0xb4; const std::string magic2 = GetArg("-messagestart2", ""); if (IsHexNumber(magic2)) btcMagic[2] = (unsigned char)strtoul(magic2.c_str(), NULL, 16);
+ btcMagic[3] = 0xd9; const std::string magic3 = GetArg("-messagestart3", ""); if (IsHexNumber(magic3)) btcMagic[3] = (unsigned char)strtoul(magic3.c_str(), NULL, 16);
+ int nLoaded = 0, blocknheight = 0, importblocksbelow = GetArg("-importblocksbelow", 0);
+ try {
+ // This takes over fileIn and calls fclose() on it in the CBufferedFile destructor
+ CBufferedFile blkdat(fileIn, 2*MAX_TX_SIZE, MAX_TX_SIZE+8, SER_DISK, PROTOCOL_VERSION);
+ uint64_t nRewind = blkdat.GetPos();
+ while (!blkdat.eof()) {
+ boost::this_thread::interruption_point();
+
+ blkdat.SetPos(nRewind);
+ nRewind++; // start one byte further next time, in case of failure
+ blkdat.SetLimit(); // remove former limit
+ unsigned int nSize = 0;
+ try {
+ // locate a header
+ unsigned char buf[CMessageHeader::MESSAGE_START_SIZE];
+ blkdat.FindByte(btcMagic[0]);
+ nRewind = blkdat.GetPos()+1;
+ blkdat >> FLATDATA(buf);
+
+ if (memcmp(buf, btcMagic, CMessageHeader::MESSAGE_START_SIZE))
+ continue;
+
+ // read size
+ blkdat >> nSize;
+ if (nSize < 80 || nSize > MAX_TX_SIZE)
+ continue;
+ } catch (const std::exception&) {
+ // no valid block header found; don't complain
+ LogPrintf("Import exception\n");
+ break;
+ }
+ try {
+ // read block
+ uint64_t nBlockPos = blkdat.GetPos();
+ if (dbp)
+ dbp->nPos = nBlockPos;
+ blkdat.SetLimit(nBlockPos + nSize);
+ blkdat.SetPos(nBlockPos);
+ std::shared_ptr pblock = std::make_shared();
+ CBlock& block = *pblock;
+ //block.new_format = false;
+ blkdat >> block;
+ //block >> blkdat;
+ nRewind = blkdat.GetPos();
+
+ LogPrintf("version: %d hash: %s prev: %s\n",
+ block.nVersion,
+ block.GetHash().ToString(),
+ block.hashPrevBlock.ToString());
+
+ // detect out of order blocks, and store them for later
+ uint256 hash = block.GetHash();
+ if (hash != chainparams.GetConsensus().hashGenesisBlock && mapBlockIndex.find(block.hashPrevBlock) == mapBlockIndex.end()) {
+ LogPrintf("%s: Out of order block %s, parent %s not known\n", __func__, hash.ToString(),
+ block.hashPrevBlock.ToString());
+ if (dbp)
+ mapBlocksUnknownParent.insert(std::make_pair(block.hashPrevBlock, *dbp));
+ continue;
+ }
+
+ if (mapBlockIndex.count(hash) == 0 || (mapBlockIndex[hash]->nStatus & BLOCK_HAVE_DATA) == 0) {
+ LOCK(cs_main);
+ CValidationState state;
+ LogPrintf("Found prev hash in index\n");
+ BlockMap::iterator mi = mapBlockIndex.find(block.hashPrevBlock);
+
+ if (mi != mapBlockIndex.end()) {
+ assert(mi != mapBlockIndex.end());
+ const CBlockIndex *pindex = mi->second;
+ LogPrintf("prev height %d\n", pindex->nHeight);
+ blocknheight = pindex->nHeight +1;
+ }
+
+ if (importblocksbelow > 0 && blocknheight >= importblocksbelow)
+ continue;
+ if (AcceptBlock(config, pblock, state, nullptr, true, dbp, nullptr)) {
+ LogPrintf("BLOCK ACCEPTEED\n");
+ nLoaded++;
+ } else
+ LogPrintf("BLOCK REJECTED\n");
+ if (state.IsError())
+ break;
+ } else if (hash != chainparams.GetConsensus().hashGenesisBlock && mapBlockIndex[hash]->nHeight % 1000 == 0) {
+ LogPrintf("Block Import: already had block %s at height %d\n", hash.ToString(), mapBlockIndex[hash]->nHeight);
+ }
+
+ // Activate the genesis block so normal node progress can continue
+ if (hash == chainparams.GetConsensus().hashGenesisBlock) {
+ LogPrintf("Found Genesis Block\n" );
+ CValidationState state;
+ if (!ActivateBestChain(config, state)) {
+ break;
+ }
+ }
+
+ NotifyHeaderTip();
+
+ } catch (const std::exception& e) {
+ LogPrintf("%s: Deserialize or I/O error - %s\n", __func__, e.what());
+ }
+ }
+ } catch (const std::runtime_error& e) {
+ AbortNode(std::string("System error: ") + e.what());
+ }
+ if (nLoaded > 0)
+ LogPrintf("Loaded %i blocks from external file in %dms\n", nLoaded, GetTimeMillis() - nStart);
+ return nLoaded > 0;
+}
+
bool LoadExternalBlockFile(const Config &config, FILE *fileIn,
   CDiskBlockPos *dbp) {
   // Map of disk positions for blocks with unknown parent (only used for

What this does is basically taking the blocks data from the block files on your harddrive and feeding them to the fork's own AcceptBlock() function. So it's not like the blocks are somehow illegitimately injected bypassing verification or any other test. If a block is accepted this way it's an identical situation as if the block was downloaded from other nodes via the Internet.

New/changed command-line options are as follows:

-loadblock=
This option was already there but allowed to import only one single block file and from only the same signature blockchain. Now it has a different meaning - with it you specify the "blocks" directory that contains blocks you want to import, e.g. -loadblock=~/.bitcoin/blocks.

-loadblockfirst=
The number of the first blk000??.dat file that you want to import from, default is 0. It's ok to repeatedly start from 0 and try to import from files you already imported from, if the block is already in blockchain it will just be ignored.

-loadblocklast=
The number of the last blk000??.dat file that you want to import from, default is 0. It's ok to specify too high number (but above some 1200 or so will probably be pointless for any current fork), if a file is not found the loop will just continue with the next file.

-importblocksbelow=
Using this option will ensure that only blocks below given number are imported. This number is typically the fork number.

-ignorecheckqueuefailed
It happened to me with one fork that I was importing blocks from the original Bitcoin and all of a sudden in the middle I got this strange 'ConnectBlock(): CheckQueue failed' error that prevented the import from continuing. I found out that if I actually ignored this error the block would then get imported successfully. So in my particular case this turned out to be a fake error. Since I knew that the blocks I was trying to import were correct and I wasn't willing to spend hours or days trying to fix somebody else's bug I implemented this option. However you should use it really as the last resort when you are stuck and only if you understand the implications.

-messagestart0=
-messagestart1=
-messagestart2=
-messagestart3=
With these options you can specify the signature (magic bytes) of the blockchain from which you are trying to import blocks. This in theory might allow you to import pre-fork blocks from any fork into any fork. The original Bitcoin blockchain signature is 0xf9 0xbe 0xb4 0xd9 which is the default. What signature is used in the particular fork which you want to import from you can find out from the definition of pchMessageStart in src/chainparams.cpp.

-logwriteblockok
An extra log of each successful WriteBlockToDisk().

Before you start importing the blocks it's advisable to first run the node and let it connect to other (fork's native) nodes and sync the headers. Once it starts downloading the actual blocks you can exit it and try this method. The import is then in two phases - first all the blocks are imported and then they are indexed. It happened to me in one case that not all blocks were imported all in one go and I had to rerun the import again (either set the -loadblockfirst appropriately or just let it go through all the files again, it's quite fast) so you may try that, too.

It's also worth noting that since all pre-fork blockchain data is identical (but not necessarily the actual blocks files are binary identical, though, so you shouldn't just cross-link them) you can very effectively use a deduplication feature of the filesystem that supports it and save nearly all the disk space. Since deduplication is not Bitcoin (forks) specific I will not go into details, I can just confirm that it works very well on both Linux (Ubuntu - zfs) and Windows (8.1 - ntfs).

Topic: You don't have to download the pre-fork blockchain again for each Bitcoin fork! (Read 193 times)