srinikethan, I think your expectations are unrealistic.
It takes a long time-- tens of seconds to hash the UTXO set on a fast host. No random peers are just going to do this for you today. (Incrementally updating a tree structured hash could be done, but only with a considerable on disk and in memory overhead.)3
Even if they do, anyone can spin up as many nodes as they want, simply asking other nodes and then refusing to run if they disagree is just a DOS vector.
If, instead, you ban ones that disagree with you this just guarantees you'll only be connected to the party that gave you the bad data.
So what has this accomplished?
This also has ignored my point that format of these databases is likely not secure against malicious input, and might well result in remote code execution for the party that gave them to you.
As far as machine specific random values go-- yes that could probably be done, though then your node would stop working when you upgraded hardware/software and it would get in the way of copying state between systems which you control where there is no trust issue.
Topic: Pasting untrusted blocks/ and chainstate/ to new pruned node safe? (Read 1111 times)
February 21, 2016, 05:16:56 PM
February 20, 2016, 11:59:51 AM
Actual result:
Core trusts index and chainstate blindly
Expected results:
Core loads the pruned index and chainstate
Core catches up
Core compares the utxo - hash with other network nodes
If some other node has different database hash the node will fail to start
EDIT: or ban the liar node
Alternative expected result:
Core loads the pruned index and chainstate
Core catches up
User is notified to add configuration option that the user is aware the chain might be untrusted
The value of the option could be dependent on some kind of hash derived from latest block or machine specific random uuid
This will prevent user to download the configuration file from untrusted source because attacker cannot predict this value
Core trusts index and chainstate blindly
Expected results:
Core loads the pruned index and chainstate
Core catches up
Core compares the utxo - hash with other network nodes
If some other node has different database hash the node will fail to start
EDIT: or ban the liar node
Alternative expected result:
Core loads the pruned index and chainstate
Core catches up
User is notified to add configuration option that the user is aware the chain might be untrusted
The value of the option could be dependent on some kind of hash derived from latest block or machine specific random uuid
This will prevent user to download the configuration file from untrusted source because attacker cannot predict this value
February 15, 2016, 06:59:31 PM
Beyond the issue of potential problems in the data itself-- there has been no auditing or effort to make sure these raw database files are safe for malicious input. (In the case of the bdb wallet files, I'm reasonably confident that they are not safe.)
Running a database (chainstate, txindex, blockindex) or wallet from an untrusted third party might result in giving them arbitrary code execution on your system.
The loadblock raw blockfile imports, otoh, are handled the same way as data from the network and are designed to handle untrusted input.
Running a database (chainstate, txindex, blockindex) or wallet from an untrusted third party might result in giving them arbitrary code execution on your system.
The loadblock raw blockfile imports, otoh, are handled the same way as data from the network and are designed to handle untrusted input.
February 15, 2016, 07:20:15 AM
Presumably for those blocks prior to the last checkpoint it is verifying that the block hash matches the actual content though - correct?
Exactly, if you trust the checkpoint, then you can trust the blocks. Having said that, it does still check everything else. It makes sure every transaction is spending money that actually exists etc.
February 15, 2016, 12:18:34 AM
It is never safe to use untrusted copies of those directories. When someone gives you an already-made block database, it's possible for them to manipulate it so that they can fork you off of the real chain at any time or do various other nasty things. By default Bitcoin Core does no checking of info in the database except for some basic checks of the last few blocks. Checkpoints are not rechecked. And even Bitcoin Core's checkblocks feature won't find all possible traps.
The database files are cross-platform, but you should copy them only if you trust the source absolutely. Also, the files are all intimately related and different per node, so it won't work to copy just some files -- it needs to be an all-or-nothing copy.
P.S. ｗａｔａｓｈｉ－ｋｏｋｏｔｏのアブターはとてもかわいい。
I wasn't advocating using someone else's copy...The database files are cross-platform, but you should copy them only if you trust the source absolutely. Also, the files are all intimately related and different per node, so it won't work to copy just some files -- it needs to be an all-or-nothing copy.
P.S. ｗａｔａｓｈｉ－ｋｏｋｏｔｏのアブターはとてもかわいい。
僕のは可愛くないの
February 14, 2016, 10:53:43 PM
It is never safe to use untrusted copies of those directories. When someone gives you an already-made block database, it's possible for them to manipulate it so that they can fork you off of the real chain at any time or do various other nasty things. By default Bitcoin Core does no checking of info in the database except for some basic checks of the last few blocks. Checkpoints are not rechecked. And even Bitcoin Core's checkblocks feature won't find all possible traps.
The database files are cross-platform, but you should copy them only if you trust the source absolutely. Also, the files are all intimately related and different per node, so it won't work to copy just some files -- it needs to be an all-or-nothing copy.
P.S. ｗａｔａｓｈｉ－ｋｏｋｏｔｏのアブターはとてもかわいい。
The database files are cross-platform, but you should copy them only if you trust the source absolutely. Also, the files are all intimately related and different per node, so it won't work to copy just some files -- it needs to be an all-or-nothing copy.
P.S. ｗａｔａｓｈｉ－ｋｏｋｏｔｏのアブターはとてもかわいい。
February 14, 2016, 10:21:58 PM
If you have pruned a copy of the blockchain and then shut down your bitcoind, you can safely copy the pruned contents of the .bitcoin directory across and use the pruned directory as is. I have done so many times over without problem.
February 14, 2016, 10:04:37 PM
The "checkpoints" don't come into this at all?
You cannot keep checkpoints and still do the signature checks. Presumably for those blocks prior to the last checkpoint it is verifying that the block hash matches the actual content though - correct?
(skipping the signature verification being an optimisation permitted by using checkpoints)
February 14, 2016, 03:25:09 PM
The "checkpoints" don't come into this at all?
The original question was just copying the /block and /chainstate directories. That is not safe, since there is no checking.
If you use the bootstrap method, then the node reads the blockchain from the bootstrap file as if it was downloaded from the internet. All blocks are checked as normal.
You can also use the -loadblock command.
bitcoind -loadblock=blk00001.dat -loadblock=blk00002.dat ....
I think you have to give them in order.
If you add a block called bootstrap.dat into the data directory, then it is automatically used for bootstrapping.
Quote
(I haven't checked the source code myself but have heard conflicting things about the validation of "old blocks")
The checkpoints have 2 effects. If a block at any of the checkpointed heights has the wrong hash, then it is rejected. This means that once a block is checkpointed, core will not accept any chain that doesn't include that block.
The second effect is with signature validation. Core doesn't check signatures until it hits the final checkpoint.
In 0.12, the last checkpointed block is at height 295000. This means that core doesn't check signatures until it hits block 295001. From them on, it checks signatures. The code is here.
You cannot keep checkpoints and still do the signature checks.
February 14, 2016, 01:36:18 PM
The "checkpoints" don't come into this at all?
(I haven't checked the source code myself but have heard conflicting things about the validation of "old blocks")
(I haven't checked the source code myself but have heard conflicting things about the validation of "old blocks")
February 14, 2016, 01:28:19 PM
Using a bootstrap file with the blockchain is safe. Pruning nodes process the entire blockchain and verify everything. Once they have processed a block, they don't have to keep the block data. In practice, they keep the most recent blocks (and always at least 2 days).
You can't use any chainstate data from the internet (UTXO and wallet data). Pruning nodes assume that the chainstate they have is correct, since they generated it while processing the block data.
[Edit]
The bootstrap file is generally a single file for bootstrapping from the internet. It isn't the directory. If the directory comes from a pruning node, then it won't have the entire blockchain anyway.
You can't use any chainstate data from the internet (UTXO and wallet data). Pruning nodes assume that the chainstate they have is correct, since they generated it while processing the block data.
[Edit]
The bootstrap file is generally a single file for bootstrapping from the internet. It isn't the directory. If the directory comes from a pruning node, then it won't have the entire blockchain anyway.
February 14, 2016, 12:42:14 PM
I ask if it's safe to download untrusted blocks and chainstate from the internet and paste it to a pruning Bitcoin Core as a quick-start method?
Does Core validate the UTXO?
Edit: Is the chain cross platform?
Does Core validate the UTXO?
Edit: Is the chain cross platform?
Jump to: