Pages:
Author

Topic: What are checkpoints in bitcoin code? (Read 14814 times)

legendary
Activity: 1232
Merit: 1094
April 10, 2014, 06:06:00 AM
#93
I'm having a hard time figuring out why checkpoints do not provide any security ....

They have positives and negatives.

A checkpoint means that miners won't jump to a secret fork.

Miners who didn't checkpoint will still jump and so will some clients if they were offline when the checkpoint triggered.

This means you split the network into 2 pieces if a major fork happens.
newbie
Activity: 2
Merit: 0
April 10, 2014, 05:19:57 AM
#92
who makes the checkpoints?
The bitcoin core devs
so ultimately, the state of the block chain depends entirely on them.
not really. they only put checkpoint for blocks that are buried at least few hundreds deep already - unlikely to be reverted by honest means.
but if you disagree with any checkpoint they add, just don't upgrade your node and you will be fine.

well that's a practical solution, but it violates the notion of 'peer-to-peer' or equality for all nodes in the network.

there is no indication of 'checkpoints' in the satoshi whitepaper.

furthermore, if we rely on such checkpoints for security, why do we even have proof of work at all?  Why don't we just rely on the checkpoints?

I'm having a hard time figuring out what you mean by "equality of all nodes".  The equality (or inequality) of nodes is not part of the system.  No one cares about nodes.  We care about the chain.  If a node has the correct chain and can give you a copy, it is "equal" to all of the other good nodes.  If it doesn't, or can't, then it equal at all, isn't even a node.

The checkpoints do not provide any security, nor were they ever intended to.  They just keep "unequal" nodes from wasting your time with bogus chains (whether by accident or malice).

Also, you are free to edit and recompile as needed.  You can strip them out entirely, or add your own, or change them.  Whatever you want.  The reference client is written for (and compiled for) working well for a wide audience.

Technically, a longer chain is the best chain, regardless of the checkpoints.  In practice, the checkpoints are far enough in the past that if a new chain showed up that overturned one, we'd almost universally consider it an attack, and roll back.

P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.


I'm having a hard time figuring out why checkpoints do not provide any security ....
legendary
Activity: 1232
Merit: 1094
April 10, 2014, 04:55:06 AM
#91
How about following rule:
when a block is generated, the block 120 blocks deep the chain is checkpointed, *if*
- current hashing power is more than half of the to-be-checkpointed block

How do you measure current hashing power?

Do you mean difficulty must be at least 50% of what it was 120 blocks ago?  That isn't much protection since difficulty doesn't change very often.

Quote
Does this guarantee solving problems with forked chains? Only the chain is checkpointed which has majority of the hashing power.

The problem with checkpointing is what happens if there is a reversal.

For example, if I launch a 51% attack starting from behind the checkpoint, then all new nodes will follow my chain.

Nodes that were offline will also follow my chain.

Nodes that were online would follow the other chain.

This forks the network and it can't heal itself.

I think sending the entire header tree would help here.  A new node that connects would see that there was a 120+ block fork.  It could warn the user that the chain is potentially unstable at the moment.

Nodes would be able to see that there was a fork that would have caused a checkpoint.

Quote
Still thinking good formula to estimate if the hashing power is more than half.. both blocks difficulty and time should be considered. And maybe some safety reserve, so variations on block times does not affect the result.

Including block time is risky, since it creates an incentive to mess with the field.  An attacker could set it to whatever he likes (but not to far in the future).

Faking high hash rate means earlier block times, so the future rule doesn't apply.  If I produce a sequence of blocks with timestamps 1 second apart, it will seem like a massive hash rate.

Quote
Edit: when a new soft checkpoint is made, the previous one can be forgotten, ie there need not to be more than one soft checkpoint at a time.

If they are both on the same chain, then keeping the old one has no effect one way or another.

What do you mean by soft?  The rule could be that a fork 120 blocks longer can displace a checkpoint.  This would at least allow the network to re-sync.

Clients could enter emergency mode using this rule.

- find longest chain (L)
- find any other chain which
-- forked more than 120 blocks ago
-- where the fork is at least 120 blocks long
-- has a weight of at least (L's weight - 120 blocks)

The hard checkpoints also speed up validation.  They could do that without acting as checkpoints.

Validation would start at the last hard checkpoint that is on the longest chain.

If the entire chain was displaced by a higher POW chain, then nodes would accept the new chain.  They would have to verify it in its entirety.
sr. member
Activity: 477
Merit: 500
April 10, 2014, 01:13:02 AM
#90
I'm quite sure this has been discussed before, but why clients does not make automatic checkpoints? I understand that transactions which are at least 6 blocks old, are considered valid. There should be no reason to re-arrange the blockchain after that, for any reason. If 6 blocks sounds too short, maybe 120 blocks then? Coinbase coins should definitely be stoned when they are spendable.


I still consider that since coinbase transactions are agreed by the network to be spendable, that means they are agreed to be existent to foreseeable future and thus should be checkpointed.

How about following rule:
when a block is generated, the block 120 blocks deep the chain is checkpointed, *if*
- current hashing power is more than half of the to-be-checkpointed block

Not sure if this checkpointing even need to be broadcasted; every client should be in full agreement of it.

Then, I also suggest that coinbase transactions *are not spendable* unless they are checkpointed.

Does this guarantee solving problems with forked chains? Only the chain is checkpointed which has majority of the hashing power.
Still thinking good formula to estimate if the hashing power is more than half.. both blocks difficulty and time should be considered. And maybe some safety reserve, so variations on block times does not affect the result.

Does this open any attack vectors?


Edit: when a new soft checkpoint is made, the previous one can be forgotten, ie there need not to be more than one soft checkpoint at a time.
newbie
Activity: 11
Merit: 0
April 09, 2014, 09:25:42 AM
#89
amazing checkpoint,still cant quitely follow
full member
Activity: 173
Merit: 100
March 10, 2014, 11:43:15 PM
#88
who makes the checkpoints?
The bitcoin core devs
so ultimately, the state of the block chain depends entirely on them.
not really. they only put checkpoint for blocks that are buried at least few hundreds deep already - unlikely to be reverted by honest means.
but if you disagree with any checkpoint they add, just don't upgrade your node and you will be fine.

well that's a practical solution, but it violates the notion of 'peer-to-peer' or equality for all nodes in the network.

there is no indication of 'checkpoints' in the satoshi whitepaper.

furthermore, if we rely on such checkpoints for security, why do we even have proof of work at all?  Why don't we just rely on the checkpoints?

Being peer-to-peer also means that everyone in the network can choose to use whatever client or whatever version of a certain client in order to be part of the Bitcoin network. As of now, we don't have too many options for running a full node, but in the future, more independent implementations will likely to emerge.
kjj
legendary
Activity: 1302
Merit: 1026
March 10, 2014, 11:40:31 PM
#87
The main issue is that in the event of a flaw which causes the network to fork you have now hard coded that fork.

For example if the db bug issue which caused a fork in the network, miners on the longer fork modified their blockchains to support the shorter fork.  So some intervention was required but for all 100,000+ other nodes they simply saw a re-org once the the shorter fork became the longer one.  If they used ratcheting checkpoints they would have been permanently locked onto a minority fork and attackers could have exploited that.

There may be some merit in using a ratcheting checkpoint system however it would have to be set much much deeper than 6 blocks.  Even 144 blocks (one day) may be insufficient.  Something like 1,024 blocks (~1 week) is more the range that would be conservative enough.

Also understand a ratcheing checkpoint system only helps existing nodes.  Bootstraping nodes would be remain vulnerable as they haven't seen the longest chain yet.

Do you mean this:
https://bitcoin.org/en/alert/2013-03-11-chain-fork

But it would have been solved simply by bootstrapping the blockchain along with the software downgrade. If it needs software update, you allways have the option to force block reload, maybe from some hardcoded checkpoint.

In that case, whoever controls the update system controls everything.
donator
Activity: 1218
Merit: 1079
Gerald Davis
March 10, 2014, 03:12:42 PM
#86
But it would have been solved simply by bootstrapping the blockchain along with the software downgrade. If it needs software update, you allways have the option to force block reload, maybe from some hardcoded checkpoint.

That is my point and until 100% of nodes upgrade their software they would be vulnerable to an isolation attack.  It creates a more fragile and exploitable system than users wallets simply being able to switch to the longest chain with no action required on the part of the user.
sr. member
Activity: 477
Merit: 500
March 10, 2014, 03:10:19 PM
#85
The main issue is that in the event of a flaw which causes the network to fork you have now hard coded that fork.

For example if the db bug issue which caused a fork in the network, miners on the longer fork modified their blockchains to support the shorter fork.  So some intervention was required but for all 100,000+ other nodes they simply saw a re-org once the the shorter fork became the longer one.  If they used ratcheting checkpoints they would have been permanently locked onto a minority fork and attackers could have exploited that.

There may be some merit in using a ratcheting checkpoint system however it would have to be set much much deeper than 6 blocks.  Even 144 blocks (one day) may be insufficient.  Something like 1,024 blocks (~1 week) is more the range that would be conservative enough.

Also understand a ratcheing checkpoint system only helps existing nodes.  Bootstraping nodes would be remain vulnerable as they haven't seen the longest chain yet.

Do you mean this:
https://bitcoin.org/en/alert/2013-03-11-chain-fork

But it would have been solved simply by bootstrapping the blockchain along with the software downgrade. If it needs software update, you allways have the option to force block reload, maybe from some hardcoded checkpoint.
donator
Activity: 1218
Merit: 1079
Gerald Davis
March 10, 2014, 02:07:37 PM
#84
The main issue is that in the event of a flaw which causes the network to fork you have now hard coded that fork.

For example if the db bug issue which caused a fork in the network, miners on the longer fork modified their blockchains to support the shorter fork.  So some intervention was required but for all 100,000+ other nodes they simply saw a re-org once the the shorter fork became the longer one.  If they used ratcheting checkpoints they would have been permanently locked onto a minority fork and attackers could have exploited that.

There may be some merit in using a ratcheting checkpoint system however it would have to be set much much deeper than 6 blocks.  Even 144 blocks (one day) may be insufficient.  Something like 1,024 blocks (~1 week) is more the range that would be conservative enough.

Also understand a ratcheing checkpoint system only helps existing nodes.  Bootstraping nodes would be remain vulnerable as they haven't seen the longest chain yet.
sr. member
Activity: 477
Merit: 500
March 10, 2014, 02:00:08 PM
#83
I'm quite sure this has been discussed before, but why clients does not make automatic checkpoints? I understand that transactions which are at least 6 blocks old, are considered valid. There should be no reason to re-arrange the blockchain after that, for any reason. If 6 blocks sounds too short, maybe 120 blocks then? Coinbase coins should definitely be stoned when they are spendable.

So I'd like to hear what is the reasoning behind the fact that there are no automatic checkpoints?

Let's *try* to imagine situations where automatic checkpoint (here I mean every node considers it chain's blocks from head-6 and earlier to be 'stoned', non-changeable) would cause harm:

1. Branch which will survive 6 blocks. Isn't this practically an error case? If it can happen, that just means 6 block validation time is too short since it cannot guarantee transactions confirmed.
2. Some node mines isolated from the others. This is the case where it would cause minor problem. However the problem is very local, and very similar problem would be regardless of the automatic checkpoint.

Basicly it would just be a non-centralised agreement between nodes after they find a new head block; we all consider blocks backwards from this-6 nodes to be immutable.
legendary
Activity: 1232
Merit: 1094
December 21, 2013, 08:14:19 AM
#82
I should look closer to your proposal.

It's not my idea, the OP in that thread thought of it.

Quote
For now I have one question:
Does "up" link points to future? If so - is it included in block hash or can it be falsified?

No, they always points backwards.

To determine where to point the up link.

- count the number of leading zeros on the previous block's hash (i.e. the block you plan to build on top of)
- starting from the block before that, find the first block with more leading zeros
- point to the block one after that

So, if the blocks had leading zeros

A: 11(up = Genesis)
B: 10 (up = Genesis)
C: 20 (up = B)   (Not both prev and up point at B)
D: 10 (up = Genesis)
E: 10 (up = D)
F: 10 (up = D)
G: 10 (up = D)
H: 15 (up = D)
I: 10 (up = D)
J: 10 (up = I)
K: 10 (up = I)

The for block L, the links would be

prev: K
up: I

The reason is that H has more leading zeros than K.  You don't link to C, even though it has more leading zeros.  Once you find that block, the up link points to the one after that.

So, finding the best block

stating at K, follow up link, until up link points to genesis
- follow up link to get to I (prev now point at H, so found block with 15 leading zeros)
- follow up link to get to D (prev now points at C, so found block with 20 leading zeros)
- up for D link points to genesis, so break from loop

follow D's previous link to get C which is the best block

You can also find all blocks with more than n leading zeros by following the previous link any time you find a block with more than that number of leading zeros.
kjj
legendary
Activity: 1302
Merit: 1026
December 21, 2013, 02:16:28 AM
#81
Do you understand that this option is protocol specification change? It effectively forks blockchain.

Heh.  This is the sort of nonsense I was talking about.

This is local control.  You, the node operator, have options.  Hardly a new thing.  Since day 1, all node operators have had the ability to fork for whatever reason and through whatever means they desire.

You have a warped view of "decentralized".  It doesn't mean that no one has any authority, it means that no one can stop you.

Who can stop you from turning checkpoints off?  No one.
Who can stop you from stripping them out of the source entirely and running your own pure node?  No one.
newbie
Activity: 28
Merit: 0
December 21, 2013, 01:20:20 AM
#80
Of course hidden fork crunched at high speed in shadow is unlikely, and maybe checkpoints do not harm due to low probability of such fork.
But they harm in other way - they compromise Bitcoin decentralization, which is in my opinion is the biggest attractive feature of Bitcoin.
Today they add checkpoints (this effectively changes protocol specification), tomorrow they could freeze BTC on your address, by adding small tweaks to validation routine (just like checkpoints validation).

Nonsense, every word.

Unbeatable arguments, thank you.

Quote
root@linuxcoin:/home/user# bitcoind --help
Bitcoin version v0.8.6.0-g03a7d67-beta

  -checkpoints           Only accept block chain matching built-in checkpoints (default: 1)

Do you understand that this option is protocol specification change? It effectively forks blockchain.
newbie
Activity: 28
Merit: 0
December 21, 2013, 01:11:47 AM
#79
In a way, bitcoin (and almost all other crypto coins) are still some what centralized. The devs are the centralization. There's not much we can do about that.

Yes, they are centralization. But as long as they do not adjust blockchain on regular basis (select which chain is good and which is not) - they gain trust of users.

The redeeming thing is that almost all of them are open source with code available to be self-compiled.

Yes, but the problem is that the client with checkpoints and one without - are not 100% compatible with each other. Checkpoints are not just cosmetic change.

While the protocol itself is decentralized, it is a requirement that honest nodes form a consensus. Remember the hard fork early this year.

Bitcoin already has mechanism to achieve consensus - voting by computing resources, it should not be substituted by any centralized authority.
kjj
legendary
Activity: 1302
Merit: 1026
December 21, 2013, 12:40:27 AM
#78
  -addtag=           Add a tag to your coinbase
  -txindex               Maintain a full transaction index (default: 0)

What do those two do? I'm guessing the tag is meant for miners? Is it useful for regular people who just run a node or wallet?

Since I just run QT like normal, I don't get the full transaction index? Does that take up additional space or isn't everything in the blockchain anyway? Or I am correct to guess that this is better used by large online wallets or something?

Heh, oops, -addtag is a custom mod that I maintain.  It does indeed add tags to the coinbase for mining.

-txindex makes the node keep an index of all transactions.  This is very helpful for RPC services, particularly those using raw transactions.
legendary
Activity: 3416
Merit: 1912
The Concierge of Crypto
December 20, 2013, 11:05:16 PM
#77
  -addtag=           Add a tag to your coinbase
  -txindex               Maintain a full transaction index (default: 0)

What do those two do? I'm guessing the tag is meant for miners? Is it useful for regular people who just run a node or wallet?

Since I just run QT like normal, I don't get the full transaction index? Does that take up additional space or isn't everything in the blockchain anyway? Or I am correct to guess that this is better used by large online wallets or something?
kjj
legendary
Activity: 1302
Merit: 1026
December 20, 2013, 10:04:57 PM
#76
I think the checkpoints are about at least a month or two back in time. Anyone who has more than 51% of 6.5 Petahash of mining power that can fork their own chain past the latest check point, is better off mining regularly and getting 51% of 3600 coins daily or about 1800 coins, at least for the next 2000 blocks when the difficulty will double.

Of course hidden fork crunched at high speed in shadow is unlikely, and maybe checkpoints do not harm due to low probability of such fork.
But they harm in other way - they compromise Bitcoin decentralization, which is in my opinion is the biggest attractive feature of Bitcoin.
Today they add checkpoints (this effectively changes protocol specification), tomorrow they could freeze BTC on your address, by adding small tweaks to validation routine (just like checkpoints validation).

Nonsense, every word.

Quote
root@linuxcoin:/home/user# bitcoind --help
Bitcoin version v0.8.6.0-g03a7d67-beta

Usage:
  bitcoind [options]
  bitcoind [options] [params]  Send command to -server or bitcoind
  bitcoind [options] help                List commands
  bitcoind [options] help       Get help for a command

Options:
  -?                     This help message
  -conf=           Specify configuration file (default: bitcoin.conf)
  -pid=            Specify pid file (default: bitcoind.pid)
  -gen                   Generate coins (default: 0)
  -datadir=         Specify data directory
  -dbcache=           Set database cache size in megabytes (default: 25)
  -timeout=           Specify connection timeout in milliseconds (default: 5000)
  -proxy=       Connect through socks proxy
  -socks=             Select the version of socks proxy to use (4-5, default: 5)
  -tor=         Use proxy to reach tor hidden services (default: same as -proxy)
  -dns                   Allow DNS lookups for -addnode, -seednode and -connect
  -port=           Listen for connections on (default: 8333 or testnet: 18333)
  -maxconnections=    Maintain at most connections to peers (default: 125)
  -addnode=          Add a node to connect to and attempt to keep the connection open
  -connect=          Connect only to the specified node(s)
  -seednode=         Connect to a node to retrieve peer addresses, and disconnect
  -externalip=       Specify your own public address
  -onlynet=         Only connect to nodes in network (IPv4, IPv6 or Tor)
  -discover              Discover own IP address (default: 1 when listening and no -externalip)
  -checkpoints           Only accept block chain matching built-in checkpoints (default: 1)
  -listen                Accept connections from outside (default: 1 if no -proxy or -connect)
  -bind=           Bind to given address and always listen on it. Use [host]:port notation for IPv6
  -dnsseed               Find peers using DNS lookup (default: 1 unless -connect)
  -banscore=          Threshold for disconnecting misbehaving peers (default: 100)
  -bantime=           Number of seconds to keep misbehaving peers from reconnecting (default: 86400)
  -maxreceivebuffer=  Maximum per-connection receive buffer, *1000 bytes (default: 5000)
  -maxsendbuffer=     Maximum per-connection send buffer, *1000 bytes (default: 1000)
  -paytxfee=        Fee per KB to add to transactions you send
  -daemon                Run in the background as a daemon and accept commands
  -testnet               Use the test network
  -debug                 Output extra debugging information. Implies all other -debug* options
  -debugnet              Output extra network debugging information
  -logtimestamps         Prepend debug output with timestamp (default: 1)
  -shrinkdebugfile       Shrink debug.log file on client startup (default: 1 when no -debug)
  -printtoconsole        Send trace/debug info to console instead of debug.log file
  -rpcuser=        Username for JSON-RPC connections
  -rpcpassword=      Password for JSON-RPC connections
  -rpcport=        Listen for JSON-RPC connections on (default: 8332 or testnet: 18332)
  -rpcallowip=       Allow JSON-RPC connections from specified IP address
  -rpcconnect=       Send commands to node running on (default: 127.0.0.1)
  -rpcthreads=        Set the number of threads to service RPC calls (default: 4)
  -blocknotify=     Execute command when the best block changes (%s in cmd is replaced by block hash)
  -walletnotify=    Execute command when a wallet transaction changes (%s in cmd is replaced by TxID)
  -alertnotify=     Execute command when a relevant alert is received (%s in cmd is replaced by message)
  -upgradewallet         Upgrade wallet to latest format
  -keypool=           Set key pool size to (default: 100)
  -rescan                Rescan the block chain for missing wallet transactions
  -addtag=           Add a tag to your coinbase
  -salvagewallet         Attempt to recover private keys from a corrupt wallet.dat
  -checkblocks=       How many blocks to check at startup (default: 288, 0 = all)
  -checklevel=        How thorough the block verification is (0-4, default: 3)
  -txindex               Maintain a full transaction index (default: 0)
  -loadblock=      Imports blocks from external blk000??.dat file
  -reindex               Rebuild block chain index from current blk000??.dat files
  -par=               Set the number of script verification threads (up to 16, 0 = auto, <0 = leave that many cores free, default: 0)
legendary
Activity: 3416
Merit: 1912
The Concierge of Crypto
December 20, 2013, 10:01:00 PM
#75
In a way, bitcoin (and almost all other crypto coins) are still some what centralized. The devs are the centralization. There's not much we can do about that.

The redeeming thing is that almost all of them are open source with code available to be self-compiled.

If it were not check points, there is also the bootstrap file uploaded by Jeff Garzik. That is sort of centralized, but distributed by torrent, for all new users who need an initial sync of the correct block chain.

While the protocol itself is decentralized, it is a requirement that honest nodes form a consensus. Remember the hard fork early this year.
newbie
Activity: 28
Merit: 0
December 20, 2013, 09:51:56 PM
#74
I think the checkpoints are about at least a month or two back in time. Anyone who has more than 51% of 6.5 Petahash of mining power that can fork their own chain past the latest check point, is better off mining regularly and getting 51% of 3600 coins daily or about 1800 coins, at least for the next 2000 blocks when the difficulty will double.

Of course hidden fork crunched at high speed in shadow is unlikely, and maybe checkpoints do not harm due to low probability of such fork.
But they harm in other way - they compromise Bitcoin decentralization, which is in my opinion is the biggest attractive feature of Bitcoin.
Today they add checkpoints (this effectively changes protocol specification), tomorrow they could freeze BTC on your address, by adding small tweaks to validation routine (just like checkpoints validation).
Pages:
Jump to: