Pages:
Author

Topic: What are checkpoints in bitcoin code? - page 4. (Read 14814 times)

newbie
Activity: 28
Merit: 0
August 25, 2013, 10:35:15 PM
#33
Instead of checkpoints, client could hard-code a MAIN_CHAIN_MIN_POW variable.  This would be the PoW of the main chain when that version of the client was released.  All clients wouldn't need to agree on a value.  It is just a spam protection value.

Clients should download headers first and only commit blocks to disk that are on a chain that has PoW higher than the hard coded minimum.

Clients should respond to "inv" messages containing unknown block hashes by sending a "getheaders" message rather than asking for the block.

If the block extends your main chain, you will be sent it.

Peer sends: "inv" containing unknown block
Send to peer: "getheaders" with standard block locator (send at most once per inv message received)
Peer sends: "headers" with new blocks on it's main chain
Add these headers to header tree
If the main chain is extended and the main chain has PoW greater than MAIN_CHAIN_MIN_POW, request those full-blocks
...
No matter what happens, the longest chain will always have more PoW than the longest one when the client was writen, so the best chain is guaranteed to have more PoW than MAIN_CHAIN_MIN_POW.

The client won't consider itself synced unless it is on a main chain with PoW that exceeds that value.

And then what if multiple chains are above this min pow?
legendary
Activity: 1232
Merit: 1094
August 25, 2013, 06:38:17 PM
#32
Thats not really what they're for— though that have the effect too. Most of their usefulness is that they prevent a dos attacker from filling up bitcoin node's disk space with long runs of low difficulty blocks forked off low in the chain.

e.g. you start off with difficulty 1 blocks at block 0, now mine-able by the millions by a single asic— _MAYBE_ a chain that starts off that way could eventually turn out to be the longest so absent the checkpoints a node would happily follow an endlessly long chain of them.

Instead of checkpoints, client could hard-code a MAIN_CHAIN_MIN_POW variable.  This would be the PoW of the main chain when that version of the client was released.  All clients wouldn't need to agree on a value.  It is just a spam protection value.

Clients should download headers first and only commit blocks to disk that are on a chain that has PoW higher than the hard coded minimum.

Clients should respond to "inv" messages containing unknown block hashes by sending a "getheaders" message rather than asking for the block.

If the block extends your main chain, you will be sent it.

Peer sends: "inv" containing unknown block
Send to peer: "getheaders" with standard block locator (send at most once per inv message received)
Peer sends: "headers" with new blocks on it's main chain
Add these headers to header tree
If the main chain is extended and the main chain has PoW greater than MAIN_CHAIN_MIN_POW, request those full-blocks

Quote
They also make is so that an attacker who has complete control of your network (and thus can prevent you from hearing the longest chain from the honest bitcoin network) from putting you on a fake (low difficulty) isolated chain unless they can also trick you into running replaced software. With the checkpoints such an attacker hast to have a ton of mining power in order to continue the chain.

No matter what happens, the longest chain will always have more PoW than the longest one when the client was writen, so the best chain is guaranteed to have more PoW than MAIN_CHAIN_MIN_POW.

The client won't consider itself synced unless it is on a main chain with PoW that exceeds that value.
kjj
legendary
Activity: 1302
Merit: 1026
June 17, 2013, 03:21:50 PM
#31
Your solution is indeed simple.  But it doesn't do the same job that checkpoints do.  Checkpoints are not about forks in the present or future, they are about helping clients avoid dead end forks in the past, while they are doing their initial sync.  Checkpoints help new nodes catch up to the network with minimal annoyance for the node operator, nothing more*.
So you are saying that checkpoints don't actually prevent my node from downloading, storing and routing an alternative block that would refer to the genesis?
Or are you saying that they do prevent it, but the important thing here is that I should care more about those who are just downloading the chain and dont know yet which branch to choose? Smiley

Well, the genesis hash was coded in from day one.  If you want an alternate chain, you need a new genesis hash, plus you'd have to clear out the checkpoints.  (My apologies if that wasn't what you meant, I'm not entirely sure that I understand you.)

The early blocks had very low difficulty.  If I wanted to, I could generate thousands of low difficulty blocks, with appropriate fake timestamps, and I could set my node to wait for new connections, then feed those bogus blocks to the noob.  They'd figure it out eventually, but I could keep them mired down for hours or days, working on my garbage.  The checkpoints give a handy shortcut to that "eventually".  If my chain doesn't include the checkpoint blocks, the new node won't waste their time on it.

The notion of "the correct chain" gets fuzzy.  If I have a lot of hashing power, like more than the rest of the world, and I start today making a new chain, but I keep my farm isolated from the rest of the network, is it "the correct chain"?  The only really hard rule is that the longer (more work, not necessarily more blocks) chain is correct.  My chain would be (work) longer, so in that sense, it is correct.  But no one knows about it but me, so the world would be working on a chain that they think is correct.

When I release my chain, it will overturn possibly thousands of blocks, possibly millions of transactions.  The hard rule says that it should happen, but almost no one would actually wants that outcome.  A checkpoint could invalidate my whole chain, if a big part of the network upgrades to a client that includes a checkpoint that happens to come after I started.

That situation is unrealistic, intentionally.  But it shows that the simple rule that we can easily write into our software is incomplete.  We humans want the rule to be "the longer chain is correct, except...", and the "except" part is hard to write.  We know that checkpoints aren't the right way to codify the exception.  If you've ever talked to Gavin, or even read his posts on the subject, you'll know that he absolutely hates that he could end up in the position of picking which chain wins.

So, we end up in the strange situation we are in.  The checkpoints are useful in the real world , but we can see that they also hold a power that no one actually wants to wield.  The devs advance them now and then, but always sorta grudgingly, and always months behind.  If someone released a 5000 block attack chain, we'd all be bitching that the checkpoint was too far behind.  Because no one has (and probably never will) hide a fork of that length, we are all bitching that checkpoints are contrary to the bitcoin design.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 01:40:58 PM
#29
Your solution is indeed simple.  But it doesn't do the same job that checkpoints do.  Checkpoints are not about forks in the present or future, they are about helping clients avoid dead end forks in the past, while they are doing their initial sync.  Checkpoints help new nodes catch up to the network with minimal annoyance for the node operator, nothing more*.
So you are saying that checkpoints don't actually prevent my node from downloading, storing and routing an alternative block that would refer to the genesis?
Or are you saying that they do prevent it, but the important thing here is that I should care more about those who are just downloading the chain and dont know yet which branch to choose? Smiley
kjj
legendary
Activity: 1302
Merit: 1026
June 17, 2013, 01:35:06 PM
#28
I'm glad you managed to dig up and read my proposal for exponential difficulty reorgs, and that you find it superior.  Not surprising, considering that yours is a degenerate case of mine where the exponent is infinite.

But it really isn't a checkpoint system.  The checkpoint system is all about the past, while both your proposal and mine are useless in that domain.
I wish I did, but as you figured, I didn't. Smiley
I'm a man of simple solutions.
Checkpoints that are there now, in the bitcoin client, are stupid.
My idea - may be simple, but at least it would do what it has to do and close the problem once and for all. And it's surely much simpler.
Your idea - I believe it's impressive, but why to complicate things that are simple and do the job?

Your solution is indeed simple.  But it doesn't do the same job that checkpoints do.  Checkpoints are not about forks in the present or future, they are about helping clients avoid dead end forks in the past, while they are doing their initial sync.  Checkpoints help new nodes catch up to the network with minimal annoyance for the node operator, nothing more*.

The real replacement for checkpoints is a smarter sync system.  Greg is working on a smarter sync (see here), but I don't think it has been implemented or tested yet, so for now, we are stuck with the dumb downloader and checkpoints as a sanity check.

* They can also be used to rectify a fork in the network, but this is an extraordinary usage that requires a lot of manual intervention by a lot of node operators.  When a signed/unsigned bug was found in the value interpretation code and someone used it to give himself billions of bitcoins, a checkpoint was used to help the corrected chain overtake the errant chain that had grown for several hours before the fix could be applied.  It was also used to unroll the more recent BDB/blocksize fork.  But again, these were not checkpoints already built into the code, we just used the same mechanism as a manually override.
hero member
Activity: 490
Merit: 501
June 17, 2013, 01:22:38 PM
#27
if you disagree with the use of checkpoints the just fork 'em.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 01:07:40 PM
#26
I'm glad you managed to dig up and read my proposal for exponential difficulty reorgs, and that you find it superior.  Not surprising, considering that yours is a degenerate case of mine where the exponent is infinite.

But it really isn't a checkpoint system.  The checkpoint system is all about the past, while both your proposal and mine are useless in that domain.
I wish I did, but as you figured, I didn't. Smiley
I'm a man of simple solutions.
Checkpoints that are there now, in the bitcoin client, are stupid.
My idea - may be simple, but at least it would do what it has to do and close the problem once and for all. And it's surely much simpler.
Your idea - I believe it's impressive, but why to complicate things that are simple and do the job?
kjj
legendary
Activity: 1302
Merit: 1026
June 17, 2013, 01:04:35 PM
#25
P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.

No, that sucks.  Sorry.
Yeah, I'm sure your checkpoints idea is much better.
I'm actually even ashamed mentioning something such stupid Smiley

I'm glad you managed to dig up and read my proposal for exponential difficulty reorgs, and that you find it superior.  Not surprising, considering that yours is a degenerate case of mine where the exponent is infinite.

But it really isn't a checkpoint system.  The checkpoint system is all about the past, while both your proposal and mine are useless in that domain.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 12:50:07 PM
#24
P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.

No, that sucks.  Sorry.
Yeah, I'm sure your checkpoints idea is much better.
I'm actually even ashamed mentioning something such stupid Smiley
kjj
legendary
Activity: 1302
Merit: 1026
June 17, 2013, 12:48:57 PM
#23
P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.

No, that sucks.  Sorry.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 12:46:08 PM
#22
Do you understand that checkpoints completely violate the entire purpose of Bitcoin?
I understand that they violate some purpose of bitcoin.
But I am also a pragmatic person and so I prefer to have a system that violates some very unlikely to happen purposes, as long as it is a way to keep my resources away from spammers.

What they do is introduce a form of approval(consensus) about the state of a chain.  This form of authority could certainly be abused, and it runs counter to practically every design principle of Bitcoin.
But your concerns are 100% theoretical and it isn't really possible to start a fork like 2000 blocks back and get with it to the top.
And even if it was possible, would you really want it? Because I would not.

Quote
at the most fundamental level, the client code should not have ANYTHING to say about the block chain.  Whether this is required in order to operate is irrelevant.  The reason people use Bitcoin is *because* it's peer-to-peer.
Again: the node, and the network, must protect itself from DoS attacks.
What are you even trying to achieve forbidding people doing this?

If you one day manage to create a branch that would beat their chosen checkpoint - then I will agree with you.
But so far, I don't get your problem, though I appreciate your innocent ideas. Smiley
sr. member
Activity: 280
Merit: 257
bluemeanie
June 17, 2013, 12:33:02 PM
#21
I think you don't understand.
Checkpoints are not there to fight proof of work - they are there to protect your node from DoS attacks.

Do you understand that if there were no checkpoints, anyone with a graphic card could just keep mining blocks that link to the genesis one, and this way fill up your hard disk?


Do you understand that checkpoints completely violate the entire purpose of Bitcoin?

What they do is introduce a form of approval(consensus) about the state of a chain.  This form of authority could certainly be abused, and it runs counter to practically every design principle of Bitcoin.

at the most fundamental level, the client code should not have ANYTHING to say about the block chain.  Whether this is required in order to operate is irrelevant.  The reason people use Bitcoin is *because* it's peer-to-peer.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 12:30:11 PM
#20
I think you don't understand.
Checkpoints are not there to fight proof of work - they are there to protect your node from attacks.

Do you understand that if there were no checkpoints, anyone with a graphic card could just keep mining blocks that link to the genesis one, and this way fill up your hard disk? Also stuffing the net with useless data.
sr. member
Activity: 280
Merit: 257
bluemeanie
June 17, 2013, 12:26:22 PM
#19
P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really god solution, if you want Wink

Just set a dynamic limit in that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever you want, but I think more than a week time would be exaggerating.


the fact is, if we've introduced a form of consensus OUTSIDE of proof-of-work, that overrides Proof-Of-Work, and that's precisely what checkpoints are- why do we have proof of work at all?

can we remove proof of work, and have a block chain that is based on consensus between N parties?  Yes!  https://docs.google.com/file/d/0BwUFHE6KYsM0ZkxLVmFwbXQ3ck0/edit?usp=sharing

-bm
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 12:22:04 PM
#18
P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
I can come out with a really good solution, if you want Wink

Just make a rule that the client would not accept any forks deeper than N blocks back from its currently known top.
Set N to whatever constant you want, but I think more than a week time would be exaggerating.
sr. member
Activity: 280
Merit: 257
bluemeanie
June 17, 2013, 12:20:21 PM
#17

 What do you think 'Peer To Peer' means?

 as in "A Peer-To-Peer Electronic Cash System"?  http://bitcoin.org/bitcoin.pdf
kjj
legendary
Activity: 1302
Merit: 1026
June 17, 2013, 12:17:41 PM
#16
who makes the checkpoints?
The bitcoin core devs
so ultimately, the state of the block chain depends entirely on them.
not really. they only put checkpoint for blocks that are buried at least few hundreds deep already - unlikely to be reverted by honest means.
but if you disagree with any checkpoint they add, just don't upgrade your node and you will be fine.

well that's a practical solution, but it violates the notion of 'peer-to-peer' or equality for all nodes in the network.

there is no indication of 'checkpoints' in the satoshi whitepaper.

furthermore, if we rely on such checkpoints for security, why do we even have proof of work at all?  Why don't we just rely on the checkpoints?

I'm having a hard time figuring out what you mean by "equality of all nodes".  The equality (or inequality) of nodes is not part of the system.  No one cares about nodes.  We care about the chain.  If a node has the correct chain and can give you a copy, it is "equal" to all of the other good nodes.  If it doesn't, or can't, then it equal at all, isn't even a node.

The checkpoints do not provide any security, nor were they ever intended to.  They just keep "unequal" nodes from wasting your time with bogus chains (whether by accident or malice).

Also, you are free to edit and recompile as needed.  You can strip them out entirely, or add your own, or change them.  Whatever you want.  The reference client is written for (and compiled for) working well for a wide audience.

Technically, a longer chain is the best chain, regardless of the checkpoints.  In practice, the checkpoints are far enough in the past that if a new chain showed up that overturned one, we'd almost universally consider it an attack, and roll back.

P.S.  No one particularly likes the checkpoints, the devs least of all.  There has been much discussion about removing them, or switching to a soft system (config file, etc).  So far, no one has come up with a really good solution.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 17, 2013, 10:34:12 AM
#15
yeah, I agree with you that it goes against an important principle, but as gmaxwell said, it does have some actual purpose, protecting your storage from being flooded by forks that would suddenly start to appear at low block numbers.
though, it does not make any sense to have multiple checkpoints (instead of i.e. a one that moves by itself), but well... who are we to argue with a solution chosen by the best bitcoin developers that satoshi left behind? Wink
sr. member
Activity: 280
Merit: 257
bluemeanie
June 17, 2013, 10:31:39 AM
#14
who makes the checkpoints?
The bitcoin core devs
so ultimately, the state of the block chain depends entirely on them.
not really. they only put checkpoint for blocks that are buried at least few hundreds deep already - unlikely to be reverted by honest means.
but if you disagree with any checkpoint they add, just don't upgrade your node and you will be fine.

well that's a practical solution, but it violates the notion of 'peer-to-peer' or equality for all nodes in the network.

there is no indication of 'checkpoints' in the satoshi whitepaper.

furthermore, if we rely on such checkpoints for security, why do we even have proof of work at all?  Why don't we just rely on the checkpoints?
Pages:
Jump to: