Topic: Creating an "official" protocol specification for the Bitcoin internet currency (Read 4701 times)

You're totally right, but we've known this for a long time; fundamentally my objections to large blocks on the basis that they can be used to flood miners on low bandwidth connections is an example of consensus failure due to network rules. It's just the "rules" in that case happen to be laws of physics rather than software.

My blockheaders over Twitter thing, while at one level an April Fools joke, is at another level not a joke at all. We do need alternate methods of block and transaction propagation on the network, and unlike validation having as much variety as possible is only a good thing. It's the fundamental security principle of Bitcoin: information is easy to spread and hard to stifle.

An alt client which implements the 95% of the network rules would probably be functional.  Maybe it wouldn't protect against spam quite so well etc.

What is the core problem that bitcoin solves?  The distributed consensus problem.

The currency aspect of bitcoin is simply a layer on top of the distributed timestamping service.  namecoin is an example of a non-currency use of distributed timestamping.

What is the core problem that bitcoin solves?  The distributed consensus problem.

There have been chains of hashes and chains of digital signatures before. What makes bitcoin different is that it is timestamping these digital messages, and protecting those timestamps against being reversed.  The currency aspect of bitcoin is simply a layer on top of the distributed timestamping service.  namecoin is an example of a non-currency use of distributed timestamping.

I disagree.  The core validation is a description of the rules for validating transactions.  This would include the script language and the rules for the encryption.

Indeed, it's full of such small but significant pitfalls, that can't just be ignored. I'm still not convinced that a human can write a complete spec (of the block chain validation, not so much the network protocol, that's trivial in comparison) that catches all of these cases and completely matches the client. But I'm happy to be proven wrong, and look forward to a well-written block chain spec.

I would claim that not even Gavin knows this completely. For example, the DER-encoded signature is not always valid according to the DER specification, because OpenSSL does some things differently. Link. This means that an implementation that rejects these invalid signatures would reject an otherwise, according to bitcoin-qt, valid block, and it would then be on a different chain than the main (bitcoin-qt) chain.

This is a nice idea, but in practice you end up creating a DoS vulnerability: if I create an invalid transaction on purpose and send it to you, then you determine it is invalid and rush to announce the fact to the rest of the network, spamming everyone with an invalid transaction.

I disagree.  The core validation is a description of the rules for validating transactions.  This would include the script language and the rules for the encryption.

Much of what is in the above rules is, as the title says, protocol rules.  They are mostly to ensure efficient operation of the network and keep spam low.

It would be nice to be able to validate the system in a distributed way.  Also, the SPV system is vulnerable to double spends, since you have to trust the node you connect to.

At the moment, there are 2 extremes, either you validate everything or validate almost nothing.  A system for notifying invalid blocks which proof of invalidity attached would allow distributed validation.  If you have 10k nodes and each node validates a random 1% of the transactions, then the probability of an invalid tx been missed is (0.99)^10000 = 2 * 10^(-44).

The only important part is the binary format of transactions and blocks, so that you can verify proof-of-work and merkle roots, other than that it really doesn't matter how you obtain them - you could get all the data you need from blockexplorer.com and bypass having to connect to the network at all if you really wanted to.

In fact, I think it speaks volumes about those complaining that every one of these threads talk about a 'protocol spec', when in fact the way that the network passes messages is largely irrelevant to the working of Bitcoin (aside from the fact that it's very well documented and simple).

There's only one thing you actually need to worry about - block validation, which is explained here.

There's also the question of why you actually want to implement a fully verifying node. The only reason I can think of is if you want to mine with it - so you don't waste your effort producing invalid blocks. Other than that, you can get all the functionality you need from SPV checks and a full blockchain (making sure to verify the merkle root) for significantly less development effort, less cpu+ram use, less likelihood of forking yourself into oblivion, for a tiny reduction in security.

As it appears to be consensus that deriving the spec is too complex and tricky for a human being

You can't send an email to an address in the format [email protected] ?

email isn't decentralized - it's relying on DNS, which is centralized.

...

the only other massive peer-to-peer decentralised comms system.. email

As it appears to be consensus that deriving the spec is too complex and tricky for a human being

Even the minimal sane behavior is surprisingly complex. But beyond that, "lock old _invalid_ blocks in with checkpoints" is really pretty hideous. You'd still have to distribute code to validate them because part of the point of the system is that none of it depends on some magic values being trustworthy. Though perhaps its preferable to relegate the legacy support to a validation tool.

Considering cases like these, it occurs to me that it might be desirable to - at some point - split the Satoshi code into three parts: "legacy", "current" and "next".

The "legacy" part would handle the odd corner cases described in the above quote. It would basically pull in all the relevant OpenSSL code into the legacy module (including the bugs in question), where it would stay untouched. This module would only be used to verify already-existing blocks in the chain; no new blocks would be verified with this code, as pulling in OpenSSL code into Bitcoin and managing future patches is next to impossible. This is the part that should be possible for future clients to not implement. They will miss the part of the block chain that follows the rules defined by this module, but I reckon that we really don't need 5+ year old blocks for more than archival purposes.

The "current" module would handle verifying current blocks, and be compatible with the "legacy" module. It would depend on OpenSSL still, and if changes are made to OpenSSL that break compatibility with "legacy", patches would need to be maintained against OpenSSL to work around this. This module cannot code-freeze OpenSSL, as vulnerabilities can become uncovered in OpenSSL, and no one must be able to produce blocks that exploit the uncovered attack vectors. Newly uncovered attack vectors aren't a problem for the "legacy" module, as it only verifies already-existing blocks, produced before the vulnerability in question was uncovered.

The "next" module would be backwards incompatible with the "legacy" and "current" module. This module changes verification rules to not accept, for example, the otherwise invalid signatures that OpenSSL accepts. The "next" module would have a block chain cut-off point into the future where, from that point on, a Bitcoin transaction would be considered invalid if, for example, it includes an invalid signature (was it a negative S-value?) even though it's accepted by the old OpenSSL code. It's sort of a staging module, where undesirable protocol behavior is weeded out. These protocol changes wouldn't take effect until some point in the future (a block number). The block number cut-off point would be advertised well in advance, and from this point on, the "next" module would become the "current" module, and the old "current" module would move into the "legacy" module (and no new blocks would be verified using this module). The new "next" module would then target fixing undesirable protocol behavior that was uncovered when running the previous "current" module, and would set a new cut-off time into the future, at which point new blocks would need to follow this improved protocol to get accepted.

Couldn this work? It would mean we could slowly (very slowly) clean up the protocol, while still maintaining backwards compatibility with clients not older than, say, 2 years, or however long into the future we choose the cut-off point for the "next" module's protocol to become mandatory.

uh. Testnet didn't prevent the hardforking we created in Bitcoin 0.8 even though we believe that the relevant cases were tested already. (there were already super large blocks there).