Author

Topic: Bitcoin protocol spec (Read 987 times)

member
Activity: 70
Merit: 10
January 01, 2014, 04:45:42 AM
#15
Ugh.  Do we really need to start a brand new thread on this every few months?

At this point, is there even a slight chance of someone making a meaningful addition to the discussion that can be found in all of the many, many other threads on this topic?

What does this comment add to the discussion? Absolutely nothing. It just shows that better efforts are needed, and that people who are in this on board for a long time, don't care about a lot of things. That's why so much things are broken and should be improved. This messageboard and the wiki are not properly searchable. Lets all go back to usenet please.

Quote
If anybody really cared to ever tune up the state of documentation in software projects,

As if Bitcoin was a regular software project. hilarious.
kjj
legendary
Activity: 1302
Merit: 1026
December 31, 2013, 11:03:00 PM
#14
Ugh.  Do we really need to start a brand new thread on this every few months?

At this point, is there even a slight chance of someone making a meaningful addition to the discussion that can be found in all of the many, many other threads on this topic?
sr. member
Activity: 406
Merit: 251
http://altoidnerd.com
December 31, 2013, 09:37:17 PM
#13

If anybody really cared to ever tune up the state of documentation in software projects, then documentation wouldn't be persistently relegated to a thankless chore dependent on the goodwill gruntwork of the few volunteers capable of writing it.

This is a human problem in general that bitcoin micropayments can solve.  I'd micropay for this! 
newbie
Activity: 32
Merit: 0
December 31, 2013, 08:50:20 PM
#12
The whole point of protocols is to get software independent descriptions, so that software can communicate over networks independently. At least that is how it works for TCP/IP, DNS, HTTP, HTML, SMTP, ECMAScript and so on. It seems Bitcoin as a network has quite a different relationship to protocols. After all the consensus build around the software, and protocols are essentially enforced consensus. At any rate, the view that bitcoind source should be the documentation is not very helpful for the future development of the network. And to say there should be no good documentation doesn't really hold up in argument. One could argue that Bitcoin should be the best documented piece of software on the planet.

Well, the reality is that we're forced to reverse-engineer the specs as with most software since it unfortunately wasn't pre-engineered by a liability-hardened steering committee but rather evolved from a buggy ad-hoc pseudonymous side project by a few volunteers.

Most of the topical work has been done, but we're left with edge-cases and idiosyncrasies and arbitrary implementation details with a high cost for reimplementing it wrong.

But what's new?

If anybody really cared to ever tune up the state of documentation in software projects, then documentation wouldn't be persistently relegated to a thankless chore dependent on the goodwill gruntwork of the few volunteers capable of writing it.
legendary
Activity: 1232
Merit: 1094
December 31, 2013, 08:11:42 PM
#11
That's sort of the definition of a "reference client", isn't it? 

Well, the reference client could be the one that implements a written spec.

I can see source code being a valid way of doing the spec.  However, the bitcoin client isn't really written that way.

It would have to compromise efficiency if that helps clarity.

However, Gavin has commented that they are not likely to do a complete re-write of the reference client.  At the moment, keeping it functional is more important.

The offending block would be logged for analysis and client authors would be expected to bring their software into compliance.

If two clients disagree about a two different blocks, which block is the "offending" one, and which is the "acceptable" one.

If there was 10 common clients, then for almost any block almost all of them would agree.  However, in that context, I would take the position that if any of them reject a block, then miners would be recommended to reject the block.
sr. member
Activity: 406
Merit: 251
http://altoidnerd.com
December 31, 2013, 10:36:22 AM
#10
But the idea that there should not even be proper documentation is absolutely ludicrous. You might as well run the project as closed source then.

The control of the code is codified through the SSH access keys. In the end the references to "community" are very vague and not based on formal trust models. I assume that miners largely don't care.

This is not a choice made, this is a result of what the meaning of trust is. The openest source project is one which the code is the only documentation for the reasons already provided.  This is because any other interpretation is by definition not necessarily trustworthy.  Descriptions are free to exist, but none have special status as "correct" except the source code.

Bitcoin is a big "company" of people who have no mutual trust.  I've got no reason to trust your description over the actual source code, which is why others here see this as moot.
member
Activity: 70
Merit: 10
December 31, 2013, 07:00:50 AM
#9
Is the long term plan that there will be one reference client forever?  Ideally, the protocol should include some kind of system for handling a situation where some of the miners are using a client that has forked.

Interesting. The question is if it is reasonable to have several alternative implementations. Say one would have two node types, say 50% of miners would run bitcoinA and 50% bitcoinX. Now the developers of bitcoinX decide to introduce a new feature for making transactions more anonymous. bitcoinA devs oppose. And so the chain splits, based on the implementations.

What would be the incentive for miners to use bitcoinX? Pretty much none, because there is only risk involved and little benefit. Everyone will stick to bitcoinA.

Essentially it probably means that true alternative implementations will be Alt-Coins and Coins will be competing against each other. I don't know how much BTC the core devs hold, and to what extent it is public knowledge, but one can imagine that introduces some biases. The general attitude towards alternative implementations and Alt-Coins is mostly negative, for good and bad reasons. But the idea that there should not even be proper documentation is absolutely ludicrous. You might as well run the project as closed source then.

The control of the code is codified through the SSH access keys. In the end the references to "community" are very vague and not based on formal trust models. I assume that miners largely don't care.
legendary
Activity: 3472
Merit: 4801
December 30, 2013, 09:08:20 PM
#8
This means that no developer can depend on the written documentation as a definitive description of intended (or actual) behavior.  As such, while the written documentation can be useful as a reference and as educational material for getting started, it is useless as a specification for exact behavior.

That is a statement of the principle that "the reference client is the spec".  It doesn't necessarily justify it.

No it doesn't justify it, it simply explains the reality of the situation.

Is the long term plan that there will be one reference client forever?

I have no idea, but I suspect so.  That's sort of the definition of a "reference client", isn't it?  The "reference client" is the one client that is used to specify the details of exactly how the protocol works.  Perhaps instead of thinking of "the reference client software is the spec", it would be more acceptable to you to think of "the spec" as being a document that has been carefully and exactly written in a language other than English.  "the spec" is a document that just happens to have been written in a computer programming language instead of an specific country's "official language".  Since bitcoin doesn't belong to any particular country, it seems fitting that it's spec isn't written in English, or Spanish, or French, etc.

Ideally, the protocol should include some kind of system for handling a situation where some of the miners are using a client that has forked.

It does. Doesn't it?  That's why it's called "forked".  The protocol states that if a miner is using a client that doesn't exactly implement the necessary protocol rules, then the output of that miner is ignored and not accepted as "bitcoin" communication.

For example, if there was customer and miner clients, then customer clients would accept a super-set of blocks relative to miners.

And how would the "chain" be maintained then?  The idea of the blockchain is to force a consensus.  If a "customer client" is willing to hold multiple competing blocks as equally valid, then the concept of a consensus is lost.

There could be a system where miners pass blocks and transactions through multiple clients and only accept blocks that are accepted by all.

How would you know when "all" clients had accepted the blocks/transactions?  How would you handle a situation where an adversary creates a client that specifically rejects the very transactions or blocks that you believe should be included?

The offending block would be logged for analysis and client authors would be expected to bring their software into compliance.

If two clients disagree about a two different blocks, which block is the "offending" one, and which is the "acceptable" one.
legendary
Activity: 1232
Merit: 1094
December 30, 2013, 07:49:47 PM
#7
This means that no developer can depend on the written documentation as a definitive description of intended (or actual) behavior.  As such, while the written documentation can be useful as a reference and as educational material for getting started, it is useless as a specification for exact behavior.

That is a statement of the principle that "the reference client is the spec".  It doesn't necessarily justify it.

Is the long term plan that there will be one reference client forever?  Ideally, the protocol should include some kind of system for handling a situation where some of the miners are using a client that has forked.

For example, if there was customer and miner clients, then customer clients would accept a super-set of blocks relative to miners.

There could be a system where miners pass blocks and transactions through multiple clients and only accept blocks that are accepted by all.  The offending block would be logged for analysis and client authors would be expected to bring their software into compliance.

I also think that an official "verifier" library would be a big improvement.  Essentially, that would accept a series of blocks and verify that it is correct.

That would be a less complex piece of software relative to an entire client including networking and orphan handling.  It just says Yes/No to a blockchain.
member
Activity: 111
Merit: 10
December 30, 2013, 07:49:22 PM
#6
A protocol is a system of rules and procedure that should be followed in a certain situation. (other definitions are available Smiley).
One of the interesting things about bitcoin is how the protocol is established and how it is maintained.
I suppose software is a set of rules and procedures but I think it is more usual to establish the rules and procedures and then design the software to implement them rather than to base the protocol on the way that the software behaves.

legendary
Activity: 3472
Merit: 4801
December 30, 2013, 07:30:16 PM
#5
- snip -
At any rate, the view that bitcoind source should be the documentation is not very helpful for the future development of the network. And to say there should be no good documentation doesn't really hold up in argument. One could argue that Bitcoin should be the best documented piece of software on the planet.

You are welcome to write any documentation that you like.

The thing to keep in mind is that any written protocol is an attempt to describe what the reference client already does (including any "bugs" or unexpected behavior).  The reference client is not written to implement a specific written protocol.

As such, if there is any discrepancy at all between what is written in the documentation and what the reference client actually does, then the bug is in the written documentation, not in the reference client.  The written documentation will in that case have failed to properly describe the protocol.

This means that no developer can depend on the written documentation as a definitive description of intended (or actual) behavior.  As such, while the written documentation can be useful as a reference and as educational material for getting started, it is useless as a specification for exact behavior.
member
Activity: 70
Merit: 10
December 30, 2013, 06:47:04 PM
#4
The whole point of protocols is to get software independent descriptions, so that software can communicate over networks independently. At least that is how it works for TCP/IP, DNS, HTTP, HTML, SMTP, ECMAScript and so on. It seems Bitcoin as a network has quite a different relationship to protocols. After all the consensus build around the software, and protocols are essentially enforced consensus. At any rate, the view that bitcoind source should be the documentation is not very helpful for the future development of the network. And to say there should be no good documentation doesn't really hold up in argument. One could argue that Bitcoin should be the best documented piece of software on the planet.
administrator
Activity: 5222
Merit: 13032
December 30, 2013, 06:19:37 PM
#3
The code of Bitcoin-Qt is the only complete protocol reference. If your protocol-related behavior doesn't exactly match the behavior of Bitcoin-Qt, even if you accurately follow the protocol specification on the wiki, then you are wrong.
member
Activity: 70
Merit: 10
December 30, 2013, 05:12:36 AM
#2
A good explanation of the block format in a more logical way

http://james.lab6.com/2012/01/12/bitcoin-285-bytes-that-changed-the-world/
member
Activity: 70
Merit: 10
December 29, 2013, 11:39:24 AM
#1
To my knowledge these are the standard references for the bitcoin protocol specification

https://en.bitcoin.it/wiki/Protocol_specification
https://en.bitcoin.it/wiki/Protocol_rules

BIPs are more detailed, but are as their name implies about improvements to the protocol, not the protocol itself

https://github.com/bitcoin/bips/

Are there are any other attempts for a full spec or lengthy detailed technical descriptions to be aware of?

History trail:

Discussion as of 11/2010: https://bitcointalksearch.org/topic/bitcoin-protocol-specification-1860
Jump to: