Topic: btcd: a bitcoind alternative written in Go - page 4. (Read 20970 times)

11 PH/s of the hashing power - how much of it are the devs actually holding?
And do you think whoever is holding it do not realize what kind of power they have?

A sooner you realize that devs have no calls on protocol changes anymore, the sooner you will avoid any further disappointments.  

If you're interested in these kinds of things I would suggest this resource: http://szabo.best.vwh.net/

When you're saying the bitcoin developers have no power, that doesn't hold up in argument. They have they keys, and they are designing the protocols. What is the input from outside of that group? where is that discussion even taking place? If you approach one of the devs with these questions you wouldn't get a straight answer. I can make a proposal, but this is subject to the lead devs views, which I don't share in many ways. We need much better trust models and infrastructure for collective decision making. we can do better than this messageboard and some mailing list.

Do you count yourself to that chosen few ? If yes, show me small piece your wisdom that I am eventually capable to grasp, as I have not seen it yet.

I think you, like almost all, are seeing this very much from the perspective of bitcoin as it is today, which is a very limited view. Bitcoin in the future can and should be infinitely more, than what it is now. As this is not necessarily btcd related, I will post longer comments elsewhere. But I think you haven't thought this through. Again, imagine two alternative implementations with 50% share. How to resolve conflicts? bitcoin's consensus mechanism is not just about the calculations. it's not just about hashes, IP addresses, ASIC's, etc. we are talking about the future of money. Unfortunately the set of people who can combine both views, the social and technological aspects, is extremely small.

Do not overestimate the power of the core team.

They can not introduce a change that breaks the lockstep immediately, since that would fork the network if not upgraded instantaneously (which is not feasible).

At max they can introduce a change that takes effect well ahead from some some future block on. If the change is highly controversial, then and this opens the time window to build alternate core team that steps up to maintain the version before the change.

grau, I believe you don't see the depth of the above statement. The fundamental question is: who makes decisions about bitcoin? The current bitcoin developers - granted by what power? They have the SSH keys and can implement more or less whatever they think is right, with rather vague references to the community. The trust and consensus system is very far from perfect. Think about it - from an economic perspective all what bitcoin does is enforce consensus for the balances. It does not enforce consensus about the properties of the network itself (devs, miners, community, users are all seperate groups). The process of development is not formalized in any way. Say an alternative implementation tracks all the features at the beginning. It slowly grows to 51% marketshare. Now the developers have the power to force new changes. There is no way to resolve a conflict.

Yes, the design depends on getting identical results in a lockstep on what the longest chain and the set of unconfirmed transactions is.

I see no argument in this against competing wallet implementations just like we have seen a variety of miner. The bottleneck in higher level feature development is imposed by poor design, not fundamental reasons.

Added the lockstep is endangered by any new version of the reference implementation itself. Recognize that the most dangerous fork we have seen was between two versions of satoshi's code. Do you need any better argument for an isolation and freeze of that ? We can however only freeze after we create an interface that enables feature development on a higher level.

What about other crypto currencies

satoshi said this about alternative clients:

In his 575 posts, that is about the strongest statement he ever made.

---

I suppose Gavin and Mike don't want to put in such strong words, but in the end the algorithm is tied up to the current developers. Which in my mind is a huge flaw in the design of the currency. If Gavin wants to design a payment protocol, he can just go ahead (BIP70-72 are terrible IMO). Any solution to this problem of decision making will be a different cryptocurrency.  


In other words you want the current devs to continue with the process as it is? All design decision will be stuck forever? I certainly hope not. As soon as one wants to do useful things, which are not possible with fiat, one is kind of stuck. Colored Coin and Mastercoin will not work, for very good and actually quite obvious reasons. One can't transfer property on the bitcoin network.

Not if what is frozen is what should not change. We need competition in higher level features not in shooting for a moving target.

I don't want Bitcoin to change. Changing and extending the protocol/standard is the worst thing we can do. There's far more important stuff that needs to be done implementation side. Moving forwards means freezing the standard, that's why I want to diversify.

This can be a great program for the future use of storing bitcoins, thanks OP!

"Freezing" is the opposite of progress.

Since we can not eliminate the bottleneck for now, we should seek to control the damage to productivity by isolating and freezing consensus relevant code base of the reference implementation.

We should take out the wallet and replace the badly designed RPC with a bus (e.g. zeromq) that allows wallets and all higher level functions to be built without touching or even understanding the P2P consensus kernel.

Added: I used the word kernel intentionally. The P2P consensus is the operating system of the network for which we need an API that is high performance, language agnostic, simple but correct representation of the current consensus for application developer who create all sorts of wallet and business functions thereby eliminating the bottleneck at those levels.

Unfortunately this is not the case.

Bitcoin can only move forward as quickly as the slowest implementation. This is even more true with the slowest implementation is the one use by virtually all the network.

There is no path forward that does not involved either fixing the reference implementation, or convincing the network to abandon it en masse.

Is it even possible to have two major reference clients in parallel? Imagine we have a full spec (bitcoinP) and two clients - bitcoinA and bitcoinX with each 50% distribution. each are SSH locked repos (that's the access model we have currently). Then somebody has a good idea on how to improve bitcoinP. The bitcoinX developers think its a great idea, bitoinA developers think its terrible. who gets to decide? very likely bitcoinX would be build as an Alt-Coin. Now you have the question, is all the infrastructure transportable? it seems to me that there is very little evolution of the primitives, and very little development activity compared to the number of people talking about bitcoin now, like silicon valley hotshots, pumping millions into inferior infrastructure. I guess if you have x millions $ in BTC you are going to be rather conservative in introducing changes.

basically the argument goes like this. we have opensource and everybody can suggest changes. therefore its free and anybody can join. this assumes that software is simply the sum of small proposed changes, not of architectural decisions. in reality the development has centralized somewhat. but the Linux/Python/whatever is not good enough as a model. we can't have the equivalent of a Linus Torvalds merging all the changes. but coming up with a better model is very hard. it seems to me that very little profits are being reused to build useful stuff. there is not even the possibility of hosting a server for 100$/year, and funding that through public means, when marketcap has reached 10 billion $. instead we have perhaps a thousand people now trying to produce a Scam AltCoin.

that async principle is a solid one to create for a bitcoin impl.

been checking in on btcd every so often, and the design/arch decisions are solid.

the code is the spec is a bit of a crap base for the future of finance and something we'll leave behind. the more of us there are, the quicker bitcoin will move forward. decentralise everything and distribute the power. we need a solid infrastructure and the bitcoind is really poor.


yes omg this is terrible. output scripts dont even need to parse. but there's plenty more like the copy paste serialization code, deeply nested locking mechanisms, cscript inheriting an std:: container (forbidden by standard), undefined behaviour (using integer loop around / invalid casts), non-modular code, ... more i can't recall right now

I appreciate what everyone at conformal is doing. Thanks for taking on this important work and producing a viable alternative.

I am confident there is sufficient willingness in the community to fund the development time it would take to push the required cleanups upstream. The only problem appears to be that the organization which you'd naturally expect to be taking that effort on does not consider it to be a priority.

In order to avoid chain forks, multiple implementations have to reproduce the same behavior exactly.  Even the slightest difference can cause a chain fork.  There are certainly some cases that can be considered weird behavior, but there are also some issues I would consider outright bugs.

For example, the transaction script language has a well-defined set of allowed opcodes.  However, it doesn't check the scripts for invalid opcodes until it actually executes them.  This means that you can craft valid scripts which have invalid and nearly arbitrary data in dead execution branches.  This is an outright bug in my mind.  There are also a couple of other things in the transaction scripts which are indisputable bugs.


The exact rules are not really documented anywhere from what I could find.  The wiki entry for the wire protocol is fairly accurate, but the block acceptance rules are largely undocumented.  There is a wiki entry for the block acceptance rules, but it's not very accurate and doesn't provide sufficient depth.  However, I don't believe a publicly editable wiki is the right place for something as important as the protocol and block acceptance rules anyways.

This is an area I believe really needs to be improved.  Unfortunately, the general consensus I have seen amongst the core bitcoind devs seems to be "the code is the spec".  I would be happy if my take on what the general consensus appears to be is wrong.  I fully understand that writing documentation and specs is not glamorous nor fun work, but I think it's paramount for the longtime survival and growth of Bitcoin as it directly plays into multiple implementations.  I can't imagine the internet would be anywhere near what it is today if there weren't legitimate specs for things like TCP/IP and instead everyone that implemented a stack had to just go read and properly interpret some mostly undocumented code.  Unfortunately, even if a spec is written, unless the majority implementations (realistically this is only bitcoind currently) choose to follow it, it is meaningless.

On the plus side, there is a block tester tool that does a pretty good job of testing the block acceptance rules programatically.  While this doesn't eliminate the need for a spec by any means, it is definitely an excellent tool that helps support multiple implementations and would be a fantastic supplement to a real spec.


Yes.  Since it is written in Go, btcd uses CSP (Communicating Sequential Processes) heavily for concurrency.  The Go mantra is "Do not communicate by sharing memory; instead, share memory by communicating", which is the model btcd strives to follow.  For example, in btcd, the server, block manager, and peers are all separate entities running in different goroutines that use channels to communicate with one another.  This provides a clear concurrency model with a clean and concise data flow.


I'm not really sure what area of security you are referring to here.  If you're referring to security issues such as exploitability of the software, I would argue that libraries written in Go eliminate an entire class of the most common causes of vulnerabilities such as stack corruption, heap corruption, uninitialized variable use, use of freed variables, and invalid frees.  This is because Go is garbage collected, all variables are initialized to their respective zero values by default, all array accesses are bounds checked by the run-time, and pointer arithmetic is disallowed.  That is not to say they're fully immune to exploits of course, but eliminating the vast majority of the most common ones by default is quite nice.

On the other hand, if you're referring more to the security issues such as coin theft, that has less to do with the implementation language and more to do with things like private key management, trusting third parties with your wallet, etc.


Thanks grau.  Reaching feature parity in that time frame is an impressive one-man effort.  Well done!  That sounds very close to what it would have taken us as well to just implement everything without the architectural changes, drive for 100% test coverage in the core packages, and heavy focus on producing fully-documented reusable development components for the community as a part of the process.  I'm really glad you posted your stats as I think this goes to show that while it's definitely complicated, it's not something that is so complicated that nobody else should bother as some comments have suggested.