Topic: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community (Read 6739 times)

It is simply a different design.

leveldb does not need per-page locks, and is not an ACID database.  leveldb has a concept called a "batch", and batches are committed atomically.

Can you explain how locking is handled better by leveldb for the particular type of database transaction that can require unbounded locks on bdb?

Probably best if we leave at that then. I'm happy the problem has been laid bare for all to see and the community is made aware of the issues in a transparent way so informed decisions can be made. Fear not, real work is happening.

We knew there was a limit. What nobody knew is that the limit was low enough to be hit with blocks smaller than 1 megabyte. Remember that 1mb blocks had been tried before and worked.

The BDB manual doesn't give a formula for selecting the number of locks, presumably because not even the BDB developers know how to do it correctly, or perhaps because it seems to vary across versions/platforms. It just says "pick a number that seems to work and then double it". That kind of nonsense is fortunately not present in LevelDB.


Seems I was right about that, wasn't I? Hence the warnings against re-implementations. Understanding and documenting all these edge case behaviours is very difficult even for people who work on the existing code. So if you rewrite everything from a wiki page it's going to be even harder to match things precisely.


In what sense was 0.8 rushed out? The lock limits issue has been in the code for a very long time and went away entirely with 0.8, so spending more time testing it would not have helped reveal the problem. You'd have to have been testing old releases to find this issue, and moreover, testing it in non-obvious ways.

I'm tired of these sorts of threads and will not be posting on the issue further. The code that Satoshi left behind is certainly not ideal in many ways, and the only way to fix it is incremental upgrades. That means real work. If you are going to make yourself useful instead of just insulting people anonymously, then get to it and start finding bugs.

My rule of thumb after 20+ years in software engineering: Never be the first to try the latest version until it has been used by others for months In fact some of my machines still run 0.3

Maybe the newer version is superior, but people could use old version to do many kind of fancy things that developer can never imagine and test. A new version typically affect many infrastructures and relations that were built upon those old version

Luckily bitcoin has not reached such maturity level that there are hundreds of other services are dependant on the blockchain

Then you don't really know a lot about large institutions, do you ?

I would bet that many banks run software written in 1999 or even before, you know why ? Because when huge risks and costs are involved, you don't change something that works properly unless there is a serious reason. Hell, some large companies still run COBOL code written 30-40 years ago !

Changing software versions is a significant risk each time it is done.

Time, money, etc to test, implement, copy over any custom code, etc.  And the consequences if something is missed.

Mike H.


Incorrect that the lock limit rule was not known about. It is stated explicitly in the pre-0.7 source code, db.cpp lines 82 and 83


But good to see that you have finally gotten to grips with BDB lock limits ... AFTER implementing the new levelDB that ignored these two lines of code in all pre-0.8 bitcoin reference code ... which recall by your own words ... to paraphrase "if you want a protocol read the source code"

0.8 maybe a superior solution but it has been rushed out. The fact you haven't fully understood the previous code and implemented the upgrade correctly may be more your deficiency than pre-0.7 code.

It baffles me why they would not have switched to 0.8. That version was out for weeks before the fork happened. Oh well, live and learn.

I'm pretty sure Mt Gox, blockchain.info, and BitPay were all on 0.7, and probably some other major sites as well. Sites like that are probably less capable of changing versions overnight compared to mining pools.

Thanks. It makes more sense to me now. Basically, 0.8 had created an unplanned hard fork, and took many users (not necessarily miners) hadn't upgraded, so transactions would have been dubious until they did so. Functionally, it sounds to me like 0.80 was a better version, but it wasn't backwards compatible and no one had planned for a hard fork.

+1

Doesn't matter which chain is longer if a majority of the people aren't on it.  Breaking changes need to be given lots of warning to be effective.  Trying to force everyone to use 0.8 would have only made the situation worse.  From the chat discussion, I don't think mtgox was using 0.8.  So trading at the largest exchange would be halted until it could be upgraded.  If that doesn't sound disastrous, I'm not sure what does.

No. OK, a bit of history. I was referring two previous hard-forking changes that I'm aware of. There were probably more.

When Bitcoin was first released, it contained two completely fatal bugs that made the entire system worthless. Fortunately, they were found and fixed before Bitcoin  actually had any serious value.

The first bug was that scripts were concatenated before being run instead of just using a shared stack. This meant that anyone could write a scriptSig that always evaluated to true and claim anyone elses coins. Fixed here in v0.3.2:

https://github.com/bitcoin/bitcoin/commit/73aa262647ff9948eaf95e83236ec323347e95d0

https://en.bitcoin.it/wiki/Incidents#CVE-2010-5141

Needless to say, if somebody when this version was first released actually wrote such a scriptSig and stole some coins, that would have caused a chain split between old and new versions. Nobody did because, why bother? I'm not even sure Mt Gox existed back then, iirc that came some months later. Many script opcodes were disabled around this time (which is also a hard-forking change).

The second bug was found some months later, and this time it was exploited:

https://bitcointalksearch.org/topic/strange-block-74638-822

A fixed version of the software was quickly released. There was a race between the old and new chains. After some time (was it 12 hours?) the new chain beat out the old one. Even then, miners who didn't upgrade kept trying to extend the broken chain for a long time.

Back then there was no talk about clearly broken behaviour being some kind of unbreakable "protocol rule". You may see some people say that bugs in the old software become a part of the protocol and yes, this is true within limits. For instance, OP_CHECKMULTISIG contains a dumb bug, but it's not dangerous and can be easily worked around. So it's better for new versions to just not fix the bug, the cost of a hard fork isn't worth fixing it. Old versions being unable to digest even quite tiny blocks is clearly far more serious and the cost of a hard fork is worth it in that case.

Maybe, but it's not that simple:


tl;dr:

+100

Thanks Mike for putting some sense on some of the crazy rants I've read in this thread.

Just one question Mike,

isn't this kind of different as on the previous forks, clients of old versions where still able to accept 100% percent of all blocks the new versions accepted, but the new versions "filtered" some out, that where created with old versions? So old versions where still able to get on the new chain as soon as it was the longer chain.

With the new soon necessary fork it's the other way around, old clients will reject new client blocks, so they will stay on their fork if they don't upgrade.

So it's kind of a new problem.

Correct me if I'm wrong.


But I totally agree with the rest +1 from me too.

We can't expect that bitcoin will never have upgrades that make it downwards incompatible. This will happen again and on the bright side this is an opportunity to test this how it can be done in a smooth way on an issue where (almost) everyone agrees that it needs to be addressed.

+1 to Mike!

Yes, unexpected crash upgrades are bad news, but if this event had happened at a later time when 0.7 was not widely used, it might have made more sense to just abandon it.

Based on what I've read my guess is that many significant non-mining businesses are still on 0.7 and couldn't upgrade quickly enough.

Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around? Technically, 0.8 was winning in chain length also.