Understanding why the call to rollback to v0.7 was made.

jgarzik

legendary

Activity: 1596

Merit: 1100

That is not the full story. There was an INTENTIONAL setting change from 0.7 to 0.8, there is no "bug" ... both version did what they were meant to but they had been made incompatible by this change below ...

https://bitcointalksearch.org/topic/m.1619546

https://github.com/bitcoin/bitcoin/commit/ae8bfd12daa802d20791e69d3477e799d2b99f45#src/db.cpp

Code:

82 - dbenv.set_lk_max_locks(10000);
83 - dbenv.set_lk_max_objects(10000);
82 + dbenv.set_lk_max_locks(40000);
83 + dbenv.set_lk_max_objects(40000);

This analysis is incorrect, as noted here when this "discovery" was posted in another thread.

johnyj

legendary

Activity: 1988

Merit: 1012

Beyond Imagination

It's cheerful that the roll back to 0.7 decision was made, the other things are not very important right now. The bitcoin's lifeline is hanging on consistency and integrity

Fork is a headache Cool

Stephen Gornick

legendary

Activity: 2506

Merit: 1010

Quote from: ?? on ??

I am unclear about the compatibility of older versions earlier than 0.7.

The BDB database lock limit issue discovered March 12th exists with every version of the reference client (Bitcoin-Qt, and prior to that WxBitcoin GUI, and bitcoind) prior to v0.8.

Quote from: ?? on ??

I assume that means all previous versions? At what point are older versions no longer usable?

That's the definition of a hard fork. Old software rejects new blocks which include the incompatible change.
- http://en.bitcoin.it/wiki/Hardfork_Wishlist

So the moment the fork starts is when any clients not supporting the new rules are no longer supported.

Quote from: ?? on ??

When that does happen how would a user know they should upgrade?

Chances are when the hard fork happens most everyone already would have upgraded.

What that means is that the client released would support both the existing rules and the new rules, with the new rules not taking effect until some future point in time (based on block number). So essentially if block_number > N then follow the new rules else follow the old rules.

And it would be released well in advance of block N occurring. [Edit: i.e., two years, according to one core dev:

Quote from: Luke-Jr on March 12, 2013, 07:25:11 AM

A hard fork like this would require the intentional support of a majority of merchants.
Short of an emergency, that means everyone will be given at least 2 years to upgrade.

]

eldentyrell

donator

Activity: 980

Merit: 1004

felonious vagrancy, personified

Thank you for posting this. However, what you write about risks on the 0.7 branch is not correct.

Quote from: DeathAndTaxes on March 12, 2013, 10:47:17 AM

However in time as hashing power fled the v0.7 fork remaining nodes would become more and more vulnerable to a variety of attacks. Not just from a potential 51% attack, but also from accepting generated coins which were valid on the v0.8 blocks and even non hashpower related double spends.

Actually this isn't true. The Satoshi client checks for long invalid chains (this line of code); if it finds one it goes into safe mode and stops processing transactions or responding to RPC calls. The message three lines below that line of code is what Pieter Wuillie is talking about in the original announcement:

Quote from: Pieter Wuille on March 11, 2013, 08:29:57 PM

If you're on 0.7 or older, the client will likely tell you that you need to upgrade.

The now-infamous OKPAY-BTCe double-spend was the reverse problem: it was an attack on a 0.8 client (or a 0.7 client that for some weird reason accepted the large block -- there are unconfirmed reports that the bug is platform-dependent). Unfortunately that problem is a lot harder to solve. The Satoshi client needs to start watching for long-and-recent-but-not-longest orphan branches.

nebulus

hero member

Activity: 490

Merit: 500

... it only gets better...

All right, thanks, man... I can get the chain fine on the 0.8. Was not sure if the problem was related.

Stephen Gornick

legendary

Activity: 2506

Merit: 1010

Quote from: nebulus on March 12, 2013, 07:19:38 PM

What exactly do you mean by roll back? I can't get the chain on 0.7 to update...

By rollback I think he meant have miners downgrade to v0.7 (i.e., roll back the version of software they are using to a prior relese).

If you are having an issue getting your client to sync with v0.7, that probably has nothing to do with the fork. You probably should open a thread on the Tech Support forum board for that (if the typical suggestions of check your connections, delete the blockchain data files and re-download, etc. don't work for you.) [Edit: Or move on over to v0.8 which doesn't use BDB for the blockchain.]

Stephen Gornick

legendary

Activity: 2506

Merit: 1010

Quote from: cypherdoc on March 12, 2013, 06:56:29 PM

what are locks?

With databases, locks are generally referring to something that must be done in a series. With multiple CPUs, there can be the situation where two pieces of code want to change the same variable at the same time. Think of an increment operation. If you have a value of four and that gets incremented by both at the same time, the result would be five, not six like it is supposed to be. So a lock basically gives one instance of the code the ability to make the change, and then after that lock is released the next instance of the code can do what it needs to do. So that value goes from four to five, and then five to six.

So in the context of the "bug" in v0.7 and prior versions, the setting used for maximum locks for BDB was below that necessary for a transaction that first made it into a block on a v0.8 node. v0.8 doesn't use BDB and therefore doesn't have that same restriction. There was a rule in the bitcoin protocol, due to a specific BDB configuration, that wasn't expressly known previously.

[Edit: And here is the info specifically on the BDB locks configuration:

Quote from: marcus_of_augustus on March 12, 2013, 03:23:19 PM

Here's the Berkeley DB tutorial for anyone who might want to do some reading on sizing your database correctly and lock limits.

http://www.stanford.edu/class/cs276a/projects/docs/berkeleydb/ref/lock/max.html

Quote

The maximum number of locks required by an application cannot be easily estimated. It is possible to calculate a maximum number of locks by multiplying the maximum number of lockers, times the maximum number of lock objects, times two (two for the two possible lock modes for each object, read and write). However, this is a pessimal value, and real applications are unlikely to actually need that many locks. Reviewing the Lock subsystem statistics is the best way to determine this value.

nebulus

hero member

Activity: 490

Merit: 500

... it only gets better...

What exactly do you mean by roll back? I can't get the chain on 0.7 to update...

Stephen Gornick

legendary

Activity: 2506

Merit: 1010

Quote from: AndyRossy on March 12, 2013, 06:40:09 PM

Not sure if you're really trying to place any blame on the merchant.... errr

No, I'm not.

It might be common sense / generally accepted protocol that when an alert is received by the bitcoin client to basically halt all payment processing until the problem is understood.

In the case of the March 12th fork, the alert didn't go out until after there had already been six or more confirmations on transactions. There's nothing an OKPay or other merchant could have done if this transaction had occurred a few blocks earlier (after the fork started but before the alert was released). The merchant can't know at five o'clock that at six o'clock miners will abandon your chain and instead help a fork become the longest chain.

cypherdoc

legendary

Activity: 1764

Merit: 1002

Quote from: Stephen Gornick on March 12, 2013, 06:29:44 PM

See above. It was a scenario (a transaction requiring 10,000 locks, or about 5,000 inputs) that hadn't been tested (again, from what conversations I've seen on it).

what are locks?

AndyRossy

sr. member

Activity: 448

Merit: 250

Quote from: Stephen Gornick on March 12, 2013, 06:20:43 PM

Quote from: Ivica on March 12, 2013, 03:38:19 PM

There shouldn't be such transactions, there can be fault at merchant end accepting double-spend transaction during fork time.

When the fork was first being looked into, at March 12 2013 and at 00:03 AM the main net at the time (mined by v0.8 clients) was at block 225439, yet other clients were stuck on the fork at the time (mined by v0.7 and prior clients) was at 225431.

Quote

00:00   sipa   ;;bc,blocks
00:00   gribble   225431

00:01   sipa   but it seems blockexplorer is stuck too...
00:01   sipa   as i'm on 225439

- http://bitcoinstats.com/irc/bitcoin-dev/logs/2013/03/12

So even before anyone knew for sure that a fork was underway, there could have been transactions with six confirmations on the v0.8 side. If for whatever reason those transactions didn't also already get included in blocks on the v0.7 side, there was the opportunity to perform a race attack to double spend the transaction that had already been included on the v0.8 side.

We now know that exact scenario is what happened with the transfer to OKPay that is being claimed to have been successfully double spent. Though in that instance, it was after the alert went out that OKPay still processed the deposit as being valid. So that specific incident could have been prevented had they halted processing once the alert went out, but there still was a window between when confirmations would occur since the fork started and when the alert eventually went out.

Not sure if you're really trying to place any blame on the merchant.... errr

Ekaros

hero member

Activity: 728

Merit: 500

Quote from: Stephen Gornick on March 12, 2013, 06:29:44 PM

Quote from: DeathAndTaxes on March 12, 2013, 10:47:17 AM

This was an undocumented issue with v0.7 ("bug").

And the term v0.7 is being misused here. It really is a pre v0.8 bug, right? i.e., it has existed since day one ... a configuration setting for BDB that has been that way since v0.3 at least, if I read correctly.

Quote from: Ekaros on March 12, 2013, 05:47:23 PM

Since when was this bug in software? So how long did the testnet have time to discover it?

See above. It was a scenario (a transaction requiring 10,000 locks, or about 5,000 inputs) that hadn't been tested (again, from what conversations I've seen on it).

Thanks.

Still, if it is that old I wonder how well scenarios with really large scale user base and amount of daily transactions is tested. As some people prophecises about global use. But wouldn't such day also mean millions to billions users and thus very large amount transactions per block... I have to look into this myself when I have time...

Stephen Gornick

legendary

Activity: 2506

Merit: 1010

Quote from: DeathAndTaxes on March 12, 2013, 10:47:17 AM

This was an undocumented issue with v0.7 ("bug").

And the term v0.7 is being misused here. It really is a pre v0.8 bug, right? i.e., it has existed since day one ... a configuration setting for BDB that has been that way since v0.3 at least, if I read correctly.

Quote from: Ekaros on March 12, 2013, 05:47:23 PM

Since when was this bug in software? So how long did the testnet have time to discover it?

See above. It was a scenario (a transaction requiring 10,000 locks, or about 5,000 inputs) that hadn't been tested (again, from what conversations I've seen on it).

Stephen Gornick

legendary

Activity: 2506

Merit: 1010

Quote from: Ivica on March 12, 2013, 03:38:19 PM

There shouldn't be such transactions, there can be fault at merchant end accepting double-spend transaction during fork time.

When the fork was first being looked into, at March 12 2013 and at 00:03 AM the main net at the time (mined by v0.8 clients) was at block 225439, yet other clients were stuck on the fork at the time (mined by v0.7 and prior clients) was at 225431.

Quote

00:00   sipa   ;;bc,blocks
00:00   gribble   225431

00:01   sipa   but it seems blockexplorer is stuck too...
00:01   sipa   as i'm on 225439

- http://bitcoinstats.com/irc/bitcoin-dev/logs/2013/03/12

So even before anyone knew for sure that a fork was underway, there could have been transactions with six confirmations on the v0.8 side. If for whatever reason those transactions didn't also already get included in blocks on the v0.7 side, there was the opportunity to perform a race attack to double spend the transaction that had already been included on the v0.8 side.

We now know that exact scenario is what happened with the transfer to OKPay that is being claimed to have been successfully double spent. Though in that instance, it was after the alert went out that OKPay still processed the deposit as being valid. So that specific incident could have been prevented had they halted processing once the alert went out, but there still was a window between when confirmations would occur since the fork started and when the alert eventually went out.

Littleshop

legendary

Activity: 1386

Merit: 1004

Quote from: Ekaros on March 12, 2013, 05:47:23 PM

Since when was this bug in software? So how long did the testnet have time to discover it?

While a bug is easy to spot in hindsight, this but looks like it would have not easily emerged even in testnet unless testnet was being used quite hard. Yes, I know complex scripts to can do that and if a series of complex to simulate transactions are not out there, maybe they should be.

Ekaros

hero member

Activity: 728

Merit: 500

Since when was this bug in software? So how long did the testnet have time to discover it?

proudhon

legendary

Activity: 2198

Merit: 1311

Sukrim

legendary

Activity: 2618

Merit: 1007

Quote from: dserrano5 on March 12, 2013, 03:57:54 PM

Quote from: DeathAndTaxes on March 12, 2013, 01:09:35 PM

It was important to resolve this before either chain was more than 100 blocks past the fork as then those coins would be spendable which would have been "chaotic".

Why? After 120 blocks, what makes those coins different from any other unspent output?

The limit to spend newly mined coins is actually 100 blocks (120 in the UI) and since they are coming "out of thin air" they would be viewed as nonexistant as in "never been there" in a different chain.

Tesla71

sr. member

Activity: 302

Merit: 252

I think its because you have to wait for 120 confimations on coins that where mined before they add up in your wallet

dserrano5

legendary

Activity: 1974

Merit: 1029

Quote from: DeathAndTaxes on March 12, 2013, 01:09:35 PM

It was important to resolve this before either chain was more than 100 blocks past the fork as then those coins would be spendable which would have been "chaotic".

Why? After 120 blocks, what makes those coins different from any other unspent output?

Topic: Understanding why the call to rollback to v0.7 was made. (Read 3298 times)