Topic: Alert: chain fork caused by pre-0.8 clients dealing badly with large blocks - page 3. (Read 155551 times)

The blocks being rejected by 0.7 are:
1. Possible to generate with 0.7 as well as 0.8
2. Perfectly valid blocks according to standard.
3. Being rejected because of a bug in 0.7

This is incorrect. The reason the fork happened is because the 0.8 clients accepted blocks that the 0.7 clients rejected. They kept on going based on that block and the 0.7 clients kept on going based on other blocks.

It doesn't matter if 0.8.1 doesn't produce incompatible blocks because solo miners are currently and will continue to use 0.8.0, which might produce those blocks (this is even if we assume that 0.7 can't produce those blocks). Then, all those 0.8.1 clients will accept the incompatible blocks and will fork from the 0.7 clients once again.

If you want to avoid forks like this, you need to ensure an acceptably-large majority of clients accept and reject the same set of blocks. I think this is best done by a scheduled hard fork, but I haven't thought too hard about it.

The use of the same term for hashers and people who actually validate and prepare blocks for hashing has been causing confusion all over the place for a while now.

Maybe clearer terminology is called for?

-MarkM-

If you're talking to regular users, it doesn't make sense to talk about 'miners' in a way that implicitly doesn't include people who mine through pools, who obviously don't need to do anything since the pool operator will handle that.

That is true, though unlikely.

I have seen it posted that 0.7 can itself produce blocks that some instances of 0.7 will reject; if that is the case it seems to me quite to be expected that 0.8 can too, even when using default settings, because I haven't heard that it ever really went out of its way not to produce just the very kind of blocks that 0.7 can produce that supposedly some 0.7's cannot handle.

I am not really all that sure yet though of the authoritativeness of the claims that 0.7 can actually produce 0.7-killers, at least not without a bit of code-hack-and-recompile shenanigans.

-MarkM-

Now we are getting somewhere. All our disagreement can be distilled into this one single "fact" that the two of us disagreed on and only now that you posted this do I find out what that "fact" is.

So, can anyone point me at evidence for or against that? as far as I knew there was no manual raising by miners and that 0.8 can, by default, generate blocks that 0.7- will reject.

This could not have happened:

For diagnostic purposes, here is a blockchain dataset built by a 0.7.2 bitcoind w/ db 4.8:

     http://gtf.org/garzik/bitcoin/chain-db48-h225429.tar.bz2

It contains blockchain + index, up to height 225429, making it easy to reproduce an injection of too-large blocks at the precise juncture where the recent chain fork occurred.

Byte size: 5755999214 (5.3G)
MD5: 479e6364352d0ec68ea5cf87200f7f1e
SHA1: 18bc8ff5e572fc4a27828341249477eb8355f012
SHA256: 41787306487da1957717d72407fc1e01ed5965202802f7d18e2930a30fc169a2

evilpete


This tries to swim against the tide of all individual incentives built into the system.

Why would anyone want to be on a version that the miners are not using? ... you are opening yourself up to the possible risk of ending up on a useless fork.

Its like a computer, you have to keep it up to date, hardware and software. if you do not upgrade you wont get allong with the rest of the world. if people refuse to upgrade (there are some verry old clients out there) then its just logic that it will stop working at some point, just like with everything else.

If they dont upgrade its there problem.

.. and 0.8 won't make a block by default that is rejected by 0.7.  Miners had to explicitly raise their -blockmaxsize, which is what happened.  It was suggested on a thread in the miners forum and a few folks tried it out.  And we saw what happened.


You're missing the point.  I'm not talking about the default configuration, which is "safe" by default everywhere.  0.8 is safe by default. The problem is when somebody maliciously makes a bdb-buster block because they either want to cause trouble, or because they figure out a financial advantage to do so.

Read what I said.  The only thing preventing another bdb-buster fork is that 51% of the hashpower are running a broken 0.7 on purpose.  If that changes, eg: miners apply the fixes to 0.6 or 0.7, or a ton of asic solo miners turn on at 0.8, then we're back in fork territory when somebody wants to cause trouble.  If the miners fix their bdb config they'll behave just like 0.8 next time a killer block comes along.

But the last thing we need is more people downgrading to 0.7.  It just delays fixing this land mine.

AND not OR.
There is no point in setting anything in 0.7-;
0.7 can NOT make a block that 0.8 rejects.
0.7 AND 0.8 can both make blocks that 0.7 will reject due to a bug in 0.7-.
0.8 can be configured to only make blocks that are accepted by everyone.

When everyone was on 0.7, if it made a block that it itself rejected than nothing bad happened. with 0.8 you had the fork risk suddenly come up.

If you are going to be messing with your install then you should do so by switching to 0.8 with the right config not by trying to mess with the bugged 0.7

Default configuration is included in the source code and as such anyone who compiles an 0.8.1 without explicitly altering said code would get the new configuration.

This wont cause trouble to anyone except for themselves, they will be generating blocks that are rejected by the chain until and unless 51+% are using 0.8+, at which point those using obsolete bugged version are forced to upgrade.

There is already a fixed 0.7, its called 0.8.

This. Most people here are focusing on the manifestation of the problem, and ignoring the continuing pattern of human behavior that increased the risk of this problem emerging. As long as bitcoin protocol is treated as a black box full of poorly documented code and unexpected behavior that is filled with more of the same with every version, the risk will remain significant.

No, the point of the big mining pools switching to non-upgraded 0.7 was to act as the lowest common denominator so that more than 51% of the hash power will (mostly) reject the same valid blocks that the other non-upgraded 0.7 users and merchants.  As close as it can be done, that is.

Quoting what I said above:  "Upgrade" includes upgrading their bitcoind/bitcoin-qt etc to 0.8+ OR setting the DB_CONFIG entries mentioned above.

Releasing a 0.8.1 won't fix anything.  All that it can do is clamp the value that you can set the max generated block size to.  The problem is the code is still there, and so is 0.8.  If somebody wants to compile a custom 0.8.1 from source with the limit at 1MB, they can.

If a person (or a pool) wanted to cause trouble, they could simply change the code in their 0.8.1 checkout and compile, or use 0.8.  Of course they need to be able to solo-mine their own blocks.  Generating a block that's complex enough to overflow the lock tables of old 0.7 bdb based systems is still valid.  If more than 51% of the hash power applies the DB_CONFIG fixes (even on 0.7) then they'll accept it and build on it.

What actually needs to happen is a new fixed 0.6.x, 0.7.x need to come out, and the people who can't/won't go to 0.8.x need to pick up one of those instead.  Or do the DB_CONFIG fixes.  But more than 51% of the hash power has to stay behind until a good 90%+ of nodes have been upgraded.  Because sooner or later somebody will generate one bdb-breaker on purpose.

To repeat, by "upgraded", I mean either switching to 0.8.x or applying the DB_CONFIG changes, or switching to a new 0.6.x or 0.7.x with the bdb changes.

Releasing 0.8.1 won't fix the vulnerable 0.3 / 0.6 / 0.7 nodes.  The moment somebody uses a a couple of avalons to feed a custom patched 0.8 in order to deliberately make a bdb-breaker block we'll be in the same mess again unless there is > 51% of the hash power on un-fixed 0.7 to cause the bdb-breaker block to be orphaned.

The moment somebody figures out a financial angle to exploit this, it'll happen.  eg: a replay of the OKPAY double spend etc.

Anyway, the pool operators have this covered.  The only reason I said anything at all was somehow end users are getting the idea they need to downgrade.  They should not.
If they're on 0.8, they should stay on 0.8.
If they're on 0.7 or earlier they EITHER need to fix their DB_CONFIG; or get a fixed 0.7.x when it comes out; or upgrade to 0.8.
If they're running mining software as part of a pool then the bitcoind they're running doesn't matter - they still need the fixed one.
Merchants particularly have to pay attention.

The only people that should be downgrading are pool operators or large solo miners.

"One scenario is that a takeover could occur where a fork is created with a fake chain faster than the real chain.  The attackers could then hold the users “hostage” and tell them to change to their client (and change the rules) or they will disrupt the system.  Another scenario is that a disagreement will occur over how Bitcoin will be used and a hard fork will be created purposely.  In this case those with the greater computing power will win.   It is speculated that many large players will maintain mining equipment kept in hibernation only to be turned on in the event of such an event."

Wow, conspiracy is fun 

Not as much as other pools. Pools running a 0.8 node were completely broken. As p2pool is decentralized the potentially lost income was proportional to the hashrate behind 0.8 nodes (some of us still run 0.7 and could have found a valid block).

In this particular case we lucked out: a block was found by a 0.8 node and orphaned when the 0.7 chain caught up and none were found by 0.7 node.

0.8 doesn't need to be taught to reject anything, the problem is 0.7 and earlier reject valid blocks due to a bug in 0.7 and earlier.
0.7 can actually CREATE blocks that it will itself reject due to that bug.

Demanding EVERYONE switch to 0.8 is not realistic either.

What needs to happen is that 0.8.1 should be released where its the same as 0.8 only it changes the default CONFIGURATION such that it will NOT create 0.7- incompatible blocks.
When over 90% of the network are using 0.8+ this limitation can be safely removed and people using 0.7 or earlier will be forced to upgrade.

Until such a hypothetical 0.8.1 is released you can MANUALLY apply the needed configuration files using the instructions in:
https://bitcointalksearch.org/topic/m.1620853

and in 0.8 .... what's good for the goose .... 0.7 has been running for much longer in production than 0.8.

Don't forget to mention that p2pool was basically broken by all this :p