Topic: Semi-soft-fork to decrease the risk of tx malleability (Read 4434 times)

do you care to elaborate on that better fix? especially on the reason why it cannot be "ported" to bitcoin. thanks in advance.

The explicitly stated goal right in the BIP62 document was a reduction in dependence on OpenSSL for consensus; which the truth and nearly the whole truth all that was missing was that the concern was less theoretical than presented.  But yes, the unstated privately known consensus vulnerability was a driving force for clearing up that ugly corner with BIP66 immediately rather than mopping it up as a side effect of BIP62. Especially due to fact that BIP62's progress has been slow-- and had grown over complicated. The fact that BIP66 accomplished one of the necessary steps will make BIP62 or its logical successor easier to move forward.

If you note from the timeline that BIP62 had already been revised to accomplish a BIP66 like goal prior to us knowing about that specific vulnerability:  We were aware of the risk class in principle before having specific knoweldge with enough concern that we thought it was worth closing off.  Knowing that it was _actually_ exposed had only an impact on timing and sequencing; and only after it was clear that BIP62 was not rapidly converging (unfortunately, in spite of much progress on BIP62 we continued to find problems with it).

It's sad, there is a vastly superior-- simpler, more comprehensive, and with secondary benefits like enabling more degrees between full and SPV-- malleability 'fix' in elements alpha but I don't have any way to get it into Bitcoin, BIP62 is disappointing by comparison (brillant ideas welcome).

Is this the real reason for BIP66, which you couldn't mention in April?

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009697.html
(Disclosure: consensus bug indirectly solved by BIP66)

Just curious, is this proposal documented anywhere? I spent a few minutes checking the usual suspects (mailing list, Reddit, this board, IRC logs, BIPs) and couldn't find anything. Maybe I missed it somehow?

Thanks.

OP_CHECKLOCKTIMEVERIFY makes it possible to make txouts that cannot be spent until after some block height.

One thing that's particularly beneficial about that is that it makes it much safer for a future BIP to create nLastTime - (by analogy with nLockTime) transactions that cannot be entered into a block AFTER a given block height.  This would be the "fill-or-kill" trade that a lot of people have asked for. 

If a tx that is not valid until after block X produces only outputs that are not spendable until block x+50, then any subsequent transactions invalidated by the tx becoming invalid in a reorg are transactions that would not be valid until 50 blocks later anyway - which removes the primary problem with nLastTime.

So If nLastTime transactions are required to produce no outputs spendable within 50 blocks after their last valid block, it's "safe" insofar as not invalidating subsequent transactions, barring a 50-block reorg.

Is there a github branch and/or description of exactly what is proposed for that (or is the plan to just manually set what BIP each bit refers to)?

Everyone who has commented has spoken up in favor of it, and plenty of people have. So I think so.

66 is in flight and must finish before another softfork can be undertaken.  I'd like further softforks to switch to the new softforking system that doesn't require serializing things.

If you'd like to see faster progress; please go nag miners to upgrade to 0.10.1.

Is there any consensus that CHECKLOCKTIMEVERIFY will be implemented??

Is the plan BIP-66 then BIP-62 and then OP_CHECKLOCKTIMEVERIFY?

It depends on how much effort is put into maintaining backwards compatibility.

Someone could have a string of transactions starting with one with a long locktime.  You couldn't spend that if after the hard fork.

The easiest is probably just to have a version number update.  The txid for transactions with version > ignores the sigScripts.

Quote

We need this to provide a upper bound of the block size. Otherwise it is impossible to determine whether the block size is under 1MB (or other limit)

Ahh ok, thought it was related to malleability.  I agree, if there is a hard fork, that should be added.  I would also add the fee sum as a 2nd piece of meta-data.  That protects against inflation.

Thanks for your reply!  And yours Cryddit.  

I see more clearly how this malleability thing can suck.  

It looks from your link like a partial solution is already proposed by GMaxwell.  An imperfect solution.  

However I see another potential solution: private TX mining.  

The trouble with malleability in the case of the refund situation is that the transaction has been broadcast publicly.  If I were to forego that and submit it privately to one of the large pools, the malleability attacks could never happen.  Sure it wouldn't be mined as quickly, but for these protocols that is probably not an issue.  

I'm sure if there were a demand large pools and/or consortiums of pools would publish pubic keys and accept TXs delivered encrypted.  After all, this means more fees for them even before they charge anything extra.  

In fact I'm a bit surprised nobody is doing that yet.  
  

You only need legacy support for those UTXOs already in the blockchain. For those after the hardfork, only the new txid format will be supported.


We need this to provide a upper bound of the block size. Otherwise it is impossible to determine whether the block size is under 1MB (or other limit)

An alternative is to have both hashes be valid ways to refer to previous transactions.  This means that legacy transactions still work.  This is needed to ensure that you don't invalidate transactions that were created in the past, but have a locktime after the hard-fork happens.

If you are creating a refund transactions, then you can use hash(transaction without sigscripts) to refer to the previous transaction.

It doubles the size of the transaction index though, since it means 2 keys for each transaction.


What is the benefit of this?  It only protects against length changing malleability attacks (which might cover a lot of them).

I had not thought of that in this context.

Well your proposed solution of having different TX types where one is flexible and the other is not or something similar seems good to me.

Forgive what may be an unusually dumb question, but how exactly does it work, that a third party can change the script but cannot change what the script does?  

EDIT:

Nevermind, I get it.  It's the spend script, not the store script.  If they change what it does then there is no transaction. 

Yes, but that would need a hardfork, or a very complicated softfork.

I proposed this last year:

(1) The txid will be the hash of the tx with all scriptSig removed.
(currently, txid is the hash of the tx, including all scriptSig)

(2) The first level of merkle root will be hash of (txid-a|size-a|txid-b|size-b), where txid-a and size-a are the txid and size of tx-a respectively
(currently, the first level of merkle root is (txid-a|txid-b))

With (1), no third party mutation of the txid is possible. Even when spending n-of-m multi-sig output, the txid is mutable only if n signers agree (currently, any 1 signers may mutate it)

However, due to scriptSig malleability, there is an infinite number of ways to record the same tx. Therefore, we need to put the tx size in the formula of the merkle root. For a tx to be valid, it must be equal or smaller than the size indicated. This guarantees that blocks in different nodes will have an upper size limit. However, the actual content the txs could be different, as long as they are valid.

The problem with not using tx hashes as a transaction ID, is that tx hashes are exactly what the block chain itself uses as a tx ID when specifying what transaction's output to use as an input.  If you don't know the transaction hash in advance, you can't make a transaction that spends its output in advance.  And if you can't make transactions that spend outputs before the outputs they spend actually get into the block chain, then a lot of escrow and other protocols don't work. 

We need transaction ID's to be stable.  Can we make the tx hash by exclusively hashing things that nobody outside the original transaction set can change? 

You're not correct, things are not that simple.

Malleability has big impact on any anything depending on transaction hash of unconfirmed transactions, e.g:

• Refund protocols: https://bitcointalksearch.org/topic/m.11149909
• Spending outputs of unconfirmed transactions. Due to third party malleability, a transaction chain could be broken even if all payers are honest

You're correct, TX malleability is only an issue if people use the TX hashes as Ids.

This is because a script saying 1=1 can be changed to saying 1=1=1 and it will still be correct (gross simplification), but the hash of the entire TX changes.

Obviously scripts cannot sign themselves because it would change the script.


"Fixing" malleability would also mean making the script system less flexible which I am against.

It has even been shown that very few of the losses of MtGox could be blamed on TX malleability - it was a total lie.

The first objective is to allow old/obsolete clients know that a soft fork has occured.  It doesn't matter what the bits are defined as.  If after the last known block, there is 950 out of the last 1000 blocks with a version bit set, then the soft fork flag can be set.

The second objective is to allow multiple soft forks to run at the same time.  The process followed could be that the reference client is updated with the soft fork rule and then is allowed to run with it being part of the reference client for 6 months.  The bit could be recovered by having a max block height rule.  If the soft fork doesn't activate before a certain block height, then it is permenently disabled.  This prevents the bit accidently activating the rule later.  If the rule is not accepted, then it is removed/amended in the reference client for the next release.

The inherent assumption is that major re-orgs don't happen.  Each new soft-fork acts as a checkpoint.