Pages:
Author

Topic: Explain the gox transaction malleability issue like you are five (Read 10241 times)

newbie
Activity: 42
Merit: 0
Good explanation. Thank you.
newbie
Activity: 14
Merit: 0
Code:
2 + 3 = 5
is a mathematically true statement.

Code:
2 + 3 + 0 = 5
is mathematically true, and in fact is the same statement.

They've got different hash values though, because hash functions care about binary representations, not mathematical equivalence.

 Cheesy Good one

If you compare the signature on the bitcoin transaction (on an input of it actually, like a transaction is composed of multiple checks.) with a traditional  Roll Eyes  wet handwritten signature, then the malleability is like the signature being done in another color of ink. A grafologist would still conclude that the purported sender could have made that signature, if he didn't know that the real sender always used a very specific color and composition of ink.
newbie
Activity: 14
Merit: 0
...

Malleability is a potential and hypothetical issue nuisance, which only became possible to exploit at MtGox for two reasons: because Gox failed to correctly implement Bitcoin specification properly, and also because it failed to implement proper workarounds for this issue. You correctly pointed out second reason, but the first is more important to point out, in my opinion, because this is why other exchanges are much less likely to be affected, if likely at all.
Gox didn't follow the specification, which required tx signature to be encoded with ASN1/DER encoding. This requirement was specified in April 2011: https://en.bitcoin.it/wiki/Protocol_specification#Signatures
Instead they used some sloppy format which was not DER encoding but was still accepted by SSL library and old reference client. When tighter checks were implemented in bitcoin reference client (the main reason for which was actually to prevent malleability issue), their transactions, which violated bitcoin spec, were rejected. Basically, their transactions looked like what hackers would employ to exploit this issue. That allowed hackers to pick these rejected transactions up, malleate them to "fix" the signature format, and re-submit. Ironically, hackers were helping MtGox to propagate their malformed transactions through the network.

I have looked up the change logs of the Bitcoin client of the previous year, and I have yet to find any sign that the client switched to more stringent checks. There are some code changes on github that you referred to earlier, but even if those made it into the default client they wouldn't go as far as to fix the problem, because these code changes still leave open lots of room for malleability. But with access to good mining equipment it would be somewhat easy to race the original transaction being transmitted on the bitcoin network with your manipulated version.

I would still like to see an actual instance of the original Mt. Gox transaction and the transaction that got in the blockchain instead and who mined that.
legendary
Activity: 1400
Merit: 1013
3. it seems like the gox omnibus acct should net out even at the end of the day.  double deliveries would raise red flags (one would think) when all has been netted

You assume MtGox is doing daily reconciliation.
Even daily reconciliation isn't really good enough. It's the 21st century - why shouldn't these kinds of services be expected to reconcile with each new block?
donator
Activity: 1218
Merit: 1079
Gerald Davis
So does that mean if you saw a transaction floating around the network that has a TX signature with junk padding, you could copy it, remove the padding and resend so the benifactor of that transaction would get paid twice? I take it that would have to be someone from Mt Gox resending the payment in order for it to work?

Yes and that is what it is assumed that attackers did.  Of course of that original pair only one could be confirmed so yes to get double paid you would also need to trick MtGox into cutting you another payment.   Of course MtGox client is horribly defective and there were thousands and thousands of legitimate reasons why they had to cut new payments so the attackers requests would hide in a sea of requests created by their incompetence.

Quote
So just to be clear it is not possible to find an Unconfirmed Transaction on the Network with junk padding, copy it and change things like recipient, amount, remove padding and resend?

Correct, you can not do that.  The inputs, outputs, value of tx, fee paid, recipients, and value to each recipient are immutable.

Quote
Jeez you would think Mt Gox would be looking for ways to speed up transactions, so you would think they would know Miners are rejecting their transactions with junk padding on the signatures and amend any script so as to remove the junk padding and speed up confirmation times. Crazy incompetence.   

The OP was a simplified explanation of what MtGox got wrong.  MtGox had a huge laundry list of fails in their client wallet. At minimum (just by observing their "missing" transactions) they
a) tried to spend immature newly mined coins which caused the tx to be dropped (until they matured if Gox was still broadcasting it) by some or all of their peers.
b) tried to make payments which violated the spam rules and thus wouldn't be relayed by some or all of their peers.
c) paid insufficient fees on low priority tx which caused them to not be relayed by some or all of their peers.
d) used non canonical signatures which caused them to not be relayed by some or all of their peers.
e) double spent their own coins which caused the tx to be dropped (correctly) by some or all of their peers.

It is very likely their wallet was deficient in other ways these are just symptoms that I and others have observed by looking at the transactions MtGox created.  As an example, obviously if MtGox is creating a tx which is spending newly mined coins less than 120 blocks from when they were mined we know their wallet is not performing that required check.  How many other deficiencies are in the "Gox Special v0" custom client.  Nobody outside MtGox knows for sure but you should not take this list to be exhaustive.
donator
Activity: 1218
Merit: 1079
Gerald Davis
3. it seems like the gox omnibus acct should net out even at the end of the day.  double deliveries would raise red flags (one would think) when all has been netted

You assume MtGox is doing daily reconciliation.  Given all the other things they either didn't do, or did wrong why would you assume that.  It is entirely possible they had absolutely no check's and balances in place and the same attacker "double withdrew" dozens or maybe even hundreds of times.  The issue with MtGox generated bad withdraws has been going on for a month.
newbie
Activity: 47
Merit: 0

So does that mean if you saw a transaction floating around the network that has a TX signature with junk padding, you could copy it, remove the padding and resend so the benifactor of that transaction would get paid twice?

No, double-spending is not possible within bitcoin protocol. MtGox didn't see that original transaction went through and re-issued it (automatically or manually - I don't know) with different inputs.

So just to be clear it is not possible to find an Unconfirmed Transaction on the Network with junk padding, copy it and change things like recipient, amount, remove padding and resend?
You cannot change recepient or amount without breaking the signature and invalidating it. You can remove padding though - thus creating a good-looking transaction. If the inputs used in this transaction were not yet spent, you can get this transaction confirmed (included into a block). So basically you will do what originator of this transaction was intending to do anyway, the only difference is that you modify transaction in a way which may be unexpected by some outdated bitcoin software like MtGox's. And that is a known issue since 2011, so reference client and probably most custom clients don't rely on that and are not affected.

Jeez you would think Mt Gox would be looking for ways to speed up transactions, so you would think they would know Miners are rejecting their transactions with junk padding on the signatures and amend any script so as to remove the junk padding and speed up confirmation times. Crazy incompetence.  

They became too big and important to care. Just like many banks.
This is my pure speculation, but probably their attempts to speed up transactions actually played role here too. They had partner miners, so they were able to send transactions directly to them, bypassing other nodes. "Network doesn't accept our transactions? Fuck the network, we will go straight to the miners". Until miners finally updated their software.
full member
Activity: 154
Merit: 100

The problem is, there is indefinite ways to alter a cheque without invalidate it. We need to live with transaction malleability and actually it's no big deal

Gox didn't follow the specification, which required tx signature to be encoded with ASN1/DER encoding. This requirement was specified in April 2011: https://en.bitcoin.it/wiki/Protocol_specification#Signatures
Instead they used some sloppy format which was not DER encoding but was still accepted by SSL library and old reference client. When tighter checks were implemented in bitcoin reference client (the main reason for which was actually to prevent malleability issue), their transactions, which violated bitcoin spec, were rejected. Basically, their transactions looked like what hackers would employ to exploit this issue. That allowed hackers to pick these rejected transactions up, malleate them to "fix" the signature format, and re-submit. Ironically, hackers were helping MtGox to propagate their malformed transactions through the network.


So does that mean if you saw a transaction floating around the network that has a TX signature with junk padding, you could copy it, remove the padding and resend so the benifactor of that transaction would get paid twice? I take it that would have to be someone from Mt Gox resending the payment in order for it to work?

So just to be clear it is not possible to find an Unconfirmed Transaction on the Network with junk padding, copy it and change things like recipient, amount, remove padding and resend?

Jeez you would think Mt Gox would be looking for ways to speed up transactions, so you would think they would know Miners are rejecting their transactions with junk padding on the signatures and amend any script so as to remove the junk padding and speed up confirmation times. Crazy incompetence.   
newbie
Activity: 47
Merit: 0
3. Not sure what you mean. But if gox really double paid some customers, they are the one to absorb the loss (or they will close and run)

Actually they may try to recover some, if they are able to figure out who did that. But I wouldn't bet much on it.
Also they may go into liquidation in a civilized way. That would be the best option for everybody. Bitcoin foundation should use all their influence to pressure Mark to do that.
newbie
Activity: 47
Merit: 0

The big question is how long has this been going on and has someone actively exploited it?

This is simply gox's problem, as they shouldn't follow the transaction flow this way in the first place.

It's wrong to think this is just Gox's problem. It's a problem of Gox's customers (large part of bitcoin community), and this is a problem of Bitcoin public image. When "the oldest and at one point the biggest bitcoin exchange" is run by such moron, that taints the whole community.
newbie
Activity: 47
Merit: 0

The problem is, there is indefinite ways to alter a cheque without invalidate it. We need to live with transaction malleability and actually it's no big deal

Malleability is a potential and hypothetical issue nuisance, which only became possible to exploit at MtGox for two reasons: because Gox failed to correctly implement Bitcoin specification properly, and also because it failed to implement proper workarounds for this issue. You correctly pointed out second reason, but the first is more important to point out, in my opinion, because this is why other exchanges are much less likely to be affected, if likely at all.
Gox didn't follow the specification, which required tx signature to be encoded with ASN1/DER encoding. This requirement was specified in April 2011: https://en.bitcoin.it/wiki/Protocol_specification#Signatures
Instead they used some sloppy format which was not DER encoding but was still accepted by SSL library and old reference client. When tighter checks were implemented in bitcoin reference client (the main reason for which was actually to prevent malleability issue), their transactions, which violated bitcoin spec, were rejected. Basically, their transactions looked like what hackers would employ to exploit this issue. That allowed hackers to pick these rejected transactions up, malleate them to "fix" the signature format, and re-submit. Ironically, hackers were helping MtGox to propagate their malformed transactions through the network.
If MtGox submitted correct DER-encoded signature in the first place, hackers would have to figure out how to malleate the transaction without breaking the spec so that the transaction is still accepted by the network, and then outrace the originator of the transaction in propagating the malleated message to the miners, or use their own miner, and be lucky enough to mine the block before somebody else mined the original transaction. A lot of trouble - and all that just to modify tx id, which is already a published known issue, and can be easily detected with manual investigation or proper workarounds in place.

It's not malleability per se which is the reason for MtGox failure, but failure to properly implement bitcoin specification + malleabiiliy + failure to implement workaround + lack of attention to abnormal behaviour in their system + inability to react quickly when the issue became glaringly apparent+not testing their system with reference implementation+....
Or, in other words, lack of competence, unacceptable for the only golden member of Bitcoin Foundation.
There was so may ways this could have been alright - and only utter stupidity and incompetence allowed this to go wrong.
sr. member
Activity: 378
Merit: 250
FYI, at least 3 of us had issues last night with double transactions. We sent a payment and it appeared a second time, one of them is unconfirmed. Not sure if it is related to this.
newbie
Activity: 1
Merit: 0
So Gox has to manually compare all previous cheques issued with photos to sort out the mess of their cashbook?

It is always not a good practice to compare the photos (transaction hash). They should check the balance of all their bitcoin accounts

A cryptographic hash is not a "photo", nor alternate representations comparable to a "dirty cheque".

They must be unique and reproducible identifiers which are unambiguous. If this is untrue of transaction IDs, why bother returning them at all? In that sense it is a protocol flaw.

Minor vulnerabilities can be used to leverage more serious exploits, this needs to be fixed.
hero member
Activity: 588
Merit: 501
Good explanation. Thank you.
Now I wonder, why someone doesn't open a bank called 'The Bitcoin Bank' and operate it in a country like Germany, Finland or the other few that actually regulated bitcoin in a positive way?

because at that point they would be a bank first, a bitcoin facilitator second, even if their only "currency" is cryptocurrency.

legendary
Activity: 1792
Merit: 1111
So Gox has to manually compare all previous cheques issued with photos to sort out the mess of their cashbook?

It is always not a good practice to compare the photos (transaction hash). They should check the balance of all their bitcoin accounts
newbie
Activity: 15
Merit: 0
So Gox has to manually compare all previous cheques issued with photos to sort out the mess of their cashbook?
legendary
Activity: 2912
Merit: 6403
Blackjack.fun
Because other exchangers aren't being run by fairies dancing around the magical pot of bitcoins when the owner is drunk dead in the basement ?

I think you mean "dead drunk", though I like your way better.


I always thought it's first the drinking and then the dying =))))  , so > drunk dead.

But right now ,It  won't be long till it he ends dead with lots of holes in it , depending on how much money people lost with his exchanger.
sr. member
Activity: 868
Merit: 250
Even worse, some customers find the gox's bug and try to exploit it. After they cashed the cleaned cheque, they complain to gox saying that they have not received a cheque. Since gox can't find the cheque in the record of Bitcoin Bank, they credit the bitcoin back to the customer's gox account so the customer doubled his bitcoin at the expense of gox's fund

That could be easily proven by gox, since there will be "double spends" to the same address in the blockchain.
hero member
Activity: 492
Merit: 503
Because other exchangers aren't being run by fairies dancing around the magical pot of bitcoins when the owner is drunk dead in the basement ?

I think you mean "dead drunk", though I like your way better.
Pages:
Jump to: