Topic: What the "average user" needs to know about Transaction Mutability - page 2. (Read 38937 times)

One transaction will always be confirmed (assuming the original tx is valid).  Even in a traditional double spend nodes (different outputs for same inputs), each node will independently rely the original and drop the duplicate.  "Original" and "duplicate" in this context are based on which version a particular node sees first.  In a race situation, it will not necessarily be the same for each node.  One of the two copies ends up in the memory pool of miners, and eventually one miner will put the copy it knows about into the next block.

Any tx can be duplicated but to be included in the next block it has to win a race with the original to at least some % of the miners.   MtGox made this race incredibly easy by creating transactions with non-canonical signatures (among other issues) which were dropped by most nodes.   So the "good original" had essentially no chance of making it to a miner which is why so many "legit" withdraw requests were broken and never confirmed.  No miner ever became aware of them because they were dropped by relay nodes between MtGox and the miners.   To spoof MtGox, the attacker didn't have to split miliseconds and work against the clock to win the "race", they could manually modify the broken tx and rebroadcast it minutes or hours later and still be included in the next block.  It wasn't any more of a "race" between two runners where one of them dies on the starting line, no matter how slow the other run is, eventually he is going to "win".

The one issue "you" as a user need to be more aware of is that the change from a tx is identified by its hash (and index).   If you spend unconfirmed change BEFORE a tx is included in a block, it is possible the duplicate will be included instead.  The original tx will not have an issue (both the tx you created and the mutated tx "do" the same thing) however the unconfirmed change is no longer valid.  It refers to the wrong tx id so the subsequent tx will fail.

What I still do not fully understand is whether capturing and manipulating the transaction ID of a transaction would result in neither the original nor the modification making it to confirmation due to the perception of a double-spend attack, or would one win out and in all cases the desired transaction is accomplished.

The former constitutes a potentially severe DOS attack.  The latter would not be that big a deal (in my own use-case and as far as I currently understand.)

Or is it the case currently that the end result of an intercept+modify_hash+redistribute attack would be somewhat left to chance due to differences in the protocol implementation and such-like.

No the attack didn't involve tx from the user or deposits as both would require confirmations and any breakdown due to malleability would have not resulted in extra payments.

Simple version:
User has x BTC on his MtGox account.
User created a withdraw request for x BTC.
MtGox creates a withdraw transfer with tx id "A".
MtGox's wallet is creating "broken" tx anyways which make legitimate issues for tx propagation.
User takes tx id "A", modifies the mutable parts and this produced tx id "B".
Tx "B" ends up in a block and user has received x BTC.
User notifies MtGox they haven't received their withdraw.
MtGox looks in blockchain and there is no tx "A" and thus assumes (incorrectly) this means user has not been paid.
MtGox creates a new tx ("C") for x BTC to the address specified by user and has now paid the user twice.

User could not either deposit the double bitcoins and pull the con again or modified tx "C" so it is tx "D" and report a second failure ("MtGox tx C didn't confirm either please pay me again, I mean pay me) to continue to the theft.

MtGox has had legitimate issues with withdraws not being confirmed (due to their own errors) for more than a month.  How many times did an attacker pull the attack described above? How many bitcoins were double paid?



Or they saw all these requests as valid normal outflows and when the hot wallet got low, moved funds from one or more cold wallets to replenish the hot wallet so the could continue to double, triple, quadruple pay withdraw requests.

Only MtGox knows how much they overpaid and they simply are not talking.

Ok, now I finally grasped the attack I think on gox.

In my own wallet(s)
Transaction A is created
Transaction B is created Based on A.
Transaction C based on B is sent to Gox
  ... gox acts idiotically and recognizes C before B is finalized
  Bitcoins bumped up in my gox account
I invalidate B so the bitcoins are still in my wallet
I withdraw bitcoins from gox and it says ok as its database of my account shows bitcoins it never really got.
I have just doubled my money

Rinse and repeat until gox notices the problem.

Sounds like gox would be empty in short order.

So then they wise up and shut the door (or they just see no money in their wallet). If they didn't lose most of their bitcoins in the process then they could just make a minor tweak to their process and only recognize confirmed transactions. Unfortunately I think they have been drained dry and are just looking for a way to open the bank with no money in the vault and prevent a run. They need to get some reserve of bitcoins back in their vaults to make payouts so people can feel calm and safe, and I suspect their bluster about waiting till bitcoin fixes itself is a delaying tactic.

Or if you want to be optimistic, gox could have kept 90% of their funds in a cold wallet and all is safe. They could easily verify such things to the public though and their lack of transparency is discouraging. A simple, "look guys, don't sweat it, here is one of our many addresses in cold storage with $1M in bitcoin and this message was signed from its private key. Chill out, and we will have things flowing smoothly in a jiffy."

Unfortunately, I am a pessimist, and this is the second time that gox has f-ed up on a massive scale. I think this will lead to a level of disappointment that wil make mybitcoin look like a small pickpocketing.

There were two different scenarios involving the mutability.

The first was the attackers were ONLY modifying their OWN transactions.  They would then report the tx failed, MtGox would lookup the tx id, see it wasn't in any block and pay them a second (and third? and twenty eight?) time.

After MtGox wised up 30 days later someone (possibly the same people, or new people) began mass modifying the tx of OTHERS.  This particular event has no direct ecnomic value and your right it is likely griefers, or someone looking to damage the credability of the Bitcoin network.  When people are talking about thieves they are talking about the former event.

My understanding is that some people were using transaction mutability to make it look like they never got paid for their BTC withdrawal. They were then convincing the exchange to (I believe manually) resend a new transaction because they claimed the original transaction never went through.

I don't know if I would call the attackers thieves. By the looks of things they aren't making off with bitcoins, they are just invalidating transactions that others are making, and thus making the db's of exchanges inconsistent with the actual bitcoins they possess. They are more like griefers.

The piece of the puzzle I don't understand is how this is a problem for exchanges. Since the very start of bitcoin it was considered that 6 confirmations were required before a transaction was considered valid for any serious financial situation. This probably could drop down to 1 or two conformations since the network is pretty robust now, but considering 0 confirmations as being worth anything? Heck, you wouldn't even do that for an e-commerce site.

My basic conceptual model of how you build any online service using bitcoins as a unit of account is you have a bitcoin address for a user, and you get bitcoins received at that address with 1 confirmation and use that to base the user's balance off of. In your db you then record transfers to and from that user and add and subtract accordingly through a transaction log. So the balance is the sum of bitcoins received + credits - debits.

Not really rocket science, and I feel something must be missing in my view as this very basic system seems the most obvious to me, and it seems like it would be immune to the problems that are plaguing all of the exchanges. Is it just that they were dancing on the edge of the razor of zero-confirm transactions because they felt that turn around was that important?

Please, tell me where I am wrong. Something is not adding up.

Well, the mutated transactions beat the original ones, even tough they were broadcasted 2,3 or 4 minutes later - not only one time, but multiple times. I guess mine is a poorly connected node (its not even a full node, 8 connections only), while the attacker has nodes very well connected to pools.

Furthermore, I run Bitcoin via Tor (as I think everybody should do) - I don't know if that could make my transactions "slower" than those of the attacker.

I'd like to understand how that worked. In 4 minutes, your TX should have propogated across the whole network. I don't see how a mutated TX could have beat yours to the pools unless the attacker mined the block himself. The mutated TX must have been transmitted before the 4 minutes.

No, the chance of an invalid transaction is not vanoshingly small if the buyer has used unconfirmed change. "Listening for N seconds" won't work, I had ALL my transactions mutated since the bot started spamming and most of them were mutated +4 minutes after the original was broadcasted, and still ALL the mutated transactions made it into blocks invalidating the original ones I broadcasted.

The attacker must have nodes very well connected with pools.

Interesting.  

So if the transaction is questionable, the BitPay receipt, instead of saying "PAID," would say "UNABLE TO CONFIRM PAYMENT UNTIL NEXT BLOCK."  It is then up to the merchant to decide what to do.  

BitPay could provide an "issue refund" button to immediately send those same coins back to the customer, should the customer be unable to wait.   Or if the merchant knows the customer, then he might say "don't worry about it, it'll be fine and if not just get me tomorrow."  Lastly, if the customer is going to be eating in the cafe anyways, they'll just wait.  

Since these questionable transactions should be very rare [especially with better wallet coin control], perhaps this is a workable solution.  

Because of the way transactions are constructed, the data that is signed is a subset of the data that is hashed. It's possible to change the transaction in ways that do not invalidate the signatures, but do change the hash.

For some types of transactions this is a useful and necessary property. For other types of transactions (most of what happens on the blockchain today), it is a nuisance that wallet software must be careful to account for.

If you're running an exchange whose wallet does not handle mutability correctly, and also makes several other errors at the same time, you can end up losing a lot of bitcoins.

I should read the code, but I would assume that the transaction ID is a 256 bit hash of the transaction details. If so, then it should take quite a birthday attack to create a valid alternate transaction ID. If the ID isn't a hash of the transaction details, then where is the hole in this idea, because it seems readily obvious.

It may not be the case that 'logs of attack machines' were needed, nor that the the software was especially based on the Bitcoin reference code.  This incident brings back and old memory:


That thread, and a sister thread where the guy used the code to map out and analyze the entire Bitcoin network are interesting and worth the read whether or not they are related to this attack.  I expect that we'll see many copy-cat attacks along the lines of the one under discussion here going forward.  I'm deeply surprised that it took this long to be honest.

Sounds like a job for a payment processor.

There are ways to improve PoS, but the PoS has to be smarter than simply accepting any zero-confirm transaction. It needs to analyze the network and the transaction: how well it is connected to peers, if it sees any malling/double-spending of the current transaction, if a fee was included, if there is a chain of unconfirmed inputs, etc.

I was responding to etotheipi's opinion that we should just write off PoS applications as not viable.

Ideally we'd have some sort of "PoS applications standard" that popular / "well behaved" mobile wallet apps would adhere to:
* never send a TX using any unconfirmed UTXO as an input
* never send a TX without the appropriate fee
* (something else?)

Andreas Schildbach's android wallet already forces #2. And we could potentially do away with #1 in future, once TX mutability is fixed properly.

All PoS systems could accept a zero conf TX only if it meets the standard, otherwise wait for a block before displaying "PAID" to the operator. ie. the PoS system should be as defensive as practical, but in a well publicised, well defined way. Then it's up to the customer to use a wallet app that conforms to this standard if he doesn't want to wait at the counter.


As per the standard I propose: ignore it until it makes it into a block.
If this behaviour was standard then customers / wallet apps would quickly learn better than to do this.


It remains to be seen how big an issue this would be in practice. As was suggested elsewhere in this thread,
a wallet could have a "fragmentation threshold" that it attempts to maintain, eg. by splitting change to itself, which would help mitigate this.

Of course, in an ideal world our wallets will all do an opportunistic CoinJoin whereever possible too ;-) That'll require a certain level of fragmentation to be effective anyway.

Maintaining a single large UTXO, slicing a little bit off every time you make a payment, and returning change in another single large UTXO... just isn't viable from a financial privacy point of view anyway.

I think you're muddling two debates into one.  Prior to the risk of malleability attack, zero-confirm transactions for purchasing beer here in Vancouver were reliable regardless of whether the transaction used confirmed or unconfirmed change outputs.  So we already know that, malleability-problem aside, zero confirm transactions work for low-value PoS purchases.  

The ability of shop-keepers (and BitPay) to continue accepting zero-confirm transactions is complicated now because when the network is under malleability attack, zero-confirm transactions built from unconfirmed change outputs are not reliable.

We can restore the reliability of zero-confirm transactions to how secure we thought they were (in our blissful ignorance) if we disallow using unconfirmed change.  But there's still a bunch of complications, for instance:

- What should BitPay do if it receives a valid transaction that was built using unconfirmed outputs while the network was under malleability attack?

- What happens when Bob has a large balance on his phone but no confirmed coins to purchase that martini for the adorable girl he just met?  [I have money, I swear.  I just need to wait for the next block to confirm     Declined my friend!]

I disagree.This is a very popular use case, and there's no reason that it can't work "well enough" in the real world.

Consider this typical scenario of a pub selling beer for bitcoin:
* we have a well connected node
* we receive a ZC txn for the beer composed entirely of confirmed UTXO as its inputs (even just 1 block deep)
* we listen for N seconds and do not observe other txns that double spend our ZC txn's inputs
* our ZC txn pays the appropriate fee to be included in a block quickly

Should we give the beer to this bloke right away, or wait for his txn to make it into a block?

I'd argue we're pretty safe just giving him his beer. It's worth all of $5.50, and the chance of a double spend invalidating that txn, given the above observations, is vanishingly small. Yes, he might:
* have a node connected to hashpower we're not aware of / well connected to
* be able to broadcast his double spend txn to that part of the network without us noticing it
* be lucky and get it into a block
* be really fast at drinking his beer and running out the door

But it's not bloody likely :-)

Honest customers will always outnumber this guy 1000:1.

I am confident that eventually tx ids will be immutable.  Sipa proposal seems to be a good way forward and it can be done gradually (favor "better over "worse", then consider "worse" nonstandard, then eventually "worse" is invalid at the protocol level).  It is a complex issue because there are multiple ways to mutate a tx.  

Long term goal should be immutable tx ids.
Short term goal should be to minimize the disruption caused by tx id mutability.

As a side note alt-coin designers if you actually want to improve on Bitcoin and not make a pump and dump, don't make the signature part of the transaction input have it follow the tx body and sign the entire transaction.

Bitcoin
-----------------------

Inputs (including signature inside)

Output


Better
--------------------------
Header
Input(s)
Output(s)
Signature(s)

The tx body is the header, inputs, and outputs.  Hash the body you get the tx id.  Use the same hash and the private key for each input to produce the signatures.  Append the signatures to the tx body and broadcast the whole thing.   You don't even need to provide pubkey in the input as it can be reconstructed from the signature.  I am not really sure if Satoshi was drunk the day he came up with how to sign transactions but it has to be the most overly complex, convoluted system which really provides no benefit and lot of edge cases.  Compared to the elegance of other parts of the design is seems out of place.  I can't think of a cryptographic system where you sign a payload and then put the signature back into the payload.  It is normally payload & signature.