Topic: Mt.Gox technical autopsy - page 2. (Read 4226 times)

Are you saying that someone can observe a transaction being submitted to one node and then quickly submit a bogus transaction to other redundant nodes?  If not, how is the original transaction information obtained?  If so, then what happens when the different nodes disagree?

I have an accounting background, so I'm still very confused about why there appears to have been no reconciliations done.

Thanks for the detailed explanation.

Do you have any idea of how high is the percentage of discarded transations under "normal" conditions? Would it be feasible for some miners to be logging those "incidents"?

Anyway, one thing is that we can't detect those "malleability attacks" afterwards and other very different that mtgox can't reconcile its internal accounting with the blockchain and detect each and all discrepancies.

Please let me know if theres some reason that a process like this would not work:

1- Balances for all accounts with no withdrawals are ok (at least from a "malleability issue" point of view)
2- Balances for all accounts with no failed withdrawal + reissue are also OK
3- Accounts with failed withdrawal transactions lets check like this:
    - For each failed transaction lets search if the transaction went through. Obviously not using the txid, but ammount, input, output. If it did, then flag the account as "abuser" and take note of the "leaked" ammount.

Now you should know exactly how much the malleability "issue" is to blame for the "missing" BTC's and also who were the users behind it, and when/where the leaked money did go to.

Is there any technical reason for this to not be a simple process to run against mtgox records or maybe the only reason they havent made this figure public is because they already know that total sum would just be negligible and that was not the real problem?

P.S.: I am aware that there have been some comments saying that one of the faults of mtgox was that maybe they were doing the reissue using different inputs...

Well, that's what makes difficult/impossible to locate the "dupes" using only the blockchain, but mtgox accounting records should have the missing information, i.e. inputs/output/ammount for each attempted transaction, original or reissued, so that it would be a matter of going through that records rechecking everything against the blockchain.

Seriously, this nonsense must stop. Understanding of the transaction malleability issue is completely wrong throughout this thread.

Transaction malleability does not result in two transactions in the blockchain.

This cannot be stressed enough, apparently.

nanonano pretty much had it right with his list.


Nothing here is incorrect, as such, but the conclusions drawn are incorrect.

When mtgox, or anyone else, submits a transaction it does not immediately enter the blockchain. This only happens once a block containing the transaction is mined, resulting in what is referred to as a confirmation.

Transaction malleability relies on the fact that while the inputs and outputs (source and destination) are fixed by cryptographic signatures, the transaction id (txid) is not. In other words, anyone can submit an identical transaction (same source and destination, same amounts etc), just with a different txid.

The way mining works, if a miner already has the "original" transaction in their memory, they will discard the modified transaction. However, the miners' perception of which is the original depends only on which one they see first.

In other words, exploiting transaction malleability relies on submitting modified transactions in the very small window before the original transaction reaches all miners... and then hoping that the modified variant is the one that gets mined first.

Only the transaction that gets mined first gets in the blockchain (with the small caveat that sometimes blocks are "orphaned" if two miners solve a block almost at the same time).

To see which transactions had multiple versions would thus require "live" access to the miners' memory pools. If no logging of these discarded transactions (whichever reached a miner last) happened, there's no way to see it after the fact.

Very interesting explanations about the malleability "issue" and its limited practical impact on mtgox assets.

The "funny" thing here is that MK has deluded himself into thinking that the "It's gone" will suffice as an explanation without offering any proofs in an environment so full of them.

Sooner or later he will have to provide internal accounting records and then it will be very easy to finally know what (and when) really happenned.

One of most important accounting records he will have to provide is the list of all the cold wallet addresses that they were using at each moment in time. Being the most valuable assets of the company (apart from bank accounts) all of them should be perfectly identified in company reports showing periodical balance.

Then we will have the full picture of what was happenning and since when.... the same full picture that Karpeles had in front of him all the time... the same that would automatically show a discrepance as soon as it started between mtgox internal BTC balances and real ("blockchained") addresses under their control.

After that basic accounting is done then it will be easy to check which explanation fits better into this case... and since when was MTgox knowingly operating in insolvency without reporting it, which I supposse its a criminal act in Japan as much as everywhere else.

Maybe then the malleability "issue" and all the bunch of excuses will be insignificant and no matter anymore even if it had any additional impact on the already criminal operation of MtGOX.

... Or maybe the next twist of this plot will be to execute another "It's gone!" this time to the accounting records? That would be even more funny.

TL;DR: The answer is in the accounting records which he will have to provide to prove the insolvency and how it developed over time. It won't be easy to fake them as they must match and be coherent with the public blockchain.

Hello, this is a topic I've been wondering about.  Please excuse me if my knowledge of bitcoin transaction details are a not up to speed at this point.

The thing that keeps sticking in my mind is how Gox went for 2 years without people noticing and screaming about it.  If Gox balances didn't agree with legit transactions balances, how do that go unnoticed?  First of all there are 2 different situations that are relevant to this issue.

1. Individual depositors transactions appear on the block chain.
2. Gox aggregated individual depositors transactions so they really only have a commitment from Gox that their bitcoins are real.

From what I've heard #2 is the way it works, and that other "exchanges" work the same way.  I find this very much against the bitcoin philosophy of a redundant trust system.  You're trusting Gox, not the redundant block chain, and maybe that's the fundamental problem with it.

But let's look at the idea that a long term attack could have gone unnoticed under these 2 different conditions.

1. Individual depositors would have recognized right away that their bitcoins were missing from their wallets if they saw they didn't appear correctly on the block chain.  Very hard to believe that they wouldn't.

2. Gox reconciliations should have picked up missing bitcoins from the block chain.  Did they not do this simple reconciliation?  This is also very hard to believe, but since it's based on inept management from a dysfunctional organization, it's easier to believe than #1.

My view on this is that the ecosystem is immature at this point and is suffering from it being primarily a fiat investment device.  People want non-sustainable things from it like excessive returns and fiat anonymity which makes people want to use exchanges and become vulnerable to trust problems with the exchange.

Another issue I have is the size of the block chain and the fact that it's not distributed, as fiat money is.  Think about if there were a block chain for dollars or euros.  It would be absolutely unmanageable and a huge breach of privacy.  These are issues that are fundamental to the viability of the bitcoin model if you ask me.

This is incorrect. Maged is describing what happens to competent exchanges with the malleability bug in the quote -- and it seems MtGox was not one of those.

The claimed explanation for MtGox was:
* MtGox sends a transaction
* attacker modifies the txid
* modified transaction goes in the blockchain, and bitcoin ownership changes
* attacker complains MtGox "I didn't receive my coins"
* MtGox searches for original txid in blockchain, doesn't find it  (<-- this is the bug)
* MtGox thinks they haven't paid, apologizes and sends a second, totally separate transaction of the same value

So there will be two transactions in the blockchain (Maged even says so in his post "Under no circumstance should you just issue a completely new transaction like MtGox did. That is how they lost some bitcoins"). Identifying the pairs could be very difficult as the sending and receiving addresses might be different.

Surmising a brief sum up.

a) Only one transaction gets registered in the blockchain -> no blockchain-based autopsy is possible.

b) There have been some  interesting contributions discussing the Gox software. I wonder whether it is available and can be inspected. Perhaps it would be reasonable to require exchanges to make their software available to public scrutiny. If a chunk of the Bitcoin ecosystem is a black hole it's hardly surprising that it swallows money.

Oh, right.  I forgot about another confounding factor.

Apparently gox's software didn't properly age coins before spending them.  Not normally a problem, but when the hot wallet is low, things can get interesting.

Compare


and


My understanding is that indeed only one transaction gets registered in the blockchain, but there seem to be different takes on this issue.

Hi,

This is my understanding of how bitcoin works. If i am misunderstanding how transactions work, please someone correct me. Also this is only about mtgox... not how malleability works in general.


I agree that more users should be fully aware as to the implications of malleability, but they are documented and somewhat limited. Developers have no excuse for not being fully aware.

I think "an open wound" is a bit excessive. this is not bitcoins fault. bitcoin gives you the rope to hang yourself, it is up to you if you do that or not.  irreversible transactions and  accidentally large transaction fees are other examples of things that could be seen by some as a 'bug' or flaw in the bitcoin code. they are not.

Trusting unsigned inputs (like transaction ID, whilst not directly controllable it is certainly modifiable.) as signed inputs is such a basic mistake - I mean really really basic.  like not pulling your trousers and underwear down before you go for a shit.

Block chain bloat, overly complex legacy that needs to be supported, almost obfuscated reference code and a very steep learning curve are open wounds. (that are healing, slowly...)

[/quote]

Yeah, but that is missing the point, there has to be 3 transactions for gox's story to hold true (even though two of them spend the same funds/are the same transaction)
- the initial transaction,
- the double spend (which must be mined before transaction 1.)
- the reissue of funds in transaction 1 by the original issuer.  it is this reissue that would set alarm bells off. (and why not reissue the _same_ funds they claim not to have received? maybe you'd have to run a multimillion dollar business and have a track record for successful technical ventures to check for that sort of thing.)

makes me wonder why the accountants did not flag the discrepancies between the sale of coins, transaction reissues and actual stock levels. they must have been doing zero stock checking for their story to hold true. (or the 'attack' was spotted very quickly) This is not even a technical thing, it is a bean counter thing. (the stock checks will not show what the issue is, just their is a major problem... _everything_ is accountable it is all right there in the block chain - bitcoin is an accountants wet dream.)


Not in the thread you linked, he even specifically states they are not connected. interesting idea though. (you would probably make more money just selling drugs...)



I think the not relaying of transactions is a distraction technique. same for the unsticking of transactions, anyone could verbatim supply the transaction to a pool they know will mine it.  Mining rules are different to passing of transaction rules.  There is no reason not to mine nonstandard transactions if you see them.


maybe, however even now it shouldnt be a big deal. gox's custom code would relay the transactions, so gox would have to be not connected to the major mining pools... this seems absurd at best. the padded transactions will be included in the next block. there is no reason not to mine them, just not to transmit them.

The best an attacker could hope for would be to connect to the same pools, but they are still massively disadvantaged because they have to see the transaction second, modify it, and get it into the transaction pool ahead of the transaction they got from the transaction pool...  for gox's account to be true the attacker would have to see the broadcast before the pools. That is until anyone can up the fees on a transaction... but that is a different can of worms. (and a really bad idea.)

So the attacker is left with having to see the transaction in memory at a major pool then modify it and pass it to another pool that gox is not connected too and hope that pool hits a block first. Unless the pools decide not to mine nonstandard transactions (not rational)

gox would never have a transaction propagation issue no matter how nonstandard yet valid their transactions are.  However this changes when we get the ability to pay for fees on other peoples transactions, making the attack easier. (assuming higher fees are more likely to appear in blocks)


I would be tempted to think this was Gox implementing a 'fix'.  A good willed person does not need to modify the transaction to be helpful, just relay it to the pools that gox doesnt talk too (okay if you arent up on bitcoin attacks, unless gox hasnt heard of a sybil attack https://en.bitcoin.it/wiki/Weaknesses they are connected to well over 75% of the mining power, hence me banging on about it.  if 1% have an issue, it is an infinitesimally small chance you will be able to out race it.) you dont need to strip the zeros.


I am not sure what you mean by era's but the issue isnt as passive as it seems. there is a difference between the rules for relaying transactions and the rules for including a transaction in a block. the attack relies on the attacker being able to send the modified transaction to the mining pools before they see the original transaction. the network propagation rules only apply to people without custom software to craft the transactions and direct connections to the major pools. technically it could have been going on the whole time.  the original transaction most likely never made it to the network if they really were attacked.


I agree completely. the story as told (have gox given an official response? I saw gesticulating and general flapping on their blog but nothing concrete.) cannot be possible.

There is a lot less than 1% chance. so much so it puts it out of the hands of even medium players.

There are other methods to take advantage of the coding error that I can think of, but they all need more access...  the only semi reasonable answer is either someone
1) MITM the connections to the major pools and rewrote every transaction, and dropping the original ones thus guaranteeing your transaction is seen first.
2) Own the transaction server, rewriting every transaction and dropping the original before it is even broadcast.

I cant wait for them to have thier solution implemented (it would seem that they have passed the trust to block explorer and block explorer say yay or nay. lol. idiots.)

Hope this helps? sorry if it goes on a bit.  but as far as I know you cannot 'update' a transaction.

kjj,

Thanks for your thoughts. I can't fully understand them (I have a language barrier and no knowledge of bitcoin mechanics / technicalities / jargon). But at least you came up with some grounds for methodology of calculating / estimating the level of malleability exploit used to trick exchanges.

I have funds at Gox (or ''had'' in case they are gone). I really want to find out what is going on / what was going on. While trying to find it out I want NOT to rely on leaked recovery plans (which may or may not represent the reality).

Any help will be appreciated that will lead to discovery of the amount that was extracted from exchanges through malleability bug.

He mentions that SR2 was a scam using this as cover.  It is quite possible that gox is the same.

This is my understanding, the best that I've been able to put together.

Gox's custom software doesn't always strip leading zero bytes from signature values.  Sipa estimated (on IRC) that only ~1% of gox transactions had excessive padding.

A while back, this was no big deal because the network would happily relay both forms, so any attempt at changing the transaction would result in a race.  Later, the default node behavior changed to not relay the padded version.  Once this changed, the original would pretty much always lose the race, if there was one.

Next, someone made a bot to "fix" transactions on the fly.  Quite possibly this person was sick of waiting for their money and/or was just being helpful.

So, we have 3 eras.

First, all transactions confirm normally.
Second, 1% of transactions never confirm.
Third, 1% of transactions confirm in modified form.

Only the third era is vulnerable, but it is a relatively short era.  I don't think anyone knows exactly how long it has been going on, but it can't have been years because the second era isn't that old either.

Now, to exploit this, you'd need to get lucky, or you'd need to keep up a massive circular flow out of and then back into gox.  Remember that each transaction has about a 1% chance.  So, 99% of the time that you withdraw your balance, it works fine.  And you have to wait 6 or 7 confirmations (minimum) before you can try again.  You'd get about 85 chances per year, so if we assume the third era was about a year long, figure the average attacker could have doubled their money about once in that time.

It just doesn't add up.  Someone is lying, or there are very important things that we don't know yet.

I would like to read more about this. AFAIU transaction malleability is still an open wound in the Bitcoin protocol. Understanding its implications is important.

I found this thread , Maged's post  about the Mt.Gox mess in particular, quite instructive.

By the way, Maged writes:

Malleability exploit uses double spend right? Why not mtgox software cannot detect double spend? I know cryptsy software will automatically freezes the trade and withdrawal once the double spend is detected on the blockchain. I don't know why mtgox did not develop this since this issue is known two years ago. They  have millions of money they can hire the brightest programmers available. Or go blind about it and make it as a scapegoat.

Okay, this assumption may not be valid, but at least we might start to try assessing the damage done to exchanges from here.

I am not the expert on bitcoin. I think a geek should conduct a study on this subject to arrive at a methodology of assessing the scale of malleability exploit.

If a good methodology is devised and a proper study run, someone might at least be able to confirm or debunk claims that were written in the ''leaked'' ''document'' that says +700k coins are gone.

This assumes that they were replacing transactions, not balances.  This assumption may not be valid.

That would mean that valid transactions do not appear on blockchain - I consider this rubbish.



I agree with this.

One needs to pair those two transactions (find them in the blockchain). There would be a few non-complex criteria for finding such malleable-looking transactions:
- amount of both must match
- date range should not be broad, e.g. the second transaction cannot be older than 1 day / week / month than the first one
- both transactions must be spent from an exchange address (not sure how to find such addresses but my guess is that such addresses generated a huge number of transactions for a short time)
- others

------------------------------

It is a perfect research project for applicants who want grants from TBF.

Both transactions would be in the blockchain - the initial withdrawal and then the re-withdrawl request several weeks later which mtgox would have allowed.  This is different than the initial gox withdrawal which had the tx id mutated and eventually made it into the blockchain albeit with this mutated tx id.

I'm not either and because of the regulations I would never set up my own exchange of any sort. However, I do benefit from it by only using coinbase for my transactions since they do follow the US regulations and are a US company. I feel much more comfortable doing bank transfers to coinbase for BTC purchases that I would with other random exchanges (be it BTC-E, Cryptsy, Coinex, or someone else). Also getting cash back from them by selling BTC is easy and direct to my bank account.

You couldn't do that, because only one of the transactions would actually be in the block chain. So you would never actually see the pair of transactions. You can only catch this stuff while neither of the equivalent transactions has entered a valid block.