Topic: [Payout Updates] Bitcoinica site is taken offline for security investigation - page 82. (Read 156711 times)

I think you should favor higher reliability.  If that means that the process will be slower, so be it.

I will post information here as I receive it.

28 May 07:00: Discussion right now is centered on the nature of selecting payess. There is the question of what nature the system for choosing payees should take with different people favouring 1 of 2 approaches (one is fast and unreliable, the other slow and reliable). I can't say more than that for technical reasons. Everyone wants to pay everyone back, but have a differing opinion how it will work.

I will post information here as I receive it.

28 May 07:00: Discussion right now is centered on the nature of payouts. There is the question of what nature the payout system should take with different people favouring 1 of 2 approaches (one is fast and unreliable, the other slow and reliable). I can't say more than that for technical reasons. Everyone wants to pay everyone back, but have a differing opinion how it will work.

Today, we have discovered a suspicious Bitcoin transaction that doesn't seem to be initiated by any one of the company owners. Some of them are not online at the moment so this is not conclusive.

Suspicious transaction:

  {
        "account" : "",
        "address" : "182tGyiczhXSSCTciVujNRkkMw1zQxUVhp",
        "category" : "send",
        "amount" : -18547.66867623,
        "fee" : 0.00000000,
        "blockhash" : "00000000000003f6bfd3e2fcbf76091853b28be234b5473a67f89b9d5bee019c",
        "blockindex" : 1,
        "txid" : "7a22917744aa9ed740faf3068a2f895424ed816ed1a04012b47df7a493f056e8",
        "time" : 1336738723
    },

We have contacted Rackspace to suspend all our servers and lock down our accounts. All your trading and financial data is safe (as far as I know), apart from the Bitcoin loss.

Thank you for your patience and understanding while we investigate this issue in detail.

No database backups. Sorry for avoiding the question.

I hoped someone else could clarify this. I don't have all the full details, and would hate to make incorrect statements. I also didn't want to jeopardise efforts to refund people.

From what I gather, there are no backups of the database. Only partial records for accounting which is being used to extrapolate balances. I'm not sure of the exact details, but I think they need a full view of the claims before payouts begin (like a big jigsaw puzzle) to properly cross match records. Hopefully someone better informed will post more details.

zhou: ah, ok. I don't know the exact details and I'll avoid commenting further.
I think Patrick assumed they were not critical hence me saying: "The assumption here was that [email protected] did not have access to critical infrastructure.". I do appreciate that several times, you told people I wasn't involved with Bitcoinica in this thread. I always assume good faith which is why I think it was a fatal miscommunication between team members.

bitcoinBullbear: that's fine. It does annoy me a little that people assume that a decentralised system like Bitcoin consists of a single piece of kosher software. bitcoin.org lists several clients. When security flaws were found, me, Mike Hearn and justmoon helped fix problems on the internal security mailing list. justmoon in fact was very instrumental in many cases for clarifying and proposing fixes for BIP 16. There was a long technical history that led to libbitcoin's creation and it has taken 8 months so far.

That picture is funny. I like it.

rjk, nope. Everyone had root. One person was installing a database, another installed Jenkins.

The anger here is justified. If this happened to me, then I would be extremely mad. I was very pissed at MtGox when they had their problems. It sucks to be no better than MtGox.


Bitcoinica took us on to help secure them.

We decided it was bad practice to make sudden disruptive changes overnight to a production system. Instead the theory was a very gradual replacing of the system while observing changes. Bitcoinica was already very fragile. I still think that was a good decision.

Step 1 - fix the code.

Flaws were already being found in the code. That was the logical first step. That the environment ended up being exploited is simply hindsight. I would prefer not changing a working environment until after knowing how the code operates. An example is that another website accidentally made out a 500 BTC payment when the file permissions were too strict. Similarly changing an aspect of Bitcoinica without proper insight could have had grave consequences.

First you understand the code. Then you run the code. You experiment with a test system. Make improvements. Deploy changes. Change production environment.

The Bitcoinica plan was to do the above while creating a new platform to replace it in the long term.

No database, a huge mass of data (much of it useless) and a number of false claims that could push out legitimate claims. The data makes sense only as a whole which makes payouts difficult (you have to build a case and gather evidence based on the known data). Being careless and paying people without being sure is stupid as you cannot reverse payments if more evidence later ends up contradicting your early guess.

That's why the initial payouts so far have been for only 50%. And only for people we're highly certain of. I support extending that to more, but the others are understandably taking more caution. If people are paid out, then it's realised there is a mistake in our assumptions, that means legitimate people will not get paid (the pool of money for payouts is limited). $1 erroneously paid out, is $1 of someone else's money. The honest and correct decision here is being as certain as possible for people you pay out, and no amount of shouting will speed up the process. The records for making the payouts are incredibly bad and inefficient. It might take 15 mins to check a single person before you realise that the records on hand for that person are useless/contradictory. Now multiply this by 100s of people.

Someone earlier mentioned hiring people but that's not an option here. I would not trust a relative unknown with this data and the time/effort involved with finding a new person who would be competent does not make it a positive tradeoff.

People are divided into different classes depending on how certain we are of their claim. As we move through verifying accounts, we become more certain. Once everyone is refunded for 50% we then double back through all refunds and refund the last 50% at the very end.

I want to explain the logic behind 50% initial payments:

For each claim you have a certainty of its validity. You have the sum of funds from before the site had the database stolen which is equal to the site's previous balance. This sum must be distributed among claims somehow.

Ideally you would mark claims that you thought were accurate and those that weren't (based on what the data indicates to us). Then you would end up with some total value which you compare to the total funds available.

If less than the total funds, then be more permissive and allow certain more claims in. Repeat the above step.

If more than the total funds, then knock out low certainty claims and maybe redistribute funds among lower certainty claims (better that someone who is maybe legitimate, gets something rather than nothing).

You then end up with claims that are refunded with a best-fit according to the available data.

However people demand funds quickly. So as a compromise, we make 50% payouts initially (for claims we are highly certain are accurate). This allows an error margin so that in the final step we can still juggle balances around to resolve payments for everybody. The only downside is that you cannot decide to pay someone 0% after you've paid them 50%.

Then once people have been refunded for 50% and the final balances are decided, the process goes back over payees and refunds them for the remaining amount. I assume the final step should not take long given that it's just making payments out for known beneficiaries.

Anyways shorts close @ 4.9 longs @ 5.1