Pages:
Author

Topic: [Emergency ANN] Bitcoinica site is taken offline for security investigation - page 9. (Read 224563 times)

hero member
Activity: 518
Merit: 500
* Goes to get the popcorn and claim $ 1 million on claims.bitcoinica.com *

What's up ? Cheesy

In all seriousness, how long do you think this will take ( months / weeks ) ?

This is all affecting the price even if I don't have anything in Bitcoinica.

You don't need anything in Bitcoinica when you just took them for 18K BTC ...

hey man i'm just doing what your sig told me to  Grin

Seems to me the hacker is a small group of people from which an individual should easily be identified.

Why has this not happened ? It is not like the hacker was some unknown entity out of the blue.

Read the thread : the only thing I had in the bucket shop is $1 bonus I got from zhoutong Cheesy

Not going to give my info to a bunch of incompetents to get it back anyway.
hero member
Activity: 686
Merit: 500
Shame on everything; regret nothing.
* Goes to get the popcorn and claim $ 1 million on claims.bitcoinica.com *

What's up ? Cheesy

In all seriousness, how long do you think this will take ( months / weeks ) ?

This is all affecting the price even if I don't have anything in Bitcoinica.

You don't need anything in Bitcoinica when you just took them for 18K BTC ...

hey man i'm just doing what your sig told me to  Grin
hero member
Activity: 518
Merit: 500
* Goes to get the popcorn and claim $ 1 million on claims.bitcoinica.com *

What's up ? Cheesy

In all seriousness, how long do you think this will take ( months / weeks ) ?

This is all affecting the price even if I don't have anything in Bitcoinica.
donator
Activity: 980
Merit: 1000
I said as much in an earlier post.

The fact that they asked for the things they asked was quite suspicious in regards to having the passwords or not.

BUT, it makes absolute sense that they didn't tell. You cannot tell people that you really don't know exactly what they had. That leads to a very obvious tragedy of the commons kind of situation if everybody starts claiming for more than they really had.
vip
Activity: 490
Merit: 502
I wasn't sure whether talking about the database was even permitted, so I skipped such questions. Now genjix has already said that, because either:

- He didn't communicate much with the rest of the team (i.e. doesn't understand why we are hiding)
Or
- He was granted the right to talk (I don't know)

Throughout the whole event, I have always been following Bitcoinica Consultancy's standard of disclosure. The reason that database deletion was not disclosed is that they were afraid of inaccurate claims that would worsen the losses.

I believe that any claims or claims modifications submitted after this point should be treated as false unless very concrete evidence has been given.

We had automated backups to back up the database and the wallet. During the hacking, I also created an emergency backup to preserve the current database. However, I was misled by one Rackspace support guy who claims that the hacker "can't do anything" to the servers which are suspended by engineers. All command buttons are disabled. I never noticed the hidden feature to delete the server. (i.e. if you're hacked, they can't log the hacker out, instead, they suspend all the servers so the hacker can't do anything but delete them.)

The hacker later restored the emergency image so he should possess a copy of the database. After that, he deleted all servers and all files in Cloud Files (like S3) including server backups.

It's my fault to not set up a offline backup schedule. Tihan used to run the accounting reports regularly (which is like offline backups) but he stopped doing so when I created a stats graph generator for him to automate the reportings. The most current record we have is his previous reports. This is my fault.

According to the information I have, returning funds to clients is not impossible. I suggested some ideas but they were rejected by Bitcoinica Consultancy for different reasons. I understand their situation though, and my offer to take over remains open.
legendary
Activity: 1232
Merit: 1076
  • Passwords are gone, so they are no use.
  • 80% of BTC funds are (I assume, please confirm) still under Bitcoinica's control.
  • 100% of USD funds are (I assume, please confirm) still under Bitcoinica's control.
  • The problem is you just have a big pool of money and no way of knowing who owns what.
  • That entirely explains the crappy claims page.
  • That entirely explains the delay in processing claims.
  • For we customers: this isn't perfect news, obviously, but it does at least give us some hope that we haven't lost everything.
  • If the investor really is doing the decent thing and funding the 20% BTC losses out of their own pocket, then we should all appreciate that and let that be an end to all the legal shouting.
  • Certainly no amount of shouting is going to recreate a database that doesn't exist.

I honestly don't know. Those more involved can hopefully clarify these points. Anything I say would be guesswork.

I'll stop posting now. I've stated everything I know already.
hero member
Activity: 504
Merit: 502
No database backups. Sorry for avoiding the question.

At last.  All is explained.

  • Passwords are gone, so they are no use.
  • 80% of BTC funds are (I assume, please confirm) still under Bitcoinica's control.
  • 100% of USD funds are (I assume, please confirm) still under Bitcoinica's control.
  • The problem is you just have a big pool of money and no way of knowing who owns what.
  • That entirely explains the crappy claims page.
  • That entirely explains the delay in processing claims.
  • For we customers: this isn't perfect news, obviously, but it does at least give us some hope that we haven't lost everything.
  • If the investor really is doing the decent thing and funding the 20% BTC losses out of their own pocket, then we should all appreciate that and let that be an end to all the legal shouting.
  • Certainly no amount of shouting is going to recreate a database that doesn't exist.

In light of some of the above comments, can I reiterate my suggestion to the Bitcoin Consultancy and Zhou Tong?  Stop airing your dirty laundry in public.  Shut up about it right now.  Regardless of whether you feel there's "just one more thing that needs addressing"... button it.  The fastest way to restore your reputations from here (and there is plenty of restoration needed) is to sort the customers out as quickly as possible.  After that is done (and only after), will it be prudent to start your war-of-logs.

I'd also second Vladimir's comment: you really shouldn't be communicating with you business partner using an Internet forum (I would guess you wanted us to be able to appreciate the difficult position you are in: tough luck, keep it to yourselves and take the insults on the chin).  It doesn't inspire confidence that you don't have (a) each other's phone numbers (b) each other's email addresses (c) a good enough relationship that you can talk with each other privately.  Even if (a), (b) or (c) are not true; a professional shouldn't let the customers see how the sausages are made.  Companies are aptly named: you are all one, and letting us see your internal fistfights is extremely damaging.
legendary
Activity: 1050
Merit: 1000
if there are no backups at all, that's a serious blow for all involved, only viable solution at this point would be to restore account balances based on deposit/withdraw records (blockchain/mtgox logs/etc)

good luck!
legendary
Activity: 1232
Merit: 1076
I personally learned from ribuck to do just that too and advise all others to consider as a standard practice to never sign any NDA's ever personally.

yeah, that was a mistake on my part.
hero member
Activity: 812
Merit: 1001
-
I personally learned from ribuck to do just that too and advise all others to consider as a standard practice to never sign any NDA's ever personally.

donator
Activity: 826
Merit: 1060
Incidentally, this is why I never sign NDAs, even though they are sometimes presented as "standard practice".

(Lots of people don't sign them, actually. If you go to a job interview at Google, you will be asked to sign an NDA. But if you don't sign the NDA, you still get the interview.)

An NDA is not needed when the parties are all acting in good faith. And if one of the parties is not acting in good faith, the NDA can be used to stop the good guys from doing what they should be doing.
legendary
Activity: 1050
Merit: 1000
assuming rackspace has no image backups either?
hero member
Activity: 574
Merit: 500
Per standard practice, Bitcoin Consultancy entered into a non-disclosure agreement which extends to Bitcoinica's proprietary systems and processes. They are free to discuss their role and history with the company.

Hi,

Thank you for this. We are incredibly happy. We will need to clear up some distinctions and make sure the account is in fact Tihan's account. He can do so via confirmation in email or on skype. We also need clarification as to what "role and history mean".

1. Are Bitcoinica Consultancy and its individual members allowed to talk about the security issues and this incident without limitation? Yes/No

The NDA extends to our persons I believe.

Finally, Tihan, people seem to have questions regarding the database.

2. Are we, Bitcoinica Consultancy and it's individual members, at liberty to discuss in full detail the nature of the database? Yes/No

3A. Are we, Bitcoinica Consultancy and it's individual members, at liberty to release relevant skype logs in full without worry that information in those logs are sensitive? Yes/No

3B. If there is a "No" answer for question 3A, could you specify and clearly what we are not allowed to post (for example, content that would violate a user's privacy) and remember to claim that the list of restrictions you post is an exhaustive list.

4. Are you willing to take the short steps to nullify any NDA we may have? Yes/No

5. Can we release a full account of the security detail and practices relevant to Bitcoinica's history and this incident? Yes/No



Finally, we can certainly see the semblance of unprofessionality that Bitcoinica Consultancy was resonating. We would like to apologise for having to go to such an extreme. We were urged against making such statements by Tihan and Zhou as they would hurt everyone's reputation, including our own. The circumstances were such that we had no real ability to respond to misinformation and misrepresentation. We full well knew that our immediate reputation would suffer greatly. In matters like this, things often need to get worse before they can get better. However, it seems we will be finally be successful in providing full disclosure for everyone. We were talking with Tihan about trying to clear up misrepresentations for a long time and with Zhou as well. Unfortunately, we were not granted the ability to clear up the relevant issues (possibly until now) and Zhou kept making and continues to make false statements and wildly misrepresenting the facts. We are very happy with the turn of events as we are certain that (as long as Tihan's comment wasn't intentionally nondescript or ambiguous) we will be able to set the record straight.


We are not pursuing this matter at the expense of the reclaims process. However, when we have time, we will (in great detail) show that many statements that have been made have been malicious and false.

Please write in simpler english or write in proper english, this post is ridden with mistakes. It's very hard to read it. I don't think a lot of people care about the "logs", such things are open to interpretations. Keep them for courts where they may matter.

Just refund Bitcoinica's customers.

edit: as per genjix's post - now I understand what happened. Complete compromise, all cloud instances were deleted, all up-to-date backups gone. It will take months to refund the customers.
legendary
Activity: 1232
Merit: 1076
No database backups. Sorry for avoiding the question.

I hoped someone else could clarify this. I don't have all the full details, and would hate to make incorrect statements. I also didn't want to jeopardise efforts to refund people.

From what I gather, there are no backups of the database. Only partial records for accounting which is being used to extrapolate balances. I'm not sure of the exact details, but I think they need a full view of the claims before payouts begin (like a big jigsaw puzzle) to properly cross match records. Hopefully someone better informed will post more details.

zhou: ah, ok. I don't know the exact details and I'll avoid commenting further.
I think Patrick assumed they were not critical hence me saying: "The assumption here was that [email protected] did not have access to critical infrastructure.". I do appreciate that several times, you told people I wasn't involved with Bitcoinica in this thread. I always assume good faith which is why I think it was a fatal miscommunication between team members.

bitcoinBullbear: that's fine. It does annoy me a little that people assume that a decentralised system like Bitcoin consists of a single piece of kosher software. bitcoin.org lists several clients. When security flaws were found, me, Mike Hearn and justmoon helped fix problems on the internal security mailing list. justmoon in fact was very instrumental in many cases for clarifying and proposing fixes for BIP 16. There was a long technical history that led to libbitcoin's creation and it has taken 8 months so far.

That picture is funny. I like it.

rjk, nope. Everyone had root. One person was installing a database, another installed Jenkins.

The anger here is justified. If this happened to me, then I would be extremely mad. I was very pissed at MtGox when they had their problems. It sucks to be no better than MtGox.

To the person above, here's what happened:
- Bitcoinica has an internet mailing list called [email protected]
- It was the email for the website and all sensitive accounts.
- You could request a password for that email. In a production system, that should never be possible.
- Several people had access to this mailing list (non-admins and business people included).
- Patrick got added.
- His personal email was compromised. Normally this shouldn't be a big deal; I use my personal email at internet cafes and public computers.
- Attacker was able to request a new password and login to rackspace.

The assumption here was that [email protected] did not have access to critical infrastructure.

Lastly, it was my fault Patrick's email server got compromised. I had a VPS for programming and development which many people had access to - randoms from #c++ IRC, people from this forum, beginners I was teaching .etc It's a public VPS for development. The SSH key on there was added to Patrick's server because we were developing the bitcoinconsultancy.com website on there (that's why it's now down). My SSH key was stolen and he ssh'ed into the box. Then had access to his emails.

Bitcoinica took us on to help secure them.

We decided it was bad practice to make sudden disruptive changes overnight to a production system. Instead the theory was a very gradual replacing of the system while observing changes. Bitcoinica was already very fragile. I still think that was a good decision.

Step 1 - fix the code.

Flaws were already being found in the code. That was the logical first step. That the environment ended up being exploited is simply hindsight. I would prefer not changing a working environment until after knowing how the code operates. An example is that another website accidentally made out a 500 BTC payment when the file permissions were too strict. Similarly changing an aspect of Bitcoinica without proper insight could have had grave consequences.

First you understand the code. Then you run the code. You experiment with a test system. Make improvements. Deploy changes. Change production environment.

The Bitcoinica plan was to do the above while creating a new platform to replace it in the long term.
hero member
Activity: 868
Merit: 1000
All this NDA talk just seems like fancy ways to sound overly important and stall things indefinitely.

My inner voyeur want to see the logs, but the whole "it wasn't our fault but we can't tell you what really happened because we're sworn to secrecy" line comes across as whiny teenager shit and it's highly unprofessional.  The best way for anyone involved in this clusterfuck to vindicate themselves and restore their reputation is to quickly process claims and ensure that users are compensated as soon as possible.  Arguing about who did what first distracts from that process and it's going to make the Bitcoin community question how professionally Bitcoinica will be run going forward.  

While it's great for Tihan to step up and accept responsibility, it's Bitcoin Consultancy who will be operating the business - it doesn't matter a damn if people trust Tihan if they don't trust the people who will be in charge of the organisation's day to day operations.  That they're not picking up the phone and talking to each other is hardly confidence inspiring.
sr. member
Activity: 258
Merit: 250
I don't know how you seem to believe that Zhou is misrepresenting everything so badly when his comments seem to correlate with the comments that were also made by genjix and Tihan (in respect to the comments made by them, that is)...

It seems kind of chickenshit to me. Just sayin'...
hero member
Activity: 504
Merit: 502
All this NDA talk just seems like fancy ways to sound overly important and stall things indefinitely.
hero member
Activity: 868
Merit: 1000
Seriously? Are you guys talking with your financier on this forum and asking him here to confirm details of your NDA terms? This is getting much closer to that russian comedy youtube videos standard now than ever before?

Who is this guy "Bitcoinica Consultancy"?

Guess they missed this part of Tihan's post.

Quote
I’m unable to follow most public postings here, but you can reach me through this forum by private message. Questions about processing of funds should be directed to Bitcoin Consultancy as they alone control that process.

It's a bit alarming if they signed an NDA without understanding exactly what "Bitcoinica's proprietary systems and processes" means - lawyers normally nail that shit down.
hero member
Activity: 812
Merit: 1001
-
Seriously? Are you guys talking with your financier on this forum and asking him here to confirm details of your NDA terms? This is getting much closer to that russian comedy youtube videos standard now than ever before!

Who is this guy "Bitcoinica Consultancy"? I was wrong, Bitcoin's entertainment value is not 50$ it is 100$ now.

Here we go guys, all those who were asking all the time what is Bitcoin backed by, you now have your answer. Bitcoin is backed by non-stop entertainment.


Per standard practice, Bitcoin Consultancy entered into a non-disclosure agreement which extends to Bitcoinica's proprietary systems and processes. They are free to discuss their role and history with the company.

Hi,

Thank you for this. We are incredibly happy. We will need to clear up some distinctions and make sure the account is in fact Tihan's account. He can do so via confirmation in email or on skype. We also need clarification as to what "role and history mean".

1. Are Bitcoinica Consultancy and its individual members allowed to talk about the security issues and this incident without limitation? Yes/No

The NDA extends to our persons I believe.

Finally, Tihan, people seem to have questions regarding the database.

2. Are we, Bitcoinica Consultancy and it's individual members, at liberty to discuss in full detail the nature of the database? Yes/No

3A. Are we, Bitcoinica Consultancy and it's individual members, at liberty to release relevant skype logs in full without worry that information in those logs are sensitive? Yes/No

3B. If there is a "No" answer for question 3A, could you specify and clearly what we are not allowed to post (for example, content that would violate a user's privacy) and remember to claim that the list of restrictions you post is an exhaustive list.

4. Are you willing to take the short steps to nullify any NDA we may have? Yes/No

5. Can we release a full account of the security detail and practices relevant to Bitcoinica's history and this incident? Yes/No



Finally, we can certainly see the semblance of unprofessionality that Bitcoinica Consultancy was resonating. we would like to apologise for having to go to such an extreme. We were urged against making such statements by Tihan and Zhou as they would hurt everyone's reputation, including our own. The circumstances were such that we had no real ability to respond to misinformation and misrepresentation. We full well knew that our immediate reputation would suffer greatly. In matters like this, things often need to get worse before they can get better. However, it seems we will be finally be successful in providing full disclosure for everyone. We were talking with Tihan about trying to clear up misrepresentations for a long time and with Zhou as well. Unfortunately, we were not granted the ability to clear up the relevant issues (possibly until now) and Zhou kept making and continues to make false statements and wildly misrepresenting the facts. We are very happy with the turn of events as we are certain that (as long as Tihan's comment wasn't intentionally nondescript or ambiguous) we will be able to set the record straight.


We are not pursuing this matter at the expense of the reclaims process. However, when we have time, we will (in great detail) show that many statements that have been made have been malicious and false.
hero member
Activity: 868
Merit: 1000

So I take this as, you or you and others you represent are Venture Capitalists that put Bitcoin Consultancy in charge of your investment. e.g. Cisco but the team you put in charge flubbed up somewhere.
 

Interestingly, Bitcoin Consultancy took over Bitcoinica the same day that Tihan's CoinLab venture secured $500,000 of VC for investment in Bitcoin projects.

http://www.forbes.com/sites/jonmatonis/2012/04/24/coinlab-attracts-500000-in-venture-capital-for-bitcoin-projects/

http://www.geekwire.com/2012/bitcoin-startup-coinlab-lands-funding-tim-draper-monetize-games/
Pages:
Jump to: