Pages:
Author

Topic: [Payout Updates] Bitcoinica site is taken offline for security investigation - page 73. (Read 156711 times)

legendary
Activity: 826
Merit: 1001
rippleFanatic
I don't think I have to mention the common sense. Of course the database backups are going to be downloaded on a constant basis. Bitcoinica has accounting records, but they are not current enough to resume trading.

Of course, now! Glad to hear you are learning. We're lucky some bitcoinica accounting records survived. But what a shame the user database is gone, or this quagmire of a claims process would've been much, much easier (this claims process is doing more to damage reputations of everyone involved than a reported theft ever would have).

However, I feel unfair for the reputation damage that wasn't even triggered by me.

You weren't the immediate trigger (the hacker was). But it was assumptions you made which allowed the situation to cascade into a catastrophe. (one assumption: hosting emergency-backup as a vps instance under the same rackspace account would be sufficient in the case of an emergency. another one: rackspace support guy is correct. why trust the word of a support guy? never should've had to.) That lack of preparedness is what damaged your reputation, and that's fair.

To think that you were building a new domain service while simultaneously operating bitcoinica! I'll trust you to build stuff, but not to operate it. (and thanks for reading).
hero member
Activity: 868
Merit: 1000
Okay, nice statement of outrage at being judged. How about answering the question regarding where are the USD funds? How about responding to user's emails since you are no longer busy running the service?

To be fair, it's Bitcoinica Consultancy which needs to answer any questions regarding the USD, MtGox deposits and BTC in cold storage - they're the ones who are legally responsible for the management of the business and Zhoutong shouldn't be asked to speak on their behalf about such matters.  Who is actually managing user deposits is certainly a relevant question as it's central to the question of legal liability for Bitcoinica's debts. 
donator
Activity: 1120
Merit: 1001
We don't hold the funds.

Then WHO HAS ALL THE USD?

And when will you respond to emails? I want to know if my account is among the claims being processed. It's not like I say to my customers "oh sorry, can't give you your money back, Bitcoinica stole it". No. I have to pay it out of my own pocket. But I can tell them, "Bitcoinica stole your money. I was dumb enough to leave your USD with them. That would be Patrick, Zhou, Tihan, and everyone else related to that project. They say they got robbed for 18K BTC, and then all their USD magically disappeared at the same time. If you ever see anything else they do, make sure to avoid it like the plague. Now I'm paying this money personally back to you on their behalf. Hopefully they'll reimburse me, but I doubt it since they haven't responded to a single one of my emails."

+1

Bitcoinica hold the users away from their fund for too long, and this has brought inconvenience to the users.
vip
Activity: 490
Merit: 502
Okay, nice statement of outrage at being judged. How about answering the question regarding where are the USD funds? How about responding to user's emails since you are no longer busy running the service?

I have all your emails in record, and Patrick has received a copy as well. I'm not supposed to give any official replies. All I can do is give you advice on how to fill the claim form properly.

I have not yet received any instructions from Bitcoinica Consultancy.
sr. member
Activity: 574
Merit: 250
Okay, nice statement of outrage at being judged. How about answering the question regarding where are the USD funds? How about responding to user's emails since you are no longer busy running the service?
vip
Activity: 490
Merit: 502
We have off-site backups in a different DC. It's managed by Rackspace.

If the server crashed, we have no problems of recovering. There are a lot of backups of all our main servers. It's just that these backups were deleted by the hacker.

I meant data center of a different company (a different admin panel with a different password, and an append-only configuration). That's what provides a level of actual redudancy. Using a cloud service from one company protects you if a truck crashes into one of their data centers, but offers zero protection if someone gets your admin password!

I can't afford a scalable solution that gives me the same reliability for a bootstrapped startup.

Not anymore you can't!

You can make backups for cheap (backups aren't accessed, so they don't scale with the rest of the site).

All financial transactions will be handled by payment gateways who will be responsible for their own security. Apart from that, no money is involved so I just want to consider scalability, performance, availability and cost. For me, I think AWS's EC2 instance with Load Balancer handling SSL termination (can't be DDOS'd) + RDS with snapshots and binary logs recoverable to 5 minutes ago are more than enough for me.

Most people choose to outsource security, it's just that in Bitcoin world everything is DIY.

I'm not going to use your domain service or any other zhoutong "cloud" service because its clear that you don't have plan B (contingencies). What happens when someone deletes your AWS instances and its snapshots and logs? Has the thought even crossed your mind to try and see if its possible on the AWS admin panel?

Its a truism that at the basic level, security can't be outsourced. You have to trust someone eventually (unfortunately for us, you happened to trust bitcoin consultancy), but catastrophes can only be averted by good planning.

I don't think I have to mention the common sense. Of course the database backups are going to be downloaded on a constant basis. Bitcoinica has accounting records, but they are not current enough to resume trading.

The requirement for currency isn't very important for a non-financial project if someone else (such as a domain registry) will keep the records anyway.

What I'm saying is that it's unreasonable to expect a mission-critical configuration for a project with limited financial interest involved. All possible efforts will be put into security for sure. The use of cloud services shouldn't be the argument here.

If I were to build another Bitcoin project, I will definitely consider the locked cages. But until I have the funds and the security expertise, I should probably stay away from money itself.

I recently got PCI Compliance for storing credit cards on the servers, and the certification has been recognized by a large business bank in Australia. AWS datacenters all have passed the physical security checks and required audits. But I still choose not to store any critical financial data at the moment. Instead I will use the vault services provided by reputable payment gateways.

For any of my commercial projects: Minute-interval binlogs and at least daily backups with weekly downloads. All firewalls configured properly. Two-factor auth on AWS accounts. No injection, XSS or CSRF possibilities.

No system is 100% secure. The uncompromised systems are the ones not being targeted. Outsourcing security to a responsible party will avoid the possibility of being a target as much as possible. We can already prove that Linode was not secure, but why only 8 accounts (all related to Bitcoin) were hacked? Bitcoin attracts cyber criminals and it's reasonable to expect a disproportionately frequent security attacks on Bitcoin-related projects.

However, I feel unfair for the reputation damage that wasn't even triggered by me. I'm always serious about security and even though I'm not a specialised security expert, I do have some knowledge and experience of maintaining a secure system that's enough for a SaaS project. You can criticise me for trusting 3rd parties too much, but it's still my belief that the so called 3rd parties have better security skills than me. It's just that they are being targeted. (Even Sony got hacked so many times.)
rjk
sr. member
Activity: 448
Merit: 250
1ngldh
Some reputations could have been salvaged, maybe, a long time back with some proper communication and some action. At this point, all is pretty much lost.

This includes the reputation of any investor(s) that may have selfishly delayed the entire process just to be sure it could have been completed all in one big lump.

Taking a month to prepare to start paying out in one batch is a huge failure. Better would have been to start the payouts as soon as possible, even if it took a month to work through them, as long as some were going out each day.

And as ssaCEO said, how did all the USD go missing? This is absolutely inconceivable. Same goes for the cold wallets. What in the world is actually going on?
hero member
Activity: 518
Merit: 500
We don't hold the funds.

Then WHO HAS ALL THE USD?

And when will you respond to emails? I want to know if my account is among the claims being processed. It's not like I say to my customers "oh sorry, can't give you your money back, Bitcoinica stole it". No. I have to pay it out of my own pocket. But I can tell them, "Bitcoinica stole your money. I was dumb enough to leave your USD with them. That would be Patrick, Zhou, Tihan, and everyone else related to that project. They say they got robbed for 18K BTC, and then all their USD magically disappeared at the same time. If you ever see anything else they do, make sure to avoid it like the plague. Now I'm paying this money personally back to you on their behalf. Hopefully they'll reimburse me, but I doubt it since they haven't responded to a single one of my emails."

No need to get angry mate. It doesn't reflect well on your business if you don't keep it civil.

I am sure they will pay it back ... at some point.

Anyone up for 4-6 weeks of waiting time Cheesy

The strangest issue is : why were the USD not paid back immediately ?

I understand the BTC issue is more delicate with an erased balance but the USD should be easy to source as none were stolen !?
hero member
Activity: 568
Merit: 500
We don't hold the funds.

Then WHO HAS ALL THE USD?

And when will you respond to emails? I want to know if my account is among the claims being processed. It's not like I say to my customers "oh sorry, can't give you your money back, Bitcoinica stole it". No. I have to pay it out of my own pocket. But I can tell them, "Bitcoinica stole your money. I was dumb enough to leave your USD with them. That would be Patrick, Zhou, Tihan, and everyone else related to that project. They say they got robbed for 18K BTC, and then all their USD magically disappeared at the same time. If you ever see anything else they do, make sure to avoid it like the plague. Now I'm paying this money personally back to you on their behalf. Hopefully they'll reimburse me, but I doubt it since they haven't responded to a single one of my emails."
donator
Activity: 452
Merit: 252
03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.

That's a bit confusing.  If all that was lost was 20% of Bitcoins on hand it should have been possible to pay everyone out 80% (the initial round of payouts) without receiving funds from Tihan - additional capital should only have been required to replace the lost Bitcoins.  You should have still been in possession of 100% of USD and 100% of Mt Gox deposits.  Or were you still waiting on capital to enable you to replace Bitcoins which were lost in the Linode intrusion as well as additional funds to cover the most recent loss?

We don't hold the funds.

03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.

Do you mean there will be another claim form to fill out, or are you just speaking about finalizing the current one?

Finalising the current one for internal (staff) usage. We need to track the payments we make more accurately for book keeping.

sounds good genjix, let us know if theres anything else we need to fill out, I have some stuff I want to buy D:
legendary
Activity: 1232
Merit: 1076
03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.

That's a bit confusing.  If all that was lost was 20% of Bitcoins on hand it should have been possible to pay everyone out 80% (the initial round of payouts) without receiving funds from Tihan - additional capital should only have been required to replace the lost Bitcoins.  You should have still been in possession of 100% of USD and 100% of Mt Gox deposits.  Or were you still waiting on capital to enable you to replace Bitcoins which were lost in the Linode intrusion as well as additional funds to cover the most recent loss?

We don't hold the funds.

03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.

Do you mean there will be another claim form to fill out, or are you just speaking about finalizing the current one?

Finalising the current one for internal (staff) usage. We need to track the payments we make more accurately for book keeping.
legendary
Activity: 826
Merit: 1001
rippleFanatic
We have off-site backups in a different DC. It's managed by Rackspace.

If the server crashed, we have no problems of recovering. There are a lot of backups of all our main servers. It's just that these backups were deleted by the hacker.

I meant data center of a different company (a different admin panel with a different password, and an append-only configuration). That's what provides a level of actual redudancy. Using a cloud service from one company protects you if a truck crashes into one of their data centers, but offers zero protection if someone gets your admin password!

I can't afford a scalable solution that gives me the same reliability for a bootstrapped startup.

Not anymore you can't!

You can make backups for cheap (backups aren't accessed, so they don't scale with the rest of the site).

All financial transactions will be handled by payment gateways who will be responsible for their own security. Apart from that, no money is involved so I just want to consider scalability, performance, availability and cost. For me, I think AWS's EC2 instance with Load Balancer handling SSL termination (can't be DDOS'd) + RDS with snapshots and binary logs recoverable to 5 minutes ago are more than enough for me.

Most people choose to outsource security, it's just that in Bitcoin world everything is DIY.

I'm not going to use your domain service or any other zhoutong "cloud" service because its clear that you don't have plan B (contingencies). What happens when someone deletes your AWS instances and its snapshots and logs? Has the thought even crossed your mind to try and see if its possible on the AWS admin panel?

Its a truism that at the basic level, security can't be outsourced. You have to trust someone eventually (unfortunately for us, you happened to trust bitcoin consultancy), but catastrophes can only be averted by good planning.
hero member
Activity: 868
Merit: 1000
03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.

That's a bit confusing.  If all that was lost was 20% of Bitcoins on hand it should have been possible to pay everyone out 80% (the initial round of payouts) without receiving funds from Tihan - additional capital should only have been required to replace the lost Bitcoins.  You should have still been in possession of 100% of USD and 100% of Mt Gox deposits.  Or were you still waiting on capital to enable you to replace Bitcoins which were lost in the Linode intrusion as well as additional funds to cover the most recent loss?
vip
Activity: 490
Merit: 502
For these reasons, I personally will never use Rackspace Cloud again unless they address all of these issues. AWS is way more secure than them.

Some guys have the fate of repeating the same mistakes over and over and over again. Roll Eyes


As I said, I won't engage in Bitcoin-related projects in the foreseeable future, you shouldn't assume that I'm going to operate a hot wallet in AWS.

I can't afford a scalable solution that gives me the same reliability for a bootstrapped startup. All financial transactions will be handled by payment gateways who will be responsible for their own security. Apart from that, no money is involved so I just want to consider scalability, performance, availability and cost. For me, I think AWS's EC2 instance with Load Balancer handling SSL termination (can't be DDOS'd) + RDS with snapshots and binary logs recoverable to 5 minutes ago are more than enough for me.

Most people choose to outsource security, it's just that in Bitcoin world everything is DIY.
hero member
Activity: 661
Merit: 500
03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.

Do you mean there will be another claim form to fill out, or are you just speaking about finalizing the current one?
vip
Activity: 490
Merit: 502
For these reasons, I personally will never use Rackspace Cloud again unless they address all of these issues. AWS is way more secure than them.

But that's still no excuse for not having offline backups. If you weren't online to notice the unauthorized rackspace session, the Rackspace admin "delete servers" bug (unable to disable) would still be an unknown bug/feature.

As for AWS, remember last year when bitomat.pl lost 17k BTC (iirc) in the blink of an eye when their AWS VPS was rebooted? MtGox bought them out and gauranteed depositor funds.

Don't trust a "cloud". (this is opposed to: first I trusted Linode, then I trusted Rackspace, and after getting burnt by Rackspace I finally decided to trust Amazon Web Services). Live and learn.

We have off-site backups in a different DC. It's managed by Rackspace.

If the server crashed, we have no problems of recovering. There are a lot of backups of all our main servers. It's just that these backups were deleted by the hacker.
legendary
Activity: 1232
Merit: 1076
03 June 2012 23:20: We're adding extra fields to the claims database (should be finished soon), we have received the funds from Tihan to make the initial payouts. Then once that's done, the first round of payments can be finished.
sr. member
Activity: 462
Merit: 250
Anyone heard from Patrick Strateman since this happened?  Anyone know him personally?  It would be interesting to hear his take on how it went down.
legendary
Activity: 1358
Merit: 1002
psy, I don't think ageis is the guy you are looking for. Just sayin'.

Then I guess he will have no problem explaining to us what does this mean
Code:
08:09.44 *** ageis materializes into BitcoinicaHacker
08:09.46 *** BitcoinicaHacker materializes into ageis
08:15.44 *** ageis materializes into B1tcoinz
08:15.51 *** B1tcoinz materializes into ageis
http://ibot.rikers.org/20120521.html.gz

Also:

Last post from him here in the forum: on: April 18, 2012, 12:06:25 AM
https://bitcointalksearch.org/topic/m.855980

Now for the final touch: https://bitcointalksearch.org/user/ageisp0lis-44466
Name:    ageisp0lis
Posts:    11
Position:    Jr. Member
Date Registered:    October 22, 2011, 02:03:34 AM
Last Active:    May 21, 2012, 08:47:58 AM

The same day of that IRC log, 30min after the username mess he made on IRC. How convenient lol
rjk
sr. member
Activity: 448
Merit: 250
1ngldh
psy, I don't think ageis is the guy you are looking for. Just sayin'.
Pages:
Jump to: