Unfortunately it is not that easy to secure a hot wallet.
The game plan should be:
1. Minimise amount of bitcoins to be held in hot wallet.
2. Have complete control over physical and management access to hardware and OS. It sucks but this requires extending trust to a few trusted people. This is the weakest link.
3. Definitely do not have ANY 3rd parties anywhere in the loop with any kind of access to the hot wallet server in either management or physical access capacity.
4. Obviously, having secure web interfacing code, with all user supplied data sanitised and hardened server helps a lot. But this is relatively trivial matter.
5. Using multisig functionality could help a lot. For example, say, a "offline" second sig server, which also has some monitoring code and freeze withdrawals based on some set of rules, until manual intervention resolves the flagged issues."
- Hosting your own email could help too in some cases.
- Securing cold wallets is another topic.
This basically means, decent colo setup with a few nuts and bolts on top of it. Hello! Big news! It was pretty much brought to Bitconica's attention in August/September 2011. I hope others will be able to learn from this.
This also means no hosting any wallets with nontrivial amounts of bitcoins on any:
- VPS's (generally, the bigger the company operating those VPS's, the more random dudes have root access, the more risk you take)
- dedicated servers are effectively the same VPS with all kinds of management access hosters have, however encrypted partitions could help a lil bit in this scenario, i.e. attacker would need to access it without rebooting.
(The above assumes VPS's and dedi's hosted by 3rd parties)
In other words, if you do not know who EXACTLY has or can have root access, say bye bye to your hot wallet.
Tough titty, "google search based sysadmins" and "flying in the clouds" generation! You simply cannot google up 10-20 years of professional experience and once you start relying blindly on all those cloud services you are screwed again.
Bitcoinica's "Zero sysadmin" policy in action. Enjoy!
Topic: [Emergency ANN] Bitcoinica site is taken offline for security investigation - page 70. (Read 224562 times)
May 11, 2012, 12:57:27 PM
May 11, 2012, 12:55:49 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
There always is a server. Some custom protocol doesn't change the fact that a server exists. When you send a command using the costom protocol where is going? Obviously bitcoind is running somewhere. Your solution is no solution. Attacker would simply bypass the stupid "interface" hit the real server and steal the private keys.
You do understand the private keys are simply numbers right? If you have the numbers you have the funds. Thieves don't need to use the lockdown bitcoind. They steal the private keys and execute a transaction from anywhere in the world.
Why would you have a custom interface but leave the bitcoind rpc port and ssh open to the public?
Are you intentionally missing their point?
Are they implying the hacker had physical access to the machine?
I don't keep my real wallet in a lockbox at a train station and I wouldn't keep a bitcoin wallet on public server at a datacenter.
May 11, 2012, 12:51:25 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
There always is a server. Some custom protocol doesn't change the fact that a server exists. When you send a command using the costom protocol where is going? Obviously bitcoind is running somewhere. Your solution is no solution. Attacker would simply bypass the stupid "interface" hit the real server and steal the private keys.
You do understand the private keys are simply numbers right? If you have the numbers you have the funds. Thieves don't need to use the lockdown bitcoind. They steal the private keys and execute a transaction from anywhere in the world.
Why would you have a custom interface but leave the bitcoind rpc port and ssh open to the public?
Are you intentionally missing their point?
Are they implying the hacker had physical access to the machine?
May 11, 2012, 12:49:00 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
There always is a server. Some custom protocol doesn't change the fact that a server exists. When you send a command using the costom protocol where is going? Obviously bitcoind is running somewhere. Your solution is no solution. Attacker would simply bypass the stupid "interface" hit the real server and steal the private keys.
You do understand the private keys are simply numbers right? If you have the numbers you have the funds. Thieves don't need to use the lockdown bitcoind. They steal the private keys and execute a transaction from anywhere in the world.
Why would you have a custom interface but leave the bitcoind rpc port and ssh open to the public?
Are you intentionally missing their point?
Are they implying the hacker would have physical access to the machine? I thought computers could only communicate through ports.
May 11, 2012, 12:47:26 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
There always is a server. Some custom protocol doesn't change the fact that a server exists. When you send a command using the costom protocol where is going? Obviously bitcoind is running somewhere. Your solution is no solution. Attacker would simply bypass the stupid "interface" hit the real server and steal the private keys.
You do understand the private keys are simply numbers right? If you have the numbers you have the funds. Thieves don't need to use the lockdown bitcoind. They steal the private keys and execute a transaction from anywhere in the world.
Why would you have a custom interface but leave the bitcoind rpc port and ssh open to the public?
Are you intentionally missing their point?
May 11, 2012, 12:45:59 PM
This is why. You guys completely ignore the magnitude of this, and completely ignore that Bitcoin is an unestablished currency. Being an unestablished currency, it needs a positive sum game far greater than those of established ones to ever hope to compete. Systematically allocating wealth from wealth creators to wealth destroyers as is being done in the Bitcoin world is unsustainable if Bitcoin is to have any value, since it represents a negative sum game in which wealth is continuosly destroyed.
Relax. It is geek currency, hackers are geeks. So they get their share one way or another. In long run it is utterly immaterial for Bitcoin how initial distribution/emission of money is done and who are "early adopters". Actually, the quicker "early adopter" lion share of Bitcoin is diluted, the better.
You say it is transferred from early adopters to geeks, but this has not at all been true lately. The only example I know of would be allinvain with his 25k coins if his report was true.
Consider also the cost of securing Bitcoins (this substracts from BTC value) and the barrier of entry it creates. I say these continuous gigantic (in terms of money supply %) thefts from large Bitcoin sites do hurt Bitcoin over the long run.
I am even thinking that this could be a similar attack to what happened with MtGox and all the other sites in summer/fall because MtGox and GLBSE were also targeted the past week. Which $$$ oriented hacker would redirect the site to Meatspin instead of a wallet.dat stealer or other malware? Ie possibly a coordinated attack on all infrastructure by someone who doesn’t like Bitcoin.
Disclosure: I hold Bitcoins and had no funds at Bitcoinica.
May 11, 2012, 12:44:19 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
There always is a server. Some custom protocol doesn't change the fact that a server exists. When you send a command using the costom protocol where is going? Obviously bitcoind is running somewhere. Your solution is no solution. Attacker would simply bypass the stupid "interface" hit the real server and steal the private keys.
You do understand the private keys are simply numbers right? If you have the numbers you have the funds. Thieves don't need to use the lockdown bitcoind. They steal the private keys and execute a transaction from anywhere in the world.
Why would you have a custom interface but leave the bitcoind rpc port and ssh open to the public?
May 11, 2012, 12:43:56 PM
So...any news on this yet? What is Zhou's twitter handle?
May 11, 2012, 12:36:18 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind.
Let me make it simple
Nobody is going to use the grace period.
Nobody is going to use the locked down bitcoind to send a tx.
They are just going to steal the private keys. No grace period on stolen keys.
Quote
There is no server to hack into when the only network-accessible thing is the custom interface.
There always is a server. Bitcoind has to be somewhere. If you have access to the server you have access to the keys. Period.*
It appear the attackers gained access to the server. Ergo they had access to the private keys.
* Well in theory maybe not with a HSM or TPM. But even then if the attacker has gained access to the wallet server your security model has already failed, it is just you could get lucky and avoid losing a lot of funds.
May 11, 2012, 12:35:09 PM
He's still 17? People don't age in Asia?
Well, they most certainly don't grow. 
we call them "little people" .....lol
I always wondered where Loompaland was, but I knew that green hair colour was fake.
So, that chocolate factory was one of the first sweat shops, eh?
/offtopicsilliness
May 11, 2012, 12:35:04 PM
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
If the keys are stolen, ANY bitcoind can make the transaction, doesn't have to be on the compromised server. May 11, 2012, 12:33:14 PM
This could have been avoided by not using the standard bitcoind rpc interface. If you have your own custom interface in between you can add large amounts of security measures such as withdraw verification and grace time. The hacker will also not be able to look up how your interface works by going to Google.
How do you know? From what we have heard, it has nothing whatsoever to do with the cracking that took place. Or do you have some inside info?18,000 BTC was withdrawn. If you had a custom interface you could make it piss red flags when it sees a transaction with such a large amount.
Which does nothing since Bitcoin is irreversible.
Quote
ALERT ALERT ALERT You have already been robbed. Have a nice day. 
The most likely attack vector was
a) gain access to rackspace admin console
b) reset root password
c) login as root
d) steal private keys
Speculation I know but we do know that just prior to the "cashout" tx hitting blockchain all the admins were notified of a password change.
So what exactly would a custom RPC do about that?
There is nothing to reverse if the transaction is canceled during the grace time before it is executed on bitcoind. There is no server to hack into when the only network-accessible thing is the custom interface.
May 11, 2012, 12:32:15 PM
This could have been avoided by not using the standard bitcoind rpc interface. If you have your own custom interface in between you can add large amounts of security measures such as withdraw verification and grace time. The hacker will also not be able to look up how your interface works by going to Google.
How do you know? From what we have heard, it has nothing whatsoever to do with the cracking that took place. Or do you have some inside info?18,000 BTC was withdrawn. If you had a custom interface you could make it piss red flags when it sees a transaction with such a large amount.
Which does nothing since Bitcoin is irreversible.
The most likely attack vector was
a) gain access to rackspace admin console
b) reset root password
c) login as root
d) steal private keys
So what exactly would a custom RPC do about that?
Anything less than an isolated dedicated, unmanaged server, is simply suicide.
May 11, 2012, 12:28:29 PM
This could have been avoided by not using the standard bitcoind rpc interface. If you have your own custom interface in between you can add large amounts of security measures such as withdraw verification and grace time. The hacker will also not be able to look up how your interface works by going to Google.
How do you know? From what we have heard, it has nothing whatsoever to do with the cracking that took place. Or do you have some inside info?18,000 BTC was withdrawn. If you had a custom interface you could make it piss red flags when it sees a transaction with such a large amount.
Which does nothing since Bitcoin is irreversible.
The most likely attack vector was
a) gain access to rackspace admin console
b) reset root password
c) login as root
d) steal private keys
So what exactly would a custom RPC do about that?
May 11, 2012, 12:26:38 PM
This could have been avoided by not using the standard bitcoind rpc interface. If you have your own custom interface in between you can add large amounts of security measures such as withdraw verification and grace time. The hacker will also not be able to look up how your interface works by going to Google.
How do you know? From what we have heard, it has nothing whatsoever to do with the cracking that took place. Or do you have some inside info?18,000 BTC was withdrawn. If you had a custom interface you could make it piss red flags when it sees a transaction with such a large amount.
Which does nothing since Bitcoin is irreversible.
Quote
ALERT ALERT ALERT You have already been robbed. Have a nice day.
The most likely attack vector was
a) gain access to rackspace admin console
b) reset root password
c) login as root
d) steal private keys
Speculation I know but we do know that just prior to the "cashout" tx hitting blockchain all the admins were notified of a password change.
So what exactly would a custom RPC do about that?
May 11, 2012, 12:25:32 PM
we call them "little people" .....lol
So do their girlfriends.
Damn, I can feel the fires of hell roasting my toes already.
May 11, 2012, 12:21:25 PM
He's still 17? People don't age in Asia?
Well, they most certainly don't grow. we call them "little people" .....lol
May 11, 2012, 12:20:39 PM
He's still 17? People don't age in Asia?
Well, they most certainly don't grow. May 11, 2012, 12:17:18 PM
He's still 17? People don't age in Asia?
I lol'd
May 11, 2012, 12:17:14 PM
Quote
lame comment is lame
yet, right
Jump to: