Topic: We thwarted a hacking attempt on Bitcoin Video Casino and learned a lot. - page 2. (Read 3284 times)

We thwarted a hacking attempt on Bitcoin Video Casino and learned a lot. We wanted to share our experience so that other site operators can be prepared.

On Monday Sep 9th, an attempt to hack into Bitcoin Video Casino was made. We initially noticed something was amiss because our server had become unreachable for several hours. When our server eventually came back online again, we noticed that the machine had been restarted. Our server has had a 100% uptime since we launched (~185 days ago) and this is the first time it was restarted.

We snooped around a bit and found an open login by a user on /dev/tty1, a local controlling terminal. That's very bad. All logins should only be from psuedo-terminals (ptyX). More looking around found a script hiding in /etc called "bitcoin.sh" and a crontab entry meant to constantly restart the script. The script was nothing but a phone home to this intruder's Amazon EC2 instance and wasn't very clever, but it did the job it was intended to. Unfortunately for the hacker, he left his private key on our servers.

We took all of the standard precautions when building our website: offline cold storage of most Bitcoins, closing down open ports, disabling password logins (ssh keys only!), but most importantly we decided to use full disk encryption, with the disk decryption key encrypted with GPG.

The downside to full disk encryption is that our site cannot come up automatically after a reboot. That's fine, really, we never reboot the machine anyway. So once the would-be hacker restarted the machine, he had no way to access our database, our hot wallet, or our game source code as they reside on the encrypted disk. The hacker couldn't even mount the encrypted disks, as the decryption key isn't stored on the server.

We had a long talk with our hosting provider on how this happened. Nobody should be able to log into our machine locally (the /dev/tty1 part). We thought that someone local to our hosting provider's data center was trying to break in, but as we pressed for answers, we learned that a KVM device was installed that allowed remote access into our machine one day prior to the hack. The evidence that the device was plugged in was present in the kernel log ('dmesg'). While the machine was rebooted, the hacker modified /etc/sudoers and changed an account password so that he could have access after the reboot. Later, auth.log would verify his activity.

So why did someone working at the datacenter install a KVM on our machine when we didn't request it? It turns out that our hosting provider's billing software was actually where the hack began. An order was fraudulently placed on our account to install the KVM through his hacking of HostBill, the software our hosting provider uses. It's a particularly popular piece of software, so it's likely other hosting providers could be vulnerable.

Anyway, the long story short is that we didn't lose any of our Bitcoins or our game software because of one smart move: full disk encryption. We highly recommend to all you developers out there to use full disk encryption if you have sensitive data to protect on remote servers. Full disk encryption doesn't help thwart hot attacks while the site is live, but it does help tremendously when hackers could possibly get physical control of your remote-hosted machine.

Note: We also posted this to Reddit, so all of the redditors on here can see the post at http://www.reddit.com/r/Bitcoin/comments/1nicl9/we_thwarted_a_hacking_attempt_on_bitcoin_video/