Topic: How are big exchanges designed from a technical POV? (Read 917 times)

You would need to ask them to know for sure, but that's one way to do it. They could also use HD wallets for that purpose.


PR speak for a hot wallet that the operators think is fairly secure .

Hi,

so yesterday I stumbled over Kraken's Security Practices: https://www.kraken.com/security/practices
They claim:


How do they do that? I would do it like this:
Generate a pool of cold wallets and store the private keys on a offline computer. Then generate all addresses and move them savely (via usb)
to the main backend. When a user now creates a deposit address it is one from the cold wallet generated addresses.
Is that correct?


What is a semi-cold wallet?

As I mentioned above - CoinKite uses something in between with their HSMs. Other than that I don't see another solution for an exchange. (well, in the past exchanges used only "hot" wallets without any safeguards if you want to count that as an solution )

There are some plain wallet services that don't utilize multisig addresses and don't store unencrypted private keys on the server. Instead, they encrypt everything in the browser and then just send it to the server for storing. This is more or less ok, but doesn't let them to provide an exchange service.

- Cool thanks for the tipp. So from my understanding Brian Armstrong said that they are splitting up their private keys into small chunks and then
place them geographically on other places. In case of a hot wallet running out of coins they get notified and are required to physically pick up their part of the private key,
meet up, put the private key together and then send a transaction from the cold wallet to e.g. the hot wallet.

But why are splitkeys considered a good idea?
With multisignature keys you have the advantage of redundancy. Even if one of the people that you trust is refusing to open the cold wallet or lost his private key,
the cold storage can still be accessed by overruling him by the majority.
It seems to me that with splitkeys this is not the case. Once a part of the private key is lost, the wallet is gone. Is this correct or am I missing something?





- Hey, thanks for pointing those articles out
I will definitely give them a read!





- You are right of course! This would be a much easier approach, thanks for pointing out





- I think you got me wrong there. Let me describe the idea once again.
You as a user need to enter a trusted address when signing up to the service. You should only enter an address that you have and will have access forever. Like an address that was created by your desktop client or the blockchain.info wallet.
You can now deposit and withdraw money and trade as long as you want, only that when you do NOT trade for, lets say a day, your bitcoins, stored on the exchange, will be then sent automatically to your trusted address.
This means less coins for the exchange to protect and gives back responsibility to the customer.
I hope I did a better job explaining this time.




- Cool, actually I did hear from bitalo once, not knowing they/you are using multisignature addresses. So I definitely have to check out how it works. Thanks
(I'll be back once I wrapped my head around it )




- Thanks! I will definitely be PM'ing the shit out of your inbox



- Just for the sake of completeness. Is there any other approach you know? I mean, a bad one is okay too since I am evaluating all of them in my thesis

Let me address some of your points:


That's more or less true. You don't have to store the whole blockchain on your "cold wallet" though. The cold wallet is just a place to store private keys. The best approach would be to create a transaction "draft" (with inputs unsigned) on an online computer, then move it on a USB stick to the offline computer with cold storage, and then use a piece of software that lets you sign this transaction with private keys. Then you move out the resulting signed transaction back to online computer and send it to the network.


Yes, hot/cold wallet scheme has many problems, and you are right on almost all those points (besides the last one, maybe).


Yes, it's currently the most (over-)used approach in the industry. Is it recommendable? No. For the afermentioned reasons.


Wrong. Multi-sig addresses are implemented in the protocol for a long time. Last year some services started to pop out using this beautiful feature of Bitcoin. One of them is Bitalo.com, which is an exchange that utilizes this technology to make user funds safe even during the trading, but, what's even more important, when they only want to store their coins on us. By "safe", I mean safe even in case of hacker attack on Bitalo servers, or even safe from the administrators themselves.

Bitalo is not the only service that implements that (but AFAIK is the only *exchange* service). Other e-Wallets that implement multisignature include: GreenAddress.it, BitGo.com and Trustedcoin.com.


You never should send Bitcoins back to the address they came from without owners consent. This is exactly because there are many Bitcoin services which use hot/cold wallets and mixup Bitcoin addresses. So the address you deposit to is different than the address you're sending from while withdrawing. So if a service would send you back Bitcoins you sent from this kind of service, it could end up on other users wallet!
One service that doesn't work that way is Coinkite - they run a full-reserve service by utilizing Hardware Security Modules, which means that they have only one wallet instead of Hot and Cold, and so they can tie addresses to specific users.

I had experience from the administrator side with both types of exchanges (both hot/cold storage one and a multisig one) so if you have any more questions, feel free to ask .

You might be interested in a piece I wrote last year on how the traditional securities settlement system works (i.e. so you can compare/contrast):

http://gendal.wordpress.com/2014/01/05/a-simple-explanation-of-how-shares-move-around-the-securities-settlement-system/

And I jotted down some thoughts on how one might improve on today's Bitcoin exchanges here:

http://gendal.wordpress.com/2014/03/02/bitcoin-exchanges-are-more-centralised-than-traditional-exchanges-we-can-do-so-much-better-than-this/

You'll see that not everybody agreed with the views in my second post but hopefully it's useful food for thought.

Richard
(@gendal)

Brian Armstrong did this awesome interview with Kevin Rose a while back. He goes into some detail about how their system works.
http://techcrunch.com/video/coinbase-founder-brian-armstrong-on-bitcoin-security-foundation/518053205/

Hi,

last week I set up to write my bachelor thesis on bitcoin.
My topic is all about "programable money" so to say.
The question that I want to be answering in my thesis is how to design and build a secure and scalable exchange/payment processor/bitcoin thingy.

Now, I would like to discuss with you several approaches on how to design a such a software.
I'd love he hear your answers. If there is something new to the approaches you tell me, I'll add them to this post to collect them all.

So after reading alot, watching some talks about it I got the idea that there is mainly one big approach for this, people call it:
(Just to get this straight, I'll tell you know how I think it works, so that you guys can correct me)



Hot Wallet, Cold Storage:
The exchange/payment processor has a hot wallet that is connected to the main payment and transactions processing server.
All updates from the blockchain (meaning transactions, blocks) come in via this wallet.


Deposit: A user sends a request to the server. Server creates a new private key, public key and an address. Server sends user the address.
Server starts listening to this address, meaning: Server checks after every block generation if there are any new inputs on this address.
User sends his bitcoins to this address. Server sees it and confirms it after 6 confirmations on the blockchain.
Then an integer/float/whatever is set on the exchange database to this amount of bitcoin.

Withdrawal: User sends request to the server containing amount of withdrawal and address. Server checks multiple addresses on their balance and builds with them
the outgoing withdrawal.

Hot Wallet: Users only interact with the wallet indirect. The backend server interacts regularly direct with the hot wallet: creating new addresses, receiving and
sending bitcoins. Hot Wallet contains only a minimum amount of bitcoins because its not considered to be save while being on a server that is connected to the internet.

Cold Storage: Once the hot wallet reaches a critical amount of bitcoins they get automatically send to a cold storage address.
A cold storage address is created on a non internet connected server/computer. There is no real wallet software running.
A cold storage can only be called a cold storage as long as its private key wasn't exposed to the internet.

So okay... This is one part where I'm struggling to understand:
Lets assume that the hot wallet is running out of money. What needs to be done for getting bitcoins out of the cold storage?
I'm basing my guess on the most secure procedure I would follow:

You need to know where your bitcoins came from in order to resend them. This is due to the input output header in the bitcoin protocol.
So because of this you would need the latest blockchain. Okay, so we take a usb stick, copy the latest blockchain on it and go to our internetless computer/server and integrate the
blockchain. We then need a tool that constructs transactions. We sign the transactions with our private key and copy them to the usb stick.

We then got back to our internet computer and send the transactions out.

So I see some problems in this approach:

• This is a very costly approach. You'll need a person that is 24/7 available doing this and it's based on how much you trust that person
• Its manual. Programmers don't like that
• It did not work out in many cases: see mtgox...
• It feels like some kind of work around
• It doesn' save your from physical assault. I mean people robbed banks for less money. Why wouldn't they just rob a bitcoin wallet copying a hex string
• You need not only to implement one wallet system but two => complicated

So my questions on hot wallet cold storage are:

• Is this in your opinion the most used approach in the industry?
• Is this a recommendable approach?

So again, this is just what I heared is the most commonly discussed approach, but I would like to hear about other approaches.
I heard alot about multi-sig addresses, but that they are yet to be implemented in the protocol.
So if you know any other approaches, then please let me know or send me a link! That would help me out big times.

One approach that I thought about my self is this:

Inactivity hot wallet:

There is just the hot wallet, which is directly connected to the internet and the backend server.
Users deposit and withdraw money from it. When signing up, users must state a bitcoin address that belongs to them.
As long as users make trades their bitcoins are stored inside of that wallet. Once the user doesn't trade for a period of time, let's say 1 day,
the system notices this and sends the users bitcoins back to the stated address from the signing process.

Pros:

• Exchange spreads risk in not holding that much bitcoins
• Inactive users, aswell as people that like to use their exchange as a save storage get kicked out
• On an actuall security breach the damage is probably significantly less than with cold storage hot wallet
• Easyer implementation

Cons:

• Users get annoyed
• Wallet connected to internet holding all coins in operation, meaning everything the company "has" ==> very bad, single point of failure
• In times of big volume: Hot wallet holding millions of dollars ==> very bad

Moving on to scalability:

bitcoind wallet:
I heard that the bitcoind wallet is mainly designed for people using bitcoin normaly and by hand. Meaning sending and receiving some coins.
It has a JSON-RPC api, I also used it from time to time, but from my understanding, reading the documentation on github, is that it saves
transactions inside of Berkeley DB. From my understanding BDB is a non relational database. I also read that it does support the ACID paradigma.
Is this the case? And if so, does the bitcoind client support the ACID paradigma?

Another question I would like to be answered is: Is the bitcoind client scalable? I mean is it fast? How many requests can it handle per second?

"industry bitcoin wallet:"
Which wallet software is the industry using. Mainly big players like bitstamp, bitfinex, bitpay, coinbase? Are they all using their custom wallet software or is there
already a usable solution to this problem?


I think this would be it for the time being. I probably will be asking you guys many more questions and I would love to hear your answers.
Also, if some of the the questions have already been answered then feel free to send me a link