Topic: Password-protected private key export format (Read 7523 times)

OP vanished, maybe he withdrawed the prize himself

someone claim prize. can you give a solution?

FYI, this is the format used by vanitygen 0.22 when you specify -e.

I had a hell of a time sorting out how to decode them, thought this might help someone else.  I'm using PHP, but PHP is easy to read.  I'm using an external PBKDF2 function.  If you use the function included with newer PHP versions, you may need to adjust the parameter order.  The mcrypt_cbc() is old too, check with the PHP manual for updated information, or use phpseclib.

The $input value is the raw base58coded string.  My base58decode function returns the binary output without verifying the check value.  If yours checks it before returning, you can skip that part.

The output value is either -1 (bad 58 check code), -2 (wrong password), or the private key as a raw binary string.


If using phpseclib, use this instead of the mcrypt_cbc call:

Keeping it short is also important if you want to store it in a QR code

This sounds very interesting.  I can see simple use cases like using one EC key both as a bitcoin account and for signing PGP messages.  Possibly using a bitcoin-disclosed public key to encrypt messages, if this is safe.  How else would integration be possible?

Another factor, GnuPG has yet to produce a stable release with support for EC keys.

It's great to see that someone is trying to establish standards in bitcoin community. A while ago I was doing a bit of research regarding the private key export and came to a conclusion that OpenPGP message format would suit this use case perfectly. I encourage you to take look into RFC 4880 if you haven't already, quite a brilliant piece of work. What I advocate is to use the existing, vast system that OpenPGP is and try fitting bitcoin into it.

Advantages to taking this approach are numerous.

• OpenPGP is a well known and tested standard
• Security is built in
• There is room for growth and modifications
• It's very robust and covers almost all use cases
• Possible use of existing OpenPGP compliant architecture

There is one great disadvantage, though. With robustness comes complexity - it's not a quick and easy fix to the problem of key sharing but rather a lengthy process of integrating bitcoin into OpenPGP ecosystem.

The primary goal of this format is to create something like PKCS#8/RFC5958.  PKCS#8 is the format commonly used to store SSL/TLS private keys, in password-protected form or otherwise.  It is a very flexible format with metadata describing of the type of private key, and the PBKDF parameters and block cipher used to encrypt it.

It's possible to convert an exported bitcoin private key in the base-58 format to and from PKCS#8.  In PKCS#8, the built-in password-protection feature can be applied to the private key.  Also, if a bitcoin key were to be imported into some other non-bitcoin application for whatever reason, perhaps as a signing key, PKCS#8 would be the best bet as an import format.

It's also possible to take a PKCS#8 representation and produce a reduced mapping that assumes certain parameters.  If we assume the use of the SECG 256k1 EC curve, a set of of PBKDF2 parameters, and the use of AES with a specific key size, the representation can be smaller.  Unfortunately, due to RFC5915, there is a practical lower limit on the size of any reduced format.  For bitcoin private keys, this is close to 135 bytes, and it includes numerous details that, for our purposes, aren't necessary or can be assumed.  This is unwieldy large.

The next step of evolution from PKCS#8 might work as follows.

privkey = Minimal EC private key representation, 32 bytes long
param = Parameter descriptor byte.  This is an index into a predefined list of parameter groups, perhaps like:

• 0: SECG 256k1 key, PBKDF2, HMAC-SHA256, 8-byte salt, 4096 iterations, AES-256-CBC
• 1: SECG 256k1 key, PBKDF2, HMAC-SHA256, 8-byte salt, 4096 iterations, Camellia-256-CBC
• 2: SECG 256k1 key, PBKDF2, HMAC-SHA256, 8-byte salt, 16384 iterations, AES-256-CBC
• etc..

pbhash = Hash function from parameter list
cipher = Block cipher function from parameter list
salt = Random salt value of size described in parameter list
iter = PBKDF iteration count described in parameter list
pbkey = PBKDF2(password, salt, pbhash, iter)
protkey = param | cipher(privkey, pbkey) | salt

This would handle the cryptography in the most conservative, commonly-used way, and should not make anyone nervous, or at least, not any more nervous than PKCS#8.  It would also make the format easily adaptable to new block ciphers and key derivation functions.  The smallest representation using AES would be 58 bytes long, and would convert to an 83-character base-58 string.  This is much more compact than a PKCS#8 interoperable format, but I still think it's too long.

I'll say don't give up on that goal. There may be an incremental way to redesign it.

1) Create a separate DB_ENV just for the wallet.
2) Change single B-tree DB into three RecNo DB-s backed by flat-text files. This is quite decently supported by the Berkeley DB, especially because the wallet database is essentially an append-only. The 3 files would be public-keys, private-keys and transaction-log.
3) I see lot of people here proposing public-key operation only for shared/managed hosts. But I have yet to see anyone discussing proper transaction auditing as done by the accountants. This is why I was thinking about 3 separate databases, the 3rd being the one given to the auditors.
4) During the transition (and possibly afterwards) the default client will continue to use Oracle BDB as its only database interface.

This incremental approach could assuage Gavin's worries that the patches/pull would try to surreptitiously change the basic protocol and principles of operation for Bitcoin.

Swapping around means storing a private key outside of a wallet.dat, either to be imported into another wallet.dat or another bitcoin client implementation.  It's an interchange format, not a specific wallet implementation.

The format proposed here is an extension that allows an exported private key to be password-protected.  Naturally, if it is to be interchangeable with another bitcoin wallet or another implementation, it shouldn't/can't be protected using the same password or master key as the wallet from which it came (if that wallet is encrypted).  The initial use case is to allow someone to use vanitygen to generate an address, and securely store the private key until it is later imported into a client.

Anyway, I think this proposal is orthogonal to the encryption of wallet.dat.

If you would rather this format follow the same password-derivation and encryption scheme as the master keys of the bitcoin wallet encryption system, that's certainly possible.  They're already quite similar, and until pixelglow came out and recommended PBKDF2, they both used EVP_BytesToKey().  I don't think the use cases between the wallet master key and a password-protected export format are quite the same though.

If special attention should go to entering the private key by keyboard, making it short and so on, shouldn't the format foresee multiple checksums to detect typing errors instead of the standard hash check provided by EncodeBase58Check?

If you would split the long private key in - let's say -  6 blocks, a checksum per block instead of one on the whole string as EncodeBase58Check does right now, gui's that allow entering the key in parts  would be capable of hilighting the specific part in red that contains a typing error, instead of invalidating the whole input.

I'm convinced at this point that the on-disk wallet format needs to be completely scrapped and redesigned from scratch. Unfortunately, I'm not likely to have the spare time to devote to working on this.

We are at the point where outside utilities are starting to need more access to the wallet.  See pywallet, for an example of a very useful tool that I wish I didn't have to shut down my client to use.

One option would be to have the client open the wallet database without a total lock, so that other things could use it at the same time.  In that case, using per-key encryption inside the wallet would be a good thing for sure.

But most of the things that would be enabled by allowing concurrent wallet access could also be done through RPC, if we can just make the right RPC commands.  Oh, and we'll probably need at least a little bit of granularity in the RPC security system.

At any rate, these encrypted single keys would always be useful as a transfer format for moving keys between servers, archiving onto paper or CDs, offline generation, etc.  Whether they are useful as an alternate storage format inside the wallet will depend on whether we go with RPC or unlocking the database.

Two different encryption schemes and/or passwords seems like a bad idea to me, both from a code maintenance/security point of view and from a usability point-of-view.

Can you expand on "private keys in base-58 export format are great for swapping around" -- what's the use case?  Who are you swapping with, and how?

Absolutely!  The goal is to handle the keying independently and not depend on the master key of a specific encrypted wallet.  So, if it were used for exporting, it wouldn't be as convenient as "backupwallet," and at least one export password would be required.  But it could be implemented to be more convenient than dumpwallet + GPG.


Thanks!  I posted a note to the list and referenced that thread.

Right, of course. I just generalized things a bit.

If you're talking about private keys, the distiction remains: either you want to export/import what is in an already encrypted wallet (useful for backup), or you want to do encryption separately. The intent here seems to be the second case, which is fine. It just means you'll need to both the wallet passphrase and the key password ready if you want to import.

Also, considering the used version byte, could you follow this thread? I'd say encrypted privkeys form a new data class.

It's not about dumping the wallet but a shared scheme for encrypting private keys.

A possible use case

You could generate an offline paper wallet containing a number of public Bitcoin addresses and their corresponding private keys. The private keys will be encrypted with a password only you know but in a standard format.

The main reason to have a standard encryption for the private key is that when you want to redeem coins against those addresses the bitcoin client or an online wallet will be able to decrypt them for you when you give your password.

Also an online wallet could store the encrypted keys for you. You don't have to worry about the service being attacked, the private keys would be worthless without the passwords. (assuming strong passwords).

Hope this helps.

I'm not sure what your use case is here. Encrypted dumps of the wallet are obviously useful, but I think they come in two categories:

1) combined with encrypted wallet, just a dump of the encrypted keys therein. This would allow you to export and import dumps without providing a password. I believe this is the safest option, but it would be inherently implementation-specific (the encryption inside bitcoin, as planned for 0.4.0, uses EVP-SHA512 keys derived from a password, used to encrypt a 256-bit master key using AES256-CBC, the wallet keys themselves are encrypted using this master key using AES256-CBC, with the hash of their pubkey as IV).

2) a portable format for exchanging wallets (hopefully between several applications). if the encryption is independent from that of the wallet storage itself, that means exporting requires knowledge of the wallet passphrase + another key to encrypt the dump with. Although there is definitely merit in doing this inside bitcoin itself, i don't think there is anything better than doing a normal wallet dump + GPG/OpenSSL/... encrypting it.

I'm only a regular programmer with a math background, hence the interest in cryptography. I once worked on a font DRM scheme which used crypto, and I'm currently working on an Australian bitcoin exchange that will feature crypto as well. The rest of it is just google  .


http://www.openssl.org/docs/crypto/EVP_BytesToKey.html seems to indicate you should move to PKCS#5 v2.0, which is PBKDF2.


Indeed, there is the PKCS5_PBKDF2_HMAC_SHA1 function in OpenSSL. I just submitted a patch to node.js to incorporate this: https://github.com/joyent/node/pull/1491


RFC 2104 (http://tools.ietf.org/html/rfc2104) seems to think that any half-decent cryptohash will do, even MD5 (!), since HMAC has a secret key which greatly expands the amount of computation you need to do to break it. I suggested HMAC-SHA1 since the PBKDF2 in OpenSSL is based on it (symmetry!), it seems to be used often enough (newer schemes have less cryptanalysis done, could have flaws you don't anticipate esp. used in tandem like with HMAC) and produces a smaller output.


Indeed. Simple schemes work best sometimes in crypto since less can go wrong. As for using XOR, check out wikipedia's article on Vernam ciphers: http://en.wikipedia.org/wiki/Vernam_cipher. TL;DR: so long as the key is truly random, as large as the plaintext and never reused in whole or part, it should be unbreakable. Obviously we can't get the first one 100% since we're starting with a human password and PBKDF2 I think doesn't make any guarantees about the randomness of the result. The second requirement is true. The last requirement should be ensured by the use of random salts.

Does bitcoin encryption use AES with a single key on the entire wallet, or AES on individual keys (without an IV or starting at the same point)? The former is pretty sensible. The latter can be dangerous since you are effectively reusing the key.

Excellent!!

Pixelglow, you really seem to know your stuff, are you a pro??

The PBKDF that I've been using isn't exactly PKCS#5 v1, it's actually OpenSSL's EVP_BytesToKey().  It's similar but will produce arbitrarily large key material.  Otherwise, there certainly wouldn't be enough key material for an AES IV using SHA256.

Regardless, using PBKDF2 with an HMAC function would seem to be a very desirable change, and also appears to be very easy to get with OpenSSL.

Regarding pwcheck, the use of an HMAC function there with some additional key material also seems like the appropriate tool for the job, more so than my hacked-together mess.  Can't really say no to that either.

Would you recommend HMAC-SHA1 over HMAC-SHA256?

Somebody smart said that a work is finished not when everything that can be added to it has been added, but when everything that can be removed has been removed.  AES may fit into this category.  However, other similar password-protection schemes, including bitcoin's built-in wallet encryption, also use AES for fixed-size private keys, and it's nerve-racking to replace it with XOR, even though it may be the correct thing to do.

Thank you very much!!

Interesting, although your layering/composing of different cryptographic schemes may come back to bite you.

As I see it, you want to encrypt the password and also verify that the encrypted bits are unchanged. Here's a slight modification of the above:

privkey = 32-byte EC private key, big-endian form
salt = 4-byte random salt value
symkey = PBKDF2(HMAC-SHA1, password, salt, 4096, 64)
cryptedprivkey = (privkey XOR symkey[0:32])
pwcheck = HMAC-SHA1(symkey[33:64],cryptedprivkey)
protkey = cryptedprivkey | pwcheck | salt

• You should use the more recent PBKDF2 (http://www.di-mgt.com.au/cryptoKDFs.html), which allows you to produce arbitrary length keys. In this case we're generating a 64-byte length key, half of which we'll use for encryption and the other half for authentication.
• Next we encrypt the privkey with half the symkey by XOR'ing them together, a simple stream cipher. This avoids the use of AES in client code (slow in Javascript?, missing an initialization vector, plaintext length < AES block size of 128 bits) and should be pretty safe. Stream ciphers like RC4 are simply a random keystream XOR with the plaintext and are difficult to break, provided the random keystream is truly unpredictable and you never repeat the key. The trouble with stream ciphers then manifests when you have a large enough chunk of data to encode, then without a proper CPRNG like RC4 the keystream may repeat etc. and thus block ciphers under CBC become useful. Since we have a 32-byte private key and a 32-byte symkey to XOR together, this should not be the issue.
• We then use a MAC, specifically a HMAC-SHA1, to act as the password check against modification. We use the second half of the symkey as the secret. HMAC's are particularly designed for message authentication.

Alternatively (safer/more paranoid) you could use two salts to produce two 32-byte keys, one for the encryption, one for the authentication.