Wallet import/export: bitkeys format

LehmanSister

member

Activity: 68

Merit: 10

High Desert Dweller-Where Space and Time Meet $

Quote from: Pieter Wuille on March 14, 2011, 07:11:50 PM

I've though about it. It's a very simple way to avoid any parsing/escaping difficulty, while making it easy to extend it further afterwards. On the other hand, CSV is so simple that you don't even need any library for parsing.

Oh how I wish this were true (I used to believe this myself). The number of extremely bizarre CSV dialects I've encountered over the years occupies far too many brain cells. Salvation can oft be found here: http://docs.python.org/library/csv.html#csv.Dialect . I'm still running into occasional Excel CSV exports that break if there's a comma in a field (not that I'm implying that Excel is an example of stellar programming).

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

Quote from: coga on May 13, 2011, 06:59:37 PM

You know what this thing would be awesome for? Gift cards.

If you need to give someone, say, 5 BTC as a birthday gift, you:

- Generate public/private key pair
- Transfer 5 BTC to that pair
- Export (possibly with delete) the key from wallet
- Convert exported key to a QR code
- print the code on a postcard

I know, it's kinda geeky, but it could be awesome way to both give something of value, as well as get more people familiar with bitcoin

See jgarzic's scratchoff patch: https://bitcointalksearch.org/topic/patch-bitcoin-scratch-off-cards-4555 and BitBills https://bitcointalksearch.org/topic/introducing-bitbills-7724.

coga

full member

Activity: 222

Merit: 100

www.btcbuy.info

You know what this thing would be awesome for? Gift cards.

If you need to give someone, say, 5 BTC as a birthday gift, you:

- Generate public/private key pair
- Transfer 5 BTC to that pair
- Export (possibly with delete) the key from wallet
- Convert exported key to a QR code
- print the code on a postcard

I know, it's kinda geeky, but it could be awesome way to both give something of value, as well as get more people familiar with bitcoin

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Code:

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="exportedWallet"

--exportedWallet
Content-Type: application/x-ecdsa-private-key
Content-Transfer-Encoding: base64
X-Bitcoin-Network-ID: 
X-Bitcoin-First-Block: 
X-Bitcoin-Txid: f2ed2f6cc779696969690696abababa4ccf185d478aa799eb7d3e561c77b1022
X-Bitcoin-Txid: f2ed2f6cc779696969690696abababa4ccf185d478aa799eb7d338712abff2b3
X-Bitcoin-Start-Block: 
X-Bitcoin-Wallet-Flags: 
X-Bitcoin-Wallet-Label: 
X-Bitcoin-Wallet-Comment: 

PGh0bWw+CiAgPGhlYWQ+CiAgPC9oZWFkPgogIDxib2R5PgogICAgPHA+VGhpcyBpcyB0aGUg
Ym9keSBvZiB0aGUgbWVzc2FnZS48L3A+CiAgPC9ib2R5Pgo8L2h0bWw+Cg==
--exportedWallet
Content-Type: application/x-ecdsa-private-key
Content-Transfer-Encoding: base64
X-Bitcoin-Network-ID: 
X-Bitcoin-First-Block: 
X-Bitcoin-Txid: f2ed2f6cc779696969690696abababa4ccf185d478aa799eb7d3e561c77b1024
X-Bitcoin-Txid: f2ed2f6cc779696969690696abababa4ccf185d478aa799eb7d338712abff2b4
X-Bitcoin-Start-Block: 
X-Bitcoin-Wallet-Flags: 
X-Bitcoin-Wallet-Label: 
X-Bitcoin-Wallet-Comment: 

PGh0bWw+CiAgPGhlYWQ+CiAgPC9oZWFkPgogIDxib2R5PgogICAgPHA+VGhpcyBpcyB0aGUg
Ym9keSBvZiB0aGUgbWVzc2FnZS48L3A+CiAgPC9ib2R5Pgo8L2h0bWw+Cg==
--exportedWallet--

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

Quote from: bitcoinex on March 21, 2011, 09:30:13 PM

(I'm not sure what it's all stored in a PEM.)

Using standart PEM gives an advantage: standard means to convert the key into binary format and then use this binary key for the barcode etc

And I agree, but I'm not convinced it's the best way to handle wallet dumps.

Anyway, my current walletdump branch (https://github.com/sipa/bitcoin/tree/walletdump) contains a "dumppemkey

" RPC call, which exports in the standard PEM format... 720 characters per key (compared to the base58 representation of a (bitcoin-specific) private key, 51 characters).

It currently does not support importing keys, or extracting only the public key of an address.

bitcoinex

sr. member

Activity: 350

Merit: 252

probiwon.com

Quote from: Pieter Wuille on March 21, 2011, 07:45:20 AM

Quote from: bitcoinex on March 21, 2011, 06:31:45 AM

Quote from: Mike Hearn on March 21, 2011, 05:47:32 AM

PEM structures have a lot of data we don't need,

?

We only need the 256-bit private key, not curve parameters, generator point and public key, which are stored in the OpenSSL structures (and some of those in .pem files too, judging by their size) as well.

(I'm not sure what it's all stored in a PEM.)

Using standart PEM gives an advantage: standard means to convert the key into binary format and then use this binary key for the barcode etc

BitterTea

sr. member

Activity: 294

Merit: 250

Quote from: Hal on March 21, 2011, 02:30:23 PM

There's other stuff in wallets, including preferences, address book entries for payees, and account information. bitkeys format won't be suitable for wallet backup if it loses all this.

That other stuff doesn't necessarily need to be in the same file or format as the private keys. I can see a use for exporting address book and account information, and I think those could be in a similar (CSV) format. I don't see much of a use for exporting settings, though it could be exported as well (INI format?).

Hal

vip

Activity: 314

Merit: 3803

Quote from: Pieter Wuille on March 14, 2011, 05:15:45 PM

Simply by adding reserve keys (which are in fact harder to filter out when exporting, since they are available in mapKeys like change keys and used keys), you have everything to have a bitkeys file function as a backup or by limiting its contents, as an interchange format.

There's other stuff in wallets, including preferences, address book entries for payees, and account information. bitkeys format won't be suitable for wallet backup if it loses all this.

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

Quote from: bitcoinex on March 21, 2011, 06:31:45 AM

Quote from: Mike Hearn on March 21, 2011, 05:47:32 AM

PEM structures have a lot of data we don't need,

?

We only need the 256-bit private key, not curve parameters, generator point and public key, which are stored in the OpenSSL structures (and some of those in .pem files too, judging by their size) as well.

bitcoinex

sr. member

Activity: 350

Merit: 252

probiwon.com

Quote from: Mike Hearn on March 21, 2011, 05:47:32 AM

PEM structures have a lot of data we don't need,

?

Quote

and lack data that is useful like block numbers, labels etc.

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

PEM looks useful as well, especially when people want to use bitcoin's keys to sign other things. Shouldn't be too hard to implement that as well.

However, what I'm trying to do is create a self-contained, compact, human readable, stable form of storage for wallets. PEM files are standard but not-so-compact ways of storing purely the cryptographic part.

Mike Hearn

legendary

Activity: 1526

Merit: 1129

PEM structures have a lot of data we don't need, and lack data that is useful like block numbers, labels etc.

bitcoinex

sr. member

Activity: 350

Merit: 252

probiwon.com

Quote from: jgarzik on March 14, 2011, 03:43:32 PM

Keys have a standard PEM format...

+1

https://bitcointalksearch.org/topic/m.34009

Quote

so the format should be :
-----BEGIN EC PARAMETERS-----
BgUrgQQACg==
-----END EC PARAMETERS-----
-----BEGIN EC PRIVATE KEY-----
MHQCAQEEII84GK/wX8stbmJWYL/WUM1nPAK1miIBDBeyNuo2vyf4oAcGBSuBBAAK
oUQDQgAE/yRZIKrOj4GBfLFtMYuocJ5QF1Tr9rWMh2ixCyfodDWRWTIU21v3ehDR
NJiAXHKwkhDqQ//i46NHoNUhjvx/lw==
-----END EC PRIVATE KEY-----

(I got this output from "openssl ecparam -out key.pem -name secp256k1 -genkey")

Mike Hearn

legendary

Activity: 1526

Merit: 1129

That's great! Keep going! :-)

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

Ok, I've since decided to use a CSV-like format anyway. It is more compact (for use in QR-encoded form, eg.), and easier to process using for example command-line tools like grep and cut.

Here is a preliminary branch that supports importing and exporting transactions: https://github.com/sipa/bitcoin/tree/walletdump

Working:

dumping a wallet to file
importing a file
watching for others doing transactions with your keys
rescanning the relevant portion of the block chain after importing
correct updating of transactions and balance in the GUI

Todo:

error-handling when a double spend has been tried anyway
importing of reserve keys
test

The file contains lines of the form:

Code:

,[,]

Current flags are:

R: reserve key
L: label is present

For example:
92263nMUE7SkR62tMn1Ucu2AxY5nANmQhXRhxHZP6MVxpfLC2EH,10801,L,"Your Address" 934xQoFPHzYkwTvJQwQSPw8QspwVB8kPd1S3wRZsZGbgLKVQPb7,10814 92xRg4At7vN25B1esc768NvW7Exrr2PcgZ78MFNmd58MzXjXGYM,10955,R

jgarzik

legendary

Activity: 1596

Merit: 1091

Quote from: Pieter Wuille on March 14, 2011, 07:11:50 PM

Still, a proposal:
[ { // moura6ybHvwzMtrWdyqiEN6n2mMakf8HxD (50.01 BTC) "privkey" : "91iwnurxhWmDF9gqwc4CbuLCB4HxuMXf5B2vEhTPDftY3rkMgDY", "firstblock" : 10806 }, { // mmFPR1oaDExBfJqpRohWBvbE2aCg86Kwcx (0.00 BTC) "privkey" : "92JsNVMJgG4RiC2ARxpduJp8DXKgdDMj9WREE5jo66Hg8UMdr3L", "firstblock" : 10870, "label" : "My Address" }, { // midiVjwgBCkMSup4X2FifFinwLhpHYe2wn "privkey" : "91s3swvX2a3F7FDoypEV7rTVhWUKiw8RCBmDuHMN8xMksLn3YtV", "firstblock" : 10870 }, { // reserve "privkey" : "93NM1B7y35d9VxqGBvc8DZcAvEB9kE5UZV7WqNGMTNWhPBdhsJT", "firstblock" : 10870, "reserve" : true } ]

Very nice. If you have this JSON, you shouldn't need anything else IMO.

Quote

Does anyone know how JSON libraries cope with comments?

According to my research, the sad fact is comments are outside JSON scope Sad

You may add faux-entries like

Code:

    "_comment1" : "privkey is the private key associated with your request",
    "_comment2" : "firstblock is the first block where it appeared",

Ugly hack, I know.

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

Quote from: BitterTea on March 14, 2011, 01:25:16 PM

Quote from: Gavin Andresen on March 14, 2011, 01:21:35 PM

I'd suggest using a standard, existing format instead of inventing a new one.

I'm assuming this functionality is going to be exposed via JSON-RPC, at least at first. Why not have it return a JSON representation of the private keys?

I've though about it. It's a very simple way to avoid any parsing/escaping difficulty, while making it easy to extend it further afterwards. On the other hand, CSV is so simple that you don't even need any library for parsing.

Still, a proposal:
[ { // moura6ybHvwzMtrWdyqiEN6n2mMakf8HxD (50.01 BTC) "privkey" : "91iwnurxhWmDF9gqwc4CbuLCB4HxuMXf5B2vEhTPDftY3rkMgDY", "firstblock" : 10806 }, { // mmFPR1oaDExBfJqpRohWBvbE2aCg86Kwcx (0.00 BTC) "privkey" : "92JsNVMJgG4RiC2ARxpduJp8DXKgdDMj9WREE5jo66Hg8UMdr3L", "firstblock" : 10870, "label" : "My Address" }, { // midiVjwgBCkMSup4X2FifFinwLhpHYe2wn "privkey" : "91s3swvX2a3F7FDoypEV7rTVhWUKiw8RCBmDuHMN8xMksLn3YtV", "firstblock" : 10870 }, { // reserve "privkey" : "93NM1B7y35d9VxqGBvc8DZcAvEB9kE5UZV7WqNGMTNWhPBdhsJT", "firstblock" : 10870, "reserve" : true } ]

Does anyone know how JSON libraries cope with comments?

Pieter Wuille

legendary

Activity: 1072

Merit: 1174

Quote from: Hal on March 14, 2011, 03:53:48 PM

I would focus on use cases.

1. Back up all the keys in a wallet, and later restore the wallet state if the wallet is deleted or corrupted.

2. Merge two wallets by exporting all the keys in one, and importing to another.

3. Export and remove selected keys from wallet, for moving to safe long-term storage in case the wallet is hacked; later, move the keys back to a wallet for spending.

4. Export selected keys for transferring their funds to someone else; the recipient imports the keys and gains access to the funds.

Any others?

I wonder whether it makes sense to use bitkeys format for case 1 and 2. Wallets do have other things, which is part of your problem. I'd suggest that case 1 & 2 should use wallet formats.

I don't like the wallet as more than an internal database for one specific implementation of a bitcoin client. It's not transparent or human readable, hard to keep updated when accessing it using external tools, and maybe very different from what other implementations like to use. It is very fast and hard to corrupt, so it is perfect for an internal database, but for all other applications, i prefer a separate, more interchangeable format.

Simply by adding reserve keys (which are in fact harder to filter out when exporting, since they are available in mapKeys like change keys and used keys), you have everything to have a bitkeys file function as a backup or by limiting its contents, as an interchange format.

Quote from: Hal on March 14, 2011, 03:53:48 PM

Finally, as far as the address and balance: I don't like putting them into CSV fields, because they are redundant and unnecessary. Your comment fields seem okay, but note that the balance may be out of date. How about a command that would take a bitkeys file, and display the address and current balance? Then you could leave the comments out.

I agree, I suggested exporting them as comments because they help making a bitkeys files also a useful way to inspect your wallet (which is maybe also a use case?), and possibly select which keys from it you want to give to someone else (essentially doing use case 3 or 4 by hand, by copy-pasting a part of a bitkeys file).

BitterTea

sr. member

Activity: 294

Merit: 250

Quote from: Gavin Andresen on March 14, 2011, 04:43:08 PM

And one usability comment:

Is "selected keys" the right way to go? Just selecting an amount of bitcoins to export/send seems like a better way to go (with a send-to-self transaction generated, if necessary, to create the right amount), with maybe an "advanced" option later that lets you select specific inputs to export/send.

Yes, this is a very good idea, and what I had in mind as a way to manage separate pools of encrypted keys. You could either let Bitcoin choose keys as it does now to create a transaction, and leave it at "around the amount you entered", or do as you say, with a send to self transaction to get the exact amount.

Gavin Andresen

legendary

Activity: 1652

Merit: 2216

Chief Scientist

Focusing on the use cases is the right approach.

Quote from: Hal on March 14, 2011, 03:53:48 PM

4. Export selected keys for transferring their funds to someone else; the recipient imports the keys and gains access to the funds.
Any others?

One variation:

5. Export keys to transfer funds to somebody else. Recipient is expected to import and then immediately send-to-self. Sender keeps the keys 'set aside' in the wallet until they are spent by recipient, or, if not spent by recipient (lost in transit maybe, or recipient chokes on a lollipop before having a chance to redeem), may be added back into the sender's balance.

And one usability comment:

Is "selected keys" the right way to go? Just selecting an amount of bitcoins to export/send seems like a better way to go (with a send-to-self transaction generated, if necessary, to create the right amount), with maybe an "advanced" option later that lets you select specific inputs to export/send.

Topic: Wallet import/export: bitkeys format (Read 19240 times)