Topic: Hardware wallet wire protocol - page 2. (Read 8676 times)

If it is own address, desktop software should indicate this by filling "address_n" field in the message structure. Device can recalculate the address from the seed and this vector, so malicious machine cannot steal coins in this way.

You can do this if you re-order the fields* in the protocol buffer messages to match the order of the Bitcoin protocol (inputs before outputs, previous output reference before input script etc.). Then, you send the entire transaction through once per required signature. Each time an output is encountered, you prompt the user about it. This is merely an extension of your "streaming" idea, but applied to the whole transaction.

You can probably see the problem with this: the user will be prompted repeatedly, for each required signature. This makes for an unnecessarily bad user experience. This is how I intend to deal with the problem in my implementation:
1. While processing the transaction, a hash for signing needs to be calculated. In addition to this "signing hash", calculate an "input invariant hash" which is just like the signing hash, except it ignores all input scripts.
2. The input invariant hash will be the same regardless of which input is being signed (that's why it's called the "input invariant hash").
3. If the user fully approves a transaction, save that transaction's input invariant hash.
4. Include a field in the SignTx message which specifies whether the required signature is for a repeat of the previous transaction.
5. If the host specifies that the transaction is repeated, do not prompt the user.
6. In order to stop the host from lying about the status of the repeated transaction, compare the input invariant hash of the current repeated transaction with the saved input invariant hash, returning a Failure message if there is a mismatch.

The host signs a series of inputs like this:

*This brings me to something which will probably be a non-issue, but we should be aware of: (according to https://developers.google.com/protocol-buffers/docs/encoding) field ordering in a protocol buffer message is not guaranteed. Every serialiser will write out fields in field number order, but certain operations may not preserve this ordering.

Even without multi-sig, some people include a large number of outputs.   Many times, because it's inconvenient to actually execute an offline transaction, they may pool together many different outgoing transactions into a single transaction.  I know Armory users do this (I do it myself, sometimes), but usually because my offline computer needs to be booted in order to execute the tx and I'm just lazy   But for this reason, you probably need the device to be able to scroll so that it can display an arbitrary number of outputs.   I even have it on my own TODO list, to put Armory's confirmation list onto a scroll area to accommodate the rare times users specify two dozen outputs... (but I haven't had anyone complain yet, so I guess it's not common to do that many).

Something else that you need to accommodate:  since the device can't recognize its own addresses, it can't recognize its own change address.  Therefore, your protocol needs some way to tell the device which output is change -- perhaps specify the address index so that it can check that the marked change address truly is in your wallet.  Without it, the device doesn't know which is which, and even if the user understands how transactions and change work, a compromised desktop computer could swap the change address and the user wouldn't realize it's malicious. 

By the way:  amusingly-relevant article on slashdot today about clocks getting off by 12 years...

http://news.slashdot.org/story/12/11/20/216234/ntp-glitch-reverts-clocks-back-to-2000

Yes, exactly. You cannot expect full bitcoin implementation from the device with 256kB flash, 32kB RAM and a passive USB HID interface ;-). The purpose of the device is:
* Do not leak seed/private keys to the computer
* Confirmation of tx signing by the user, by displaying the transaction on the display and waiting for button press.

...and nothing more.


Exactly!


I designed something similar for input transactions, I call it "input streaming". In this way device don't need to store all inputs in the memory so the device can sign transaction with unlimited counts of inputs.

Although very high count of inputs may happen naturally (like when people are consolidating tiny pool payouts), having so many output transactions isn't so common and usually such complex transaction can be split into more smaller txes. However I over-looked the case of multisig transactions, where having very high number of outputs may be required as well. I'll try to propose some solution for this case too. Thank you for pointing to the problem...


Not at all. I'd like to have a protocol which allow signing of data streams, without any limit to input and output transactions. As I described above, I already have a solution for input streaming, now I just need to think about output streaming. But this can be definitely solved...

So the real divergence here was that I assumed the device would be somewhat "independent."  It sounds like, instead, your device will be wholly dependent on a host computer to do most of the work for it, and the device will literally only sign.  There will always be a computer-in-the-loop with your watching-only wallet to reformat the thing-to-be-signed into this low-level format for the device.   I had anticipated the device take, from any computer anywhere, a BIP10-like data stream, recognize it's own public keys/addresses, and get user confirmation to sign it.

Okay, we're on the same page now.  My point is valid, that a BIP10+ could potentially serve this purpose if we added extra complexity to accommodate devices that don't even know their own address pool, but it's not necessary.

So, if you have some enormous transaction to sign, let's say with 100 inputs and the transaction data was 100 kB, can you at least hold 100 OutPoint-Value pairs in RAM?  i.e. you feed in first supporting transaction and output index (because your transaction will be spending that output from that tx) and the device will hash it and confirm that the hash and then store the hash and output value in RAM.  Then repeat for all supporting transactions to collect a list of "verified" outpoint-value pairs.  Then when you feed in the transaction to be signed, it can go through it's local memory to confirm each outpoint that belongs to you to be signed.

Then, it sounds like, if that transaction is actually 100 kB, then you can't even hold the whole thing at once.  If you have to sign 30 inputs, you would pass it in 30 times, each time with a different txin-script-in-txout-script-place-and-all-others-blanked.  The thing can accumulate the hash as it goes, and then pass out the signatures one at a time when it's done reading the final transaction.

Is that too crazy?  Or should the device be limited to handling transactions that will fit nicely into its cache?  It could be a requirement of the software that interfaces it (Armory, Electrum, etc), that it will prevent the user from sending tx bigger than X kB...?  However, even if this tx is within the size limit, there's no guarantee that any of your supporting transactions is within the size limit.

The protocol which I proposed is built on top of Protocol buffers. Having a separate parser for another format just to comply with BIP10 doesn't look smart enough.


The conversion from BIP10 to something built on top of PB can be quite straightforward, so even BIP10-compatible software might convert the format for the device easily. To be honest, I didn't study internals of the BIP10 protocol yet, I just watched the protocol format.


You still need some desktop software to work with the device. As far as Armory or Electrum will support this device and Armory or Electrum implements BIP10, there's really no need why device itself must understand BIP10. Or any simple desktop (command line) tool can convert BIP10 format and pass the transaction into the device over USB.


It will be computed by the computer, not by the device. I don't see problem here.


This protocol is low-level layer for communication between the device and desktop software. This protocol must be optimized for low-resource devices and must be really minimal. There's no need why this should use BIP10 at all.

I think we diverged here.  I was kind of under the impression that we were coming up with a BIP 10+ that was more versatile (like, having a compact binary representation) so it could also be implemented in your device.   I'd really like to see this "protocol" standardized across devices and applicaitons -- which was the original goal of BIP 10, but I was the only one using it, so it never really got the versatility of a real "standard"... I stopped developing it when it worked for Armory, knowing it would be expanded later.

So, I was hoping that your device would simply implement the same BIP 10+, and everyone is happy.  I'm looking for the possibility that someone who creates a TxDP on their desktop, and requiring 3 other signatures, some of which may come from regular desktops, offline computers and/or your devices, that there won't need to be any special conversion for your device to be able to understand it, recognize its own keys, and sign it (after user confirmation).

With that in mind, I think someone42 mentioned the device won't hold its own address list, and thus this protocol requires telling the device which keys belong to it and what address index those keys are in the wallet.   I really wanted to avoid that, if possible, since it complicate other use cases.  It sounded like you said the device could regenerate all of its addresses/public keys from the root ... but given the other limitations of the device, I would think that calculating 200 public keys from a root private key every time would take a while.  And once you've used it a lot, 200 may not be enough.  If this is an unavoidable implementation, then we may be required to split the use cases into different "standards", but I hope you recognize the value of aiming for a single standard.

tl;dr  The protocol can easily be unified with all other devices/applications (i.e. BIP 10+) if you can have the device recognize its own addresses/pubkeys.  I was hoping for that unification, but it may not be achievable/efficient to do so.

Why we should store addresses on the device, when the address can be generated on the request from the seed?


Generate such list of addresses is a matter of second for the desktop machine. Using the device's memory for storing the list will limit the usability. For example we're using platform with 256kB memory for both application and data. Thanks to memory fragmentation, only limited part of that 256kB can be used for data storage.


As far as I understand the BIP10, it is out of the scope for such device. Device requires very minimal interface and it needs prepared data by the computer. For example it requires streaming of inputs in separate messages to allow signing of huge transactions. However desktop software using the device for signatures can be BIP10 compatible, of course.


Public keys can be derived from master public keys stored in offline wallet, so yes, desktop software can compute indexes just from public keys.


Yes, this is expected. Software can download master public key from the device, store it offline and then handle it as an offline wallet, generate addresses or so.

Yeah, we don't need to get too sidetracked on that topic.  I was just stating that even if you could store timestamps with each address, I'm not sure you're getting too much value out of it.

However, let me get back to the previous topic I brought up -- what's the chance you could store a raw addr160 list on the device?  Here's what I said before:


If the hardware device can only store the root, then it's going require either:  (1) a computer holding the watching-only wallet for the offline device to create the TxDP to be signed, or (2) When you give someone your public key for a multi-sig tx you gotta supply an address index with it somehow so they can include it, or (3) The TxDP must pass through the computer holding the watching-only wallet so that it can be amended with the correct address index. 

Implementing NTP directly in the device is without a chance, it is completely out of the focus.

However there can be some API for storing/retrieving timestamp to/from the device. But it will be just a dumb storage of the value and it will be up to the computer's software if it will use it or not. I cannot imagine anything useful what can be done with the timestamp by the device.

You're right that timestamps will be probably more and more in the future as people typically use recent wallets. Except that the device is designed more like a cold storage, so that timestamp optimization won't improve user experience for most of the users. Still, if storing timestamp of the seed generation will be useful, we can include it.

Yeah, I know.  On the other hand, DHCP can provide local NTP servers.

Timekeeping is one of the things that really pisses me off.  There is absolutely no excuse for any network operator to fail to provide and advertise local NTP servers.  GPS receivers with PPS interfaces have been dirt cheap and quite common for 10 or 15 years now, and ntpd supports them all.  Doing it right is well within the technical and financial means of any network provider.  Hell, within the means of most college students.

And device makers suck.  It isn't hard at all to implement NTP properly, but no one seems to until they end up on slashdot.  And times are changing; even the dinosaurs that have kept PBX systems in the dark ages have started to catch on.

I'm fully with you on the use of ISO format for dates, but I disagree on the NTP for autoconfiguration.

NTP abuse is so rampant that it now has its own notable Wikipedia article:

http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse

And thats just a tip of iceberg. In many locales servers like "time.windows.com" are unreachable because of the rampant abuse by the various software licensing/subscription enforcement schemes. McAfee private-label products were implicated in DDoS-ing Swiss government NTP servers, if I recall correctly. And apparently the situation is slowly getting worse, not better.

The european system is just as stupid, just incompatibly so.  ISO8601 puts magnitude in the proper order on all scales, and there hasn't been a valid excuse for using anything else in a long, long time (when you need actual dates, not seconds-from-epoch).

If you don't know the proper birthdate of a key or wallet, set it to zero so you get a full rescan.  Key birthdates is something I wish for about twice a week.  I do a lot of private key importing because I work with offline key generation.  The raw transaction API lets me bypass a lot of that, but not all.

If you don't think that key birthdays are important, just give it a few more years.  If bitcoin is going to take off, transaction volume (and thus scan complexity) is going to grow much faster than Moore's law, and the rescan that you think is just fine now will be downright painful soon.

P.S.  NTP has been around since 1985, if someone doesn't know if their clock is right or not, they deserve what they get.

I personally hate the American system.  It doesn't make any sense.  Though, I'm not sure how relevant it is here:  UTC unix time is the "path of least resistance" for storing timestamps in binary files, and thus wouldn't have that problem.

Part of the reason I avoided this is because there's a lot that can go wrong with offline computers.  The clocks could be totally wrong with no access to the internet, and thus they don't know the block height at birth, either.  I could insert special cases where I know one or the other based on what information is available at the time it was recorded in the log file.  But then I have to treat all conditions of {have/donthave} {block/time}.  And as I said... I think getting it wrong and silently missing parts of the user's wallet could be a disaster -- those could literally be lost forever, because no program that ever reads the file would check before the recorded time.

The end result is that users would just want to scan everything anyway, just to be sure. 

Hardcore Americanism detected!  

The month-day-year versus day-month-year ambiguity with dates is an endless source of the errors that stick is thinking about. People in USA don't see is as often because most of the BIOSes and other low-level tools use the American layout. But it trips over many Europeans.

Wait till Americans start importing Czech-made wallets that use day-month-year order...

I'm not sure.  The "cost of misclassification" is not symmetric.  It takes 5 minutes to do it right 99.9% of the time, but it takes 7 min to do it right 100.0% of the time.  And given the heartache associated when it's done wrong, why not just take the 7 minutes?  At least, that's been my attitude, even if you don't agree with the trade-off. 

You're stretching.  Anything is "possible" but using a 0.01% case to justify giving new users a bad experience is not reasonable.

That's wrong assumption. If the battery on your motherboard is empty the date would be 1.1.1970. I can come up with some other scenarios where the date/time is seriously incorrect.

The clock might be wrong... but it's not gonna days or weeks wrong.

You have to include variability anyway, because blocks can vary.  So given birthday X, start scanning at X minus 2 days, or X minus 1 week.