Topic: Hardware wallet wire protocol - page 3. (Read 8770 times)

I've been battling this concept in Armory with its deterministic wallets, for a while.  I implemented "blockCreated" and "timeCreated" variables for each address, but I have been hesitant to actually use them for anything.  Because, I determined that getting it wrong was not worth the performance benefit of when it is right:  if you trust the timestamps are created and stored correctly, but it turns out that something was wrong (incorrect clock when it was stored), the user could be missing a big chunk of their wallet and it could be a mess trying to figure out what happened (during recovery of a deterministic wallet, that is).   Plus, even if it is right, unless the user knows exactly how much they had they don't know that it's right unless you do the full scan anyway.  At least for right now, doing the full scan isn't that much more cumbersome than a partial scan, so it's easier to just do it.

I don't know how the HW device will be dealing with addresses (or what is possible memory-wise), but I have found it extremely useful to store the keys even though they are derived from the root.  Not only does it allow me to store birthdays and comments/labels, but (most importantly) I can search for an address and know if it's in my wallet without re-doing all the computation.  If it's possible to at least maintain addr160 list in non-volatile memory, it will make everything quite a bit easier.

I am hesitant to "endorse" requiring key-indexes to be included in the TxDPs (as I call them in BIP 10, perhaps another phrase+acronym would be better).  The reason is to do with multi-sig, which makes sense to accommodate here -- since there's like 90% overlap between what's needed for offline wallets and multi-sig.   There will be situations where the TxDP will be created by another party who doesn't have your watching-only wallet, they are only supplied a public key to include in the transaction.  They don't know how to specify what your address index is, and your offline device will have no way to detect what public keys belong to you.  I guess you could have an optional field so that your online computer could "amend" the TxDP with your address index, but that sounds complicated.

I know it sounds premature to think about multi-sig for your HW device, but I don't think it makes sense to over-complicate that use case if it can be avoided.  If we must accommodate devices that can't even store their own address list, then we have to deal with it, but it's worth trying to avoid.  And in the end, even with 5000 addresses, that's only about 100 kB to store a raw addr160 list...

Birthday is of continuing relevance to:  new users, new wallets.

...except the device doesn't with the keys, but with the seed (BIP32/Electrum). Device don't store private keys or any metadata for particular addresses.

However such metadata can be stored on the computer, using master public key as a wallet identifier.

Edit: Device can teoretically store timestamp of the seed generation, as some quick-and-dirty workaround. However this still must be provided by the computer, which will probably just ask the user.

All this is predicted on an invalid assumption:  the common case will be older, existing bitcoin users carrying around one big wallet.

First, wallet creation date is much different than individual key creation dates.

Keys are constantly created, therefore a portion of the wallet state is always created recently.  As blocks get bigger and bigger, key birthday reduces the number of keys one must scan in each block, especially if key retirement is also supported.

Long term you will have a rolling window of keys, no matter when the wallet is created.  Key birthdays will always be relevant and useful, for this reason.

Furthermore, there will always be new users creating new wallets, and users with multiple wallets.

It has significant advantage as your wallet has been created recently. You created wallet around block 200k, so you need to synchronize only around 10k of blocks (20x speedup). However in few years of usage, on block 1M, you'll need to synchronize 800k of blocks. Then the advantage of timestamp is almost neglible, because the speedup will be only 0.2x (probably a lot less, as blocks are bigger and bigger).

Can you elaborate?

All the major client developers agree this is a good idea.  The benefits into the future increase, as the block chain gets larger.  It makes an even bigger difference in the future.

While it might make a huge difference now (because seeded private keys are relatively new concept), it won't make a big difference in a future. I agree it might help to avoid unnecessary rescanning, but relying on that feature is flawed ...

Generally speaking, yes, you should store the key birthday (or seed birthday) somewhere.  When [re]scanning, this makes a huge difference in startup and network-sync times.

Correct.

Yes I think I will have to ask the user or get it from a non P2P server.

I presume the device does not even have a battery and gets it's power from the USB so it wouldn't have a real time clock.

Interesting ideas, Jim. Technically this is not a problem, however thanks to LoadDevice() method, device can contain seed used before the device was created. For example when user lost the device and will try to reload seed to new one. As this is not 100% bulletproof, we should look at another solution...


Looks like this is issue not directly related to Multibit, but to chainless clients generally. Maybe we should encode something into the mnemonic format itself? Full timestamp isn't possible because it requires too much bits, but maybe some lighter solution?

Edit: After quick brainstorming with Stick I realized that this is stupid idea, for many various reasons. So I think that asking user for some date is the best and the most transparent solution so far.

I have been starting to think about what UI changes will be needed for MultiBit for BIP32 wallets.

One of the limitations of MultiBit/ bitcoinj is that currently you do not have quick access to the complete set of unspent outputs for an arbitary key (or BIP32 sequence of keys).

MultiBit has to 'replay the blocks' i.e. get the blocks starting with a far-enough-back-in-time block and set what transactions are relevant. It is very useful to have a date "slightly before the first transaction or the birthdate of the wallet". You can then set the blockchain head to then and get the blocks from a bitcoind. Bloom filters will speed this up but it is conceptually the same.

Anyhow, is it possible to know the birthdate of a wallet ? By this I mean the date when the wallet seed was created.
Is there a real-time network-synced clock on the device that could be remember this and expose it? Or the host computer could give the device the current time perhaps.

Otherwise I will have to ask the user for the creation date of the wallet (which of course they will not know accurately) or get it from another channel other than bitcoind (possible but then I am leaking pretty important information to some server like blockchain.info or an electrum server and I am no longer strictly P2P).

After having a closer look at nanopb's API, I withdraw any request for limited sizes of messages or fields. It is indeed possible to decode smaller chunks of messages. There is even a mechanism to decode smaller chunks of individual fields.

Just a suggestion (I found default TI code quite bugged, especially under linux).
Since you want to use Stellaris Launchpad. The best way would be, ofc, to use a different chip for USB com which also be in charge to reset the arm chip by toggling its reset pin. E.g. if you use a USB<->SER converter, like one made by FTDI, you can use its RTS and DTR pins to perform reset.
If you want to have the ARM chip running wallet to do the USB device part too (which is bad for security imho), you can do something I tested already with TI code. In my case, the ARM chip runs a CDC, not HID, but it can easily converted to some HID command.
In usb_dev_serial, there is ControlHandler() which calls SetControlLineState() on USBD_CDC_EVENT_SET_CONTROL_LINE_STATE event (this happens when /dev/ttyACM is opened and closed).
In SetControlLineState() I added:
   
   
In main() I added:
   

I really don't want to interfere with your design, but my feeling is that a rarely-changing session_id will not be sufficient.

Fundamentally, you have a problem of synchronizing two state machines over a channel with erasures.

I'm not sure that the above problem has any simpler solution that the "sequence number" patterned after TCP over IP (or X-MODEM or Kermit protocol.)

The session_id would be then equivalent to seqence numbers starting with a value different than zero and different for each session.

Again, those are my feelings. I don't have a concrete proof, neither for nor against.

Edit: On the other hand I still remember the lesson you taught me in the Stratum thread: the protocol has to be conceptually simple enough to be reasonably implemented by the majority of the Bitcoin application programmers. If the price is an occasional, rare failure then it is worth to pay it.

I just added "session_id" variable into Initialize/Features messages, so the computer can easily restart the device's internal state and check that all buffers have been cleared, because computer received the same session id back.

Handling buffers and delayed responses may be a bit tricky as I already see in my serial transport prototype. However session_id/ping messages are quite effective way how to detect possible issues on the wire.

Hard-encoding maximum message size into the protocol sounds problematic. It already requires hacks for message signing, so you still need some workaround in the code. Why to limit other messages then?

PB message can be decoded in smaller chunks, i.e. you don't need to allocate receiving buffer to fit whole message into it.

I think what we try to establish in this thread is an application protocol. Physical transport is not that important (RS232, PS/2, USB HID, Infrared, ...)

It should speak RS232. Almost all point of sale systems speak it and the usb bridges are ubiquitous.

Watching.