Pages:
Author

Topic: Hardware wallet wire protocol - page 3. (Read 8753 times)

legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 19, 2012, 07:17:32 PM
#45
First, wallet creation date is much different than individual key creation dates.

...except the device doesn't with the keys, but with the seed (BIP32/Electrum). Device don't store private keys or any metadata for particular addresses.

However such metadata can be stored on the computer, using master public key as a wallet identifier.

Edit: Device can teoretically store timestamp of the seed generation, as some quick-and-dirty workaround. However this still must be provided by the computer, which will probably just ask the user.

I've been battling this concept in Armory with its deterministic wallets, for a while.  I implemented "blockCreated" and "timeCreated" variables for each address, but I have been hesitant to actually use them for anything.  Because, I determined that getting it wrong was not worth the performance benefit of when it is right:  if you trust the timestamps are created and stored correctly, but it turns out that something was wrong (incorrect clock when it was stored), the user could be missing a big chunk of their wallet and it could be a mess trying to figure out what happened (during recovery of a deterministic wallet, that is).   Plus, even if it is right, unless the user knows exactly how much they had they don't know that it's right unless you do the full scan anyway.  At least for right now, doing the full scan isn't that much more cumbersome than a partial scan, so it's easier to just do it.

I don't know how the HW device will be dealing with addresses (or what is possible memory-wise), but I have found it extremely useful to store the keys even though they are derived from the root.  Not only does it allow me to store birthdays and comments/labels, but (most importantly) I can search for an address and know if it's in my wallet without re-doing all the computation.  If it's possible to at least maintain addr160 list in non-volatile memory, it will make everything quite a bit easier.

I am hesitant to "endorse" requiring key-indexes to be included in the TxDPs (as I call them in BIP 10, perhaps another phrase+acronym would be better).  The reason is to do with multi-sig, which makes sense to accommodate here -- since there's like 90% overlap between what's needed for offline wallets and multi-sig.   There will be situations where the TxDP will be created by another party who doesn't have your watching-only wallet, they are only supplied a public key to include in the transaction.  They don't know how to specify what your address index is, and your offline device will have no way to detect what public keys belong to you.  I guess you could have an optional field so that your online computer could "amend" the TxDP with your address index, but that sounds complicated.

I know it sounds premature to think about multi-sig for your HW device, but I don't think it makes sense to over-complicate that use case if it can be avoided.  If we must accommodate devices that can't even store their own address list, then we have to deal with it, but it's worth trying to avoid.  And in the end, even with 5000 addresses, that's only about 100 kB to store a raw addr160 list...



legendary
Activity: 1596
Merit: 1100
November 19, 2012, 03:54:11 PM
#44

Birthday is of continuing relevance to:  new users, new wallets.

legendary
Activity: 1386
Merit: 1097
November 19, 2012, 02:39:24 PM
#43
First, wallet creation date is much different than individual key creation dates.

...except the device doesn't with the keys, but with the seed (BIP32/Electrum). Device don't store private keys or any metadata for particular addresses.

However such metadata can be stored on the computer, using master public key as a wallet identifier.

Edit: Device can teoretically store timestamp of the seed generation, as some quick-and-dirty workaround. However this still must be provided by the computer, which will probably just ask the user.
legendary
Activity: 1596
Merit: 1100
November 19, 2012, 02:15:48 PM
#42
Can you elaborate?

It has significant advantage as your wallet has been created recently. You created wallet around block 200k, so you need to synchronize only around 10k of blocks (20x speedup). However in few years of usage, on block 1M, you'll need to synchronize 800k of blocks. Then the advantage of timestamp is almost neglible, because the speedup will be only 0.2x (probably a lot less, as blocks are bigger and bigger).

All this is predicted on an invalid assumption:  the common case will be older, existing bitcoin users carrying around one big wallet.

First, wallet creation date is much different than individual key creation dates.

Keys are constantly created, therefore a portion of the wallet state is always created recently.  As blocks get bigger and bigger, key birthday reduces the number of keys one must scan in each block, especially if key retirement is also supported.

Long term you will have a rolling window of keys, no matter when the wallet is created.  Key birthdays will always be relevant and useful, for this reason.

Furthermore, there will always be new users creating new wallets, and users with multiple wallets.

legendary
Activity: 1386
Merit: 1097
November 19, 2012, 01:58:39 PM
#41
Can you elaborate?

It has significant advantage as your wallet has been created recently. You created wallet around block 200k, so you need to synchronize only around 10k of blocks (20x speedup). However in few years of usage, on block 1M, you'll need to synchronize 800k of blocks. Then the advantage of timestamp is almost neglible, because the speedup will be only 0.2x (probably a lot less, as blocks are bigger and bigger).
legendary
Activity: 1596
Merit: 1100
November 19, 2012, 01:38:05 PM
#40
Generally speaking, yes, you should store the key birthday (or seed birthday) somewhere.  When [re]scanning, this makes a huge difference in startup and network-sync times.

While it might make a huge difference now (because seeded private keys are relatively new concept), it won't make a big difference in a future. I agree it might help to avoid unnecessary rescanning, but relying on that feature is flawed ...

Can you elaborate?

All the major client developers agree this is a good idea.  The benefits into the future increase, as the block chain gets larger.  It makes an even bigger difference in the future.

sr. member
Activity: 441
Merit: 268
November 19, 2012, 12:29:27 PM
#39
Generally speaking, yes, you should store the key birthday (or seed birthday) somewhere.  When [re]scanning, this makes a huge difference in startup and network-sync times.

While it might make a huge difference now (because seeded private keys are relatively new concept), it won't make a big difference in a future. I agree it might help to avoid unnecessary rescanning, but relying on that feature is flawed ...
legendary
Activity: 1596
Merit: 1100
November 19, 2012, 11:47:05 AM
#38
Anyhow, is it possible to know the birthdate of a wallet ? By this I mean the date when the wallet seed was created.
Is there a real-time network-synced clock on the device that could be remember this and expose it? Or the host computer could give the device the current time perhaps.

Generally speaking, yes, you should store the key birthday (or seed birthday) somewhere.  When [re]scanning, this makes a huge difference in startup and network-sync times.
legendary
Activity: 1386
Merit: 1097
November 19, 2012, 11:45:34 AM
#37
I presume the device does not even have a battery and gets it's power from the USB so I wouldn't have a real time clock.

Correct.
legendary
Activity: 1708
Merit: 1066
November 19, 2012, 11:43:46 AM
#36
Yes I think I will have to ask the user or get it from a non P2P server.

I presume the device does not even have a battery and gets it's power from the USB so it wouldn't have a real time clock.
legendary
Activity: 1386
Merit: 1097
November 19, 2012, 10:06:26 AM
#35
Anyhow, is it possible to know the birthdate of a wallet ?

Interesting ideas, Jim. Technically this is not a problem, however thanks to LoadDevice() method, device can contain seed used before the device was created. For example when user lost the device and will try to reload seed to new one. As this is not 100% bulletproof, we should look at another solution...

Quote
Otherwise I will have to ask the user for the creation date of the wallet (which of course they will not know accurately) or get it from another channel other than bitcoind (possible but then I am leaking pretty important information to some server like blockchain.info or an electrum server and I am no longer strictly P2P).

Looks like this is issue not directly related to Multibit, but to chainless clients generally. Maybe we should encode something into the mnemonic format itself? Full timestamp isn't possible because it requires too much bits, but maybe some lighter solution?

Edit: After quick brainstorming with Stick I realized that this is stupid idea, for many various reasons. So I think that asking user for some date is the best and the most transparent solution so far.
legendary
Activity: 1708
Merit: 1066
November 19, 2012, 09:37:43 AM
#34
I have been starting to think about what UI changes will be needed for MultiBit for BIP32 wallets.

One of the limitations of MultiBit/ bitcoinj is that currently you do not have quick access to the complete set of unspent outputs for an arbitary key (or BIP32 sequence of keys).

MultiBit has to 'replay the blocks' i.e. get the blocks starting with a far-enough-back-in-time block and set what transactions are relevant. It is very useful to have a date "slightly before the first transaction or the birthdate of the wallet". You can then set the blockchain head to then and get the blocks from a bitcoind. Bloom filters will speed this up but it is conceptually the same.

Anyhow, is it possible to know the birthdate of a wallet ? By this I mean the date when the wallet seed was created.
Is there a real-time network-synced clock on the device that could be remember this and expose it? Or the host computer could give the device the current time perhaps.

Otherwise I will have to ask the user for the creation date of the wallet (which of course they will not know accurately) or get it from another channel other than bitcoind (possible but then I am leaking pretty important information to some server like blockchain.info or an electrum server and I am no longer strictly P2P).

member
Activity: 78
Merit: 11
Chris Chua
November 19, 2012, 08:29:32 AM
#33
My only request is that, for all the non-signing messages, there is a maximum message size specified. A reasonable value would be something like 256 bytes. This allows buffers to be statically allocated (i.e. without the use of malloc). The limit cannot apply to signing messages, because transactions can be very large.

Hard-encoding maximum message size into the protocol sounds problematic. It already requires hacks for message signing, so you still need some workaround in the code. Why to limit other messages then?

PB message can be decoded in smaller chunks, i.e. you don't need to allocate receiving buffer to fit whole message into it.
After having a closer look at nanopb's API, I withdraw any request for limited sizes of messages or fields. It is indeed possible to decode smaller chunks of messages. There is even a mechanism to decode smaller chunks of individual fields.
sr. member
Activity: 406
Merit: 250
LTC
November 19, 2012, 06:48:21 AM
#32
Just a suggestion (I found default TI code quite bugged, especially under linux).
Since you want to use Stellaris Launchpad. The best way would be, ofc, to use a different chip for USB com which also be in charge to reset the arm chip by toggling its reset pin. E.g. if you use a USB<->SER converter, like one made by FTDI, you can use its RTS and DTR pins to perform reset.
If you want to have the ARM chip running wallet to do the USB device part too (which is bad for security imho), you can do something I tested already with TI code. In my case, the ARM chip runs a CDC, not HID, but it can easily converted to some HID command.
In usb_dev_serial, there is ControlHandler() which calls SetControlLineState() on USBD_CDC_EVENT_SET_CONTROL_LINE_STATE event (this happens when /dev/ttyACM is opened and closed).
In SetControlLineState() I added:
Code:
        //when /dev/ttyACM is closed, RTS ->0 is emulated. When this happens, we will reset the entire chip

//old_can_send, can_send some boolean/char vars initialized to 0

can_send = 0;
if((usState & 2) != 0) //D1  Carrier control for half duplex modems. This signal corresponds to V.24 signal 105 and RS-232 signal RTS.
can_send  = 1;
if(old_can_send != can_send ){
if(old_can_send == 1){
MAP_SysCtlPeripheralReset(SYSCTL_PERIPH_USB0); //explicit peri reset, system reset wont reset peripherials too
SysCtlReset(); //system reset
}
//other blah blah

}
old_can_send = can_send;
   
   
In main() I added:
Code:
    USBDCDCInit(0, (tUSBDCDCDevice *)&g_sCDCDevice);

//after initing USB stuff or it will segfault
    //system reset and peri reset dont signal usb host anything so we have to do something about
    //USB host will see this device disconnecting and another device connecting, so everything will be reinitialized host side
    MAP_USBDevDisconnect(USB0_BASE);
    for(ulLoop = 0; ulLoop < 100000000; ulLoop++){;}
    MAP_USBDevConnect(USB0_BASE);
   
legendary
Activity: 2128
Merit: 1073
November 19, 2012, 06:03:19 AM
#31
I just added "session_id" variable into Initialize/Features messages, so the computer can easily restart the device's internal state and check that all buffers have been cleared, because computer received the same session id back.

Handling buffers and delayed responses may be a bit tricky as I already see in my serial transport prototype. However session_id/ping messages are quite effective way how to detect possible issues on the wire.
I really don't want to interfere with your design, but my feeling is that a rarely-changing session_id will not be sufficient.

Fundamentally, you have a problem of synchronizing two state machines over a channel with erasures.

I'm not sure that the above problem has any simpler solution that the "sequence number" patterned after TCP over IP (or X-MODEM or Kermit protocol.)

The session_id would be then equivalent to seqence numbers starting with a value different than zero and different for each session.

Again, those are my feelings. I don't have a concrete proof, neither for nor against.

Edit: On the other hand I still remember the lesson you taught me in the Stratum thread: the protocol has to be conceptually simple enough to be reasonably implemented by the majority of the Bitcoin application programmers. If the price is an occasional, rare failure then it is worth to pay it.
legendary
Activity: 1386
Merit: 1097
November 19, 2012, 05:43:37 AM
#30
...There could be also an opposite situation where the USB device is plugged to the powered USB hub. The user interaction could cause hub disconnection, but the device's internal state would not be affected...

I just added "session_id" variable into Initialize/Features messages, so the computer can easily restart the device's internal state and check that all buffers have been cleared, because computer received the same session id back.

Handling buffers and delayed responses may be a bit tricky as I already see in my serial transport prototype. However session_id/ping messages are quite effective way how to detect possible issues on the wire.
legendary
Activity: 1386
Merit: 1097
November 19, 2012, 05:28:55 AM
#29
My only request is that, for all the non-signing messages, there is a maximum message size specified. A reasonable value would be something like 256 bytes. This allows buffers to be statically allocated (i.e. without the use of malloc). The limit cannot apply to signing messages, because transactions can be very large.

Hard-encoding maximum message size into the protocol sounds problematic. It already requires hacks for message signing, so you still need some workaround in the code. Why to limit other messages then?

PB message can be decoded in smaller chunks, i.e. you don't need to allocate receiving buffer to fit whole message into it.
sr. member
Activity: 441
Merit: 268
November 18, 2012, 04:46:56 PM
#28
It should speak RS232. Almost all point of sale systems speak it and the usb bridges are ubiquitous.
I think what we try to establish in this thread is an application protocol. Physical transport is not that important (RS232, PS/2, USB HID, Infrared, ...)
newbie
Activity: 56
Merit: 0
November 18, 2012, 01:15:55 PM
#27
It should speak RS232. Almost all point of sale systems speak it and the usb bridges are ubiquitous.
legendary
Activity: 3920
Merit: 2349
Eadem mutata resurgo
November 16, 2012, 06:20:01 PM
#26
Watching.
Pages:
Jump to: