Pages:
Author

Topic: Wallet Import Format - page 2. (Read 6698 times)

TT
member
Activity: 77
Merit: 10
November 16, 2011, 11:47:11 AM
#26
The signature is:

304502205ea291ce55ecc95f346f6be2c198993dcb1a72cc4eddf520f173ed9ac85a1e03022100d 0ae6c394d014de8fecb44d034904a0c6142e6335a394aa4629d7a839aaaa2 <--- 70 bytes

a301  <---- two additional bytes. what hashTypeCode does this correspond to and
how do we expand it to 4 bytes?

Or am I missing something?
TT
member
Activity: 77
Merit: 10
November 16, 2011, 11:45:00 AM
#25
Right, I saw your diagram. I think I get all of it except for the hashTypeCode.

So NewTx broken down looks like:

NewTx
-----
01000000 <---version (big endian)
01           <----tx_in count

fbe470cf995c04ecaa82fc2d4ae598075e21986700b544f660ffea93a6a82fe5 <-PrevTx hash (big endian)
01000000    <------ index (big endian)

8b      <----- scriptSig length (139 bytes = 2 opcodes + 72 for sig + 65 for pubKey)
48      <---- push the next 72 bytes onto stack OpCode
304502205ea291ce55ecc95f346f6be2c198993dcb1a72cc4eddf520f173ed9ac85a1e03022100d 0ae6c394d014de8fecb44d034904a0c6142e6335a394aa4629d7a839aaaa2a301 <---- 72 bytes (sig)
41      <----- push the next 65 bytes onto stack OpCode
043ef1593aa79bab3c6a21f4f82f348b12e68d107f95f577e610466aca7d0f2e4ebcfd9a9bafcad 7207618ec4e5e78358dbac3dff4a432b833bf8a9cee85834ac0 <---- 65 bytes (pubKey)

ffffffff     <------sequence
01        <----- tx_out count
8096980000000000  <------- value (big-endian)
19     <--- pk_script length (25 bytes)
76a9148073e789954e05c5938c5cc493308f9021539bb588ac <---- pk_script
00000000   <----- lock time

.....
so where exactly is the hashTypeCode?

-TT
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 16, 2011, 08:35:53 AM
#24
Sorry to break the news to you:  but you're asking about OP_CHECKSIG which is really complicated.  But, see my post here for a description of how it works (about halfway down the page).  It tells you exactly how to construct the string to be signed.

And everything should be in little-endian unless otherwise stated, including the hashcode.  Just make sure the final hash is big-endian before signing.  Yeah, it's ridiculous...that's why I made this diagram!

-Eto
TT
member
Activity: 77
Merit: 10
November 16, 2011, 01:36:25 AM
#23
Specifically, I couldn't find the hash type and wasn't sure what to enter for the hash type code. where is the hashtype byte? and what 4 bytes do I need to enter for the code? and do these 4 bytes need to be big-endian? or little-endian?

-TT
TT
member
Activity: 77
Merit: 10
November 16, 2011, 01:21:16 AM
#22
OK, I have the following raw data:
Quote
Private key: 24ed089647b7f330588c491309e527c44cbf5e04444540782d6b88f8c44b3105 length: 64
Public key hash: 211f0c809a1a14f46af53ae59aa32d02aaf72724 length: 40
------------------------
Wallet import format: 5J6YocBZpn5j9hcPWv1wPEGtfXvHP8g2ZPSSTrjgr9PxUhALeYM length: 51
Address: 1428VeCoiJR81vVjdtXe9sb5G15qjYyLwo length: 34

PrevTx - Raw Data
-----------------
0100000001e1877fe168c04e1f91a170c37320d8d6e6dbac94cb1edf56eab2d075f548cb9300000 0008c493046022100b0ac6689455d95fb81f0012f38b9285d44ae75f64b4c82ea9d1e96c2541392 7c022100bcf31e15dde4d83b567f848cf6b4a708a23f0a71a206d858bfaea0285fca350f014104c 6420d1b499b277a1f4e284cb4bc4cc327539adfc24bc6fd212577af5665395886660c9777484448 745868e8e5c5159d34c929706941e941f9de2fa6a18817f4ffffffff023000c901000000001976a 9145d9536d605d7ddf4f51f57006d1dddc38bb3f79c88ac80969800000000001976a914211f0c80 9a1a14f46af53ae59aa32d02aaf7272488ac00000000

PrevTx - Human Readable
-----------------------
    Hash: e52fa8a693eaff60f644b5006798215e0798e54a2dfc82aaec045c99cf70e4fb
    Data format version: 1
      Input 0 - 1LL8GeU5AxAhG7NuopgSrfeKCnzz46AaGM
          Previous out: 93cb48f575d0b2ea56df1ecb94acdbe6d6d82073c370a1911f4ec068e17f87e1#0
          scriptSig: 493046022100b0ac6689455d95fb81f0012f38b9285d44ae75f64b4c82ea9d1e96c25413927c022 100bcf31e15dde4d83b567f848cf6b4a708a23f0a71a206d858bfaea0285fca350f014104c6420d 1b499b277a1f4e284cb4bc4cc327539adfc24bc6fd212577af5665395886660c977748444874586 8e8e5c5159d34c929706941e941f9de2fa6a18817f4
          sequence: 0xffffffff
      Output 0 - 19XpbRe7XRT2c9FKGP6jTwcbjVwyyGBKiS
          Value: 0.29950000
          scriptPubKey: 76a9145d9536d605d7ddf4f51f57006d1dddc38bb3f79c88ac
      Output 1 - 1428VeCoiJR81vVjdtXe9sb5G15qjYyLwo
          Value: 0.10000000
          scriptPubKey: 76a914211f0c809a1a14f46af53ae59aa32d02aaf7272488ac


NewTx - Raw Data
----------------
0100000001fbe470cf995c04ecaa82fc2d4ae598075e21986700b544f660ffea93a6a82fe501000 0008b48304502205ea291ce55ecc95f346f6be2c198993dcb1a72cc4eddf520f173ed9ac85a1e03 022100d0ae6c394d014de8fecb44d034904a0c6142e6335a394aa4629d7a839aaaa2a30141043ef 1593aa79bab3c6a21f4f82f348b12e68d107f95f577e610466aca7d0f2e4ebcfd9a9bafcad72076 18ec4e5e78358dbac3dff4a432b833bf8a9cee85834ac0ffffffff0180969800000000001976a91 48073e789954e05c5938c5cc493308f9021539bb588ac00000000

NewTx - Human Readable
----------------------
    Hash: 460fcfa566eaf7906cf7768f22d624c4f2e8dc1ba00474b497ad7bbacd696f14
    Data format version: 1
      Input 0 - 1428VeCoiJR81vVjdtXe9sb5G15qjYyLwo
          Previous out: e52fa8a693eaff60f644b5006798215e0798e54a2dfc82aaec045c99cf70e4fb#1
          scriptSig: 48304502205ea291ce55ecc95f346f6be2c198993dcb1a72cc4eddf520f173ed9ac85a1e0302210 0d0ae6c394d014de8fecb44d034904a0c6142e6335a394aa4629d7a839aaaa2a30141043ef1593a a79bab3c6a21f4f82f348b12e68d107f95f577e610466aca7d0f2e4ebcfd9a9bafcad7207618ec4 e5e78358dbac3dff4a432b833bf8a9cee85834ac0
          sequence: 0xffffffff
      Output 0 - 1CiCLjhX1291hwjxZxBNCq1k9Ptkb4YNgR
          Value: 0.10000000
          scriptPubKey: 76a9148073e789954e05c5938c5cc493308f9021539bb588ac

Can you show me exactly what needs to be hashed and signed to generate the signature?

-TT
TT
member
Activity: 77
Merit: 10
November 13, 2011, 12:39:06 AM
#21
As for ECDSA key generation, the following OpenSSL commands work, which I got from that bitcoin-off-the-grid link casascius gave:

#To generate the key and save it to the file ecKey.pem:
openssl ecparam -genkey -name secp256k1 -out ecKey.pem

#To pull out a 32-byte private key as hex:
openssl ec -text -noout -in ecKey.pem | head -5 | tail -3 | fmt -120 | sed 's/[: ]//g'

#To get ripemd160(sha256(public key)), which is what bitcoin uses for addresses, as hex:
openssl ec -in ecKey.pem -pubout -outform DER | tail -c 65 | openssl dgst -sha256 -binary | openssl dgst -rmd160 -binary | xxd -p -c 80

There's surely a more efficient method for performing these steps, but they seem to do the trick for now. Perhaps
later I'll document usage of the OpenSSL API from within a single process, to avoid the overhead of starting additional processes.

Once you have the 32-private key and the 20-byte hash of the public key, apply the base58Check steps to them.

-TT
TT
member
Activity: 77
Merit: 10
November 13, 2011, 12:24:19 AM
#20
Better documentation is pending further research. Unfortunately, this stuff doesn't pay my bills...yet.

-TT
TT
member
Activity: 77
Merit: 10
November 13, 2011, 12:20:14 AM
#19
I think the wiki base58Check article https://en.bitcoin.it/wiki/Base58Check_encoding covered it pretty well...but I'll sum up:

1) Add the version byte as the most significant byte to the data
2) Compute checksum = sha256(sha256(data))
3) Take the first four bytes of checksum, append them to the end of data
4) Convert to base58
5) Pad with necessary leading zeros (represented with 1's in base58)

And yes, etotheipi, you did cover a lot of the details. It's nice to see the steps summed up succinctly, though Smiley
Endianness is as you say...little endian for this part of the process.

-TT

php code that does the above:
Quote
function hexToBase58Check($version, $payload, $length, $padding) {
   // prepend version
   $data = $version . $payload;
   
   // compute checksum
   $checksum = hash("sha256", hex2str($data));
   $checksum = hash("sha256", hex2str($checksum));
   $checksum = substr( $checksum, 0, 8 );
   
   // append checksum
   $data .= $checksum;
   return str_pad(bcdec58(bchexdec($data)), $length, $padding, STR_PAD_LEFT);
}
TT
member
Activity: 77
Merit: 10
November 05, 2011, 06:25:50 AM
#18
Patience  Cool
sr. member
Activity: 437
Merit: 415
1ninja
November 04, 2011, 09:31:30 PM
#17
so, where's the new doc? I struggled with this too initially Cheesy
TT
member
Activity: 77
Merit: 10
November 03, 2011, 02:36:02 AM
#16
I got it!

Thank you very much, you guys!

I appreciate the help.

-TT
TT
member
Activity: 77
Merit: 10
November 02, 2011, 10:32:48 PM
#15
Thanks, I think I got it. I'm still not sure why that bash script calls
Quote
openssl ec -text -noout -in data.pem | head -5 | tail -3 | fmt -120 | sed 's/[: ]//g'
multiple times...but it does seem to give me 32-byte or 33-byte output.

I guess it just ignores the 33-byte cases rather than removing the leading zero byte. But the output is in hex, so it doesn't really matter since leading zeros are ignored when I convert it into big-endian integers.

-TT
vip
Activity: 1386
Merit: 1140
The Casascius 1oz 10BTC Silver Round (w/ Gold B)
November 02, 2011, 09:59:32 AM
#14
Definitely big endian.

Must be unsigned. (OpenSSL may add a 0x00 byte to represent a positive sign for a total of 33 bytes and you must remove this)

Must be 32 bytes (OpenSSL may give you 31 bytes if the first byte was 0x00... Or 30 bytes if 0x00 twice... Etc.  You must readd missing zero bytes.
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 02, 2011, 08:57:30 AM
#13
My only missing pieces are getting the correct encoding for the keys prior to that whole Base58Check thing. I've got both the pem stuff and the Base58Check thing down.

-TT

If I understand correctly, doesn't the first half of my post cover that?  Convert private key to 32-byte big-endian binary string.  Prefix a 0x80 byte to it.  Then double-sha256 and add the first 4 bytes of the result to the 0x80+PrivKey string.  

I am not familiar with these import formats, but my experience with address strings and the description you provided makes me pretty confident that's what you're looking for (if it's not right try encoding the private-key in little-endian before hashing).

TT
member
Activity: 77
Merit: 10
November 02, 2011, 08:52:30 AM
#12
My only missing pieces are getting the correct encoding for the keys prior to that whole Base58Check thing. I've got both the pem stuff and the Base58Check thing down.

-TT
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 02, 2011, 07:12:53 AM
#11
What is the goal here?  Are we creating new keys?  Trying to insert them into your wallet?  Extract existing ones from your wallet?  I can't help you with the openssl library, or PEM formats or anything.  But I can offer you some code and diagrams that will help you figure out what's going on.  The middle link of my signature has a diagram of the address-conversion process (halfway down the page), starting from the two 32-byte public-key integers and ending up at the final BTC address.  It sounds like to me, that the private key format is very similar except you're skipping the top half of that diagram and using a full 32-byte repr of the private key, instead of a hash.

So a regular address looks like (where netbyte==0x00 for main network)
    
Code:
[NetByte  |  20-byte Hash  |  4-byte-hash-of-first-21-bytes]  ==> 25 bytes

Instead, the private key format would look like:
    
Code:
[0x80  |  32-byte private key (BE)  |  4-byte-hash-of-first-33-bytes] ==> 37 bytes

If it's anything like the address conversion, the private key will probably have to be big-endian before the hash is applied.  My signature has a a link to PyBtcEngine which has a ton of great python tools for playing with this stuff (you can actually ignore all the C++ code, you only need pybtcengine.py and nothing else).  Then you can do something like the following:

Python/PyBtcEngine Example code on gist

And here's the output of the above code:
Code:
Creating new address:
  BTC Address:      13KHLsKV1wVSFGAR7ohMWmvdEdcJsVdK52 (BinaryLE=19656185d9ba7af25bb8a3f3182833a39b79ae45)
  Have Public Key:  True
  Have Private Key: True
  Public Key Hex (Big-Endian):  
     04 869499bad9e3b4bf1c94dc772faf8a37d1182581697947b902e7d246dd7e7517
        9f6ffe719917b111f87253dda6c4beb31808bcfe45d1a6bd8272fd65f102258d
  Private key (integer): 89186423366759378937952085459266693389487184404252697411501348684075646220810
  Private key (hex, BE): c52dba0d191411cc7142da6979e47c8faa65ac9708f553c8fb6ce0517e4e7a0a
  Encoded privkey: 80c52dba0d191411cc7142da6979e47c8faa65ac9708f553c8fb6ce0517e4e7a0a4095c005
  Base58 privkey:  5KK8F9VQF6KMxDMgM1juEDd11XZXhLxVbeMb8hTPtt1bpV58diL

Recovering private key from base58 encoding:
  Valid checksum!
  Recovered address information:
  BTC Address:      13KHLsKV1wVSFGAR7ohMWmvdEdcJsVdK52 (BinaryLE=19656185d9ba7af25bb8a3f3182833a39b79ae45)
  Have Public Key:  True
  Have Private Key: True
  Public Key Hex (Big-Endian):  
     04 869499bad9e3b4bf1c94dc772faf8a37d1182581697947b902e7d246dd7e7517
        9f6ffe719917b111f87253dda6c4beb31808bcfe45d1a6bd8272fd65f102258d

Obviously, then you can examine what the python code is doing (just remember my hash256() == sha256(sha256())).  Be aware, that the python random-number-generator is probably not the best PRNG to be using for real money, but it is perfectly sufficient for playing around.  In any software I produce, I'll probably be making calls to the C++ cryptopp libraries to get random numbers. (you could also generate your own random 32-bytes offline (or integer less than 2^256), and then plug it in at the appropriate line above.

P.S. in the python code, I called the method "binary_to_addrStr" and "addrStr_to_binary" because I was expecting to only use it for address strings, but it works fine for private keys, too.
TT
member
Activity: 77
Merit: 10
November 02, 2011, 04:40:46 AM
#10
So this, huh?
Quote
hexsize=$(openssl ec -text -noout -in data.pem | head -5 | tail -3 | fmt -120 | sed 's/[: ]//g' )

while [ ${#hexsize} -ne 64 ]
do
openssl  ecparam -genkey -name secp256k1 | tee data.pem &>/dev/null && hexsize=$(openssl ec -text -noout -in data.pem | head -5 | tail -3 | fmt -120 | sed 's/[: ]//g' )
done

openssl ec -text -noout -in data.pem | head -5 | tail -3 | fmt -120 | sed 's/[: ]//g'
TT
member
Activity: 77
Merit: 10
November 02, 2011, 04:35:50 AM
#9
Wouldn't it just be better to dissect bitcoind, in that case? We know that that implementation seems to work pretty well. I'm pretty sure I could figure it out if I spent enough time poking into the innards of the OpenSSL library and certain source files like key.h, wallet.h, wallet.cpp, and all that...but I was hoping this stuff was already documented somewhere so I don't have to do that.

-TT
vip
Activity: 1386
Merit: 1140
The Casascius 1oz 10BTC Silver Round (w/ Gold B)
November 02, 2011, 01:33:33 AM
#8
Someone else wrote a shell script a while back that successfully generates bitcoin addresses using openssl.  It was called "BOTG" or Bitcoins-Off-The-Grid.

It may have some imperfections (e.g. it throws out some keys and rolls the dice again if the keys don't fit certain convenient criteria, rather than trying to format them correctly) and is probably untested (any address generator should be rigorously tested to avoid possible risk of bad addresses and permanent LOSS of bitcoins) but it certainly serves as a good place to start.

https://bitcointalksearch.org/topic/bitcoin-off-the-grid-botg-secure-savings-script-v011-23081
TT
member
Activity: 77
Merit: 10
November 02, 2011, 12:51:03 AM
#7
Everything in https://en.bitcoin.it/wiki/Base58Check makes perfect sense except for:

Quote
"Base58Check encoding is also used for encoding private keys in the Wallet Import Format. This is formed exactly the same as a Bitcoin address, except that 0x80 is used for the version/application byte, and the payload is 32 bytes instead of 20 (a private key in Bitcoin is a single 32-byte unsigned big-endian integer)."

The output of RIPEMD160 will always be 20 bytes. Obviously, the private key format cannot be merely a hash. So what encoding for the Bitcoin private key should I use to generate the Base58Check? And what OpenSSL command can I use to get it?

Something like
Quote
openssl ecparam -genkey -name secp256k1 -out ecKey.pem
generates 160 characters of a base64 encoding, but I imagine this encoding also contains the curve parameters, initial point, etc...

So how do I get the 32-byte private key from this?

-TT
Pages:
Jump to: