AHHH!!!
So that's what steps 5-9 here are: https://en.bitcoin.it/wiki/Address
Thank you!
-TT
Topic: Wallet Import Format - page 3. (Read 6642 times)
Look on the Wiki for "Base58Check".
I understand the mathematics behind ECDSA for the most part. Same idea used in a lot of crypto: Find a representation for a cyclic group of known order for which it's relatively simple to compute powers but very difficult to compute the log. In the case of ECDSA, the representation is point multiplication on an elliptic curve.
What I still don't get well enough are the specific encodings. And most urgent for my current application is understanding the wallet import formats specifically.
-TT
What I still don't get well enough are the specific encodings. And most urgent for my current application is understanding the wallet import formats specifically.
-TT
Thank you! You're probably going to end up saving me a lot of time!
-TT
-TT
I have broken down the entire process from the bit level, to the application level, in an effort to write my own client. And it took me a while, but I eventually did figure out everything you seek to find, except for the mini private keys. And actually, Base58 private keys... I'm not sure how those are generated, but I do know how ECDSA works...
First of all, for BTC addresses, start with my post in the middle of this page: https://bitcointalksearch.org/topic/on-the-wire-byte-map-opchecksig-diagram-knowledge-donation-29416. The other two images on that page are useful for understanding BTC, but not as important. That image in the middle shows you exactly how to get from an ECDSA public key to a Base58 address. It's not trivial, but I've been able to recreate it from scratch on the first try with the aid of that diagram.
Second of all, a fellow forum goer "Lis" wrote this very useful, dependency-less ECDSA module in python: https://bitcointalksearch.org/topic/solved-python-secp256k1-23241. I ended up using this in my code and it works beautifully. While I understand ECDSA, it would be a pain to reimplement it. FYI: a private key is just a 32-byte integer and a public key is two 32-byte integers cryptographically related to the private key. There's lots of different ways to encode these things, but in the end they are just big numbers.
Additionally, in that first link, you can see the "OP_CHECKSIG" diagram. That explains how you get from an unsigned transaction you just created, to a binary string that you can sign with your ECDSA key. It is stupidly complicated... but that's why I made these diagrams!
One of the big trip-ups in everything BTC related is the endianness. Most things are little-endian, except for three things: mainly whatever is in TxIn/TxOut scripts, the DER-encoded public key integers before they are converted to B58 addresses, and the final hash that you sign right at the end of the OP_CHECKSIG.
I'll be happy to answer any questions, or perhaps you will benefit from looking at the pybtcengine.py script in my github repo (look at the link in my signature). That python file has everything-- it allows you to supply raw numbers for pub/priv keys and verify linked transactions directly from the blockchain. I've spent a lot of time figuring this stuff out the past couple months. I hope I can help someone avoid the pain I went through!
EDIT: oh and to answer one of your questions directly: you only ever apply hashes to binary representations of data. Never on the hex or base58 representations. It must always be in binary and you must make sure the endian-ness is right for whatever you are hashing (usually little-endian, but in the case of tx-signing, big-endian)
First of all, for BTC addresses, start with my post in the middle of this page: https://bitcointalksearch.org/topic/on-the-wire-byte-map-opchecksig-diagram-knowledge-donation-29416. The other two images on that page are useful for understanding BTC, but not as important. That image in the middle shows you exactly how to get from an ECDSA public key to a Base58 address. It's not trivial, but I've been able to recreate it from scratch on the first try with the aid of that diagram.
Second of all, a fellow forum goer "Lis" wrote this very useful, dependency-less ECDSA module in python: https://bitcointalksearch.org/topic/solved-python-secp256k1-23241. I ended up using this in my code and it works beautifully. While I understand ECDSA, it would be a pain to reimplement it. FYI: a private key is just a 32-byte integer and a public key is two 32-byte integers cryptographically related to the private key. There's lots of different ways to encode these things, but in the end they are just big numbers.
Additionally, in that first link, you can see the "OP_CHECKSIG" diagram. That explains how you get from an unsigned transaction you just created, to a binary string that you can sign with your ECDSA key. It is stupidly complicated... but that's why I made these diagrams!
One of the big trip-ups in everything BTC related is the endianness. Most things are little-endian, except for three things: mainly whatever is in TxIn/TxOut scripts, the DER-encoded public key integers before they are converted to B58 addresses, and the final hash that you sign right at the end of the OP_CHECKSIG.
I'll be happy to answer any questions, or perhaps you will benefit from looking at the pybtcengine.py script in my github repo (look at the link in my signature). That python file has everything-- it allows you to supply raw numbers for pub/priv keys and verify linked transactions directly from the blockchain. I've spent a lot of time figuring this stuff out the past couple months. I hope I can help someone avoid the pain I went through!
EDIT: oh and to answer one of your questions directly: you only ever apply hashes to binary representations of data. Never on the hex or base58 representations. It must always be in binary and you must make sure the endian-ness is right for whatever you are hashing (usually little-endian, but in the case of tx-signing, big-endian)
Hi,
I've been trying to find some documentation on bitcoin ECDSA key formats, the exact specifications,
the reason for why the particular hash steps were chosen, the algorithm to verify whether keys are valid,
how the checksums work, etc...and have only been able to find little tidbits here and there but no nice
comprehensive document.
This article is pretty helpful: https://en.bitcoin.it/wiki/Address
I've managed to implement those exact steps and it works, even though I can see many developers who
haven't had a lot of experience working with crypto libraries tripping over certain details
(i.e., are the hashes on strings of hex digits? if so, is it case-sensitive? or are the hashes on binary numerical
formats? what about dealing with base64 encoding? should the base64 encoded string be hashed? or its
numerical value? etc...).
Not that this stuff is particularly difficult to grasp, but a concise description with rigorous steps
for generating/validating key pairs seems desperately needed.
Even worse, though, when looking up the private key specification, all I can find on the bitcoin wikis is
https://en.bitcoin.it/wiki/Private_key#Base_58_Wallet_Import_format
and
https://en.bitcoin.it/wiki/Wallet_import_format
which say almost nothing...
And the documentation on the mini private key format is even more lacking...and quite frankly doesn't
make sense to me from a security standpoint...after all, the whole point of having a long private key
is that it makes it difficult for an attacker to guess by brute-force. Having mini private keys seems to
defeat the whole purpose of this...or am I just missing some brilliant insight?
Could someone please point me to an exact set of steps for generating wallet import format base 58
privkey? And also, I'd love to understand the validation algorithms/checksums/etc...
If nobody has yet put together a good document on all this, I volunteer to write one and to make it
available for the bitcoin community.
Thanks.
- TT
I've been trying to find some documentation on bitcoin ECDSA key formats, the exact specifications,
the reason for why the particular hash steps were chosen, the algorithm to verify whether keys are valid,
how the checksums work, etc...and have only been able to find little tidbits here and there but no nice
comprehensive document.
This article is pretty helpful: https://en.bitcoin.it/wiki/Address
I've managed to implement those exact steps and it works, even though I can see many developers who
haven't had a lot of experience working with crypto libraries tripping over certain details
(i.e., are the hashes on strings of hex digits? if so, is it case-sensitive? or are the hashes on binary numerical
formats? what about dealing with base64 encoding? should the base64 encoded string be hashed? or its
numerical value? etc...).
Not that this stuff is particularly difficult to grasp, but a concise description with rigorous steps
for generating/validating key pairs seems desperately needed.
Even worse, though, when looking up the private key specification, all I can find on the bitcoin wikis is
https://en.bitcoin.it/wiki/Private_key#Base_58_Wallet_Import_format
and
https://en.bitcoin.it/wiki/Wallet_import_format
which say almost nothing...
And the documentation on the mini private key format is even more lacking...and quite frankly doesn't
make sense to me from a security standpoint...after all, the whole point of having a long private key
is that it makes it difficult for an attacker to guess by brute-force. Having mini private keys seems to
defeat the whole purpose of this...or am I just missing some brilliant insight?
Could someone please point me to an exact set of steps for generating wallet import format base 58
privkey? And also, I'd love to understand the validation algorithms/checksums/etc...
If nobody has yet put together a good document on all this, I volunteer to write one and to make it
available for the bitcoin community.
Thanks.
- TT
Jump to: