Topic: How does wallet.dat work? (Read 3232 times)

Thanks for the clarification on how protocol handles both key forms.

When you said "no it isn't" that didn't refer to the key size did it?  My understanding is that compressed keys are 33 bytes. 32 bytes for y-value plus another byte to indicate which "side" of the curve the point is located on.

On edit:

More info on compressed keys (including wrong answer about compressed keys being incompatible with the protocol)
http://bitcoin.stackexchange.com/questions/3059/what-is-a-compressed-bitcoin-key

No it isn't.  We pass the pubkey as a binary blob to openssl, and openssl will validate the signature using either form.  The same private key can actually correspond to two different addresses, because the hash of the compressed pubkey is different from the hash of the uncompressed pubkey.

Typos typos typos.  Your right Danny.  Although the compressed key is more than just the y value (32 bytes).  It also includes a byte to indicate which side of the curve the point lies on (ECDSA curve is like a U so a single y value can represent two points).

The protocol itself isn't aware of compressed keys and to make it aware would require a hard fork.  All that potential bandwidth and storage savings wasted.  Even if a compressed key is used in the client it is converted to a full key when creating a transaction on the network. - Incorrect.

Don't "compressed key" addresses just use half of that pair?  The pair together (as indicated in your graphic) is 512 bits, but in the case of a compressed key address, only one of the 2 values is represented (and therefore only 256 bits), right?

You could also die tomorrow getting eaten by a shark while struck by lightning in a desert.  The odds of that are probably higher than a 160 bit collision.  You could also buy one ticket, win the powerball lottery jackpot, next day buy a ticket win that powerball lottery jackpot, next day buy another ticket win that powerball lottery jackpot, next day buy a ticket, win that jackpot, next day buy a ticket, buy that jackpot, next day buy a ticket, win that jackpot, next day buy a ticket, win that jackpot.  The odds of doing that is more probably than generating a 160bit collision.


Well not exactly. The address isn't the public key.  The address is a checksummed hash of the public key.

So in Bitcoin you have:
* private key - 256 bit number
* public key - 512 bit ECDSA (x,y pair) -essentially a number
* public address - 200 bits (8 bit version identifier) + (160 bit hash of public key) + (32 bit checksum - to prevent sending funds to invalid addresses)

Public addresses are simply 200 bit numbers.  We put them into Base58 format to make them human readable but they are just numbers to the network.  

All three of the following numbers are the same and represent a bitcoin address, they are just in different formats.

00010966776006953D5567439E5E39F86A0D273BEED61967F6 (in hexadecimal)

0110011101110110000000000110100101010100000000000000000000000000
0110101000001101001001110000000000000000000000000000000000000000 (in binary)

16UwLL9Risc3QfPqBUvKofHmBQ7wMtjvM (in Base58 format)

The Base58 is the most human readable (and compact) but all three formats have the same value.

Someone made this awesome graphic to show the details


More details:
https://en.bitcoin.it/wiki/Technical_background_of_Bitcoin_addresses

The public key is a 32 byte (256 bit) integer.  So that part is 2²⁵⁶.

The bitcoin address on the other hand is built off of a 20 byte (160 bit) hash of the public key.

This results in 2¹⁶⁰ possible addresses (1.46 x 10⁴⁸).

Then a 1 byte version number is tacked on to the front of the address, and a 4 byte checksum is appended to the end, resulting in a 25 byte number.

This 25 byte number is then encoded with Base58Check encoding using [1-9,A-H,J-N,P-Z,a-k,m-z] (note that 0, I, O, and l are not used).

That encoding typically results in 33 or 34 characters made up of 58 different symbols.

34⁵⁸ = 6.7 x 10⁸⁸.  You'll notice that is significantly larger than 2¹⁶⁰.  This is because 5 bytes (the version and the checksum) are dependent on the 160 byte hash.  There are many combinations of [0-9,A-Z,a-z] that are not valid bitcoin addresses, leaving only 1.46 x 10⁴⁸ that any well written wallet will recognize.

There are no real owners of coins. Having the only person/entity with the private key is the closest you can get to ownership with Bitcoin.

Having said that, if you think it is possible, you probably haven't grasped the improbability yet. This comes up pretty often, so you can search the forums to find very nice and funny analogies to explain the situation.

Basically, if 1 billion people began generating addresses at the rate of 1 million addresses per second, in about a century they would generate a total of 2⁸¹ addresses. 2¹⁰⁷ addresses until the death of our sun. Of course these aren't unique addresses, so you will never have all addresses at this speed. It might feel as if you could be unlucky enough to have your private key discovered in your lifetime, but you really aren't. The probability of you getting kidnapped and beaten until you reveal your password is much much higher.


This is a good measure, not only because your private key might accidentally get generated (it won't), but as a general precaution. If you store money in various private keys and divide them with separate access restrictions, it will make it harder for attackers to access all your money, even through physical torture. A more sci-fi scenario is; if an address never initiated a transfer, its public key is not revealed, which makes it practically immune to quantum computing attacks that we know can be possible in the far future; so the true paranoid might want to store their savings in virgin addresses.


A public key is 256 bits. An address is a 160-bit hash of the public key, plus version and checksum. What you are describing seems to be the encoding, which can be 27-34 characters long.

I understand the probability, but say you have all your savings on an address, and, god forbid, someone else "mints" an address that has the same public key, isn't that really HORRIBLE?! How would/could the system then help the real owner of these coins?

So I guess the only real hedge is to spread coins over many addresses?

I mean putting enough energy behind trying to generate an address with alot of coins on it would make sense if bitcoin value rises higher!

the public key is [0-9a-zA-Z] so its not really 2^160 right? It's 32 characters with 10+28+28 mutex no?

Yes, it stores the transaction passively. The client is able to scan the blockchain to re-discover these transactions (the -rescan command-line option forces this), but storing them in the wallet is more convenient.


Of course. The private key can even be on paper or even in your head.

Ok, one last thing: if the wallet stores "only" the private keys, why does the wallet.dat filesize change upon every transaction to an address in it? It stores some reference to transactions there? Because you can make payments to an address in a wallet that isn't online without problems right?

Yes the odds are 1 in 2^256 however what really matters is that the odds of two public keys producing the same address (which is a 160bit hash of the public key with other meta data like version and checksum) are 1 in 2^160.  Note even the 1 in 2^160 is a very small number and for all practical purposes it can be considered 0%.

It's a lot bigger than a gazillion.  It is so unlikely that you can really consider it impossible.  A 160 bit number is a lot bigger than most people can wrap their minds around.  But yes, the only thing in bitcoin that prevents two attempts at generating an address from generating the same address is the extreme unlikelihood of it happening.

Ok, thanks! I think I get everything about bitcoin now!

But so there must be a chance in one gazillion that two private keys generate the same public key hash no?

First, you pray.  Then you pull 256 bits at random from the entropy source that you just prayed to.  These 256 bits are your private key.  You check that the private key is not in excess of the modulus of the field, which would make it insecure, if it is, you start over.  Otherwise, the private key gets encrypted (you have encryption enabled, right?) and stored in your wallet.  Then, an operation called EC point multiplication is done using that private key and a well known constant.  The result of that multiplication is a pair of 256 bit numbers (x and y), the pair together known as the public key.*  The public key is put into a special representational form and stored in the wallet.  Finally, the pubkey representation is hashed, and then hashed again.  The second hash is then encoded with a checksum (more hashes!), and the result is an address.

* side note, the y-coordinate can be reconstructed, so it isn't needed, and can be discarded, but we have to store 2 different flags.  This changes the representation, and thus the hash, meaning that a private key can have two different addresses.  One flag is a marker along with the private key, telling the software which of the two possible addresses to use, and the other flag is a parity indication, coded into the public key representation to resolve the ambiguity of the quadratic used to reconstruct y.

Yes.


Actually, the private key is created first. It's basically just a 256-bit random number. Then the ECDSA public key corresponding to that private key is generated. This public key can be used to verify a signature. This way, you can prove that you have the private key without having to reveal it.

The public key is then hashed a couple of times using different algorithms and decorated with a version number and checksum to produce the address, which makes it easy to distinguish Bitcoin addresses (e.g. from test network addresses) and very hard to create a valid one by mistake (e.g. typo).

so basically, when you create the address there is a private key created that is stored in the wallet? how are addresses created?

Anyone can claim to own the address. 

But if you really have the private key for it, only you can spend coins sent to it, and only you can sign messages using that key.  Those are really the same thing.  A bitcoin transaction is really just a special message to be signed.

Yes.  The important thing is knowing the private key.  The way the client works internally isn't so important, but basically those coins are yours as soon as the transactions is confirmed.  When your client gets back online and catches up on the block chain, then it will update the balance that it shows you.

But how does then the addresses work:

1) I have 1Eu6P1eRSewoqf8GZcYoBtMtXG1865Umjh, it has never received any money. How do I prove ownership of this address? Why can't somebody just claim to own this address?

2) If I receive money on this address without having bitcoin client running, will that money just magically appear when I launch the bitcoin client in the future (IE is it the ownership of the address that is stored in the wallet or the transactions/coins?)

That one tripped me up when I was learning how this worked.  I couldn't figure out why a peer couldn't change the address in the output before relaying the transaction.  I figured the input was signed (or rather a hash of the input), so any peer could do whatever they want with the output.  Then it was explained to me (as you have just done here) that the entire transaction is signed (or rather a hash of the entire transaction).  Changing an output would change the signature, and without the private key a peer can't generate a valid signature.  Light bulb goes on and it all makes sense.

I offer this in hopes that it helps someone find a way to think about bitcoin's inner workings in a way that is useful to them.

There are no addresses. 

There are no balances.

Bitcoin is a chain/mesh of transactions.  Those transactions go by many different names, sometimes "inputs" or "outputs", sometimes "vins" or "vouts", sometimes "coins".  I like to think of them as "coins", but it requires special metaphors.

Your wallet is a collection of "coins", each of which has a specific value.  To complicate things, sometimes we think of the unit that the value is expressed in as "coins" too, but try not to do that for a little while.  When you want to spend, you select a bunch of your "coins" such that their value is equal or greater than the amount you want to spend.  You then prove that you "own" those coins by signing them* , and you bundle them together and specify one or more outputs, each with a specific value.

Think of it like melting a bag of "coins" of different values and minting them into a new bag of "coins", with the network checking to make sure that you don't try to make the value of the output bag more than the value of the input bag.

Of course, these aren't real "coins", there isn't a thing being acted upon.  These are abstractions built upon abstractions.

* Technical note, typically the signature is calculated based on a subset of the entire event (the transaction), rather than just signing the input by itself.  This is because you usually care more about where the spend is going than where it came from.