Author

Topic: Recover wallet passphrase - special circumstance? (Read 1067 times)

hero member
Activity: 672
Merit: 504
a.k.a. gurnec on GitHub
Guess I have a lot of work ahead of me. Grr. Thanks for the helpful answers guys. I was really hoping to be able to get the LTC back and not have to do as much manual work.

The LTC might be a lost cause, assuming you don't remember anything about the password, but why is there a lot of manual work for your BTC? It should just be a single dumpwallet, rename the wallet.dat file, and then a single importwallet, correct?
newbie
Activity: 8
Merit: 0
Aah ok. I was under the impression that there was always a master key, and it was unencrypted until you added a passphrase.

unencrypted wallet.dat just stores private keys (amongst other things).

Encrypted wallet.dat has private keys which are encrypted with a master key (that was not there before it was encrypted), which itself has been encrypted with the derived_key thats made from the password.

Very elegant. You can store the decrypted master key in memory, and while you may be able to extract that, it wont get your the password nor be useful on any other wallet. That makes the nerd in me happy (while the financial advisor inside is still upset Wink

Thanks again, amazing explanation.
donator
Activity: 1218
Merit: 1079
Gerald Davis
You said that before the wallet is encrypted, the master key is just a random 32 bit number.

If the wallet has no password it also has no master_key.  The master_key is not randomly generated until the user initially encrypts the wallet.  The decrypted master_key is never stored on disk.

Quote
After you encrypt it with a password, the password decrypts it, then is wiped.

This sentence is unclear.

To initially encrypt the master_key, the client asks the user for a new password.  The password_derived_key is created from the passphrase.  The master_key is encrypted with the password_derived_key.  The password & password_derived_key is removed from memory.

To "unlock" the wallet the client prompts the user for the password.  The password_derived_key is created from the passphrase.  The encrypted_master_key is decrypted with the password_derived_key.  The master_key is stored in memory.  The password & password_derived_key is removed from memory.

The "lock" the wallet, the client deletes the master_key from memory.

Quote
I'm assuming that it's the same as the unencrypted private key... finding the original unencrypted master key wouldn't do any good, you'd still need to brute force it, correct? Which is no easier than just brute forcing the wallet.

I am not sure I understand the question.  If you find the (decrypted) master_key you can use it to decrypt the individual private keys.  No brute force is required, you have the decryption key.

The reason for not encrypting the individual_private_key with the password_derived_key is if the user changes the password then you would need to decrypt and re-encrypt every single private_key.  By using a random "master_key" if the user changes the password you just need to decrypt and re-encrypt the master_key.  

Quote
Just curious at this point. What got us into bitcoin originally (ok, me really) is the cryptography used. It's fascinating to me. You can't copy BTC or fake a transaction (well, easily, 51% and all that) or cheat the system. It's quite beautiful.

Cryptography done right is pretty amazing.  With Bitcoin we are only at the beginning.
newbie
Activity: 8
Merit: 0
I'd imagine it would have to be... otherwise you'd need to enter the password to re-lock it.

Neither the password nor the password_derived_key is stored in memory.   It is only used temporarily to decrypt the master_key.  The encrypted version of the master_key is stored in the wallet and it is just a random number.  When the wallet is "unlocked" it means the master_key (not password or password_derived_key) is in memory.  The client uses the master_key to decrypt individual private keys but only as needed.  So most of the time even when "unlocked" everything in the wallet is encrypted, it just means the master_key is in memory so individual elements can be decrypted.  Even if you recover the master_key it won't be the same master_key on any other wallet even if that wallet has the same passphrase. When a client decrypts an individual private key it deletes the decrypted version when it done using it (i.e. after signing a transaction).  When the wallet application closes or the unlock times out then the master_key is deleted from memory as well. 

PRNG -> random 256 bit number -> master_key
Wallet Password -> Key Derivation Function -> password_derived_key

master_key -> AES_Encrypt(w/ password_derived_key) -> encrypted_master_key
encrypted_master_key -> AES_Decrypt(w/ password_derived_key) -> master_key

For each of the bitcoin private keys in the wallet
private_key -> AES_Encrypt(w/ master_key) -> encrypted_private_key
encrypted_private_key-> AES_Decrypt(w/ master_key) -> private_keys

Stored in wallet.dat:
encrypted_private_key(s)
encrypted_master_key

In memory while wallet is unlocked (deleted when wallets locks or application terminates):
master_key

In memory only temporarily (deleted as soon as the task is completed):
password
password_derived_key
individual private_keys

This is a pretty common arrangement.  It allows one to change the password without changing the master_key.  If the master_key was changed it would require decrypted and re-encrypting every single private key.

Quote
I got some help on reddit and have been doing some testing and it's not looking good.
If someone on reddit told you that you can recovery from memory something which isn't there then it wasn't "help".

Didnt notice your reply before I quoted the one above yours (it was all the answer I needed to see to realize that resistance is futile!)

But after reading it, I now have a technical question.

You said that before the wallet is encrypted, the master key is just a random 32 bit number. After you encrypt it with a password, the password decrypts it, then is wiped.

I'm assuming that it's the same as the unencrypted private key... finding the original unencrypted master key wouldn't do any good, you'd still need to brute force it, correct? Which is no easier than just brute forcing the wallet.

Just curious at this point. What got us into bitcoin originaly (ok, me really) is the cryptography used. It's fascinating to me. You can't copy BTC or fake a transaction (well, easily, 51% and all that) or cheat the system. It's quite beautiful.
newbie
Activity: 8
Merit: 0
Firstly, you're hopefully aware if your wallet is unlocked, you can export all the keys at once (unencrypted) with dumpwallet (which can then be imported all at once into a new wallet with importwallet)?

Regarding your password, sorry to be the bearer of bad news, but it is not stored anywhere in memory and cannot be retrieved. When the wallet is unlocked, the password is used to generate a decryption key which is then used to decrypt your Master Key, and then the password is removed from memory (it is securely overwritten). The decrypted Master Key (which is simply 32 random bytes) alone is stored in memory while the wallet is unlocked, and it is used to decrypt individual private keys on demand. When the wallet becomes locked, the decrypted Master Key is securely overwritten.

So to get your LTC back, I imagine your only options would be recovering your password, restoring your LTC wallet from a backup should you have one, or brute-forcing your LTC wallet if the password is short enough and/or you remember some portion of it...

Poop. Thanks for the insight. Makes sense they'd do it that way. And I posted this on reddit as well and found out that knowing the decrypted private key wont help either.

Guess I have a lot of work ahead of me. Grr. Thanks for the helpful answers guys. I was really hoping to be able to get the LTC back and not have to do as much manual work.
donator
Activity: 1218
Merit: 1079
Gerald Davis
I'd imagine it would have to be... otherwise you'd need to enter the password to re-lock it.

Neither the password nor the password_derived_key is stored in memory.   It is only used temporarily to decrypt the master_key.  The encrypted version of the master_key is stored in the wallet and it is just a random number.  When the wallet is "unlocked" it means the master_key (not password or password_derived_key) is in memory.  The client uses the master_key to decrypt individual private keys but only as needed.  So most of the time even when "unlocked" everything in the wallet is encrypted, it just means the master_key is in memory so individual elements can be decrypted.  Even if you recover the master_key it won't be the same master_key on any other wallet even if that wallet has the same passphrase. When a client decrypts an individual private key it deletes the decrypted version when it done using it (i.e. after signing a transaction).  When the wallet application closes or the unlock times out then the master_key is deleted from memory as well.  This is a pretty common arrangement.  It allows one to change the password without changing the master_key.  If the master_key was changed when the password changes, it would require decrypted and re-encrypting every single private key.

Pseudocode of how it all fits together:
PRNG -> random 256 bit number -> master_key **
Wallet Password -> Key Derivation Function (involves millions of SHA256 operations) -> password_derived_key

master_key ** -> AES_Encrypt(w/ password_derived_key) -> encrypted_master_key
encrypted_master_key -> AES_Decrypt(w/ password_derived_key) -> master_key

individual_private_key -> AES_Encrypt(w/ master_key **) -> individual_encrypted_private_key*
individual_encrypted_private_key*-> AES_Decrypt(w/ master_key **) -> individual_private_key

* stored in the wallet.dat
** stored in memory while wallet is unlocked (deleted when wallet locks or the application terminates)
The password, password_derived_key, and decrypted individual_private_keys are only kept in memory temporarily and deleted as soon as they are no longer needed.

Quote
I got some help on reddit and have been doing some testing and it's not looking good.
If someone on reddit told you that you can recovery from memory something which isn't there then it wasn't "help".
hero member
Activity: 672
Merit: 504
a.k.a. gurnec on GitHub
Firstly, you're hopefully aware if your wallet is unlocked, you can export all the keys at once (unencrypted) with dumpwallet (which can then be imported all at once into a new wallet with importwallet)?

Regarding your password, sorry to be the bearer of bad news, but it is not stored anywhere in memory and cannot be retrieved. When the wallet is unlocked, the password is used to generate a decryption key which is then used to decrypt your Master Key, and then the password is removed from memory (it is securely overwritten). The decrypted Master Key (which is simply 32 random bytes) alone is stored in memory while the wallet is unlocked, and it is used to decrypt individual private keys on demand. When the wallet becomes locked, the decrypted Master Key is securely overwritten.

So to get your LTC back, I imagine your only options would be recovering your password, restoring your LTC wallet from a backup should you have one, or brute-forcing your LTC wallet if the password is short enough and/or you remember some portion of it...
legendary
Activity: 3472
Merit: 4801
Let me be clear then: I HAVE ALREADY exported the keys. No kidding. Thats the only backup I have.

Not sure why you ask a very specific question, and you get tons of people telling you to do something else.

I know. Export the keys. Save the bitcoin in them. I know that.

Lets take a look at why people thought you were not getting started on the most important and simple thing you could do to at least preserve access to your bitcoins, shall we?

The biggest reason I'm asking here instead of just getting right in on exporting the keys is that


listaddressgroupings rpc command will export address list, and you can dump the private key one-by-one with dumpprivkey command.

Right. Understood. But I also very clearly stated in my OP that I'm aware of that, but thats a LOT more work . . .

I'm fully aware I can export all the info. Exporting it all, . . . is a lot more work than recovering the password if possible.

Because of this, you invited many people who didn't know how to help you with your "very specific question" to at least offer their advice that you should stop putting off this simple step.

cp1
hero member
Activity: 616
Merit: 500
Stop using branwallets
Then you should have stated instead that you already exported the keys, since in your post you wrote that it was too big of a hassle to do that.
newbie
Activity: 8
Merit: 0
You could probably do elbandi's solution just using a text file if you don't want do code it (and it probably would be faster):
1. dump the address list/dump the wallet
2. Copy the results and open in a text file.
3. Insert dumpprivkey in front of each line using your text editors commands to do it for every line.  In Unix it is pretty easy and in many other editors too.
4. Copy and paste results into the console, and I would think it would execute each line - I haven't tried and am not in front of it to try it.
5. Copy the results of the dumpprivkey list.
6. Edit as needed to reimport into a new wallet.

Then you would not have to do each one individually.  If pasting into console doesn't work, you could always try sending each via RPC automatically with a script.

I do not believe that the password is stored in memory after the wallet is unlocked.  I thought I read this during the heartbleed bug discussion, but have not verified it.


I'd imagine it would have to be... otherwise you'd need to enter the password to re-lock it.

I got some help on reddit and have been doing some testing and it's not looking good.

But I really want to thank everyone who suggested that I do the things that I already know are options, and those who are insisting that I export those keys RIGHT NOW, as if I havent yet done it.

Let me be clear then: I HAVE ALREADY exported the keys. No kidding. Thats the only backup I have.

I was asking to see if there was an EASIER WAY than manually rebuilding the wallet, AND IN ADDITION, being able to recover the 81 LTC. Not sure why you ask a very specific question, and you get tons of people telling you to do something else.

I know. Export the keys. Save the bitcoin in them. I know that.

My QUESTION was about recovering the passphrase, to both make this easier, and save the LTC.
staff
Activity: 4284
Merit: 8808
You want the dumpwallet rpc.

Your computer doesn't know the passphrase.

Are you sure it was the dog and instead you didn't happen to hack some poor person's system  and find their wallet encrypted but unlocked and forget that you have a moral obligation to report the issue and not steal from them? Just checking… Smiley
cp1
hero member
Activity: 616
Merit: 500
Stop using branwallets
You're really asking for trouble by not exporting the private keys right now.
legendary
Activity: 4256
Merit: 1313
You could probably do elbandi's solution just using a text file if you don't want do code it (and it probably would be faster):
1. dump the address list/dump the wallet
2. Copy the results and open in a text file.
3. Insert dumpprivkey in front of each line using your text editors commands to do it for every line.  In Unix it is pretty easy and in many other editors too.
4. Copy and paste results into the console, and I would think it would execute each line - I haven't tried and am not in front of it to try it.
5. Copy the results of the dumpprivkey list.
6. Edit as needed to reimport into a new wallet.

Then you would not have to do each one individually.  If pasting into console doesn't work, you could always try sending each via RPC automatically with a script.

I do not believe that the password is stored in memory after the wallet is unlocked.  I thought I read this during the heartbleed bug discussion, but have not verified it.


You want to do it while you still can.  ;-)


newbie
Activity: 8
Merit: 0
listaddressgroupings rpc command will export address list, and you can dump the private key one-by-one with dumpprivkey command.

Right. Understood. But I also very clearly stated in my OP that I'm aware of that, but thats a LOT more work than it would be IF the password were recoverable. In addition, 81 LTC sitting in a locked wallet that uses the same passphrase that I'd prefer not to lose.

I'm fully aware I can export all the info. Exporting it all, doing it by hand, manually reimporting all the addresses is a lot more work than recovering the password if possible. And again, 81 LTC in another wallet.

I understand most lost passwords are a lost cause without months of brute forcing. This is a special case in that I have the unlocked wallet with the password in memory, and I also have a before and after private key to compare.

Thats what I'm tring to see if possible.
legendary
Activity: 4256
Merit: 1313
elbandi had a good solution, just do it right away - I would recommend starting with the largest balances first too in order to save as much as possible so that if you have a power failure, crash, passphrase timeout etc like you mentioned, you'll lose less.

And finally, as my cousin always asks her husband: "Now what have you learned from this?"     Wink

hero member
Activity: 525
Merit: 531
listaddressgroupings rpc command will export address list, and you can dump the private key one-by-one with dumpprivkey command.
newbie
Activity: 8
Merit: 0
I have an odd situation and am looking for some guidance that may save me an absolutely mindblowing amount of work.

I have a bitcoin wallet which has stored a ton of addresses. Many customers and clients pay to their own address, and there is an automated system that wathces those payments and acts on them.

The flash drive that contained the passphrase has been, lets just say it's been irrevocably damaged. (which may or may not have involved our asshole dog). I have it being looked at by a friend who knows about data recovery and electronics, so maybe irrevocable isnt the best term, but, my hopes are not high.

So here's the special circumstance. We lost the passphrase. However, the wallet right now IS OPEN AND UNLOCKED.

Question one: Before we have a power issue or my machine decides to reboot, is it possible to extract the passphrase from memory? It has to be in there somewhere to keep it open. Anyone know how and where to look and with what tool? I used to have a tool back in my video gaming days that would scan the memory of your PC and look for data (42,528gp) and then you change that data (spend some gp) and have it search for the new value. Once it found it, it could edit the data in memory and suddenly you have 999,999gp.

Is anything like that possible with bitcoin?

Second question, not sure if this is harder or easier: We also have a backup of the original wallet BEFORE we added the encryption key. Only a few addresses are in it - so we cant use that (we'd lose over 200 customer private keys). But given we have a wallet with a UNENCRYPTED private key, and the same exact key that IS encrypted, could there be a tool that knowing the before and after, could more easily brute force/figure out the passphrase?

Now before the replies with other alternatives come in, we've already considered them. We cannot just send the balance out to a new address - it's the loss of the current private keys. Issuing each customer a new address and updating all records and the database will be a massive pain in the ass.

Someone else suggested a better option: Export all the private keys, and then re-import them into a new wallet. A better option than above, as we retain all addresses, but it's still a huge amount of work to do by hand (or to hire someone to write the scripts to do it).

That is the option we will go with if we have no other choice. The biggest reason I'm asking here instead of just getting right in on exporting the keys is that we used that passphrase on our LTC wallet also, which is NOT open. The balance in there is not huge - about 81 LTC or so, which is just about 1.2 BTC or so - we could live with losing that, but obviously would rather not, if we could possibly recover the current key.

Is something like this feasiable? Any insight or help would be appreceiated. Thanks in advance.
Jump to: