I was not around when BIP 39 was first specified, so I cannot say why it was not implemented then, but I can speculate based on what I know. However I can say why it is still not implemented today.
Firstly, Bitcoin Core is an open source software where the developers work on things that they care about. There is no roadmap nor is there a project manager saying "go implement that". So if no developer wants to implement a feature, then the feature does not become implemented. So clearly, developers do not want to work on BIP 39. So we need to see why multiple independent developers do not want to implement BIP 39.
When BIP 39 first came out, Bitcoin Core had not yet implemented BIP 32, which is a prerequisite for using BIP 39. BIP 32 was first specified in 2012, but the Bitcoin Core wallet did not receive support for it until 4 years later in 2016. In that time, there had been multiple attempts to implement BIP 32, but those were much more complicated. What ended up being merged was the absolute bare minimum that could be considered BIP 32 support. The wallet was able to generate private keys deterministically, and that was about it. It did not allow users to set the BIP 32 seed, users could not import xprvs or xpubs, and users could not change the derivation path used. It ended up being this bare minimum and taking so long just for the bare minimum simply due to how the wallet is structured. It would take a lot of work to overhaul the wallet structure to actually support BIP 32 properly, so what we got was the absolute bare minimum, and that's what we have even till today, for the most part.
Only after BIP 32 was implemented could we start looking at BIP 39. The technical challenge post BIP 32 is not the big, although BIP 39 would still require significant changes to support. However by this time (in 2016), BIP 39 was already showing its weaknesses. Electrum used BIP 39 for some time, but they moved to their own mnemonic system (which is not really specified). One of the major issues of BIP 39 is that it is touted as a full wallet backup but fails to provide several key pieces of information. BIP 39 fails to provide the derivation paths used, and it fails to provide the type of scripts and addresses to use. The latter point was less evident prior to Segwit, but when Segwit was released in 2016, people began to realize the importance of specifying address types in backups.
People often point out the lack of version numbers in BIP 39, and usually this is referenced as a solution to the aforementioned derivation paths and address types. Personally, I don't think that versioning would help with that, but versioning would certainly help if there were improvements to BIP 39 such as using a different KDF.
Another frequently mentioned issue is the choice of KDF. The authors of BIP 39 chose to use PBKDF2, however this has long been considered to be a fairly weak KDF. I have heard some developers (of other wallets) indicate they don't want to use BIP 39 because of this, but I don't have a strong opinion on that.
There is also the fact that BIP 39 itself says that software should still accept mnemonics with invalid checksums, which is a really weird design decision to make. The checksum could definitely be better and the BIP should not be recommending that invalid checksums be basically ignored.
Lastly, the other main reason that Electrum moved away from BIP 39 was due to the requirement of fixed wordlists. In order for BIP 39 to generate the BIP 32 seed to use, it uses PBKDF2 on the mnemonic itself. This means that the mnemonic isn't really an encoding of the raw entropy used to generate it and that the same entropy used to create mnemonics with different wordlists (e.g. for other languages) would result in different wallets. This significantly reduces the portability and usability of BIP 39.
So with all of these shortcomings in mind, BIP 39 is not really the ideal single string wallet backup solution that people want, and so many developers feel like it is not worth it to invest the time to implement something that has been shown to be insufficient.
Now the question is what will we implement in lieu of BIP 39. This is something that I have been actively working on for the past several years. Pieter Wuille came up with the idea of strings of text called Output Script Descriptors which describe everything needed for a wallet to actually regenerate all of its keys and addresses. Unlike BIP 39, descriptors explicitly specifies the type of addresses (output scripts really, hence the name) to generate. They also explicitly provide the BIP 32 derivation paths. These two facts resolve the two major complains with BIP 39 - the lack of derivation paths , and the lack of address types. Furthermore, descriptors can be extended in the future with new types of descriptors. This can be done easily and was recently done to add taproot support. Bitcoin Core's wallet was completely overhauled so that we could use these descriptors as the basis of the wallet itself.
Currently, descriptors are not like BIP 39 in that they are much longer and aren't a mnemonic. I intend on proposing a generic mnemonic scheme which would allow for descriptors to be represented as a mnemonic, but that's still a ways off. There is still work to be done with descriptors themselves. BIP 39 is also insufficient for a generic mnemonic scheme since the way it is specified is really specific to entropy and generating a random seed for use in something like BIP 32. We would have to make a wholly new mnemonic scheme.
Topic: Why Core does not allow to use words to "spawn" your wallet? (Read 348 times)
Im still not 100% sure as to why this idea was discarded in Core. I've read some other claims regarding how it was discarded because people may by accident and due lack of information use a non secure seed, for instance, removing words to make it easier to remember, thus compromising the security.
@achow101 posts here. Could you possibly explain this in detail?
@achow101 posts here. Could you possibly explain this in detail?
because of the shortcomings of the mnemonic scheme.
Do you have more links with discussions on this? im sure this has been discussed and there is a lot of literature to read as to why this was discarded. I would like to know gmaxell, luke jr, laanjw, peter wiulle and more opinions on this to get a better idea.
Wait this went over my head. So "cT2ozo5xcj6iCe4dTfywan3qJnib" is the "12 words" that Electrum uses so to speak? With "cT2ozo5xcj6iCe4dTfywan3qJnib" you could spawn it all? Including transaction history, name for each address, etc etc? What does it exactly spawn?
No. Electrum generates a master key using a 12 word mnemonic, with the extended keys being in BIP32 format. By creating a blank wallet and importing a HD seed in the format above, your addresses will be generated using that HD seed and thus you can restore your wallet using that HD seed without having any deviation in addresses. You will only get the same addresses and labels or any data stored locally are only able to be backed up using the wallet file.
And I still dont understand why Core cannot generated 12 words instead of cT2ozo5xcj6iCe4dTfywan3qJnib assuming Electrum does it safely. At least make it optional.
The stackexchange link that you've referenced basically encapsulates the reason why Bitcoin Core isn't implementing it. Just because other wallets are able to implement a mnemonic system, it doesn't mean Bitcoin Core has to follow. I believe that it has been brought up several times but it wasn't implemented primarily because of the shortcomings of the mnemonic scheme. So basically using Electrum as a way to store BTC is not safe?
I did not say that.Both BIP39 and the algorithm Electrum uses are safe as long as the initial entropy was generated randomly using a strong RNG.
Quote
the conclusion is that using any "remember these words and spawn your wallet" methods are not safe and that we must carry the wallet.dat file?
The "remember something" methods are always unsafe because you can't trust your memory after a very long time.The point of mnemonic algorithms is NOT to let user "remember" the words but to simply write down a human readable form of the random octet string.
In other words it is easier for the user to without mistakes write down (with pen and paper)
Code:
hamster diagram private dutch cause delay private meat
slide toddler razor book happy fancy gospel tennis
maple dilemma loan word shrug inflict delay length
Code:
68a79eaca2324873eacc50cb9c6eca8cc68ea5d936f98787c60c7ebc74e6ce7c
That's more than 12 words. When it comes to 12 words it's pretty doable to memorize without being a genius. Imagine that there's a flood and you lose all of your backups or something. Or once again, you try to cross a border and you don't want to carry anything digital with you encrypted, or governments become increasingly tyrannical and they knock on your door looking for keys and so on. I see many use cases where you wouldn't want to have any encrypted data and being able to memorize 12 words and know your wallet is safe gives you a peace of mind in these extreme situations where carrying any data becomes a liability.
Bitcoin Core makes HD wallets by default, which means you can generate all your private keys via the wallet's HD seed instead of using mneumonic words.
To get the HD seed, you can open the console and type "dumpwallet ", and it will be within the first few lines (this is a testnet wallet, but same principle applies for mainnet):
Then to restore it, you can create a new wallet (do not use an existing wallet) and open the console and run sethdseed
To get the HD seed, you can open the console and type "dumpwallet ", and it will be within the first few lines (this is a testnet wallet, but same principle applies for mainnet):
Code:
# Wallet dump created by Bitcoin v0.21.0
# * Created on 2021-01-21T16:59:50Z
# * Best block at time of backup was 1905234 (00000000e26170d0f846a334a1bfdfebe8da906c5),
# mined on 2021-01-21T17:05:05Z
# extended private masterkey: tpR2efnJMh85ufpQcYSTwrrKrYJ
cTdH4SbvTR 1970-01-01T00:00:01Z label= # addr=mnb8RSpTqvhLh8Q6,2MudYFE2awqc,tbvhpur8zf35ql3yx5h9nu
cRaKzuR9MCVwAxVa3aAZrVh15YbV 2019-08-26T01:19:00Z reserve=1 # addr=tb1q3qc52q hdkeypath=m/0'/0'/222'
cT2ozo5xcj6iCe4dTfywan3qJnib 2019-08-26T01:19:00Z hdseed=1 # addr=tb1qzqtu25qsue0a5pp3hg8lkftclf8ds <-- The first string of text on this line is the HD seed because it has 'hdseed=1'
cNb2K9tUhQaeXiVz2Jt2Wq9DMLdf 2019-08-26T01:19:00Z reserve=1 # addr=tb14d7lx hdkeypath=m/0'/0'/117'
Then to restore it, you can create a new wallet (do not use an existing wallet) and open the console and run sethdseed
Wait this went over my head. So "cT2ozo5xcj6iCe4dTfywan3qJnib" is the "12 words" that Electrum uses so to speak? With "cT2ozo5xcj6iCe4dTfywan3qJnib" you could spawn it all? Including transaction history, name for each address, etc etc? What does it exactly spawn?
And I still dont understand why Core cannot generated 12 words instead of cT2ozo5xcj6iCe4dTfywan3qJnib assuming Electrum does it safely. At least make it optional.
So basically using Electrum as a way to store BTC is not safe?
I did not say that.Both BIP39 and the algorithm Electrum uses are safe as long as the initial entropy was generated randomly using a strong RNG.
Quote
the conclusion is that using any "remember these words and spawn your wallet" methods are not safe and that we must carry the wallet.dat file?
The "remember something" methods are always unsafe because you can't trust your memory after a very long time.The point of mnemonic algorithms is NOT to let user "remember" the words but to simply write down a human readable form of the random octet string.
In other words it is easier for the user to without mistakes write down (with pen and paper)
Code:
hamster diagram private dutch cause delay private meat
slide toddler razor book happy fancy gospel tennis
maple dilemma loan word shrug inflict delay length
Code:
68a79eaca2324873eacc50cb9c6eca8cc68ea5d936f98787c60c7ebc74e6ce7c
It uses PBKDF2 which is generally regarded to be a fairly weak KDF so it isn't considered to be good for the secure storage of all of your Bitcoin. Some software (such as Electrum) used BIP 39 in the past but have switched to using their own mnemonic algorithm because of this weakness in BIP 39.
PBKDF2 is not there to provide any security, it is there to provide a way to derive keys using a combination of the initial entropy (ie. the mnemonic) and a user input (referred to as passphrase). When we use a 256-bit entropy for example, the security is guaranteed by the randomness and size of that entropy regardless of what KDF is used to derive the BIP32 seed.Electrum is also using the same algorithm including PBKDF2. The only differences are the checksum, string normalization and lack of dependence on a fixed length word list.
So basically using Electrum as a way to store BTC is not safe? the conclusion is that using any "remember these words and spawn your wallet" methods are not safe and that we must carry the wallet.dat file?
Damn, in this case, how do people deal with this when crossing borders? you never know when you are going to get stopped and forced to decrypt. And they make perpetual copies of your drives anyway and store them.
Then to restore it, you can create a new wallet (do not use an existing wallet) and open the console and run sethdseed
I missed this part just now, sethdseed only takes a WIF key (ie. your private key), not the master key.Actually, although it looks like a WIF, it really is the seed. My post was based on this answer:
...
An important thing to note here is that Bitcoin Core does not use an extended private key. Rather it uses a HD seed (as defined in BIP 32). When keys are being generated, that seed is hashed to get the extended master private key, and from there the child keys are derived. That extended master private key is never stored in the wallet file itself.
Instead of backing up the master private key, you could instead backup the HD seed. This can also be found in the dumpwallet output and it will be listed as hdseed=1. It will look like a normal private key and be presented in Wallet Import Format.
Once you have the seed, you can create a new wallet and set it to use your backed up seed using the sethdseed command.
An important thing to note here is that Bitcoin Core does not use an extended private key. Rather it uses a HD seed (as defined in BIP 32). When keys are being generated, that seed is hashed to get the extended master private key, and from there the child keys are derived. That extended master private key is never stored in the wallet file itself.
Instead of backing up the master private key, you could instead backup the HD seed. This can also be found in the dumpwallet output and it will be listed as hdseed=1. It will look like a normal private key and be presented in Wallet Import Format.
Once you have the seed, you can create a new wallet and set it to use your backed up seed using the sethdseed command.
The HD seed generates the master private key which cannot be set manually and changing it should generate the subsequent private keys. Because knowledge of what are the "subsequent private keys" is arcane - due to the fact there is one keypool for change and receiving addresses and that they use different paths - it's better to just make a new wallet file from scratch before you run sethdseed to avoid all this complexity.
Yes it is (regtest dump):
You might be talking about the "extended private masterkey"?
-snip-
Why do you need to rescan after flushing the unused keys?
Oh okay. I stand corrected. I thought both of you were talking about importing using an xpriv key, since the topic is about using BIP39 in Bitcoin Core. Was thinking of using a workaround to use the mnemonic for that. You might be talking about the "extended private masterkey"?
-snip-
Why do you need to rescan after flushing the unused keys?
AFAIK, the "hdseed" in the dumpfile is WIF.
It isn't. The hdseed in the dumpfile is xpriv, you can't use the command with that.You might be talking about the "extended private masterkey"?
But NotATether's suggested command should be sethdseed true "hdseed" - "true" so that the previous unused keys will be flushed. And then, rescanblockchain
Why do you need to rescan after flushing the unused keys?AFAIK, the "hdseed" in the dumpfile is WIF.
It isn't. The hdseed in the dumpfile is xpriv, you can't use the command with that.But NotATether's suggested command should be sethdseed true "hdseed" - "true" so that the previous unused keys will be flushed. And then, rescanblockchain
Why do you need to rescan after flushing the unused keys? Then to restore it, you can create a new wallet (do not use an existing wallet) and open the console and run sethdseed
I missed this part just now, sethdseed only takes a WIF key (ie. your private key), not the master key.But NotATether's suggested command should be sethdseed true "hdseed" - "true" so that the previous unused keys will be flushed and also for it to accept the next argument.
And then, rescanblockchain
If the source wallet contains an "inactive hdseed" (usually set before the wallet was encrypted), I'd suggest to set it first and rescan before setting the active hdseed.
It uses PBKDF2 which is generally regarded to be a fairly weak KDF so it isn't considered to be good for the secure storage of all of your Bitcoin. Some software (such as Electrum) used BIP 39 in the past but have switched to using their own mnemonic algorithm because of this weakness in BIP 39.
PBKDF2 is not there to provide any security, it is there to provide a way to derive keys using a combination of the initial entropy (ie. the mnemonic) and a user input (referred to as passphrase). When we use a 256-bit entropy for example, the security is guaranteed by the randomness and size of that entropy regardless of what KDF is used to derive the BIP32 seed.Electrum is also using the same algorithm including PBKDF2. The only differences are the checksum, string normalization and lack of dependence on a fixed length word list.
We should remember that they don't have to implement BIP39 and I guess OP is not asking about why this BIP is not implemented but why isn't core providing a mnemonic (a human readable encoding of the master key used to generate the child keys).
Yeah. But the current only standardized solution that we have is BIP39, don't think there is any other proposal within the BIPs and I assume Bitcoin Core would be primarily interested in having a mnemonic generated using a known standard.Then to restore it, you can create a new wallet (do not use an existing wallet) and open the console and run sethdseed
I missed this part just now, sethdseed only takes a WIF key (ie. your private key), not the master key.https://bitcoin.stackexchange.com/questions/88237/is-there-a-reason-to-why-bitcoin-core-does-not-implement-bip39
Andrew Chow explains here BIP39 isn't secure, but then he also explains Electrum does not use this anymore, yet Electrum still gives you the option to spawn wallets with 12 words. On the Electrum website they say: "Electrum 2.0 derives keys and addresses from a hash of the UTF8 normalized seed phrase with no dependency on a fixed wordlist.". So the question would be, why isn't Core using this method for wallets then? Electrum devs still consider secure the "show 12 words for seed" method when creating wallet, why not Core?
What would the addition of those features do, exactly?
The features are quite secondary. BIP39 has always relied on a list for the checksum which can be quite troublesome and presents a problem with the various locale that users may choose to use. The shortcomings are pretty much highlighted in the link I've posted.As the "reference implementation" bitcoin core is and should be defining these things not being limited to what is exists!
There isn't any other viable proposal currently and BIP39 is probably the only finalized proposal out there. If there is an alternative proposal to that, then it won't be limited to BIP39. There isn't any point to making a mnemonic system specific to Bitcoin Core without providing an appropriate BIP for other wallets to adopt as well. I'm not saying Bitcoin Core should only be using an existing standard, but there has to at least be a BIP to better refine it before implementing it in Bitcoin Core. I assume Bitcoin Core would be primarily interested in having a mnemonic generated using a known standard.
As the "reference implementation" bitcoin core is and should be defining these things not being limited to what is exists!What would the addition of those features do, exactly?
They will add additional functionality.Quote
Would adding version within the seed phrase prevent older wallet clients from deriving the correct wallets from newer seeds?
Generally a version in an encoding could help the client make sure that it understands the input.For example take bitcoin addresses. A P2PKH address has a version 0, if you change it to 1 your wallet will complain. Same with P2WPKH, if you enter a bech32 address with version 2 the wallet can immediately tell you that it doesn't have the code for version 2 witness program and either reject it or warn you that it may be non-standard because it can't validate.
Same with seed phrase. Your wallet can immediately figure out that the input is not something it understands and either reject it or ask for additional user input (like manually adding the derivation path or address type).
Quote
Would adding derivation path feature lock the seed to a specific path, not allowing the seed to be used to generate other wallets?
Yes and No.Just like the version the wallet can add more flexibility. For example after you enter your seed phrase it can show you a new window that has the modifiable derivation path, address type or for multi-coin wallets the coin type. The user can simply modify these if they wanted to.
Besides there is no reason to use multiple derivation paths, if you need more "branches" you simply use a different "account".
Code:
m / purpose' / coin_type' / account' / change / address_index
The flaws is in "lack of features" such as not having a version or not having any indication of the derivation path
What would the addition of those features do, exactly? Would adding version within the seed phrase prevent older wallet clients from deriving the correct wallets from newer seeds? Would adding derivation path feature lock the seed to a specific path, not allowing the seed to be used to generate other wallets? Just asking out of curiosity. I can see how adding these features might make seeds more user friendly, even if they are less versatile.
We should remember that they don't have to implement BIP39 and I guess OP is not asking about why this BIP is not implemented but why isn't core providing a mnemonic (a human readable encoding of the master key used to generate the child keys).
Yeah. But the current only standardized solution that we have is BIP39, don't think there is any other proposal within the BIPs and I assume Bitcoin Core would be primarily interested in having a mnemonic generated using a known standard.Then to restore it, you can create a new wallet (do not use an existing wallet) and open the console and run sethdseed
I missed this part just now, sethdseed only takes a WIF key (ie. your private key), not the master key. If you look at the BIP itself, it is actually unanimously discouraged from implementation.
We should remember that they don't have to implement BIP39 and I guess OP is not asking about why this BIP is not implemented but why isn't core providing a mnemonic (a human readable encoding of the master key used to generate the child keys).Im assuming the Electrum wallets are impossible to crack and the method is safe,
The question, is it safe? or im missing some security compromises using Electrum's method?
Mnemonic generation algorithms (BIP39, Electrum, AEZeed) are all secure algorithms that use different approaches to encode a randomly generated entropy to a set of words and use those words in a certain way to derive the BIP32 seed.The question, is it safe? or im missing some security compromises using Electrum's method?
The flaws is in "lack of features" such as not having a version or not having any indication of the derivation path or only supporting a fixed word list. Some of these algorithms address some of these missing features though. For example Electrum can accept any word list of any length. Or AEZeed adds a timestamp and uses a different KDF.
If you look at the BIP itself, it is actually unanimously discouraged from implementation. The reason is outlined here: https://github.com/bitcoin/bips/wiki/Comments:BIP-0039.
Jump to: