Topic: Question about restoring and syncing bitcoin-qt wallets (Read 1483 times)

Nice!  I didn't know about "Coin Control".  That gives me something to search for.

At least once every 100 addresses.  The wallet uses an address every time you send a transaction, AND every time you click the "New Address" button.

Also, anytime you use the "Encrypt Wallet" option or "Change Passphrase" option in the menus, for security reasons, the wallet discards all 100 unused addresses in the pool, and generates 100 new addresses.  Therefore, you should always backup your wallet immediately after encrypting or changing the passphrase.

It would be a good idea to have a few recent backups just in case one is damaged somehow.  As such, it might be a good idea to keep a rough idea of how many addresses you've used since your last backup, and do a new backup anytime you exceed approximately 25 addresses.  Then keep the 3 most recent backups.


Seeing this info (and having the ability to control which of these outputs the wallet spends) is commonly called "coin control".  I believe there is a non-standard version of Bitcoin-Qt that can be sued for that.  If you search the forum for "Coin Control", you'll probably find it.


It can be.  I think it's done by creating a file in the Bitcoin data directory (where the wallet.dat is stored) named "bitcoin.conf" and adding a line "keypool=#" to that file (where # is the number of pre-generated addresses you want the wallet to maintain)


It might. The devil is in the details.  You'd have to find a secure way to transmit the private keys between the two instances of the wallets often enough to avoid situations where one wallet has expired all the shared keys and uses an address that the other wallet hasn't been made aware of yet.


Not for the average user.  You can run a "Coin Control" version of the software if you trust it, or you can use "createrawtransaction" in the Console interface.  You need to be certain you know what you are doing, and you need to be very careful when using "createrawtransaction".  Very knowledgeable and experienced people have been known to make a mistake that results in accidentally paying a huge transaction fee.  I think I heard of one programmer who should have known better accidentally paying a transaction fee in excess of 100 BTC.


Possibly, but there is a lot of opportunity for error there.  It is much safer to simply acknowledge that the Bitcoin-Qt wallet is not designed to be used with two concurrent instances sharing the same originating wallet, and not try to find a way to do it.  Given the amount of care you have to take, and the risks of making a mistake, it would seem that it isn't worth the effort.


That's the safe way to do it.  It only becomes a problem if you intend to re-use some of the addresses from an old wallet and/or believe that someone may still send bitcoins to an address that you've given out in the past.

That's a great reason.  That would imply that a backup must be taken at least once every 100 transactions, otherwise there is a risk of loss if something happens to your installation, right?


Would it be useful to anybody else if these change addresses were also displayed, perhaps with the current balance of each address?  That might be useful for truly seeing how your holdings are spread across your addresses.

Also, it might be useful if the amount of pregenerated addresses could be set to something other than 100.  If each transaction is going to consume an address, for example, then it might be useful to set this to 1000 or something like that, to guarantee a backup will be good for a long time.  Conversely, if the user wants a smaller wallet, and doesn't plan to spend very many transactions before taking another backup, this could be lowered.


Would this problem go away if the Bitcoin client were made aware of every address within both wallets, including change addresses, and merged each list together so that all addresses were unified into a single wallet, including the private key for each address?  It seems that a "Merge Wallet" feature would be useful.  It would result in one wallet containing the combined balance of both previous wallets.


Is it possible to control from which address(es) the coins are spent from?  If so, the user could be careful to select different addresses for each outgoing transaction, during the time the installations would be out of sync.  That should solve the problem of double-spending from the same address.

Thanks for the warnings.  They have given me good ideas for adding features to the Bitcoin client in the future, but for now, if I start a second Bitcoin installation, I'll do it from scratch.  I can then easily transfer a balance by paying myself.

If it didn't, your backup wouldn't be any good anymore as soon as you request a new address in the "Receive Coins" section as well as every time you send a transaction.  By pre-generating 100 addresses, you can restore a backup that you created 5 addreses and 40 transactions ago and still have access to all your bitcoins.


Yes.


The directory where you need to put it depends on the operating system you are using (and any additional configuration you may have done).  In general you can find the appropriate directory here:
https://en.bitcoin.it/wiki/Data_directory


As long as you've never sent any transactions at all, yes.  As soon as you send a transaction, the wallet will likely move some of the bitcoins to a "change" address that it does not display to you.  There are commands that you can use in the Console interface to find a list of change addresses that have bitcoins associated with them.


See my comments about change addresses above.


If you've never sent any transactions, then it will match up with the addresses where you initially received each transaction.  However, when you send bitcoins, it is likely that the wallet will send some of the value tracked by the wallet to a change address.


As an example, if one wallet sends change to an address that the other wallet doesn't know about, and then later the second wallet sends change to an address that the first wallet doesn't know about, the two wallets will show differing amounts.  If you restore a backup to try to get it back to the last time you saw it in sync, or you copy one of the wallet.dat files over the other, you can lose all bitcoins that are associated with the change address in the wallet that gets overwritten.  There may be other things that can happen as well, that is just one that I know I've seen other people do in the past in this situation.


The "magic state" you refer to is the 100 pre-generated addresses and the change addresses.  Once you get to the 101st address the wallets will have differing sets of addresses internally.

You can see other "oddities" as well.  If there is a soft-fork in the blockchain for a block or two, you can see differing balances or confirmations between the two wallets.  As another example, it is possible to send a transaction from one wallet, and if the other wallet hasn't caught up synchronizing it won't know those coins are spent.  Then if you try to send a transaction from the second wallet it can accidentally attempt to double-spend the already spent bitcoins.

I'm not sure I know what all the issues are that you can run into, but it is generally enough to know that you shouldn't try to operate two separate instances of Bitcoin-Qt that are both based off the same wallet.dat.  If you are going to use a wallet.dat from one Bitcoin-Qt to bootstrap another instance, then you should abandon the original instance as soon as you know that the new instance is working properly.

Interesting.  The Bitcoin client generates addresses in advance?  I wonder why it does that: non-vanity address generation shouldn't take very long at all.  I thought it generated addresses on the fly, whenever needed.


Thanks.  Does it have to be named exactly "wallet.dat"?  I assume the Bitcoin client will pick it up, when that file is placed in the Bitcoin client's main directory that it manages (not any subdirectories)?


Interesting.  I didn't mean for it to remain in continuous synchronization, but rather, to use the wallet.dat file from one installation to bootstrap another installation.

Are the displayed addresses in my Bitcoin wallet all it takes to represent my holdings?  Or, are there more addresses than what is displayed?  All the Bitcoin client displays is my total balance across all addresses, it doesn't let me break it down by address (however I verified a few addresses against what blockchain.info displays, and it seems to add up OK).

How would it lose coins, if running multiple installations?  If the network is consistent, then all of my addresses should have a balance associated with them, and anybody could add them all up to determine my total balance, correct?  Or, is there some magic state associated with each installation?


Ouch.  It would be very useful to be able to see all of these hidden addresses, as well as a way to see balance by address (not just an overall total balance).


Interesting.  I wonder if there would be interest from others in adding this feature, to be done automatically?  An opportunity for a project.  Solving the problem of making it easier to merge wallets, would also make it easier to restore wallets, as doing a merge into an empty wallet is the same thing as doing a restore.

Unless you've changed the defalut settings, the wallet.dat stores the next 100 addresses (and private keys) that you will use in the future.  It also stores all the "change" addresses fro every transaction you've ever sent.  I'm not certain, but I think it also stores any unconfirmed transactions that you've sent or received.


That would be nice, wouldn't it?  Instead, you have to find the directory on your operating system where the Bitcoin-Qt client stores the wallet.dat file.  Then you have to shut down Bitcoin-Qt (make sure you wait for it to shut down completely), copy the backup wallet.dat to the appropriate directory, and start Bitcoin-Qt up again.


It is a bad idea to run in this configuration for very long.  You will encounter unexpected behavior and the potential to accidentally lose bitcoins.  The Bitcoin-Qt wallet is not designed to remain in synchronization with another copy that has the same wallet.dat.


At the moment you start them up?  Yes, probably assuming that the backup is recent.  However, the Bitcoin-Qt wallet moves bitcoins around to new hidden addresses that it creates every time you send a transaction.  Eventually one wallet will send bitcoins to an address that the other one doesn't know about.  Then the balances will be different.  If you aren't careful, you can end up accidentally losing bitcoins permanently while trying to get them back in sync.


It is not designed to do this easily, however, using the console interface you can issue a "dumpprivkey" command for each and every address in the wallet that has any bitcoins associated with it.  Then you can "importprivkey" each of those private keys into the wallet that you are attempting to merge into.  Make sure you understand exactly how the Bitcoin-Qt wallet works, or you can miss some of the addresses that you need to import.

Been wondering the same thing myself.

Hoping this isn't the super obvious newbie question that I just overlooked.

I've done "Backup Wallet" in my bitcoin-qt client, it wrote a wallet.dat file just fine.  I repeated that a few times, backing up to various drives and USB sticks.

I noticed the file is rather large for what I thought was just the private keys for my Bitcoin addresses.  What is contained in the wallet file?  Mine was around 300K, which is still very small, but large for just a few private keys.  What else is in the wallet file?

Anyway, my question is, how do I then restore the wallet that I've backed up?  Thinking "Restore Wallet" would be right next to "Backup Wallet" in the menu, it wasn't.

I want to also set up a bitcoin-qt client on another computer, and want them to sync up, so that they both see the same addresses and show my balance.  This will also be a good test of whether or not the wallet.dat file truly does a full restore.  Ideally, the account balance should always match, assuming both computers are fully caught up to the latest blockchain, right?

Also curious: How can I merge two wallets together, in the bitcoin-qt client?  That would also be useful.

Thanks!
Josh