auto backing up of wallet.dat | Bitcointalksearch.org

nelisky

legendary

Activity: 1540

Merit: 1002

Quote from: BioMike on September 07, 2010, 12:10:36 AM

Nice. How about being able to set as a config option and have the function called after every send(/receive?) of bitcoins?

I'm doing backups every time I create a new address and when I send btcs. Receiving should be safe? Also, I'm collecting other stuff and doing offsite scp based backups of the wallet, meaning I have the trigger embedded in my program logic, but auto backups would be great, as one can look for changes in the backup file and use that as trigger to do the offsite.

But what I *really* think would rock is some kind of event mechanism

BioMike

legendary

Activity: 1658

Merit: 1001

Nice. How about being able to set as a config option and have the function called after every send(/receive?) of bitcoins?

satoshi

founder

Activity: 364

Merit: 7423

rpc backupwallet is in SVN rev 147.

satoshi

founder

Activity: 364

Merit: 7423

Sorry, I've been so busy lately I've been skimming messages and I still can't keep up.

We want to avoid Windows API calls whenever possible. They usually take about 6-8 parameters and a lot of testing to get right, it takes a page of code to do something simple.

I usually shy away from iostreams. Seems like I too often hit limitations. They kind of botched the C++ streams standard in the 90's, which is too bad, streams can be very powerful and useful when done right. Using it in rpc.cpp may still turn out to be a mistake.

Bottom line is I'd rather call an existing file copy function than make and test my own.

jgarzik

legendary

Activity: 1596

Merit: 1100

Quote from: satoshi on August 26, 2010, 09:54:07 PM

I doubt there's an mmap(2) on Windows.

The Windows version of mmap was mentioned in the message to which you replied: CreateFileMapping

In my earlier message, I mentioned how to use this from boost: Boost::Iostreams already has a mapped_file Source.

satoshi

founder

Activity: 364

Merit: 7423

I doubt there's an mmap(2) on Windows. I'd rather call an existing file copy function than make and test my own.

Quote from: nelisky on August 26, 2010, 08:21:09 PM

But if you are already using features from boost::filesystem you can use copy_file from that. I just think that, if not already required for something else, it's a tad overkill.

Thanks. I thought it would be in there somewhere.

We already use boost::filesystem in a dozen places. It's not a new added dependency. It gives us a lot of portable stuff that we would otherwise have to have a #ifdef for each OS and test everywhere.

jgarzik

legendary

Activity: 1596

Merit: 1100

Quote from: satoshi on August 26, 2010, 08:13:42 PM

If you read it into memory and write it out, it could fail in tight memory situations.

I think you misunderstand what mmap does? mmap / CreateFileMapping does not read a file into heap memory: http://en.wikipedia.org/wiki/Mmap

nelisky

legendary

Activity: 1540

Merit: 1002

Quote from: satoshi on August 26, 2010, 08:13:42 PM

If you read it into memory and write it out, it could fail in tight memory situations.

I'm looking for something like copyfile(const char* from, const char* to) or copyfile(path from, path to), preferably something in Boost if it has it. If you find it for me, it's more likely I'll get to implementing it.

Quote from: nelisky on August 25, 2010, 08:21:57 PM

As for the file copy, why add to the boost dependency? I for one would love to get a core lib with very little deps.

We require Boost for JSON and a dozen things replacing dependencies on wxWidgets. Boost is good, portable stuff, we should not shy away from it.

Ok, so what's wrong with the simple standard fstream usage from that snippet I mentioned? I think simple is best

But if you are already using features from boost::filesystem you can use copy_file from that. I just think that, if not already required for something else, it's a tad overkill.

satoshi

founder

Activity: 364

Merit: 7423

If you read it into memory and write it out, it could fail in tight memory situations.

I'm looking for something like copyfile(const char* from, const char* to) or copyfile(path from, path to), preferably something in Boost if it has it. If you find it for me, it's more likely I'll get to implementing it.

Quote from: nelisky on August 25, 2010, 08:21:57 PM

As for the file copy, why add to the boost dependency? I for one would love to get a core lib with very little deps.

We require Boost for JSON and a dozen things replacing dependencies on wxWidgets. Boost is good, portable stuff, we should not shy away from it.

jgarzik

legendary

Activity: 1596

Merit: 1100

Quote from: satoshi on August 25, 2010, 07:57:40 PM

What's the simplest portable way to copy a file? Is there something in Boost?

mmap(2) + memcpy(3) ? Boost::Iostreams already has a mapped_file Source.

Quote from: satoshi on August 25, 2010, 07:57:40 PM

What should it be named? maybe:
backupwallet

Sounds quite useful!

nelisky

legendary

Activity: 1540

Merit: 1002

Quote from: satoshi on August 25, 2010, 07:57:40 PM

I started posting in the other topic but I'll repeat here, this thread seems more specific to the topic.

The main backup improvement will be a pre-generated pool of keys and a rescan at load to scrape missed transactions from the block history. Then a backup will last forward for a long time.

I was starting to post the same idea you said nelisky.

Yes, I saw your other post and I really like the poll of addresses, but we still need a way to easily back up when those get all used, don't we? I know the address space is huge, but there might be applications that deliver thousands of addresses per day.

Quote from: satoshi on August 25, 2010, 07:57:40 PM

How about a json-rpc command that locks the wallet, flushes it, copies wallet.dat to a location you specified, then unlocks it? That would be a smaller project than the pooled keys, so maybe it could be done first.

What's the simplest portable way to copy a file? Is there something in Boost?

What should it be named? maybe:
backupwallet

Great on all accounts, name and implementation approach.

As for the file copy, why add to the boost dependency? I for one would love to get a core lib with very little deps. In C++ you can just use standard file streams, can't you? Something like http://www.dreamincode.net/code/snippet2306.htm (out of a quick google, didn't try but looks correct).

To put the cherry on top, why not add a trigger that makes the copy whenever there's a change to the wallet? Well, this would depend on file locking to work properly and that sucks on windows iirc. Been a while since I had to code there

satoshi

founder

Activity: 364

Merit: 7423

I started posting in the other topic but I'll repeat here, this thread seems more specific to the topic.

The main backup improvement will be a pre-generated pool of keys and a rescan at load to scrape missed transactions from the block history. Then a backup will last forward for a long time.

I was starting to post the same idea you said nelisky.

How about a json-rpc command that locks the wallet, flushes it, copies wallet.dat to a location you specified, then unlocks it? That would be a smaller project than the pooled keys, so maybe it could be done first.

What's the simplest portable way to copy a file? Is there something in Boost?

What should it be named? maybe:
backupwallet

mizerydearia

hero member

Activity: 574

Merit: 513

https://bitcointalksearch.org/topic/m.10728 is somewhat related

nelisky

legendary

Activity: 1540

Merit: 1002

So, as many others, I need to backup wallet.dat. And since this one is on a server, I need it to happen unattended. And because this server is being used by the lottery, I must not shutdown bitcoind.

So far I just copy the file over, and do it often so I can outlive one corrupted file. But this is not ideal, as I should backup after every transaction (sent, right? no new addresses are created on receiving a transfer ) or after everytime I create a new address.

For that I thought about instead of copying the file I could use db_dump to dump it's contents, using the -r flag just in case. Would that work?

The ideal solution would be an rpc call that would either:
- flush and lock updates until a new rpc call (any call, didn't need to be an unlock command) would come over
- flush and cp wallet.dat to a side file
- flush and dump through jsonrpc. If each key would come separate in an array, we could then call this with 'lastseen=X' to just get new keys

Could this work? Which would work best?

Topic: auto backing up of wallet.dat (Read 9410 times)