Author

Topic: Recovered .db files using Testdisk (Read 247 times)

sr. member
Activity: 356
Merit: 268
July 11, 2023, 09:45:43 PM
#9
found this in my old notes,

Code:


    
        
            \x61\x15\x06\x00
            \x00\x06\x15\x61
            \x62\x31\x05\x00
            \x00\x05\x31\x62
            \x61\x15\x06\x00
            \x00\x06\x15\x61
            \x62\x31\x05\x00
            \x00\x05\x31\x62
            \x53\x22\x04\x00
            \x00\x04\x22\x53
            \x88\x09\x04\x00
            \x00\x04\x09\x88
        

        
    



i think it's supposed to be used in https://www.google.com/search?q=R-Studio


I can't vouch for it, but like i said before if you recover using photorec anything that resembles a wallet database (bdb) it will have a .db extension and be missing the first 8kb or less, that will probably be seperared, the wallet wont read correctly but you can fix it








edit: this post: https://bitcointalksearch.org/topic/how-i-rescued-my-walletdat-2637884 is also very useful but i have yet to test properly, had to take a break from trying to recover my old coins.


Two years ago I formatted my harddisk and installed Windows 10 on it. Before this I did a backup, but unfortunately the backup was broken. So I lost my wallet.dat, with a few Bitcoins in it. I could restore some files with RStudio, and I had older backups for the rest, but seemed to be that the latest wallet.dat was already overwritten, and I frequently add new addresses. So I gave up, not a big deal, maybe $200 lost. But I didn't use the harddisk and bought a new one.

Fast forward to December 2017: Now a few Bitcoins is some serious money, so I decided to give it another try. I tried any option I could find in RStudio, checking the dozens of filesystems it reported after scanning it for hours (only a few where valid from previous installations), but I couldn't restore it. Ok, this needed some more work.

My assumption was, that the file headers were broken, so I wrote a small C program myself, which scanned the whole harddisk for the wallet.dat signature (testing for the first 16 bytes). The filesystem was NTFS, which has 4k sector sizes and a file starts always at sector start, if I understand it correctly, which makes things easier. Also usually if there is enough space, contiguous sectors are used to save a file. My hope was that somewhere I could find old version of the wallet.dat, but not too old that the new keys were missing.

This is the very simple and straightforward scan program I hacked together:

Code:
#include
#include
#include
#include

uint8_t buf[4096];
char filename[1000];

uint8_t search[] = {
    0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x62, 0x31, 0x05, 0x00
};

int main(int argc, char** argv)
{
    uint64_t pos = 0;
    FILE* f = fopen("/dev/sdd", "rb");
    FILE* w = NULL;
    int walletNumber = 0;
    int walletIndex = 0;
    uint64_t max = 1000000000000ULL;
    while (1) {
        int c = fread(buf, 1, 4096, f);
        if (c != 4096) break;
        if (!w) {
            if (memcmp(search, buf, 16) == 0) {
                sprintf(filename, "wallet%i.dat", walletNumber++);
                walletIndex = 0;
                w = fopen(filename, "wb");
                printf("found: %" PRIu64 "\n", pos);
            }
        }
        if (w) {
            fwrite(buf, 1, 4096, w);
            walletIndex++;
            if (walletIndex == 256) {
                fclose(w);
                w = NULL;
            }
        }
        pos += 4096;
    }
    fclose(f);
    return 0;
}

I used it on Linux as my host system and the old harddisk was visible as /dev/sdd (you can see this with dmesg). I compiled it with "gcc -O3 scan.cpp -o scan" and started it with "sudo ./scan", and a few hours later (it was a 1 TB harddisk) I got a wallet0.dat to wallet9.dat, each 1 MB in size (it doesn't matter if there is crap after the wallet data). This was a nice start Grin

Then I tried to copy it to a wallet.dat of a current Bitcoin installation, but most of the time it said the wallet was corrupt, once it even crashed at start and when it said it could salvage some information, no keys were in it.

My rescue was https://github.com/joric/pywallet This program could decode all files and output it in JSON format. It needs the wallet.dat in a bitcoin-qt installation in the .bitcoin directory. I knew one of my old addresses, so I wrote a script which did test all files (actual key changed) :

Code:
for i in $( ls wallet*.dat ); do
    echo item: $i
    cp $i .bitcoin/wallet.dat
    ./pywallet.py --dumpwallet --datadir=.bitcoin | grep -i 12QDRXssT63Pv5KTGBN2kyAvfLW3s7jxBs
done

The output looked like this:

Code:
item: wallet0.dat
item: wallet10.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet11.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet1.dat
WARNING:root:encrypted wallet, specify password to decrypt
item: wallet2.dat
item: wallet3.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet4.dat
Traceback (most recent call last):
  File "./pywallet.py", line 1706, in
    main()
  File "./pywallet.py", line 1683, in main
    read_wallet(json_db, db_env, True, True, "")
  File "./pywallet.py", line 1556, in read_wallet
    parse_wallet(db, item_callback)
  File "./pywallet.py", line 1287, in parse_wallet
    for (key, value) in db.items():
bsddb.db.DBPageNotFoundError: (-30986, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')
item: wallet5.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet6.dat
WARNING:root:encrypted wallet, specify password to decrypt
            "addr": "12QDRXssT63Pv5KTGBN2kyAvfLW3s7jxBs",
item: wallet7.dat
WARNING:root:encrypted wallet, specify password to decrypt
item: wallet8.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet9.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.

So the address was in wallet6.dat, success! I then used "/pywallet.py --dumpwallet --datadir=.bitcoin --password=mysecrectpassword > keys.txt" and I got all my keys back. In the bitcoin client I could import it with importprivkey (don't forget the "false" parameter as the last parameter, to avoid rescanning after each import, if you import multiple keys) and after the final rescan, I got my Bitcoins back. One day work for like 2 Bitcoins, which I already sold, that's a nice hourly rate Cool

Maybe this will help some other people as well. In case you rescue a lot of Bitcoins, I would really love it if you would send me some to 1ieKggPzp2DfroFBNie4ib77kHKNbJMkw.

Marilyn wishes you a merry Christmas, a merry Christmas, And a happy New Year!
newbie
Activity: 28
Merit: 1
July 10, 2023, 05:45:35 PM
#8
Are there other good data recovery tools I can try? Just deleted some important files by mistake and now looking for a solution.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
February 26, 2022, 06:43:44 AM
#7

 
I thought magic numbers were only the first bit of a file
But my knowledge is minimal..

Can I confirm what OP is please? And by copying bytes to a new file will it change to a .dat like it originally was or will it stay at a .db?

You are brilliant thank you so much I have been stuck on this for some time now

Appreciate your time so much!

OP is forum-speak for Original Poster.

Copying the bytes, if the correct number of bytes is copied, will produce a valid .db file, otherwise, it'll be still be a .db file but it cannot be opened by applications because of the junk at the end of the file.

A bitcoin core wallet.dat starts with the following bytes:

Code:
000000  \0  \0  \0  \0 001  \0  \0  \0  \0  \0  \0  \0   b   1 005  \0
000010  \t  \0  \0  \0  \0 020  \0  \0  \0  \t  \0  \0  \0  \0  \0  \0

(Generates using od -Ax -v -c on the wallet.dat file)

And doesn't appear to end with a predictable sequence of bytes, but it's always aligned to a 4096-byte boundary. That means the size will be a multiple of 4KB. Perhaps if I look at some Berkeley DB source code I will find what kind of stuff is written at the end of a file.
sr. member
Activity: 356
Merit: 268
February 25, 2022, 03:02:04 PM
#6
Databases(Berkeley) have more than one database in it, I noticed with photo wreck usually calves them separately if so like they’ll be a small and large Output. The database wouldn’t also load properly and would need manual cleaning.

If you can find the correct headers and config feel free to share I think it’s on Google somewhere


Also dont share any screenshots if they include data from your wallet, especially ones with 01 01 04 20 hex str
newbie
Activity: 3
Merit: 0
February 25, 2022, 01:00:33 PM
#5

 
I thought magic numbers were only the first bit of a file
But my knowledge is minimal..

Can I confirm what OP is please? And by copying bytes to a new file will it change to a .dat like it originally was or will it stay at a .db?

You are brilliant thank you so much I have been stuck on this for some time now

Appreciate your time so much!
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
February 24, 2022, 10:56:11 PM
#4
testdisk recovered a few GB worth of files in .db format

Size of wallet generated by Bitcoin Core shouldn't be that big, unless it contain LOTS (could be few million) of address/transaction. If your USB storage is really old (2013 or earlier), it's possible you recovered Berkeley DB files which contain transaction/block on Bitcoin blockchain. Bitcoin Core (used to be called Bitcoin Qt) use Berkeley DB (rather than LevelDB) before Bitcoin Qt 0.8.1 to store transaction/block on Bitcoin blockchain.

That's probably because photorec could not determine the end of the Bitcoin Core wallet file, so it just appended whatever data it thought was part of it until it became that size.

I'm pretty sure there are some magic bytes at the end of wallet.dat files that OP can find to trim (by copying the bytes out to a new file) the wallet size and make it readable by programs, but I have to go manually inspect some wallet.dat's I have first, for this data.
newbie
Activity: 3
Merit: 0
February 24, 2022, 12:17:11 AM
#3
Thanks for your reply.

I ran file and it did print

"Berkeley DB (Btree, version 9, native byte-order)"

Then I ran hexdump -C

https://ibb.co/zSgSh2X

So Testdisk (photorec) found the files based on the photorec.sig for -
"Berkeley DB (Btree, version 9, native byte-order)"

And the file command confirmed that is it Berkley DB

So now how do I turn these files into wallet.dat or what can I do testdisk recovered a few GB worth of files in .db format

What I have noticed is the first file will be few KB and out put "b1 main" and the next file will be 741 MB of heaps of data keys pool ect and then again the next file is few KB "b1 main" and so on...if that makes any sense
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
February 23, 2022, 09:30:03 AM
#2
If you have a Unix (that's Mac or Linux) box, open a terminal and run the file command against the file to check if it's in a format that is used by one of the well-known wallets:

Code:
file full_path_to_file

It will print

"Berkeley DB (Btree, version 9, native byte-order)"

If it's a Bitcoin Core wallet,

"ASCII text, with very long lines"

If it's an Electrum wallet. Any other output means it's either not a wallet file or it was made by obscure wallet software (in particular, file just prints Armory ".wallet" and ".lmdb" files as "data").
newbie
Activity: 3
Merit: 0
February 23, 2022, 08:55:53 AM
#1
Hi,

I ran testdisk (photorec) to recovery wallet files from a formatted usb. Photorec was able to recovery .db files using berkeley signature which is great but now I don't know how to extract just the wallet file out of the .db files and convert it to .dat or make it readable for bitcoin core.
Any suggestions would be greatly appreciated?
Jump to: