Pages:
Author

Topic: [overview] Recover Bitcoin from any old storage format - page 3. (Read 8117 times)

legendary
Activity: 1039
Merit: 2783
Bitcoin and C♯ Enthusiast
Since I just added this "optimization" to FinderOuter I though I should give an update on this post regarding the 7 characters and the checksum (I can't believe it's been 2 years!).

The key posted in OP (5KMWmYkn5YWkJnUDG4utD9L1HXQv3DBseqqCGsQXmthcEerbA7k) is missing more than just its checksum. In fact there are 515 valid WIFs with the same starting 44 characters. I did a little modification to the code and printed all of them. Interestingly enough the valid key is found on line 122.
https://gist.github.com/Coding-Enthusiast/8339f64c736053107f0d918675c186e0

I ran a lot of tests with different keys before releasing this code and I came up with these numbers representing the number of possible valid keys with the same initial characters in case the missing ones are from the end. In case of compressed keys checksum could be up to 6 chars and only 5 for uncompressed ones.
Code:
        Uncompressed ;     Compressed
1-5 ->             1 ;              1
6   ->             9 ;              1
7   ->           514 ;              3
8   ->        29,817 ;            117
9   ->     1,729,387 ;          6,756
10  ->   100,304,420 ;        391,815
11  -> 5,817,656,406 ;     22,725,222
Here is a non-permanent link to the code (https://github.com/Coding-Enthusiast/FinderOuter/blob/master/Src/FinderOuter/Services/Base58Sevice.cs#L130-L139) which may be updated in the future if I find any new information.


While I'm here, is there any wallet or tool (apart from bitaddress.org) that has ever used Base64 encoding for private keys?
Since looking at OP, FinderOuter already has an option to recover everything listed there (Base58 WIFs, Base16, mini keys, BIP38, even addresses and finally mnemonics) but the only things missing right now are Base64, BIP38 password brute force and wallet file recovery which I have plans for implementing the last two after releasing 0.5.0 with all the optimizations but not Base64 and don't know if I should add that to the list or not.


PS. your list is missing Version Extended Wallet Import Formats (WIFs) outlined in BIP-178 and has a briefly used alternative different implementation by Electrum.
It is basically the same looking WIFs that when decoded have used either a suffix or prefix (depending on the implementation) to "extend" the key with extra bits to indicate the corresponding pubkey script type derived from that key. Such keys can't be imported in wallets without converting them first.
I also have an implementation of it in Bitcoin.Net with a bunch of test vectors (first 3 WIFs are BIP-178 and the remaining 5 are Electrum specific) which also supports the Electrum script type format (scripttype:wif).
member
Activity: 180
Merit: 38
QR codes are also a widely used storage format, the well known paper wallets, but it's not in that list.

When you look at the name BITcoin then one would think that you would start with the base2 or raw 0b binary 256 Bit key format, the famous flip of a coin example, because after all that is where it got it's name, and that is what it's all about.
But it's not in there.

The base10 key format is also missing, this is the most widely used system on the planet, this is the storage format most people alive today use.
When you say any storage format it has to include them all.
It might not be a commonly used bitcoin storage format but it is certainly a valid storage format and there are several bitcoin applications that use this format internally.
This one could likely rise to the surface in a considerable amount of time, because it is already used on a global scale and people are familiar with it.
It has the potential to lower or even break the tech barrier.

It seems the format will only make it to the list if it has a known recovery url...  Cry
That's not any/all formats, like in the title, but just the ones you have a recovery url for.
Its a bit incomplete that way.
Besides it's perfectly possible to recover a damaged QR code i just don't have or need a url for that.
jr. member
Activity: 41
Merit: 41
Missing storage format: 32 bytes of raw binary.

Since random data is incompressible, it even remains unchanged if your wallet is in an (unencrypted) zip file.

If your wallet is on an old hard drive or USB stick, back up and scan a raw image of the ENTIRE DRIVE. Even if your wallet is encrypted or damaged, you might just find your private key in a chunk of an invisible cache or page file somewhere else on the disk!

As a last resort, I made a highly optimized tool to exhaustively scan a raw disk image for a binary private key, using a sliding window of 32 bytes. It still took *months* to scan a big hard drive, though.

To be sure, I recommend making a disk image using ddrescue with SystemRescueCd, then make a backup of that image, then scan the image.



Another missing storage format: ASN.1 DER.

Early versions of Bitcoin used OpenSSL and its related overengineered storage standards. It might be binary, as above, with various leading bytes, or it might be a base64 pem file (with newlines in the middle of it).

References:
https://github.com/dsharhon/bitcoinj-minimal/blob/master/core/ECKey.java#L66
https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER
https://www.hanselman.com/blog/DecodingAnSSHKeyFromPEMToBASE64ToHEXToASN1ToPrimeDecimalNumbers.aspx
newbie
Activity: 26
Merit: 3
Hi can tell me how to use the software on a macbook or send me the exact link.

You'll have to compile the code yourself to be able to run it on macOS, my released binaries are only for Linux since I want to encourage users to run it on a live Linux and offline.
The readme file of the project has the required information for building: https://github.com/Coding-Enthusiast/FinderOuter#build-from-source-code
If you have Visual Studio, all you need to do is to open the .sln file and then build.

Please post your additional questions in the project's announcement (https://bitcointalksearch.org/topic/the-finderouter-a-bitcoin-recovery-tool-v0160-2022-09-19-5214021) or open an issue on GitHub. Let's not hijack this topic anymore.

Hi From your previous answer you provided me were do i put in my private key and could kindly write the code for me as i am not a developer 
https://github.com/Coding-Enthusiast/FinderOuter/blob/85d5fdaa8d931bc8709c6e5058704ca0fca94ab4/Src/FinderOuter/Services/Base58Sevice.cs#L338
legendary
Activity: 1039
Merit: 2783
Bitcoin and C♯ Enthusiast
Hi can tell me how to use the software on a macbook or send me the exact link.

You'll have to compile the code yourself to be able to run it on macOS, my released binaries are only for Linux since I want to encourage users to run it on a live Linux and offline.
The readme file of the project has the required information for building: https://github.com/Coding-Enthusiast/FinderOuter#build-from-source-code
If you have Visual Studio, all you need to do is to open the .sln file and then build.

Please post your additional questions in the project's announcement (https://bitcointalksearch.org/topic/the-finderouter-a-bitcoin-recovery-tool-v0160-2022-09-19-5214021) or open an issue on GitHub. Let's not hijack this topic anymore.
newbie
Activity: 26
Merit: 3
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Hi Can you tell me exactly where do i input the private key with missing characters?
I haven't tried the software myself, but based on The FinderOuter (initial release 2020-01-01), you can just enter the private key in the GUI:
newbie
Activity: 26
Merit: 3
Hi Can you tell me exactly where do i input the private key with missing characters?

As right now all i am getting a return of "faee560d9"  ( i am only providing part of it this is only the beginning)?  this is after i had some code already written. for base 58 missing characters.
legendary
Activity: 1039
Merit: 2783
Bitcoin and C♯ Enthusiast
newbie
Activity: 26
Merit: 3
Hi All

in the following script where do input the private key with the missing characters? is it Bitcoin Public Key?

That is not a script, it is an old helper class to be used in different places within the program (from one of my old projects). It doesn't have any option to accept any "private" keys.
If you want to know how things are done you have to check the link I previously posted to the project called the FinderOuter.
For quick reference the entry point is here where "key" is the base58 encoded private key string containing missing characters that are replaced by a character defined by the missingChar char.
Initialize() method converts what it can from the given string to base-256 and sets the missing indexes. Then depending on key type (compressed or uncompressed) a LoopComp() or LoopUncomp() is called where different base-58 characters are placed in missing indexes and they are checked against the checksum. The rest are SHA-256 details.

hi In the link you have provided where do input the private key with the missing characters?
legendary
Activity: 1039
Merit: 2783
Bitcoin and C♯ Enthusiast
Hi All

in the following script where do input the private key with the missing characters? is it Bitcoin Public Key?

That is not a script, it is an old helper class to be used in different places within the program (from one of my old projects). It doesn't have any option to accept any "private" keys.
If you want to know how things are done you have to check the link I previously posted to the project called the FinderOuter.
For quick reference the entry point is here where "key" is the base58 encoded private key string containing missing characters that are replaced by a character defined by the missingChar char.
Initialize() method converts what it can from the given string to base-256 and sets the missing indexes. Then depending on key type (compressed or uncompressed) a LoopComp() or LoopUncomp() is called where different base-58 characters are placed in missing indexes and they are checked against the checksum. The rest are SHA-256 details.
newbie
Activity: 26
Merit: 3
Hi All

in the following script where do input the private key with the missing characters? is it Bitcoin Public Key?

Using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Numerics;
using System.Security.Cryptography;
using System.Text;


namespace CommonLibrary
{
    ///
    /// https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses
    ///

    public class BitcoinConversions
    {
        ///
        /// Satohi is the smallest amount of bitcoin, 10^-8
        ///

        public const decimal Satoshi = 0.00000001m;


        ///
        /// Characters used in Base58Encoding which is all chars excluding "0OIl"
        ///

        private const string Base58Chars = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz";




        ///
        /// Converts a hex string to intiger.
        ///

        /// hex value to convert.
        ///
        //public static UInt32 HexToInt(string hex)
        //{
        //    byte[] ba = HexToByteArray(hex);
        //    if (!BitConverter.IsLittleEndian)
        //    {
        //        ba = ba.Reverse().ToArray();
        //    }
        //    // Prevent out of range exception by adding zeros.
        //    if (ba.Length < 4)
        //    {
        //        ba = ba.Concat(new byte[4 - ba.Length]).ToArray();
        //    }
        //    return BitConverter.ToUInt32(ba, 0);
        //}




        ///
        /// Converts a hex string to bye array.
        ///

        /// hex to convert.
        /// byte array of hex
        public static byte[] HexToByteArray(string hex)
        {
            int NumberChars = hex.Length;
            byte[] bytes = new byte[NumberChars / 2];
            for (int i = 0; i < NumberChars; i += 2)
                bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
            return bytes;
        }


        ///
        /// Converts a byte array to hex string.
        ///

        /// byte array to convert.
        ///
        public static string ByteArrayToHex(byte[] b)
        {
            StringBuilder result = new StringBuilder();
            for (int i = 0; i < b.Length; i++)
            {
                result.Append(b.ToString("x2"));
            }
            return result.ToString();
        }




        ///
        /// Converts the Hash160 (RIPEMD-160) to Base58 encoded string.
        ///

        /// Hash160 string value.
        /// Base58 Encoded result.
        public static string Base58ToHash160(string data)
        {
            // Decode Base58 string to BigInteger
            BigInteger intData = 0;
            for (var i = 0; i < data.Length; i++)
            {
                intData = intData * 58 + Base58Chars.IndexOf(data);
            }


            // Encode BigInteger to byte[]
            // Leading zero bytes get encoded as leading `1` characters
            var bData = intData.ToByteArray();
            if (BitConverter.IsLittleEndian)
            {
                bData = bData.Reverse().ToArray();
            }
            bData = bData.SkipWhile(b => b == 0).ToArray();
            var hash160 = BitConverter.ToString(bData).Replace("-", "").ToLower();
            // Remove checksum (4 bytes = 8 last chars)
            return hash160.Remove(hash160.Length - Cool;
        }




        ///
        /// Computes the hash value of the specified byte array.
        ///

        /// byte array to compute hash of.
        ///
        private static byte[] DoubleShaByte(byte[] b)
        {
            using (SHA256 sha = new SHA256Managed())
            {
                byte[] hash1 = sha.ComputeHash(b);
                byte[] hash2 = sha.ComputeHash(hash1);


                return hash2;
            }
        }


        ///
        /// Returns Transaction Id for the given Raw Transaction hex.
        ///

        /// Raw Transaction hex.
        /// Transaction Id
        public static string GetTxId(string txHex)
        {
            byte[] ba = HexToByteArray(txHex);
            byte[] sha2x = DoubleShaByte(ba);


            return ByteArrayToHex(sha2x.Reverse().ToArray());
        }




        ///
        /// Converts Public Key to Hash160 (RIPEMD-160)
        /// A.K.A. payload
        ///

        /// Bitcoin Public Key
        /// hash160 (RIPEMD-160)
        public static byte[] PubKeyToHash160(string pubKey)
        {
            using (SHA256 sha = new SHA256Managed())
            {
                byte[] pubKeyBytes = HexToByteArray(pubKey);


                // 2. Perform Sha256 hashing on public key
                byte[] hash1 = sha.ComputeHash(pubKeyBytes);


                // 3. Perform RIPEMD-160 hashing on step 2
                using (RIPEMD160 r160 = new RIPEMD160Managed())
                {
                    byte[] hash2 = r160.ComputeHash(hash1);


                    return hash2;
                }
            }
        }


        ///
        /// Converts Hash160 (RIPEMD-160) to Base58 bitcoin Address.
        ///

        /// Hash160 (RIPEMD-160) bytes
        /// Base58 encoded bytes
        public static string Hash160ToBase58(byte[] hash160)
        {
            // 4. Add version byte in front of RIPEMD-160 hash (0x00 for Main Network)
            byte[] ver = { 0x00 };
            byte[] hash160Extended = ver.Concat(hash160).ToArray();


            // 5&6. Perform SHA-256 hash on the extended RIPEMD-160 result x2
            byte[] sha2x = DoubleShaByte(hash160Extended);


            // 7. The first 4 bytes are address checksum
            byte[] checkSumByte = new byte[4];
            checkSumByte[0] = sha2x[0];
            checkSumByte[1] = sha2x[1];
            checkSumByte[2] = sha2x[2];
            checkSumByte[3] = sha2x[3];


            // 8. 25-byte binary Bitcoin Address = RIPEMD-160 extended + Checksum
            byte[] hash160WithCheckSum = hash160Extended.Concat(checkSumByte).ToArray();


            return Hash160WithCheckSumToBase58(hash160WithCheckSum);
        }
        public static string Hash160ToBase58(string hash160)
        {
            byte[] hash160Bytes = HexToByteArray(hash160);
            return Hash160ToBase58(hash160Bytes);
        }


        ///
        /// Converts hash to Base58 Encoded string.
        ///

        /// 1-byte_version + hash_or_other_data + 4-byte_check_code
        /// Base58 encoded result
        public static string Hash160WithCheckSumToBase58(byte[] hash)
        {
            // Decode byte[] to BigInteger
            BigInteger intData = hash.Aggregate(0, (current, t) => current * 256 + t);


            // Encode BigInteger to Base58 string
            StringBuilder result = new StringBuilder();
            while (intData > 0)
            {
                var remainder = (int)(intData % 58);
                intData /= 58;
                result.Insert(0, Base58Chars[remainder]);
            }


            // Append '1' for each leading 0 byte
            for (var i = 0; i < hash.Length && hash == 0; i++)
            {
                result.Insert(0, '1');
            }


            return result.ToString();
        }
        public static string Hash160WithCheckSumToBase58(string hash)
        {
            byte[] hashByte = HexToByteArray(hash);
            return Hash160WithCheckSumToBase58(hashByte);
        }


        ///
        /// Converts Public Key to Base58 encoded Bitcoin Address.
        ///

        /// Bitcoin Public Key.
        /// Base58 encoded Bitcoin Address.
        public static string PubKeyToBase58(string pubKey)
        {
            byte[] hash160 = PubKeyToHash160(pubKey);


            string base58 = Hash160ToBase58(hash160);


            return base58;
        }
    }
}
jr. member
Activity: 119
Merit: 1
Thanks is there a tool i can use to recover ? as i must have missed a few characters off or could someone help me via private message  as i do not want to disclose it on here !!

You could use The FinderOuter. Compile it from source code or download the released version, run it offline. The program only has 2 options for now and you need the second option ("Missing Base58"). I believe the rest is self explanatory.
It should take a second to find the right key(s) with 3 missing characters.

Disclaimer: I am the developer of The FinderOuter. The project is brand new and in beta, I'll try to add new features (eg. not knowing the missing character locations) to it soon.

Where can I download your software ( The FinderOuter ) for testing?
legendary
Activity: 1039
Merit: 2783
Bitcoin and C♯ Enthusiast
Thanks is there a tool i can use to recover ? as i must have missed a few characters off or could someone help me via private message  as i do not want to disclose it on here !!

You could use The FinderOuter. Compile it from source code or download the released version, run it offline. The program only has 2 options for now and you need the second option ("Missing Base58"). I believe the rest is self explanatory.
It should take a second to find the right key(s) with 3 missing characters.

Disclaimer: I am the developer of The FinderOuter. The project is brand new and in beta, I'll try to add new features (eg. not knowing the missing character locations) to it soon.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Thanks is there a tool i can use to recover ? as i must have missed a few characters off or could someone help me via private message  as i do not want to disclose it on here !!
Just another warning: you shouldn't disclose your private key through PM either!
You could make an exception if you really trust the person and your balance isn't high, but as always: stay vigilant.
newbie
Activity: 26
Merit: 3
1) i have a private key which begins with K and is 49 Characters long base58
2) i have a private key which begins with K and is 54 Characters long base58
PS: If i use the the script on here and use this in github https://pastebin.com/S8WARrRn? will it work or do i have to use it on pastebin to run the script ?

no it won't work because it is hard-coded for a special case and none of the settings match your issue.
- only has a fixed number of missing characters: 5
- is an uncompressed private key (starting with 5)
- you know the location of missing characters

your private keys are compressed that start with K and their length should have been 52. and you don't seem to know the location of the missing characters.
now are you sure what you have are actually private keys? how did you lose the rest of the characters? for example if it is from the end then recovering it is going to be trivial as it would take a fraction of a second.

p.s. if my calculation is right you have to check 22.94 billion keys if you miss 3 characters and don't know their location.

Thanks is there a tool i can use to recover ? as i must have missed a few characters off or could someone help me via private message  as i do not want to disclose it on here !!
jr. member
Activity: 119
Merit: 1

Could someone kindly help as i could use the BTC's right now!!!!!
PS: If i use the the script on here and use this in github https://pastebin.com/S8WARrRn? will it work or do i have to use it on pastebin to run the script ?
Many Thanks
J

Interestingly, something similar tools for private key recovery is available under PYTHON_27, maybe on Github ?
legendary
Activity: 3430
Merit: 10505
1) i have a private key which begins with K and is 49 Characters long base58
2) i have a private key which begins with K and is 54 Characters long base58
PS: If i use the the script on here and use this in github https://pastebin.com/S8WARrRn? will it work or do i have to use it on pastebin to run the script ?

no it won't work because it is hard-coded for a special case and none of the settings match your issue.
- only has a fixed number of missing characters: 5
- is an uncompressed private key (starting with 5)
- you know the location of missing characters

your private keys are compressed that start with K and their length should have been 52. and you don't seem to know the location of the missing characters.
now are you sure what you have are actually private keys? how did you lose the rest of the characters? for example if it is from the end then recovering it is going to be trivial as it would take a fraction of a second.

p.s. if my calculation is right you have to check 22.94 billion keys if you miss 3 characters and don't know their location.
newbie
Activity: 26
Merit: 3
Hi All

I just wanted some help form anyone i am not sure the balance of my private key but that is not the main issue....

The main issues are as follows
1) i have a private key which begins with K and is 49 Characters long base58
2) i have a private key which begins with K and is 54 Characters long base58

I am not sure how to resolve this and kind the actual private key i have even wrote BTC address down wrong...

Could someone kindly help as i could use the BTC's right now!!!!!
PS: If i use the the script on here and use this in github https://pastebin.com/S8WARrRn? will it work or do i have to use it on pastebin to run the script ?
Many Thanks
J
Pages:
Jump to: