Topic: [HIRING] 1.5 BTC for wallet.dat inspection tool [Raised from 0.5 to 1 to 1.5BTC] (Read 2029 times)

Quote from: cointra on May 20, 2014, 09:37:05 AM

Fixed the passphrase segfault. Make sure you install pycryto python module or else decryption will be dead slow. Everything should work as expected now.

Great. I might not have time to test for a while, so please send me your BTC address and I'll send 0.75 BTC now and the rest after I test.

In the one test I did I got an error message:

Code:

$ python wi.py --is-encrypted --wallet=/home/user/wallet1.dat
Traceback (most recent call last):
  File "wi.py", line 1823, in 
    db_env = create_env(db_dir)
  File "wi.py", line 1158, in create_env
    (DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_THREAD | DB_RECOVER))
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- mmap: Invalid argument')

Tried it with a couple wallet files and got the same error. The user has read/write permission to the directory. I have the python-crypto package installed.

Could it be the bsddb module version you have? I tried the exact same command and got no error. I will just post a set up guide to make sure we're installing the correct python/module versions.

Actually just did a quick search, it could be python version issue as mentioned http://stackoverflow.com/questions/15958917/bsddb-4-2-52-dbinvalidargerror-22-invalid-argument

Can you try with python 2.7? If not I will downgrade to python 2.5

Actually, I figured out what is causing the problem. I'm using VirtualBox and the wallet files are on a shared drive, and for some reason BSDDB can't work with that directory. All other tools I've used can read/write fine to it, they treat it just like any other directory. My suggested fix will actually fix two issues:

- Suggested fix: Create a temporary working directory in /tmp, do the operation, remove the directory.

This addresses the issue with the shared drive and also fixes the issue that currently lots of __db.00X files are left behind in the directory after the script finishes. If you can do the operation using the temporary directory and leaving the wallet file in its current place, that is best. If you need to have the wallet file in the temporary directory, then copying it over would be ok.

And please send me your Bitcoin address so I can send you half now. I've been slow to test but I want to make sure you know I'm good for the BTC and to keep you motivated to make these fixes.

/tmp is a good idea and I will implement that. I am completely fine with receiving the whole amount once you're happy with the script, however if you really want to send half now then you can use the address on my profile i.e. 16xe1bCgnDLe27FAWHb4pmKzjiEWzMxLG8

Thanks

coiner8

member

Activity: 65

Merit: 10

Quote from: coiner8 on May 20, 2014, 08:09:15 AM

Quote from: coiner8 on May 20, 2014, 08:09:15 AM

Fixed the passphrase segfault. Make sure you install pycryto python module or else decryption will be dead slow. Everything should work as expected now.

Great. I might not have time to test for a while, so please send me your BTC address and I'll send 0.75 BTC now and the rest after I test.

In the one test I did I got an error message:

Code:

$ python wi.py --is-encrypted --wallet=/home/user/wallet1.dat
Traceback (most recent call last):
  File "wi.py", line 1823, in 
    db_env = create_env(db_dir)
  File "wi.py", line 1158, in create_env
    (DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_THREAD | DB_RECOVER))
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- mmap: Invalid argument')

Tried it with a couple wallet files and got the same error. The user has read/write permission to the directory. I have the python-crypto package installed.

Could it be the bsddb module version you have? I tried the exact same command and got no error. I will just post a set up guide to make sure we're installing the correct python/module versions.

Actually just did a quick search, it could be python version issue as mentioned http://stackoverflow.com/questions/15958917/bsddb-4-2-52-dbinvalidargerror-22-invalid-argument

Can you try with python 2.7? If not I will downgrade to python 2.5

Actually, I figured out what is causing the problem. I'm using VirtualBox and the wallet files are on a shared drive, and for some reason BSDDB can't work with that directory. All other tools I've used can read/write fine to it, they treat it just like any other directory. My suggested fix will actually fix two issues:

- Suggested fix: Create a temporary working directory in /tmp, do the operation, remove the directory.

This addresses the issue with the shared drive and also fixes the issue that currently lots of __db.00X files are left behind in the directory after the script finishes. If you can do the operation using the temporary directory and leaving the wallet file in its current place, that is best. If you need to have the wallet file in the temporary directory, then copying it over would be ok.

And please send me your Bitcoin address so I can send you half now. I've been slow to test but I want to make sure you know I'm good for the BTC and to keep you motivated to make these fixes.

cointra

member

Activity: 83

Merit: 10

Fixed the passphrase segfault. Make sure you install pycryto python module or else decryption will be dead slow. Everything should work as expected now.

Great. I might not have time to test for a while, so please send me your BTC address and I'll send 0.75 BTC now and the rest after I test.

In the one test I did I got an error message:

Code:

$ python wi.py --is-encrypted --wallet=/home/user/wallet1.dat
Traceback (most recent call last):
  File "wi.py", line 1823, in 
    db_env = create_env(db_dir)
  File "wi.py", line 1158, in create_env
    (DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_THREAD | DB_RECOVER))
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- mmap: Invalid argument')

Tried it with a couple wallet files and got the same error. The user has read/write permission to the directory. I have the python-crypto package installed.

Could it be the bsddb module version you have? I tried the exact same command and got no error. I will just post a set up guide to make sure we're installing the correct python/module versions.

Actually just did a quick search, it could be python version issue as mentioned http://stackoverflow.com/questions/15958917/bsddb-4-2-52-dbinvalidargerror-22-invalid-argument

Can you try with python 2.7? If not I will downgrade to python 2.5

coiner8

member

Activity: 65

Merit: 10

Quote from: coiner8 on May 19, 2014, 08:31:09 AM

Fixed the passphrase segfault. Make sure you install pycryto python module or else decryption will be dead slow. Everything should work as expected now.

Great. I might not have time to test for a while, so please send me your BTC address and I'll send 0.75 BTC now and the rest after I test.

In the one test I did I got an error message:

Code:

$ python wi.py --is-encrypted --wallet=/home/user/wallet1.dat
Traceback (most recent call last):
  File "wi.py", line 1823, in 
    db_env = create_env(db_dir)
  File "wi.py", line 1158, in create_env
    (DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_THREAD | DB_RECOVER))
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- mmap: Invalid argument')

Tried it with a couple wallet files and got the same error. The user has read/write permission to the directory. I have the python-crypto package installed.

cointra

member

Activity: 83

Merit: 10

Fixed the passphrase segfault. Make sure you install pycryto python module or else decryption will be dead slow. Everything should work as expected now.

cointra

member

Activity: 83

Merit: 10

Committed with all the fixes. Passphrase decryption is not working and gives segmentation fault, found the problem statement will be fixing it soon. In the meantime if you can check if all your previously mentioned issues have been resolved. Also note I have put a different blockchain explorer for zetacoin so balance should work now.

cointra

member

Activity: 83

Merit: 10

Quote from: cointra on May 18, 2014, 11:27:44 PM

Quote from: coiner8 on May 15, 2014, 11:42:32 PM

Quote from: cointra on May 18, 2014, 11:27:44 PM

Please check https://github.com/unicoinex/walletinfo

What works:

~~Everything except balance (just need to put regex for each explorer. Mostly trivial work left)~~ Balances also added.

Only included top 10 coins for this initial release. New coins will be added gradually.

So essentially the app is complete and just needs the remaining coins addition

Please note that since its in python, for the sake of completeness, I have kept the coin details within the file.

Thanks cointra! I'll be able to check it out in detail in this weekend and will post any feedback here.

Hi coiner8, did you get the chance to check it out?

Hi cointra,

I can't test it easily or fully because it uses pywallet's strange and broken --datadir. The script needs to be able to work by providing just the path to a wallet file, no --datadir option. Also pywallet's implementation only opens files named "wallet.dat", even if you specify a different file. The script should be able to handle a wallet with any filename. If it requires a working directory it should handle that internally.

--list-keys: The format of the public and private keys should be the "usual" format. For the public key, that is the format shown in the client and block explorers (i.e. 1GoK6fv4tZKXFiWL9NuHiwcwsi8JAFiwGK). For the private key, it should be the format used by the client's "dumpprivkey" and "importprivkey" commands. For long term planning perhaps adding a --format option where the default is the "usual" format would be a good way to go.
--list-keys: When I do --list-keys (or pretty much any other command) the script needs to output the warning/notice about the encryption status as the first line of stdout. See the OP for details.
--is-encrypted: This is outputting "Encrypted" even for wallets I'm sure are unencrypted. I haven't been able to get it to say "Unencrypted" yet.
--is-encrypted: When I ran this on a newly created encrypted Blackcoin wallet I got this error:

Code:

Traceback (most recent call last):
  File "wi.py", line 1685, in parse_wallet
    item_callback(type, d)
  File "wi.py", line 1323, in item_callback
    if passphrase:
NameError: global name 'passphrase' is not defined
ERROR parsing wallet.dat, type mkey
key data: mkey
key data in hex: 046d6b657901000000
value data in hex: 3086c443a92ac09f438ff7d70d9caea2d35bf0de063cf8b8e948540c95e3196f39a5a02201eb0db22e03d20f93977f00f108ac026eb2a53908a200000000a861000000

--test-passphrase:

--list-contacts: The labels should be in double quotes. If there is no label for an address, output "".
--balances: I tested with a Zetacoin wallet and it gave an error for each address: "Error reading balance for address". The URL in the script returns a 404 when I visit it in my browser.
--explore-url: Should be named --explorer-url. Also it output "None" as the last line. Not sure why that was, but each line should either be a URL or an error message.

This list of issues doesn't represent a full thorough test. The inconvenience of the --datadir/wallet.dat filename issue was enough that I only tried a few different wallet files. Once that issue is fixed I'll do a more complete test of all features with more wallets.

And despite the list of issues, I do think the tool is coming along well. It seems like it has the core routines it needs, it just needs to be polished for user friendliness.

Resolved most of your reported issues:
- is-encrypted now works
- datadir is removed (use --wallet and pass wallet path, works with any name)
- Zetacoin is an issue with zetachain.cc but I don't blame them they clearly mention that they're in beta. Will see if I can find any other zetacoin explorer
- list-contacts now show labels in double quotes

Some things still left, I will commit the whole later today once they are all done.

coiner8

member

Activity: 65

Merit: 10

Quote from: coiner8 on May 15, 2014, 11:42:32 PM

Quote from: coiner8 on May 15, 2014, 11:42:32 PM

Please check https://github.com/unicoinex/walletinfo

What works:

~~Everything except balance (just need to put regex for each explorer. Mostly trivial work left)~~ Balances also added.

Only included top 10 coins for this initial release. New coins will be added gradually.

So essentially the app is complete and just needs the remaining coins addition

Please note that since its in python, for the sake of completeness, I have kept the coin details within the file.

Thanks cointra! I'll be able to check it out in detail in this weekend and will post any feedback here.

Hi coiner8, did you get the chance to check it out?

Hi cointra,

I can't test it easily or fully because it uses pywallet's strange and broken --datadir. The script needs to be able to work by providing just the path to a wallet file, no --datadir option. Also pywallet's implementation only opens files named "wallet.dat", even if you specify a different file. The script should be able to handle a wallet with any filename. If it requires a working directory it should handle that internally.

--list-keys: The format of the public and private keys should be the "usual" format. For the public key, that is the format shown in the client and block explorers (i.e. 1GoK6fv4tZKXFiWL9NuHiwcwsi8JAFiwGK). For the private key, it should be the format used by the client's "dumpprivkey" and "importprivkey" commands. For long term planning perhaps adding a --format option where the default is the "usual" format would be a good way to go.
--list-keys: When I do --list-keys (or pretty much any other command) the script needs to output the warning/notice about the encryption status as the first line of stdout. See the OP for details.
--is-encrypted: This is outputting "Encrypted" even for wallets I'm sure are unencrypted. I haven't been able to get it to say "Unencrypted" yet.
--is-encrypted: When I ran this on a newly created encrypted Blackcoin wallet I got this error:

Code:

Traceback (most recent call last):
  File "wi.py", line 1685, in parse_wallet
    item_callback(type, d)
  File "wi.py", line 1323, in item_callback
    if passphrase:
NameError: global name 'passphrase' is not defined
ERROR parsing wallet.dat, type mkey
key data: mkey
key data in hex: 046d6b657901000000
value data in hex: 3086c443a92ac09f438ff7d70d9caea2d35bf0de063cf8b8e948540c95e3196f39a5a02201eb0db22e03d20f93977f00f108ac026eb2a53908a200000000a861000000

--test-passphrase: Same error as above.
--list-contacts: The labels should be in double quotes. If there is no label for an address, output "".
--balances: I tested with a Zetacoin wallet and it gave an error for each address: "Error reading balance for address". The URL in the script returns a 404 when I visit it in my browser.
--explore-url: Should be named --explorer-url. Also it output "None" as the last line. Not sure why that was, but each line should either be a URL or an error message.

This list of issues doesn't represent a full thorough test. The inconvenience of the --datadir/wallet.dat filename issue was enough that I only tried a few different wallet files. Once that issue is fixed I'll do a more complete test of all features with more wallets.

And despite the list of issues, I do think the tool is coming along well. It seems like it has the core routines it needs, it just needs to be polished for user friendliness.

cointra

member

Activity: 83

Merit: 10

Please check https://github.com/unicoinex/walletinfo

What works:

~~Everything except balance (just need to put regex for each explorer. Mostly trivial work left)~~ Balances also added.

Only included top 10 coins for this initial release. New coins will be added gradually.

So essentially the app is complete and just needs the remaining coins addition

Please note that since its in python, for the sake of completeness, I have kept the coin details within the file.

Thanks cointra! I'll be able to check it out in detail in this weekend and will post any feedback here.

Hi coiner8, did you get the chance to check it out?

cointra

member

Activity: 83

Merit: 10

Added more coins bringing a total to 21 coins (from the top 50 market cap list excluding the ones that don't have a blockchain or blockchain explorer)

coiner8

member

Activity: 65

Merit: 10

Please check https://github.com/unicoinex/walletinfo

What works:

~~Everything except balance (just need to put regex for each explorer. Mostly trivial work left)~~ Balances also added.

Only included top 10 coins for this initial release. New coins will be added gradually.

So essentially the app is complete and just needs the remaining coins addition

Please note that since its in python, for the sake of completeness, I have kept the coin details within the file.

Thanks cointra! I'll be able to check it out in detail in this weekend and will post any feedback here.

cointra

member

Activity: 83

Merit: 10

Please check https://github.com/unicoinex/walletinfo

What works:

~~Everything except balance (just need to put regex for each explorer. Mostly trivial work left)~~ Balances also added.

Only included top 10 coins for this initial release. New coins will be added gradually.

So essentially the app is complete and just needs the remaining coins addition

Please note that since its in python, for the sake of completeness, I have kept the coin details within the file.

coiner8

member

Activity: 65

Merit: 10

Now raised the bounty to 1.5 BTC (+0.25 BTC bonus)! That's pretty reasonable I think. $660-$770 at current price of $440/BTC. And it's a pretty fun project that will be widely used after you make it!

Any takers? All the info you should need is in the OP. Just get started, maybe put it up on Github during development and take my money!

Anyone else want to contribute to the bounty? If so state the amount in the thread.

coiner8

member

Activity: 65

Merit: 10

https://github.com/jackjack-jj/pywallet does this already.
* on second look pywallet works with block explorer. The tough part is that they want the tool to be able to work with new coins by adding coinname/blockchain url in a config file, but that would work only if all block explorers use a common api which is sadly not the case.

Not all block explorers use the same API, but there are certainly common ones. ABE is one but I've noticed other ones that are used by many coins. Also there are services that handle many coins like blockr.io and cryptocoinexplorer.com and those use the same API for each coin. As long as the tool supports adding new coins for the major APIs/sites that is fine.

coiner8

member

Activity: 65

Merit: 10

Pywallet might be a good place to start, but it doesn't handle most of the functions I want. I use pywallet a lot and I end up having to do lots of operations manually (like figuring out which coin the wallet file is for, what the otherversion is, etc.).

@lyth0s. I'll PM you too, but stating the amount you're willing to contribute here will be most helpful.

I'll raise my bounty to 1BTC (+0.25BTC bonus). Does that work for you E.exchanger? If you start off by putting it on Github there maybe be multiple people willing to chip in code & bounties.

cointra

member

Activity: 83

Merit: 10

Quote from: lyth0s on April 27, 2014, 11:24:59 PM

https://github.com/jackjack-jj/pywallet does this already.

I don't see the function listed where it can decrypt a wallet with a given password. Is there an option for that within that python script?

You can just use openssl decrypt the private keys (which is the only things encrypted in the wallet) it uses the AES-256-CBC.

pywallet works on wallets that are updated by some Bitcoin client, the OP wants a tool that works with only the wallet.dat file and not require presence of a local blockchain. The only for that is as the OP mentions by getting balance updates from block explorers. * on second look pywallet works with block explorer. The tough part is that they want the tool to be able to work with new coins by adding coinname/blockchain url in a config file, but that would work only if all block explorers use a common api which is sadly not the case.

lyth0s

legendary

Activity: 1260

Merit: 1000

World Class Cryptonaire