[PULL] private key and wallet export/import - page 3.

Red Emerald

hero member

Activity: 742

Merit: 500

Quote from: casascius on November 20, 2011, 06:26:21 PM

Quote from: mndrix on November 11, 2011, 07:37:15 PM

Quote from: casascius on November 11, 2011, 02:46:39 PM

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched

If key import is the use case, you could also make the index smaller by pointing at only the first block that mentions an address. If I send coins today, such an index would immediately eliminate 152,000 blocks from the search. I'd guess that imported keys typically have a short time between initial funding and sweeping to a semi-permanent address.

I would recommend the index point to all of the blocks that mention an address, otherwise I could DoS websites offering private key import simply by repeatedly trying to deposit private keys from transactions done a long time ago, even if those keys had no funds.

But there is one place I would put a derivative of the idea you just mentioned: in the -rescan command. I frequently need to import large numbers of private keys. I have patched my own bitcoind so that importprivkey doesn't rescan the wallet immediately, so it returns instantly. After importing keys, I shutdown and do a rescan. I have it so that I can do bitcoind -rescan=150000, that it will only rescan from block 150000 if I know I'm only importing recent keys... saves me tons of time importing private keys in the absence of that index. (in a nutshell, I added an optional "nSkip" parameter to ScanForWalletTransactions()). Of course, having the index would be much sweeter.

Do you have these rescan changes as another pull request?

I'm also looking forward to this pull request. I want to merge some wallets and would like to use official code.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

just try'd to import/merge a encrypted wallet, i cant neither import a privkey to a encrypted wallet, nor can i merge a unencrypted wallet into a encrypted one.
any plans on implementing this? is there a way (i google'd, couldnt find anything) to decrypt my wallet.dat so i got a unecrypted one where i can add privkeys and later reecrypt it?

greetings

casascius

vip

Activity: 1386

Merit: 1136

The Casascius 1oz 10BTC Silver Round (w/ Gold B)

Quote from: mndrix on November 11, 2011, 07:37:15 PM

Quote from: casascius on November 11, 2011, 02:46:39 PM

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched

If key import is the use case, you could also make the index smaller by pointing at only the first block that mentions an address. If I send coins today, such an index would immediately eliminate 152,000 blocks from the search. I'd guess that imported keys typically have a short time between initial funding and sweeping to a semi-permanent address.

I would recommend the index point to all of the blocks that mention an address, otherwise I could DoS websites offering private key import simply by repeatedly trying to deposit private keys from transactions done a long time ago, even if those keys had no funds.

But there is one place I would put a derivative of the idea you just mentioned: in the -rescan command. I frequently need to import large numbers of private keys. I have patched my own bitcoind so that importprivkey doesn't rescan the wallet immediately, so it returns instantly. After importing keys, I shutdown and do a rescan. I have it so that I can do bitcoind -rescan=150000, that it will only rescan from block 150000 if I know I'm only importing recent keys... saves me tons of time importing private keys in the absence of that index. (in a nutshell, I added an optional "nSkip" parameter to ScanForWalletTransactions()). Of course, having the index would be much sweeter.

runeks

legendary

Activity: 980

Merit: 1008

Adding the following line to bitcoinrpc.cpp

Code:

#include

fixes the above issue for me.

runeks

legendary

Activity: 980

Merit: 1008

I get the following error when I try to build the showwallet branch with gcc 4.6:

Code:

g++ -c -pthread -Wno-invalid-offsetof -Wformat -g -DNOPCH  -DUSE_SSL -fno-stack-protector -fstack-protector-all -Wstack-protector -Wl,-z,relro -Wl,-z,now -D_FORTIFY_SOURCE=2 -O2 -MMD -o obj/nogui/bitcoinrpc.o bitcoinrpc.cpp
bitcoinrpc.cpp: In function 'void ThreadRPCServer2(void*)':
bitcoinrpc.cpp:2213:9: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2213:26: error: expected ';' before 'certfile'
bitcoinrpc.cpp:2214:14: error: 'certfile' was not declared in this scope
bitcoinrpc.cpp:2214:49: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2215:13: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2215:32: error: 'certfile' was not declared in this scope
bitcoinrpc.cpp:2217:9: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2217:26: error: expected ';' before 'pkfile'
bitcoinrpc.cpp:2218:14: error: 'pkfile' was not declared in this scope
bitcoinrpc.cpp:2218:45: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2219:13: error: 'filesystem' has not been declared
bitcoinrpc.cpp:2219:32: error: 'pkfile' was not declared in this scope
make: *** [obj/nogui/bitcoinrpc.o] Error 1

I'm using gcc 4.6.1, libc6 2.13 and libboost 1.46 from Debian sid (the Linuxcoin distribution).
The regular bitcoin branch builds without issues.

EDIT: I just tried with gcc 4.5.2 in Ubuntu Natty as well, gives me the same error.

runeks

legendary

Activity: 980

Merit: 1008

^ I see. I really should go read more about the inner workings of Bitcoin.
Or maybe I should post questions in various threads on the questions I have about it until I understand it fully...

mndrix

vip

Activity: 447

Merit: 258

Quote from: runeks on November 11, 2011, 06:29:47 PM

If we have to search through the whole block chain in order to know the balance of a single address, how can the Bitcoin network verify or discard transactions as fast as it can?

Transactions don't spend funds from an address. They spend funds from a previous transaction. So it's using a different index than what we'd need here.

mndrix

vip

Activity: 447

Merit: 258

Quote from: casascius on November 11, 2011, 02:46:39 PM

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched

If key import is the use case, you could also make the index smaller by pointing at only the first block that mentions an address. If I send coins today, such an index would immediately eliminate 152,000 blocks from the search. I'd guess that imported keys typically have a short time between initial funding and sweeping to a semi-permanent address.

SgtSpike

legendary

Activity: 1400

Merit: 1005

Quote from: runeks on November 11, 2011, 07:13:31 PM

^ Ah, I see your point. The address is stored so there are no redundant bits, or else the block chain would take up too much space. Looks like he uses a variable called uint160, which is a base_uint class with the size of 160 bits. So it should work just like a 160 bit unsigned integer variable.

Vanity addresses can make the "take first x chars"-approach match more addresses than what would be calculated if we assumed all random address generation.
Ie., using the first 32 bits of an address as the ID doesn't give us a 1/2^32 chance of that matching two addresses (or whatever probability random address generation would lead to, I'm not that good a probability math

), because of vanity addresses. But this may not be relevant since, as casascius already said, we would just be searching through maybe two or three times as many blocks, but that would still be way way less than searching through all blocks.

Interesting, thanks for the info!

runeks

legendary

Activity: 980

Merit: 1008

^ Ah, I see your point. The address is stored so there are no redundant bits, or else the block chain would take up too much space. Looks like he uses a variable called uint160, which is a base_uint class with the size of 160 bits. So it should work just like a 160 bit unsigned integer variable.

Vanity addresses can make the "take first x chars"-approach match more addresses than what would be calculated if we assumed all random address generation.
Ie., using the first 32 bits of an address as the ID doesn't give us a 1/2^32 chance of that matching two addresses (or whatever probability random address generation would lead to, I'm not that good a probability math

), because of vanity addresses. But this may not be relevant since, as casascius already said, we would just be searching through maybe two or three times as many blocks, but that would still be way way less than searching through all blocks.

SgtSpike

legendary

Activity: 1400

Merit: 1005

Quote from: runeks on November 11, 2011, 06:29:47 PM

^ It's base58 so about 5.85 bits per char.
We should also consider vanity addresses though. Not all addresses are "random". Some are selected based on aesthetics.

If we have to search through the whole block chain in order to know the balance of a single address, how can the Bitcoin network verify or discard transactions as fast as it can? If it has to search through the whole block chain when a transaction comes in, to see if the sending address has at least the amount it wishes to send, how can the network ever handle multiple transactions per second?

Well, technically, yes, 5.85 bits. I was talking more about how those chars are stored though, since that's what is relevant when casascius says "the first 32 bits". Are they actually stored in some 6-bit formation, or are they stored in 8-bit ASCII chars, or 16-bit Unicode, or something else?

Also, how do vanity addresses change anything?

runeks

legendary

Activity: 980

Merit: 1008

^ It's base58 so about 5.85 bits per char.
We should also consider vanity addresses though. Not all addresses are "random". Some are selected based on aesthetics.

If we have to search through the whole block chain in order to know the balance of a single address, how can the Bitcoin network verify or discard transactions as fast as it can? If it has to search through the whole block chain when a transaction comes in, to see if the sending address has at least the amount it wishes to send, how can the network ever handle multiple transactions per second?

SgtSpike

legendary

Activity: 1400

Merit: 1005

Quote from: casascius on November 11, 2011, 02:46:39 PM

Quote from: btc_artist on November 11, 2011, 11:48:00 AM

Just to make sure I understand, the index would be to look up a bitcoin address and get all the blocks that that address has transactions in, so only those blocks would need to be scanned for transactions to add to your wallet, correct?

Exactly.

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched. My estimate of 20% of the block chain was very liberal in the side of "biggest impact" - practically, I think it would be far less. The index would point to all blocks that contained any kind of reference to the address, whether input, output, or whatever, possibly even in orphan blocks.

You're probably safe pulling it just based on the first 7 chars of any address. There are very few addresses that share the same first 7 chars. 8 would be virtually none. I don't know how many bits there are per char in the address though... Is it just the same 8 bits per char as ASCII?

casascius

vip

Activity: 1386

Merit: 1136

The Casascius 1oz 10BTC Silver Round (w/ Gold B)

Quote from: btc_artist on November 11, 2011, 11:48:00 AM

Just to make sure I understand, the index would be to look up a bitcoin address and get all the blocks that that address has transactions in, so only those blocks would need to be scanned for transactions to add to your wallet, correct?

Exactly.

It would be fine if this index were built on the beginning of the address (e.g. the first 32 bits) rather than the entire thing, in order to save on space, at the minor expense that a few blocks might occasionally be consulted that don't actually contain transactions for the bitcoin address being searched. My estimate of 20% of the block chain was very liberal in the side of "biggest impact" - practically, I think it would be far less. The index would point to all blocks that contained any kind of reference to the address, whether input, output, or whatever, possibly even in orphan blocks.

N.Z.

sr. member

Activity: 427

Merit: 250

I didn`t get anyway...any chance to see official import feature in 0.5? 0.6? When? Undecided

btc_artist

full member

Activity: 154

Merit: 101

Bitcoin!

Just to make sure I understand, the index would be to look up a bitcoin address and get all the blocks that that address has transactions in, so only those blocks would need to be scanned for transactions to add to your wallet, correct?

casascius

vip

Activity: 1386

Merit: 1136

The Casascius 1oz 10BTC Silver Round (w/ Gold B)

Quote from: Gavin Andresen on November 09, 2011, 06:30:07 PM

And it seems like 'sweep private key' and 'merge wallets' is really the functionality most people want, not import private key/wallet keys. The only issue I have with them is they are slow because of the rescanning of the block chain, and they may not work or may not be secure if you don't happen to have the whole block chain downloaded.

They wouldn't be slow if there were an index to speed it up. Such an index would have numerous other uses besides sweeping.

This would be an optional index that would be disabled by default, because it would consume a lot of disk space but benefit only the relatively small percentage of users who would use it. Of course, for those who use it regularly, having Bitcoin take up another 20% of disk space is far better than having to wait 3-5 minutes each time you want to import a key.

I've put up a bounty for this index to be implemented. jarpiain appears to have made considerable progress on it in his github fork of bitcoin. I probably should up the bounty to account for decrease in BTC value since it was posted.

SgtSpike

legendary

Activity: 1400

Merit: 1005

Quote from: bulanula on November 09, 2011, 07:41:03 PM

I would like functionality to actually see your private keys without additional software. Just using the main client. Is this possible now ?

That would be nice too... just so long as it requests your wallet encryption password for every private key request.

bulanula

hero member

Activity: 518

Merit: 500

I would like functionality to actually see your private keys without additional software. Just using the main client. Is this possible now ?

SgtSpike

legendary

Activity: 1400

Merit: 1005

Quote from: btc_artist on November 09, 2011, 07:35:04 PM

Thanks for the update Gavin. I'm not sure what you're referring to with "sweep private key", but the functionality I would like to see is (1) merge wallets, and (2) import private key. Merge wallets is pretty self-explanatory. Importing private keys is needed to be able to "redeem" bitcoins from offline storage such as paper wallets or physical bitcoins. Rescanning the block chain and/or waiting for the rest of the block chain to download is slow, but it's inevitable. Have a progress bar with a note saying it may take a while, and let the user interact with the rest of the application while it's working.

Yes this.

Importing private keys is a must for things like offline storage of bitcoins, etc. It may seem like most users just want merged wallets, but I know there's a bunch who need the ability to import private keys easily as well!

Topic: [PULL] private key and wallet export/import - page 3. (Read 39497 times)