Topic: Sweep/import private key feature request - page 3. (Read 10250 times)

yep agree with DeathAndTaxes, maybe we should think that any priv key is more like a prepaid card. The company or vendor that made the card know the priv code and you only have to scratch the protecting film to redeem the value.
In theory you could save the card in case they send any amount of value in the future but we know that isn't the reality. Every time i refill my mobile credit, that card is trashed or guarded few days just in case. Same thing could be done with "untrusted" priv keys, save them in a separate cache file that you can purge regularly.

For private keys that are known to be secure, I want the option to import them as first-class citizens in my wallet (think a securely-generated vanity address I want to be using).  For private keys given to me by someone else, I want to transfer the funds to a new key in my wallet, but I want to keep a list of those private keys that have been transferred from, so that if any more funds ever shows up in any of them, I can also transfer those funds to my wallet.

I think part of the inherent problem is that the Satoshi client shows transactions and your total balance. This is fundamentally wrong. It should show all your key pairs and the funds that are in each. When you send a payment, you choose one or more keys to send the funds from. If it was done this way, each key/address pair could be marked as generated in the client or imported.

If I have a bunch of USD in my wallet in my pants, the actual physical selection of bills I have is important to some degree. What if I want to tip someone with a small bill? What if I need to make a small purchase where large bills are not accepted? What if one of the bills might be counterfeit and worthless? All of this is important information and if shown in the client, mitigates many of the problems of how to deal with imported/swept private keys.

I misunderstood your position before your re-edit (EDIT: and Etotheipi's re-clarification). I suppose sweep and purge is at least consistent with respect to the data model. As is pure import.

Call me a pack-rat then, I abhor the idea of throwing a published address into the aether. But, my use cases are admittedly obscure. If I want to keep an untrusted key, I could just import it into a untrusted wallet. Everyone is happy. OK, I'm on board for exclusively both IMPORT and SWEEP (and forget).

Netrin, I sympathize with your arguments, but I can't concede to them.  As an application developer where I will be implicitly responsible the for well-being of a lot of peoples' money, I just simply cannot do it.  It's not to say that it shouldn't ever be done: maybe another developer would have the patience and creativity to design such a system and add more value than risk, but that's not me.  When something goes wrong, my fault or the users', I'll still be dealing with it.  And I will deal with it by putting in the 90% solution as we just discussed... so far I haven't been compelled by this discussion otherwise.

But it's not a lost cause for the argument you made.  Yes, throwing away the key is strange.  The default for "sweep" will be to for them to put in the private key, the client sweeps it, and then it's never seen again.    But that's not really the end of the story... the user still has the private key.   If they disagree with the fact that they don't have the option to sweep and import... then just run the dialog again and make it a permanent part of your wallet.  Done.   

So my client effectively "supports" that option and advanced/smart users who want this behavior can get it.  I just refuse to directly support/encourage it, because as has been described above, it opens up much more risk to the user than the value it adds.

I sure as hell would (see post above).  

You either trust a private key or you don't.  By trust I mean you are 100% absolutely certain that nobody else on this planet has ever seen the private key.

If you trust a key why do you need to sweep it?
If you don't trust a key why are you putting something you don't trust in your wallet?

Then that can and must be handled at the user interface level. Not twisting up the data model.

Flag the address in the wallet as untrusted/imported and mark it with skull and cross bones in the GUI or: "100 BTC (20 untrusted)" in the HIGHLY UNLIKELY case of this occurring. You could also perpetually import coins sent to imported addresses to new addresses, but second class keys in the data model is a backassward design.


It can and should be optional, recommended or not. It is far worse to have strange behavior ALWAYS, rather than strange behavior in the unlikely and potential malicious case of someone planting a trojan-address in your wallet.


How is this 'new class' of flagged key different from your 'second class' swept key?


That's draconian and at best it could be optional though I am sure no one would elect to throw a key away. Why should anyone ever throw a key away? On this point Satoshi would agree with me.

+1.  My thoughts exactly.  I don't really see the case where someone would want to import and sweep. Sweep is a mechanism used to secure funds from a compromised (or potentially compromised) key. If you can't guarantee the security of the key it shouldn't be imported.  If you can guarantee the security of the key why do you need to create an unnecessary transaction to sweep the value to another equally secure key in the same wallet?

It is a good idea for everyone to get into the mindset that things we don't trust completely don't belong in the wallet.  Period.

IMPORT = FULL TRUST
Example you generated an vanity address and want to integrate it into an existing wallet.  No need to transfer funds (if any) as you implicitly trust this key. Import is essentially saying "Yes I am absolutely sure there is no chance this key has been compromised and I am confident risking future coins into perpetuity with that assertion."

SWEEP = UNTRUSTED.
Example would be someone gives you a funded private key.  It has value right now but it might not in the future.  Sweeping ensures the transaction can't be reversed but the same risk that necessitated the "sweep" remains, thus the key can NEVER be trusted. There is no reason to risk future attack by integrating a compromised (even potentially compromised) key into your wallet.    Sweep is essentially saying "Hell no I don't trust this key I have no idea where it has been.  Still I would like those coins please."

While there may be edge cases I think they are rare and not worth the confusion it would create.  Two cases is already potentially confusing.  The SWEEP vs IMPORT having a clear line helps to reinforce the broader concept that your wallet (and keys inside it) should remain private and secure.  You shouldn't put insecure stuff in there and if you feel your wallet is compromised you should stop using those keys.  The risk isn't just in the here and now but all future value in those keys into perpetuity.

My only concern with this is that it does leave open an attack vector for the person who gave you the key.  By importing that key into your wallet permanently, it's funds look like they belong 100% to you.  The person who gave you that key, can then send you money to that key to make you believe you received it, and then transfer the money back to themselves once the exchange is complete.

You could argue for an "auto-sweep" function/option, but I am not fond of any kind of automated transaction... anything that moves money when the user isn't looking, especially when it might require a tx fee, is asking for people to stop trusting your software.  Not to mention, I would have to create a new class of keys/wallets in my software to handle "kind of trusted" private keys, and it would confuse users who now have to understand this new class of information.

I prefer the option of sweep once, throw away the key.  If the user is importing an untrusted key to be used once, then they should only use it once to sweep, and not leave themselves open to such games as above.  Otherwise, it leaves the door open for not-even-so-creative attackers to plant a key in your wallet that they own, and then start pulling your funds out from under you when you use it for other transactions.  By throwing it away, we avoid the mess entirely.

Why not the third option, where "SWEEP" means, "Import and immediately transfer coins"? I think "SWEEP" with second class keys is the worst of all worlds: confusing to newbies, disempowering to geeks, and complicated to maintain.

Thus with import implied, the only question is: "Do you want to transfer funds to a new trusted address?" Yes (recommended) or No (I know what I'm doing).

Yeah the way I see it there are three good choices.

Default IMPORT (option in menu to change to SWEEP).

Default SWEEP (option in menu to change to IMPORT).

Default prompt the user (w/ option "check to always xxxx").

I guess it really depends on what people are using it for.  I would recommend any wallet explain to the user what is happening (maybe w/ a help icon for more detailed info).

Something like (sweep version as summing no auto-sweep)

It depends what kind of tree structure you're using.   If you keep a tree of every address as the key, and a list of block numbers as the values, then a trie/patricia tree will have constant access time (actually, a function of the length of the addresses).  But, it requires prefix searching and C++ doesn't have this kind of tree available in the STL.

So I have to settle for map which is a binary tree ~ O(log(n)).  But in the grand scheme of things O(log(n)) is much "closer" to O(1) than O(n).  In this application, it's probably indistinguishable.

Isn't a radix hash index O(1), or constant with respect to chain size? The performance look-up penalty is the number of instances of a single address, not the total number of addresses nor blocks.

I don't disagree with you, and my goal for the first release of Armory is feature-based proof-of-concept, with no concern for how heavy the client is.  In the future, I will be converting to index-based approaches, as you suggested.  I actually already have a method for indexing all addresses, I just don't have it integrated as the "go-to" source of information for addresses.  That will be investigated for the next release.

But it doesn't change the fact that is still works right now, and even it's 30s-60s for a scan, it's a still "workable" as a client feature for users that want their money.  If someone wants to use the library in their own client for this purpose, I can help them get it integrated/linked.

Somehow, I suspect that this scan will be highly dependent on the blockchain file not being fragmented on disk, or being on flash memory.  If overly fragmented, the hard drive simply may not be able to deliver the whole file in 15 seconds regardless of how well the code performs.

I think an index is desirable regardless, because at best, an optimized blockchain scan scales O(n) and an index scales O(log n)... as the block chain gets bigger, the index scan will be faster than the full scan by increasing orders of magnitude.

This is a good discussion, and thanks for letting me know how I can add $500 income to my first release of Armory.  I hope that bounty is still valid   But I'm also interested to converge on a good way to "responsibly" implement these features...

Regardless of the Armory software itself, I have proven with the PyBtcEngine/ArmoryEngine that it's possible to scan the entire blockchain for a new address within a reasonable amount of time:

• If you're holding the entire blockchain in memory, the code can find all transactions in less than one second. 
• If it's not in memory, then I can load the blockchain from disk, index it, and scan for the address in less than 15s.

(the top-layer is Python, but I have a super-optimized C++ layer doing all the blockchain operations).

It might be possible to wrap the C++ engine and link it as a dll, purely for doing these types of scans.  It sidesteps a lot of the issues expressed in this thread (in the medium term, when it's still feasible to hold the entire blockchain), as you don't have to trust anyone more than you trust the blockchain itself in order to find and sweep funds for a given address/key. 

As for the Armory software itself, I already have the ability to import external private keys, and appropriate warnings have been displayed on the import dialog.  You can always add a new wallet just for importing untrusted private keys, though I recognize that it can still be dangerous for folks who don't quite understand the under-the-hood stuff.   The default behavior is to import the key as a permanent address in your wallet, but after reading this thread, I might make "sweeping" be the default behavior.  I might also move this feature to the "advanced" mode.  Recommendations are welcome.

Private key shouldn't be considered green.  It should be considered unverified.

A dishonest customer could give you the private key and at the same time create a transaction using that private key to transfer funds to another address.  A variant on the classic double spend.  If his transaction gets into the block first then your transaction is nullified.  Alternatively is he is a bad miner/customer he could use Finney attack involving a private key.

The only private keys which should be considered "green" would be those in physical form protected by hologramic (or other countermeasure) where you both trust the integrity of the physical key AND the issuer.  If the issuer and counter measuers can be trusted then you can assume that nobody has seen the private key except you and there is no risk of a double spend.

I asked about this on the pywallet thread to verify. Just because the wallet is unlocked for 10-15 minutes doesn't imply that pywallet can then access the wallet to import a key. No one answered or confirmed that this works. I don't think it will. Does unlocking the wallet actually re-write it to disk so that pywallet can use it? And if it does this is a security concern since an unencrypted wallet exists on disk even if briefly and potentially more so if the client crashed while the wallet was open. My assumption was that unlocking simply keeps the unencrypted wallet in memory or even just sets a flag that says it's ok to decrypt it as needed. I that case pywallet can not update the wallet.

It seems the only way now is to create a new wallet, import all your old keys (for me that includes previously created vanity keys) into it and then encrypt it again. And in that case I'd probably just leave the wallet unencrypted and use gpg manually (to encrypt it) as that gives me more flexibility.

I personally think that things like this should just be put on an advanced menu for advanced users and not have developers trying to protect the masses by dumbing down tools. Or, an rpc command is fine too. Advanced users can easily use that and novices will never do dumb things that way.

In the specific case of importing a new vanity address/key you are never worried about past transactions and rescanning the chain.

From a merchant's perspective is there any difference between these scenarios?:

1) customer sends bitcoin from a Green address; merchant can see the unconfirmed transaction in the aether.

2) customer gives private key to merchant who then sweeps and transfers coins and can see the unconfirmed transaction in the aether.

The biggest reason I want to see this in the client - at least as an RPC call - is that every bitcoin-accepting merchant will suddenly be able to accept typed private keys as payment.  A sweep RPC command will make implementing private key acceptance drop-dead simple for any site operator, since they'll just pipe the "Redeem Private Key" screen into a sweep RPC call, sweep the funds to a new address like they already generate for normal BTC transactions, and treat it like a normal incoming payment from elsewhere.

Average Joe will then be able to be a full citizen in the bitcoin economy using only paper wallets.  He can buy bitcoins at retail, and enter them at the marketplace or merchant of his choice like a gift card.  Pywallet isn't feasible for merchants to automate that on their website quite yet, since it can't add funds to a running instance of bitcoind.  Sweep will be a BIG deal for Bitcoin.

there is a way to import keys using pywallet... link