Topic: Armory - Discussion Thread - page 207. (Read 521829 times)

Hi Alan,

I figure we can only juggle so many things at one times in our brains so the more you focus on one area, the less you see elsewhere.  All part of the joy of being human.

:-)

I can't believe I f***ed that up so badly!  I was sooo focused on getting some logic right for 1/2 the overall problem, that I completely botched the other half of the program along the way!  But not badly enough to notice it in testing that first half, that the second half was totally wrong!

Armory 0.70 is not ready.  It's more than not ready, I need to un-overhaul some of the blockchain logic, as I broke instant rescans for users with lots of RAM, and apparently 32-bit systems have a high risk of seg-faulting due to rescans (due to weirdness with MMAP on large files on 32-bit systems).

Sorry about that!  Hopefully it won't take me more than a day or two!   (this is what I get for solving half the problem 2 months ago, then forgetting all the context I needed when I returned...)


P.S. -- My original mmap branch never even did a remap, which was the correct solution before I recklessly changed it.  Even though I'm going back, I'm wondering why the botched solution failed on 32-bit systems and maybe someone can help:

-- MMAP'ing 1.1 GB blockchain file always succeeds on first load.  
-- Remapping 1.100001 GB blockchain file always fails.  I attempted to unmap the original and close the file, before re-opening and remapping.

I bet it has something to do with virtual address spaces... there's enough virtual RAM to map a single 1 GB file, but not two? (perhaps I'm not actually unmapping the first, and it's trying to open a second)    Does this mean that even if the blk0001.dat file splits at 2 GB and I can map it okay, that it will fail when I try to mmap two such files at the same time?   I thought, as long as each file was less than 2^32 bytes in size, that mmap would work. If so, I might want to abandon all mmap-based solutions and do a lower-level overhaul (or just switch entirely to another library for blockchain management)

Geez, the moment I hit the post button on this post, the whole thing started imploding.   See next post for details...



***Armory RAM-Reduction is Nearly Complete!  (Linux & OSX only)***

Tested and working on an Ubuntu VM with 1GB of RAM!

-------

Linux & OSX users:  Please please please pretty please help me test it!   Especially on systems with 1GB of RAM (I wonder if 512MB would work...?  I don't see why not...)  While people are testing it in *nix, I will be working on the code branch to make the same thing happen in Windows.  I don't think it should take me too long, but who knows what kind of dragons I'm going to run into...

Armory looks mostly the same, and the only difference is that it will ask you if you are willing to wait, before it executes a full blockchain rescan from disk.  Armory is very smart about it, too -- if you remove an address or wallet then re-import it, no rescan will be required.  And the C++ tools can predict when a rescan is or is not required and will ask you only if necessary.  The only downside is that you can't cancel out of the scan once you've started.  But at least you get a pretty blue "Please Wait..." window

Usually, if you hit cancel before doing a rescan, Armory will either abort the operation or tell you that balances will be incorrect until you restart.  There shouldn't be any behavior that isn't explained.   But a LOT of stuff changed under the hood, and I'm sure I didn't catch everything in my own testing.   So please help me test!

I'm declaring this to be the alpha version of Armory 0.70-beta.    While none of my testing wallets have gotten corrupted, it's always a good idea to backup before trying the new version.  In order to test it in Ubuntu, follow the regular directions on the Building Armory from Source page,  but switch to the mmap branch first.  Modified instructions are below:

• $ sudo apt-get install git-core build-essential libcrypto++-dev swig libqtcore4 libqt4-dev python-qt4 python-dev python-twisted
• $ git clone git://github.com/etotheipi/BitcoinArmory.git
• $ cd BitcoinArmory/cppForSwig
• $ git checkout mmap
• $ make swig
• $ cd ..
• $ python ArmoryQt.py

BTW:  I have not tested 0.70 in offline mode yet.  I'm actually quite sure I broke something minor, but you can still use 0.61 or less on the offline computer even if you are using 0.70 on the online computer.   I really wanted to get the testing release out as soon as is reasonable, and offline systems are not the target of this release![/s]

Oh yeah.   I should clean up that part of the display for offline mode...

That must be the auto-donation feature. 

The program automatically gives me one Satoshi for each generated wallet. How generous.
I'm running it without a blockchain.

EDIT :
Oh wait, it means I get one negative Satoshi for every created wallet.

Thanks. I posted here instead of starting a new thread because etotheipi probably already has everything it needs in Armory.

Nice idea.

Feature request:
Well not really a feature. I would like to see a client that works without a stored wallet. If a simple (ha!) client could be created that only works with paper Bitcoin bills that can print and scan, but never shows any keys on the computer screen itself. It could show an amount being scanned or transferred, and can also verify the bitbill in the blockchain, but unless there is a physical Bitcoin present, no transaction can be made. I think this would go a long way toward taking the geek factor out of the equation. I also think that people could then print their own bitbill "checks" that can be very secure with multisig protection. If someone wants to spend them online, there can be sweeping software that sends the hash string and automatically alerts the user when the transaction has acquired the required confirmations.

Coincidentally, just now I was working on merging my RAM-reduction branch into the master, which I forked a month or two ago.   Apparently, I made the the exact changes I just said I thought I had made, but I guess I made them in the RAM-reduction branch by accident instead of master.  So, that might actually be fixed in the RAM-reduction release...

Hopefully Friday, I will have version (0.70-alpha)-alpha -- the alpha version of version 0.70-alpha   (for Linux/OSX only)

Interesting.  That was not intentional:  I intended for them to be shown and marked as unconfirmed for 120 blocks.  But it seems I botched something in the unconfirmed count, anyway -- I put in logic for change-to-self and sent-to-self as automatically confirmed, but it didn't seem to work.  Therefore, if you have only one unspent output and you send 1/10th of it, your entire balance will look unconfirmed because you sent 9/10 back to yourself.  Perhaps the two problems are related. (I even have a unit test for this, I'll have to dig it up)

I'm adding that to my list of stuff to do after RAM reduction:  p2pool/mining outputs, and unconfirmed balances.

ooh noes

Ack... that's why I didn't do it yet...

Because Armory currently relies on the blk0001.dat produced by the Satoshi client (Satoshi client maintains it, Armory only reads it).  That means that what I just said I would do... won't work.  I can't wait until Armory doesn't rely on the Satoshi client anymore, but it's going to be a lot of work to cut that umbilical cord. 

One day...

perfect, i can add the -flag to a shortcut
or do you need actual access to the block file?
also, the satoshi and armory wallets NEVER mingle?

You know,  I never put this in but I don't know why not.   I can add that as a CLI option.   I'll do that tonight if I think of it.

How can I use a satoshi client on a different PC? I have a good node running elsewhere in my network. Can I pass the RPC info?

+1

I've got a (hopefully) small feature request.  Armory doesn't seem to show generated transactions until they are confirmed.  It would be nice if generated funds that were younger than 120 blocks counted into the "unconfirmed" total.

^ Very interesting, etotheipi. Also, for those of us running Linux thinking we're safer, that's not necessarily true. Here's a guy writing a USB exploit for Linux that circumvents the lock screen of a running computer. I'm sure he could have managed to copy some data over to the USB stick if that was his objective: http://www.youtube.com/watch?feature=player_detailpage&v=cNK2PDZI8fs#t=2913s

Offline wallets are not threatened by the physical security of the data moving back and forth.  I don't care if an attacker has access to the data I'm moving back and forth between the computers because it's only raw transaction data, which is all going to end up in the public blockchain anyway.  What I'm concerned about is what nasty things an attacker can do to a USB key to "autorun" and compromise the offline system when it is plugged in.  While autorun.inf is the most vulnerable way to induce arbitrary code execution, there's other creative ways attackers have been able to compromise system.

In Windows, there was a vulnerability in the ...png?...library that is used to render the icons embedded in files to be displayed in the file explorer next to the file name (for instance, you always see .exe files that have custom icons when displayed in file browser or on the desktop).  An attacker only needed to modify the icon header of the file to exploit that vulnerability which gives it root access to your system.  And simply the act of viewing the file in a file browser--you don't even click on it--will cause the malicious icon code to be executed and compromise your system.  This is not USB-specific, as it could be easily executed via email if you can get the person to simply download the file... compromised as soon as the download directory is viewed in a file explorer)

And to do this, the attacker wouldn't even be adding any files to the key: it would simply be bloating one of the existing files with a new icon header.  In the same way, he could pull a private key off the system by embedding it in slack space in one of the existing files, or injecting it into the icon header, or using steganography to hide it in an existing image file (if there were any on the USB key).  Unfortunately, private keys are small enough that they could be embedded just about anywhere.

EDIT: How could I forget NTFS alternate data streams!.  Windows was trying to improve compatibility using them, but opened up a massive security hole in their filesystem:  the ability to embed data and even executables in files in such a way that it doesn't even show up in file explorer!


I'm not suggesting any of this is likely -- it requires very resourceful attacker.  But these things are possible.  I'd rather come up with a 100% solution now and never think about it again.  That's why I started this thread.  It seems that USB-serial port cables are good way to move raw ASCII back and forth without any risk of the online computer inducing remote code execution on the offline system.  i.e. the offline system doesn't even receive a file via serial, it listens on the serial port for an ASCII-only stream of bytes that can be parsed successfully as a BIP 0010 packet.  The only potential vulnerability there would be if the "unserializeAscii" method in Armory had an exploitable vulnerability in it -- but that is ludicrously far-fetched for an text-parsing-only method that fails to do anything else unless the received data passes a sha256 checksum and matches the BIP 0010 format.

EDIT: On that last note, anything that will be processing the BIP 10 "packet" would be vulnerable, not just the BIP validation method.  If for instance, I had an "eval(packet[i:i+j])" as part of the BIP 10 parser, an attacker might be able to construct a valid BIP 0010 packet that has malicious executable [python] code between byte i and byte i+j.  This is one reason I avoid "eval()" function at all costs in all python programs.  But as you can see, the available attack vectors are cut down to virtually nothing with a few precautions.