Is armory really that slow after all when having 10s of thousands of addresses in each wallet? While trying several of the armory functions for a system we are creating, it seems armory becomes too slow to work with when the number of addresses becomes significant.
IE. As in a previous topic i started, armoryd.py seems to take ages to start since when it calls checkWallet() and the wallet contains many addresses, it goes through each one of the addresses to check them.
The same happens in a few other cases as well like when trying to sign a transaction ("signasciitransaction"), when it tries to unlock the wallet given the passphrase, the call to "self.curWlt.unlock(securePassphrase=passwd)" which tries to unlock the wallet again has to go through each address individually taking a significant amount of time in big wallets.
Does this make armory unusable when wallets get significantly big or are there ways to overcome these limitations like disabling these checks every time or creating new wallets every X addresses to keep the wallets responsive. All considerations views on the subject are welcome, thank you.
IE. As in a previous topic i started, armoryd.py seems to take ages to start since when it calls checkWallet() and the wallet contains many addresses, it goes through each one of the addresses to check them.
The same happens in a few other cases as well like when trying to sign a transaction ("signasciitransaction"), when it tries to unlock the wallet given the passphrase, the call to "self.curWlt.unlock(securePassphrase=passwd)" which tries to unlock the wallet again has to go through each address individually taking a significant amount of time in big wallets.
Does this make armory unusable when wallets get significantly big or are there ways to overcome these limitations like disabling these checks every time or creating new wallets every X addresses to keep the wallets responsive. All considerations views on the subject are welcome, thank you.
1) armoryd's checkWallet: This code was originally designed for the UI, to run in its own thread at startup, so there was no intent to make it fast or resource efficient. It performs tons of ECDSA calculations to cover extreme edge cases of corruption. The rational is that we don't care for the time it takes as long as it verifies the wallets under every angle. When this code was factored in armoryd, the first iteration is blocking. The idea is armoryd shouldn't be allowed to start with broken wallets. Subsequent calls are threaded (checkWallet is called periodically with armoryd).
We have a couple solutions on this front. The first is that the new wallets will be a lot easier to check (BIP32 n all), so it will reduce the overall cost of the consistency check. Also there are plans to move this entire code to the C++ end with the new wallets to give it a speed buff (the current code mostly runs in Python).
2) Unlocking wallets: we did a change a while ago to only unlock necessary private keys when signing a transaction. This is in the UI, and I'm not sure it was leverage with armoryd (I haven't worked much with armoryd). This comes with a caveat anyways: if you are trying to sign coins for which the private was never computed, it's going to unlock the entire wallet the first time around.
This will change drastically with the new wallets too: encryption/decryption will be on an address entry basis, no more decrypting whole wallets in any case.
Generally all this stuff will be packed along the new back end, which itself is a lot faster and scalable. We're planning to distribute an alpha release of the new stuff early next month, to let adventurous users start toying with it and hunting down bugs we missed. If it all goes well, our next official release should have both the new backend and new wallets, at which point I would invite you to migrate your old wallets to the new ones and enjoy the improved performance.
In the short term I recommend you make sure all your private keys are computed in your signing wallets (changing the wallet encryption passphrase will do that). You could disable the wallet check in armoryd, but I don't recommend doing that, and you'd have to modify armoryd source for that purpose. There is currently no built in option to skip the check in there.
