Author

Topic: Armory - Discussion Thread - page 164. (Read 521829 times)

legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 04, 2012, 11:59:19 AM
I can add that there is definitely some kind of race condition when it comes to Armory reading the blockchain database while the Satoshi client writes to it.  I have several times seen Armory crash in that situation.  It typically happens when the Satoshi client is catching up, and I start Armory while this is happening.  Then suddenly Armory crashes.  But it is rare, I cannot reproduce it on demand, and I was stupid enough not to save info about the crash (not that it seemed particularly informative, I think it was just a segfault with no info about where in the code it happened).

Definitely, Armory doesn't like "drinking from the firehose" if Bitcoin-Qt is still updating (and 0.84.1 will detect this condition and produce a warning).  I'm about to run off for an hour, so I put a one-hour sleep command after my load-blockchain call in my code, to try to guarantee there is a new block by the time it "finishes" scanning.  I haven't had time to hunt down the code path, but at least if I can reproduce what OpenYourEyes was seeing, that's a great start.

so i just downloaded 0.84.  upon launch i get this error:  the specified module could not be found.  LoadLibrary (pythondll) failed.

also something about python27.dll error

Bizarre.  Can you try manually deleting "C:\Program Files (x86)\Armory" and then reinstall?  

Of course, I just had my first crash of Armory in Windows, after it was running for about a day!  Gah, everything looks so good, then starts falling apart!

that won't erase any wallet data will it?  i migrated in a satoshi wallet.

Wallets are stored in C:\Users\\AppData\Roaming\Armory.  You can manually back them up to another directory to be sure.  But it's intentionally kept separate so that you can install/upgrade/uninstall without affecting the wallets.
legendary
Activity: 1764
Merit: 1002
November 04, 2012, 11:40:17 AM
I can add that there is definitely some kind of race condition when it comes to Armory reading the blockchain database while the Satoshi client writes to it.  I have several times seen Armory crash in that situation.  It typically happens when the Satoshi client is catching up, and I start Armory while this is happening.  Then suddenly Armory crashes.  But it is rare, I cannot reproduce it on demand, and I was stupid enough not to save info about the crash (not that it seemed particularly informative, I think it was just a segfault with no info about where in the code it happened).

Definitely, Armory doesn't like "drinking from the firehose" if Bitcoin-Qt is still updating (and 0.84.1 will detect this condition and produce a warning).  I'm about to run off for an hour, so I put a one-hour sleep command after my load-blockchain call in my code, to try to guarantee there is a new block by the time it "finishes" scanning.  I haven't had time to hunt down the code path, but at least if I can reproduce what OpenYourEyes was seeing, that's a great start.

so i just downloaded 0.84.  upon launch i get this error:  the specified module could not be found.  LoadLibrary (pythondll) failed.

also something about python27.dll error

Bizarre.  Can you try manually deleting "C:\Program Files (x86)\Armory" and then reinstall?  

Of course, I just had my first crash of Armory in Windows, after it was running for about a day!  Gah, everything looks so good, then starts falling apart!

that won't erase any wallet data will it?  i migrated in a satoshi wallet.
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 04, 2012, 10:38:53 AM
I can add that there is definitely some kind of race condition when it comes to Armory reading the blockchain database while the Satoshi client writes to it.  I have several times seen Armory crash in that situation.  It typically happens when the Satoshi client is catching up, and I start Armory while this is happening.  Then suddenly Armory crashes.  But it is rare, I cannot reproduce it on demand, and I was stupid enough not to save info about the crash (not that it seemed particularly informative, I think it was just a segfault with no info about where in the code it happened).

Definitely, Armory doesn't like "drinking from the firehose" if Bitcoin-Qt is still updating (and 0.84.1 will detect this condition and produce a warning).  I'm about to run off for an hour, so I put a one-hour sleep command after my load-blockchain call in my code, to try to guarantee there is a new block by the time it "finishes" scanning.  I haven't had time to hunt down the code path, but at least if I can reproduce what OpenYourEyes was seeing, that's a great start.

so i just downloaded 0.84.  upon launch i get this error:  the specified module could not be found.  LoadLibrary (pythondll) failed.

also something about python27.dll error

Bizarre.  Can you try manually deleting "C:\Program Files (x86)\Armory" and then reinstall?   

Of course, I just had my first crash of Armory in Windows, after it was running for about a day!  Gah, everything looks so good, then starts falling apart!
legendary
Activity: 1764
Merit: 1002
November 04, 2012, 10:30:26 AM
so i just downloaded 0.84.  upon launch i get this error:  the specified module could not be found.  LoadLibrary (pythondll) failed.

also something about python27.dll error
hero member
Activity: 547
Merit: 500
Decor in numeris
November 04, 2012, 04:02:50 AM
I can add that there is definitely some kind of race condition when it comes to Armory reading the blockchain database while the Satoshi client writes to it.  I have several times seen Armory crash in that situation.  It typically happens when the Satoshi client is catching up, and I start Armory while this is happening.  Then suddenly Armory crashes.  But it is rare, I cannot reproduce it on demand, and I was stupid enough not to save info about the crash (not that it seemed particularly informative, I think it was just a segfault with no info about where in the code it happened).
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 03, 2012, 11:18:56 PM
Btw what happens when bitcoind receives a new block in the middle of armory's scanning at startup?

Nothing.  Armory will continue scanning the blkxxxx.dat files as they were before the new block was received, and Armory will still consider the end of the previous block to be the end of the file.  

Once Armory is done scanning, it will find the new block on the first "heartbeat" (every 1s) and update itself accordingly.

In the past hour or so, I've been trying to pin down the bug I mentioned previously, opening and closing armory in between deaths (5-10 minutes interval) in Kingdom Rush game. I've observed conclusively that armory does not re-sync afterwards when a new block is received by bitcoind in the middle of the scan at startup.


If it were truly skipping that block, then the first block it would receive after the scan would be two higher than the top block, and appear to be an orphan.  This would be an excellent explanation for what OpenYourEyes is seeing when running Armory.  It doesn't explain why I've never run into it, though...

I'll do some tracing on my own to see if can find a faulty code path.  I'm afraid it might be an OS-specific bug -- as I've started Armory, literally, more than 100 times in the last week and never witnessed this.  Or maybe I just never noticed (or my system scans the blockchain too fast for it to ever happen to me!)
full member
Activity: 160
Merit: 100
November 03, 2012, 11:12:41 PM
Btw what happens when bitcoind receives a new block in the middle of armory's scanning at startup?

Nothing.  Armory will continue scanning the blkxxxx.dat files as they were before the new block was received, and Armory will still consider the end of the previous block to be the end of the file.  

Once Armory is done scanning, it will find the new block on the first "heartbeat" (every 1s) and update itself accordingly.

In the past hour or so, I've been trying to pin down the bug I mentioned previously, opening and closing armory in between deaths (5-10 minutes interval) in Kingdom Rush game. I've observed conclusively that armory does not update itself afterwards when a new block is received by bitcoind in the middle of the scan at startup.
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 03, 2012, 09:36:28 PM
Btw what happens when bitcoind receives a new block in the middle of armory's scanning at startup?

Nothing.  Armory will continue scanning the blkxxxx.dat files as they were before the new block was received, and Armory will still consider the end of the previous block to be the end of the file. 

Once Armory is done scanning, it will find the new block on the first "heartbeat" (every 1s) and update itself accordingly.
full member
Activity: 160
Merit: 100
November 03, 2012, 09:18:30 PM
Installed 0.84 on Ubuntu 10.04 desktop. So far, the bug that sometimes made the wallet not update (block count not changing) when maybe switching workspaces in Gnome or restoring from minimized state has not appeared yet. I could not always reproduce that bug in 0.82.2, so I can't say precisely what was causing it.

The bug is still there, and it has nothing to do with switching workspaces or minimized state. This time, I have not done anything with armory yet, just leaving it open while I do other things on my desktop.

Btw what happens when bitcoind receives a new block in the middle of armory's scanning at startup?
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 03, 2012, 06:41:36 PM
Quote
Quote
By the way, if you don't mind re-downloading the blockchain in Bitcoin-Qt, I would appreciate you trying that.
I'll certainly do that, it probably won't be for a few days though as it takes close to two days to download the blockchain on my laptop.

Could it be an issue of me no leaving enough time between starting bitcoind and then armory?
I normally start bitcoind, wait 20-30 seconds and then start armory.
Just started up armory and again it was stuck on not having the correct block number. with the same repative error messages present in log that you pointed out), so I restarted armory and it's working fine.
Code:
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (ERROR) -- armoryengine.py:9143 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103

It might be best to start bitcoin, wait until you stop downloading blocks, then start armory.  If you need to download a few hours worth of blocks, waiting 30 seconds is definitely not enough.  Especially if the blockchain normally takes days to download.

Red Emerald, if you look at those info lines, you'll notice that all the block numbers are the same.  That means that something truly wacky is going on with his system:  it is detecting new blocks in the blkxxxx.dat files, but concludes that none of them extend the main chain (hence, Armory believes that block 206103 is always the top block even after reading newer blocks).  That's not a download issue, that's a real WTF issue.  One I haven't seen before...
hero member
Activity: 742
Merit: 500
November 03, 2012, 06:17:05 PM
I use Armory from git, which branch should I be testing?
Use the "threading" branch.  Although, I have been committing some updates directly to the threading so you are probably going to get a few more features than the others testing the installers I released (but hopefully no extra bugs!)

I just build the threading branch under Mac OS X Mountain Lion.  It is the first version of Armory that builds completely without any modification to the Makefile.  It is very nice that it starts right away!  I have only tried it very lightly (moved a few BTC between wallets), but so far it looks good!  Great work, Mr. -1  Smiley


You should post your steps over at https://bitcointalksearch.org/topic/building-armory-on-osx-73648

I've been having weird problems where I have to compile against the system python, but set the PYTHONPATH to brew.  I would love it if I could figure out how to compile against brew's python.
hero member
Activity: 742
Merit: 500
November 03, 2012, 06:15:45 PM
etotheipi: Thanks for the reply.

Quote from: etotheipi
FYI, Armory connects to bitcoin-qt/d via sockets (via python-twisted) on localhost over port 8333.  Is that potentially an issue?
It shouldn't be: the first rule within my iptables script allows traffic on the loopback interface.

Quote
I just looked through the log file you sent me, and I see the most interesting thing:
2012-10-30 15:10 (INFO) -- ArmoryQt.py:2339 - New Block! : 205678
...
maybe I'll figure out what's so special about your system!
If it helps: I'm running Arch Linux 32bit, encrypted (LUKS) EXT4 FS, my .bitcoin/* files are all only readable by me (chmod 700), can't think of any thing else that would differ from others.
That should be fine assuming Armory is also running as your user.

Quote
Quote
By the way, if you don't mind re-downloading the blockchain in Bitcoin-Qt, I would appreciate you trying that.
I'll certainly do that, it probably won't be for a few days though as it takes close to two days to download the blockchain on my laptop.

Could it be an issue of me no leaving enough time between starting bitcoind and then armory?
I normally start bitcoind, wait 20-30 seconds and then start armory.
Just started up armory and again it was stuck on not having the correct block number. with the same repative error messages present in log that you pointed out), so I restarted armory and it's working fine.
Code:
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (ERROR) -- armoryengine.py:9143 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103

It might be best to start bitcoin, wait until you stop downloading blocks, then start armory.  If you need to download a few hours worth of blocks, waiting 30 seconds is definitely not enough.  Especially if the blockchain normally takes days to download.
full member
Activity: 238
Merit: 100
November 03, 2012, 05:47:55 PM
etotheipi: Thanks for the reply.

Quote from: etotheipi
FYI, Armory connects to bitcoin-qt/d via sockets (via python-twisted) on localhost over port 8333.  Is that potentially an issue?
It shouldn't be: the first rule within my iptables script allows traffic on the loopback interface.

Quote
I just looked through the log file you sent me, and I see the most interesting thing:
2012-10-30 15:10 (INFO) -- ArmoryQt.py:2339 - New Block! : 205678
...
maybe I'll figure out what's so special about your system!
If it helps: I'm running Arch Linux 32bit, encrypted (LUKS) EXT4 FS, my .bitcoin/* files are all only readable by me (chmod 700), can't think of any thing else that would differ from others.

Quote
By the way, if you don't mind re-downloading the blockchain in Bitcoin-Qt, I would appreciate you trying that.
I'll certainly do that, it probably won't be for a few days though as it takes close to two days to download the blockchain on my laptop.

Could it be an issue of me no leaving enough time between starting bitcoind and then armory?
I normally start bitcoind, wait 20-30 seconds and then start armory.
Just started up armory and again it was stuck on not having the correct block number. with the same repative error messages present in log that you pointed out), so I restarted armory and it's working fine.
Code:
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (ERROR) -- armoryengine.py:9143 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
2012-11-03 22:17 (INFO) -- ArmoryQt.py:2339 - New Block! : 206103
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 03, 2012, 05:31:30 PM
I guess it's a testament to the fragility of multi-threaded programming (or my own implementation of it).

I am amazed what you can do with multithreading in Python.  Python is a fantastic language, but multithreading is definitely not one of its strengths, with stuff like the "global interpreter lock" etc coming in the way.  I am impressed!


"Multi-threading" is actually fairly easy to use in Python (doing it "correctly" is another story).  The problem with the GIL is more related to computational efficiency:  if I have two compute-heavy tasks, the GIL is not actually helping me parallelize them, and in fact may complicate and slow things down.  However, parallelizing computation is not the goal here:  it's to make sure there is CPU devoted to managing the GUI while the block data mgr is doing its thing.  In this sense , Python threading is actually quite good (separating the user context from the compute context).

Overall, I've had a very good experience with Python multi-threading, but a rough experience with the general concept of converting a complicated single-threaded app into a multi-threaded, server-client type of application (the BlockDataManager is the server, the GUI is the client).  I'm so glad it's done now...
hero member
Activity: 547
Merit: 500
Decor in numeris
November 03, 2012, 05:16:41 PM
I guess it's a testament to the fragility of multi-threaded programming (or my own implementation of it).

I am amazed what you can do with multithreading in Python.  Python is a fantastic language, but multithreading is definitely not one of its strengths, with stuff like the "global interpreter lock" etc coming in the way.  I am impressed!
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 03, 2012, 04:32:51 PM
I use Armory from git, which branch should I be testing?
Use the "threading" branch.  Although, I have been committing some updates directly to the threading so you are probably going to get a few more features than the others testing the installers I released (but hopefully no extra bugs!)

I just build the threading branch under Mac OS X Mountain Lion.  It is the first version of Armory that builds completely without any modification to the Makefile.  It is very nice that it starts right away!  I have only tried it very lightly (moved a few BTC between wallets), but so far it looks good!  Great work, Mr. -1  Smiley


After some complaining/suggestions from Red Emerald, I was able to add a OS-detection branch to the Makefile that applies the necessary mods for Mac.  Unfortunately, I haven't yet figured out how to distribute for OSX, but at least it's easier to build manually.

And I must say that I'm thrilled that it's working for everyone:  including the OSX users!   It's amazing how discouraged I was a month ago when everything was falling apart and seg faulting.  But then I found that one bug that was throwing everything out of sync, and now it seems pretty darned stable.  I guess it's a testament to the fragility of multi-threaded programming (or my own implementation of it).

So, only a couple more subtle features before 0.84.1.  This will probably be one of those "official" releases that ends up being followed the next week by a bug-fix release.  That bug-fix release will be 0.85-beta.  I have decreed it be time for beta!  Any last requests?  (small suggestions only, please... I'm not going to do anything crazy before beta).

I think one of the first things after beta will be the new, universal wallet format.  It will support compressed public keys (and thus, importing Satoshi wallets),hierarchical deterministic wallets, sub-wallets, linked wallets (for multi-sig and split-keys), much faster, merging wallets, and external comment/script store.  It will be a slow upgrade, but will open up a lot of opportunities to expand what Armory is capable of (like multi-sig!).

hero member
Activity: 547
Merit: 500
Decor in numeris
November 03, 2012, 01:34:21 PM
I use Armory from git, which branch should I be testing?
Use the "threading" branch.  Although, I have been committing some updates directly to the threading so you are probably going to get a few more features than the others testing the installers I released (but hopefully no extra bugs!)

I just build the threading branch under Mac OS X Mountain Lion.  It is the first version of Armory that builds completely without any modification to the Makefile.  It is very nice that it starts right away!  I have only tried it very lightly (moved a few BTC between wallets), but so far it looks good!  Great work, Mr. -1  Smiley

hero member
Activity: 496
Merit: 500
November 03, 2012, 12:51:06 PM
I use Armory from git, which branch should I be testing?

Use the "threading" branch.  Although, I have been committing some updates directly to the threading so you are probably going to get a few more features than the others testing the installers I released (but hopefully no extra bugs!

I had some problems before I realized I needed to rebuilt swig. Now it works great! It's so nice having it start up instantly rather than sit around waiting forever on my slow laptop.
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
November 03, 2012, 10:11:35 AM
I use Armory from git, which branch should I be testing?

Use the "threading" branch.  Although, I have been committing some updates directly to the threading so you are probably going to get a few more features than the others testing the installers I released (but hopefully no extra bugs!)
hero member
Activity: 496
Merit: 500
November 03, 2012, 10:10:15 AM
I use Armory from git, which branch should I be testing, threading? Or has it been merged into another branch already?
Jump to: