"Parsing Tx Hashes" all night, hung or wait?

PhoenixZephyrus

full member

Activity: 155

Merit: 102

Quote from: jojo69 on January 07, 2021, 05:15:32 PM

well god dammit, lol

This whole exercise was precipitated by trying to NOT re-download the entire chain, but alas, it appears you are correct and while I do have ample space, it appears that the folder is only 6 gigs in size...

ok then...see you all in a few days after I get that sorted

I did the same exact mistake...pruned it and downloaded ~85% of the blockchain and then read on here that pruned mode wont work, so had to redownload the whole thing again for 12 hours. Even after that it was stuck on a few steps, like "Scanning Transaction history" and "Parsing Tx Hashes". So, if you run into further troubles, here's a list of things you should ensure (just so that you dont have to troubleshoot for hours like I did lol):

1. Check off the auto bitcoind in armory and run Bitcoin Core yourself (and just make sure you're on latest version, I think the latest version is 0.96.5 on Github)
2. Make sure you dont run Chrome in the background with a lot of stuff open, that tends to eat up a huge chunk of RAM and Armory used up ~14GB/16GB for me.
3. Building database seems to take quite a lot of time, even on my Ryzen 7 4800H 8c/16t CPU it took ~8 mins with an all core turbo of 4.17 Ghz for the whole almost (though I was running some other tasks in the background)
4. If nothing seems to work even after making sure of these, rebuild the database. That seemed to fix everything for me.

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

well god dammit, lol

This whole exercise was precipitated by trying to NOT re-download the entire chain, but alas, it appears you are correct and while I do have ample space, it appears that the folder is only 6 gigs in size...

ok then...see you all in a few days after I get that sorted

HCP

legendary

Activity: 2086

Merit: 4361

This is your problem right here:

Quote from: https://pastebin.com/0ZCpn4ua

2021-01-07 11:02:07 (WARNING) -- SDM.py:518 - bitcoind exited, bitcoind STDERR:
2021-01-07 11:02:07 (WARNING) -- SDM.py:520 - : You need to rebuild the database using -reindex to go back to unpruned mode. This will redownload the entire blockchain.
2021-01-07 11:02:07 (WARNING) -- SDM.py:520 - Please restart with -reindex or -reindex-chainstate to recover.
2021-01-07 11:02:07 (WARNING) -- SDM.py:520 -

Quote

-ERROR - 10:59:34: (lmdb_wrapper.cpp:1503) Headers DB has no block at height: 0
-ERROR - 10:59:34: (lmdb_wrapper.cpp:1483) No headers at height 0
-ERROR - 10:59:34: (BlockchainScanner.cpp:445) Missing file map for output scan, this is unexpected
-ERROR - 10:59:34: (BlockchainScanner.cpp:447) Has the following block files:
-ERROR - 10:59:34: (BlockchainScanner.cpp:451) Was looking for id #4294967295

It seems that you're running Bitcoin Core in "pruned" mode... Armory needs Bitcoin Core to have a full, unpruned copy of the blockchain... do you have 350+Gigs of free diskspace to be able to sync the full blockchain? Huh

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

netstat -ltnp returns

Code:

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      2443/cupsd          
tcp        0      0 0.0.0.0:445             0.0.0.0:*               LISTEN      2915/smbd           
tcp        0      0 127.0.0.1:8223          0.0.0.0:*               LISTEN      6385/python2        
tcp        0      0 0.0.0.0:139             0.0.0.0:*               LISTEN      2915/smbd           
tcp        0      0 0.0.0.0:49709           0.0.0.0:*               LISTEN      1876/rpc.statd      
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1847/rpcbind        
tcp6       0      0 ::1:631                 :::*                    LISTEN      2443/cupsd          
tcp6       0      0 :::445                  :::*                    LISTEN      2915/smbd           
tcp6       0      0 :::36579                :::*                    LISTEN      1876/rpc.statd      
tcp6       0      0 :::6566                 :::*                    LISTEN      2744/saned          
tcp6       0      0 :::139                  :::*                    LISTEN      2915/smbd           
tcp6       0      0 :::111                  :::*                    LISTEN      1847/rpcbind

this while armorycplog.txt is spamming

Code:

-ERROR - : (StringSockets.cpp:351) FcgiSocket::writeAndRead FcgiError: unknown fcgi header request byte

fresh armorylog.txt https://pastebin.com/0ZCpn4ua

fresh dblog.txt

Quote

Log file opened at 10:59:34: /home/stg/.armory/dbLog.txt
-INFO - 10:59:34: (main.cpp:32) Running on 8 threads
-INFO - 10:59:34: (main.cpp:33) Ram usage level: 50
-INFO - 10:59:34: (BlockUtils.cpp:915) blkfile dir: /media/BITCORE/Blockchain/blocks
-INFO - 10:59:34: (BlockUtils.cpp:916) lmdb dir: /home/stg/.armory/databases
-INFO - 10:59:34: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 10:59:34: (BDM_Server.h:263) Listening on port 59328
-INFO - 10:59:34: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO - 10:59:34: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN - 10:59:34: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO - 10:59:34: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO - 10:59:34: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO - 10:59:34: (Blockchain.cpp:248) Organizing chain
-INFO - 10:59:34: (Blockchain.cpp:370) Organized chain in 0s
-INFO - 10:59:34: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO - 10:59:34: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 10:59:34: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO - 10:59:34: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO - 10:59:34: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking
-INFO - 10:59:34: (BDM_Server.cpp:1121) registered bdv: efd166360b74055c3d4d
-INFO - 10:59:34: (BDM_supportClasses.cpp:401) Starting address registration process
-ERROR - 10:59:34: (lmdb_wrapper.cpp:1503) Headers DB has no block at height: 0
-ERROR - 10:59:34: (lmdb_wrapper.cpp:1483) No headers at height 0
-ERROR - 10:59:34: (BlockchainScanner.cpp:445) Missing file map for output scan, this is unexpected
-ERROR - 10:59:34: (BlockchainScanner.cpp:447) Has the following block files:
-ERROR - 10:59:34: (BlockchainScanner.cpp:451) Was looking for id #4294967295

Log file opened at 11:17:09: /home/stg/.armory/dbLog.txt
-INFO - 11:17:09: (main.cpp:32) Running on 8 threads
-INFO - 11:17:09: (main.cpp:33) Ram usage level: 50
-INFO - 11:17:09: (BlockUtils.cpp:915) blkfile dir: /media/BITCORE/Blockchain/blocks
-INFO - 11:17:09: (BlockUtils.cpp:916) lmdb dir: /home/stg/.armory/databases
-INFO - 11:17:09: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 11:17:09: (BDM_Server.h:263) Listening on port 50407
-INFO - 11:17:09: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO - 11:17:09: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN - 11:17:09: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO - 11:17:09: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO - 11:17:09: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO - 11:17:09: (Blockchain.cpp:248) Organizing chain
-INFO - 11:17:09: (Blockchain.cpp:370) Organized chain in 0s
-INFO - 11:17:09: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO - 11:17:09: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 11:17:09: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO - 11:17:09: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO - 11:17:09: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking
-INFO - 11:17:09: (BDM_Server.cpp:1121) registered bdv: bbff82b8b99c0c3a6db6
-INFO - 11:17:09: (BDM_supportClasses.cpp:401) Starting address registration process
-ERROR - 11:17:09: (lmdb_wrapper.cpp:1503) Headers DB has no block at height: 0
-ERROR - 11:17:09: (lmdb_wrapper.cpp:1483) No headers at height 0
-ERROR - 11:17:09: (BlockchainScanner.cpp:445) Missing file map for output scan, this is unexpected
-ERROR - 11:17:09: (BlockchainScanner.cpp:447) Has the following block files:
-ERROR - 11:17:09: (BlockchainScanner.cpp:451) Was looking for id #4294967295

I have gone so far as to uninstall armory and reinstall it and this behavior persists.

HCP

legendary

Activity: 2086

Merit: 4361

You could try using netstat...

Code:

netstat -ltnp

Have a look through the list for the port number you're interested in... You may need to run it using sudo to see all the system processes etc.

Or maybe one of the methods here: https://www.tecmint.com/find-out-which-process-listening-on-a-particular-port/

The thing is that if you let Armory spawn ArmoryDB (ie. you just start with armory command), then it should spawn ArmoryDB with a "random" port number... and it won't use 9001.

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:52811         0.0.0.0:*               LISTEN      4965/ArmoryDB       
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:8223          0.0.0.0:*               LISTEN      4852/python2        
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$

on next run:

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:62284         0.0.0.0:*               LISTEN      5234/ArmoryDB       
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:8223          0.0.0.0:*               LISTEN      5121/python2        
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$

You can see the port changed from 52811 to 62284... however, if I spawn ArmoryDB manually using ArmoryDB from the terminal, it always tries for 9001 unless I use the --fcgi-port command:

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:9001          0.0.0.0:*               LISTEN      5314/ArmoryDB       
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$

and with ArmoryDB --fcgi-port=52828:

Code:

hcp@hcp-VirtualBox:~$ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      536/systemd-resolve 
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      4720/cupsd          
tcp        0      0 127.0.0.1:52828         0.0.0.0:*               LISTEN      5326/ArmoryDB       
tcp6       0      0 ::1:631                 :::*                    LISTEN      4720/cupsd          
hcp@hcp-VirtualBox:~$

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

I know that this is veering off topic of strictly Armory support into general linux tutoring, but is there a way to discover what this other process is that is squatting my port?

bob123

legendary

Activity: 1624

Merit: 2481

Quote from: jojo69 on January 05, 2021, 11:35:27 AM

would I also have to change the port that QT is sending on, or am I misunderstanding the whole thing?

I don't think so, no.
The problem seems to be between the 2 armory processes: armoryqt and armorydb.

Armoryqt tries to connect to armorydb on a specific port but receives a weird reply -> Some other process is listening on that port.
As far as i am aware, there is no problem between the communication of armory and bitcoin core.

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

would I also have to change the port that QT is sending on, or am I misunderstanding the whole thing?

bob123

legendary

Activity: 1624

Merit: 2481

Quote from: jojo69 on January 05, 2021, 10:53:40 AM

ok, is there a guide to modifying the port settings?

You can start armorydb with a custom port by using the --fcgi-port=X argument:

Code:

armorydb.exe --fcgi-port=X

(Replace X with your port, e.g. 55555).

Or you can add this to your config file:

Code:

fcgi-port=X

(Replace X with your port, e.g. 55555).

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

ok, is there a guide to modifying the port settings?

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

Quote from: jojo69 on January 04, 2021, 08:03:03 PM

Code:

-ERROR - : (StringSockets.cpp:351) FcgiSocket::writeAndRead FcgiError: unknown fcgi header request byte

is this a clue?

Something else than ArmoryDB is listening on the port that ArmoryQt connection to.

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

Code:

-ERROR - : (StringSockets.cpp:351) FcgiSocket::writeAndRead FcgiError: unknown fcgi header request byte

is this a clue?

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

thanks, I did not know that

bob123

legendary

Activity: 1624

Merit: 2481

Quote from: jojo69 on January 03, 2021, 01:14:57 PM

just to be clear, am I good to ctrl C the DB process and close that console? The DB doesn't need any special shutdown does it?

Ctrl + C sends a SIGINT signal to the process. This is not a termination signal like SIGKILL or SIGSTOP.
What the software does after receiving a SIGINT completely depends on the software. It can be ignored, or used as a "shutdown command".

Pressing ctrl + c does not terminate the process. It tells the process to stop / terminate itself. So, yes. you are fine using ctrl + c as its just asking the software to stop.

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

thanks HCP,

just to be clear, am I good to ctrl C the DB process and close that console? The DB doesn't need any special shutdown does it?

HCP

legendary

Activity: 2086

Merit: 4361

Quote from: jojo69 on May 23, 2020, 01:31:35 PM

So I created an armorydb.conf in /home/stg/.armory/

Code:

--dbdir="/media/BITCORE/armory"
--satoshi-datadir="/media/BITCORE/Blockchain"

That armorydb.conf looks wrong... I believe it should just be:

Code:

dbdir="/media/BITCORE/armory"
satoshi-datadir="/media/BITCORE/Blockchain"

You don't include the --'s in the .conf files. If you do, the options get ignored. It seems the comment in the pathing docs is slightly incorrect:

Quote from: jojo69 on January 02, 2021, 10:36:28 PM

that seems to return the correct pathing no?

Yep, that seems to be picking up the correct pathing when you run it from the commandline with the "/media/BITCORE/..." paths.

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

terribly sorry for the delay

Code:

stg@UD5:~
$ cd /usr/bin
stg@UD5:/usr/bin
$ ./ArmoryDB --dbdir="/media/BITCORE/armory" --satoshi-datadir="/media/BITCORE/Blockchain"
/home/stg
logging in /home/stg/.armory/dbLog.txt
-INFO  - 19:34:04: (main.cpp:32) Running on 8 threads
-INFO  - 19:34:04: (main.cpp:33) Ram usage level: 50
-INFO  - 19:34:04: (BlockUtils.cpp:915) blkfile dir: /media/BITCORE/Blockchain/blocks
-INFO  - 19:34:04: (BlockUtils.cpp:916) lmdb dir: /media/BITCORE/armory
-INFO  - 19:34:04: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 19:34:04: (BDM_Server.h:263) Listening on port 9001
-INFO  - 19:34:04: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO  - 19:34:04: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN  - 19:34:04: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO  - 19:34:04: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO  - 19:34:04: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO  - 19:34:04: (Blockchain.cpp:248) Organizing chain 
-INFO  - 19:34:04: (Blockchain.cpp:370) Organized chain in 0s
-INFO  - 19:34:04: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO  - 19:34:04: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 19:34:04: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO  - 19:34:04: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO  - 19:34:04: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking

that seems to return the correct pathing no?

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

Quote from: jojo69 on May 23, 2020, 01:31:35 PM

Is this what you wanted?

Yes. The results are surprising, it is not detecting the pathing at all. Try running the DB directly with the cli args. From /usr/bin run:

Code:

./ArmoryDB --dbdir="/media/BITCORE/armory" --satoshi-datadir="/media/BITCORE/Blockchain"

Quote

then I navigated to /usr/bin and ran ArmoryDB

If you've installed Armory you shouldn't not need to browse to the bin folder to run the binary, it should be in your path, i.e. you can run ArmoryDB, whatever path your terminal is pointing at. Since you bothered browsing to the binary's folder, run with ./ preprended to guarantee you're using the local copy of ArmoryDB. Who knows, maybe you have installation snafu. Also, try the cli args without the quotes (") and lmk.

jojo69

legendary

Activity: 3318

Merit: 4606

diamond-handed zealot

OK, I am going to be very methodical here because it is probable that I am making really dumb mistakes. Honestly, goatpig, I do not know what motivates you to continue putting up with people like me but bless you.

So I created an armorydb.conf in /home/stg/.armory/

Code:

--dbdir="/media/BITCORE/armory"

--satoshi-datadir="/media/BITCORE/Blockchain"

then I navigated to /usr/bin and ran ArmoryDB

Code:

stg@UD5:~
$ cd .
stg@UD5:~
$ cd /usr/bin
stg@UD5:/usr/bin
$ ArmoryDB
/home/stg
/home/stg
logging in /home/stg/.armory/dbLog.txt
-INFO - 11:15:14: (main.cpp:32) Running on 8 threads
-INFO - 11:15:14: (main.cpp:33) Ram usage level: 50
-INFO - 11:15:14: (BlockUtils.cpp:915) blkfile dir: /home/stg/.bitcoin/blocks
-INFO - 11:15:14: (BlockUtils.cpp:916) lmdb dir: /home/stg/.armory/databases
-INFO - 11:15:14: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 11:15:14: (BDM_Server.h:263) Listening on port 9001
-INFO - 11:15:14: (BlockUtils.cpp:1108) Executing: doInitialSyncOnLoad
-INFO - 11:15:14: (DatabaseBuilder.cpp:199) Reading headers from db
-WARN - 11:15:14: (lmdb_wrapper.cpp:1241) No headers in DB yet!
-INFO - 11:15:14: (DatabaseBuilder.cpp:238) Found 1 headers in db
-INFO - 11:15:14: (DatabaseBuilder.cpp:71) updating HEADERS db
-INFO - 11:15:14: (Blockchain.cpp:248) Organizing chain
-INFO - 11:15:14: (Blockchain.cpp:370) Organized chain in 0s
-INFO - 11:15:14: (DatabaseBuilder.cpp:76) updated HEADERS db in 0s
-INFO - 11:15:14: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 11:15:14: (DatabaseBuilder.cpp:1231) verifying txfilters integrity
-INFO - 11:15:14: (DatabaseBuilder.cpp:1314) done checking txfilters
-INFO - 11:15:14: (BDM_supportClasses.cpp:1891) Enabling zero-conf tracking

Is this what you wanted?

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

Create an armorydb.conf, put your --dbdir and --satoshi-datadir args in there, then ArmoryDB from the terminal. What do you get?

Topic: "Parsing Tx Hashes" all night, hung or wait? (Read 449 times)