Okey :-)
Will give DB_BARE a try then
Topic: Missing tx hashes (Read 634 times)
What you need is the hash of your utxos (covered by DB_BARE), and potentially the hashes of the supporting transactions (supported in DB_FULL).
If I use DB_BARE instead of DB_FULL won't I miss all tx hashes? I think function that created unsigned transaction requires tx data from database (missing hash is my orignal problem)...
Quote
Still missing that one tx hash. Any other ideas how to resolve it?
Maybe there is a way of asking DB to process specified hash?
Maybe there is a way of asking DB to process specified hash?
No, it failed to record the filter during the build. You'll have to rebuild & rescan, although at this point I'd recommend you use DB_BARE.
Thanks.
Unfortunately rescanning didn't help.
I did --rescan with log as:
After that run --rescanSSH with log:
Still missing that one tx hash. Any other ideas how to resolve it?
Maybe there is a way of asking DB to process specified hash?
Unfortunately rescanning didn't help.
I did --rescan with log as:
Code:
Log file opened at 2017:02:05 19:12:15.526
: /home/armory/.armory/dbLog.txt
-INFO - 2017:02:05 19:12:15.526
: (main.cpp:23) Running on 8 threads
-INFO - 2017:02:05 19:12:15.526
: (main.cpp:24) Ram usage level: 20
-INFO - 2017:02:05 19:12:15.526
: (BlockUtils.cpp:1338) blkfile dir: /home/armory/.bitcoin/blocks
-INFO - 2017:02:05 19:12:15.526
: (BlockUtils.cpp:1339) lmdb dir: /home/armory/.armory/databases
-INFO - 2017:02:05 19:12:15.526
: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 2017:02:05 19:12:15.527
: (BlockUtils.cpp:1530) Executing: doInitialSyncOnLoad_Rescan
-INFO - 2017:02:05 19:12:15.528
: (BitcoinP2P.cpp:790) Connected to Bitcoin node
-INFO - 2017:02:05 19:12:15.884
: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 2017:02:05 19:12:15.943
: (DatabaseBuilder.cpp:169) Reading headers from db
-INFO - 2017:02:05 19:12:18.197
: (DatabaseBuilder.cpp:208) Found 451808 headers in db
-INFO - 2017:02:05 19:12:20.041
: (DatabaseBuilder.cpp:51) updating HEADERS db
-INFO - 2017:02:05 19:12:20.093
: (DatabaseBuilder.cpp:477) Found next block after skipping 571648bytes
-INFO - 2017:02:05 19:12:20.475
: (DatabaseBuilder.cpp:268) parsed block file #763
-DEBUG - 2017:02:05 19:12:20.475
: (Blockchain.cpp:242) Organizing chain
-INFO - 2017:02:05 19:12:20.503
: (DatabaseBuilder.cpp:56) updated HEADERS db in 0.46233s
-INFO - 2017:02:05 19:12:20.734
: (DatabaseBuilder.cpp:106) scanning new blocks from #-1 to #451711
-INFO - 2017:02:05 19:12:31.799
: (BlockchainScanner.cpp:650) scanned from height #0 to #194837
-INFO - 2017:02:05 19:12:40.880
: (BlockchainScanner.cpp:650) scanned from height #194838 to #218463
-INFO - 2017:02:05 19:12:51.318
: (BlockchainScanner.cpp:650) scanned from height #218464 to #234311
-INFO - 2017:02:05 19:13:02.796
: (BlockchainScanner.cpp:650) scanned from height #234312 to #254650
-INFO - 2017:02:05 19:13:13.238
: (BlockchainScanner.cpp:650) scanned from height #254651 to #271579
-INFO - 2017:02:05 19:13:25.038
: (BlockchainScanner.cpp:650) scanned from height #271580 to #285631
-INFO - 2017:02:05 19:13:38.616
: (BlockchainScanner.cpp:650) scanned from height #285632 to #297097
-INFO - 2017:02:05 19:13:52.749
: (BlockchainScanner.cpp:650) scanned from height #297098 to #308100
-INFO - 2017:02:05 19:14:07.961
: (BlockchainScanner.cpp:650) scanned from height #308101 to #319348
-INFO - 2017:02:05 19:14:21.416
: (BlockchainScanner.cpp:650) scanned from height #319349 to #328437
-INFO - 2017:02:05 19:14:35.887
: (BlockchainScanner.cpp:650) scanned from height #328438 to #335732
-INFO - 2017:02:05 19:14:49.211
: (BlockchainScanner.cpp:650) scanned from height #335733 to #342335
-INFO - 2017:02:05 19:15:05.388
: (BlockchainScanner.cpp:650) scanned from height #342336 to #350095
-INFO - 2017:02:05 19:15:19.602
: (BlockchainScanner.cpp:650) scanned from height #350096 to #356791
-INFO - 2017:02:05 19:15:32.244
: (BlockchainScanner.cpp:650) scanned from height #356792 to #362068
-INFO - 2017:02:05 19:15:46.109
: (BlockchainScanner.cpp:650) scanned from height #362069 to #366204
-INFO - 2017:02:05 19:15:59.933
: (BlockchainScanner.cpp:650) scanned from height #366205 to #371141
-INFO - 2017:02:05 19:16:15.055
: (BlockchainScanner.cpp:650) scanned from height #371142 to #376280
-INFO - 2017:02:05 19:16:30.648
: (BlockchainScanner.cpp:650) scanned from height #376281 to #380524
-INFO - 2017:02:05 19:16:42.135
: (BlockchainScanner.cpp:650) scanned from height #380525 to #384480
-INFO - 2017:02:05 19:16:54.423
: (BlockchainScanner.cpp:650) scanned from height #384481 to #388103
-INFO - 2017:02:05 19:17:08.109
: (BlockchainScanner.cpp:650) scanned from height #388104 to #392098
-INFO - 2017:02:05 19:17:23.900
: (BlockchainScanner.cpp:650) scanned from height #392099 to #396090
-INFO - 2017:02:05 19:17:39.980
: (BlockchainScanner.cpp:650) scanned from height #396091 to #399826
-INFO - 2017:02:05 19:17:56.319
: (BlockchainScanner.cpp:650) scanned from height #399827 to #403442
-INFO - 2017:02:05 19:18:12.595
: (BlockchainScanner.cpp:650) scanned from height #403443 to #407192
-INFO - 2017:02:05 19:18:28.813
: (BlockchainScanner.cpp:650) scanned from height #407193 to #410790
-INFO - 2017:02:05 19:18:45.109
: (BlockchainScanner.cpp:650) scanned from height #410791 to #414163
-INFO - 2017:02:05 19:19:01.296
: (BlockchainScanner.cpp:650) scanned from height #414164 to #417411
-INFO - 2017:02:05 19:19:17.776
: (BlockchainScanner.cpp:650) scanned from height #417412 to #420847
-INFO - 2017:02:05 19:19:33.427
: (BlockchainScanner.cpp:650) scanned from height #420848 to #424036
-INFO - 2017:02:05 19:19:49.348
: (BlockchainScanner.cpp:650) scanned from height #424037 to #427429
-INFO - 2017:02:05 19:20:05.604
: (BlockchainScanner.cpp:650) scanned from height #427430 to #430966
-INFO - 2017:02:05 19:20:22.107
: (BlockchainScanner.cpp:650) scanned from height #430967 to #434351
-INFO - 2017:02:05 19:20:38.560
: (BlockchainScanner.cpp:650) scanned from height #434352 to #437519
-INFO - 2017:02:05 19:20:55.218
: (BlockchainScanner.cpp:650) scanned from height #437520 to #440632
-INFO - 2017:02:05 19:21:12.137
: (BlockchainScanner.cpp:650) scanned from height #440633 to #443669
-INFO - 2017:02:05 19:21:29.056
: (BlockchainScanner.cpp:650) scanned from height #443670 to #446712
-INFO - 2017:02:05 19:21:46.034
: (BlockchainScanner.cpp:650) scanned from height #446713 to #449889
-INFO - 2017:02:05 19:21:56.407
: (BlockchainScanner.cpp:650) scanned from height #449890 to #451711
-INFO - 2017:02:05 19:21:56.836
: (BlockchainScanner.cpp:225) scanned transaction history in 908.183s
-INFO - 2017:02:05 19:26:15.336
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 21:30:55.562
: (BlockchainScanner.cpp:1616) 40168 blocks hit by tx filters
-INFO - 2017:02:05 22:19:23.777
: (BlockchainScanner.cpp:1705) found 119822 missing hashes
-INFO - 2017:02:05 22:19:24.562
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 10390s
-INFO - 2017:02:05 22:19:24.706
: (DatabaseBuilder.cpp:156) scanned new blocks in 11224s
-INFO - 2017:02:05 22:19:24.706
: (DatabaseBuilder.cpp:160) init db in 11229s
-INFO - 2017:02:05 22:19:24.708
: (BlockUtils.cpp:1636) Enabling zero-conf tracking
-INFO - 2017:02:05 22:19:26.487
: (BlockchainScanner.cpp:650) scanned from height #451712 to #451725
-INFO - 2017:02:05 22:19:27.398
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:19:38.250
: (BlockchainScanner.cpp:1616) 32 blocks hit by tx filters
-INFO - 2017:02:05 22:19:43.887
: (BlockchainScanner.cpp:1705) found 167 missing hashes
-INFO - 2017:02:05 22:19:43.932
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 83.1203s
-INFO - 2017:02:05 22:27:51.145
: (BlockchainScanner.cpp:650) scanned from height #451726 to #451726
-INFO - 2017:02:05 22:27:51.297
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:27:51.506
: (BlockchainScanner.cpp:1616) 3 blocks hit by tx filters
-INFO - 2017:02:05 22:27:51.510
: (BlockchainScanner.cpp:1705) found 3 missing hashes
-INFO - 2017:02:05 22:27:51.515
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 1.45913s
-INFO - 2017:02:05 22:28:22.726
: (BDM_Server.cpp:803) registered bdv: aa00f9a7c123f30bb49e
-INFO - 2017:02:05 22:28:22.780
: (BlockchainScanner.cpp:650) scanned from height #451727 to #451727
-INFO - 2017:02:05 22:31:40.562
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:31:40.564
: (BlockchainScanner.cpp:1616) 0 blocks hit by tx filters
-INFO - 2017:02:05 22:31:40.564
: (BlockchainScanner.cpp:1705) found 0 missing hashes
-INFO - 2017:02:05 22:31:40.570
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 0.015369s
After that run --rescanSSH with log:
Code:
Log file opened at 2017:02:05 22:33:55.788
: /home/armory/.armory/dbLog.txt
-INFO - 2017:02:05 22:33:55.788
: (main.cpp:23) Running on 8 threads
-INFO - 2017:02:05 22:33:55.788
: (main.cpp:24) Ram usage level: 20
-INFO - 2017:02:05 22:33:55.788
: (BlockUtils.cpp:1338) blkfile dir: /home/armory/.bitcoin/blocks
-INFO - 2017:02:05 22:33:55.788
: (BlockUtils.cpp:1339) lmdb dir: /home/armory/.armory/databases
-INFO - 2017:02:05 22:33:55.788
: (lmdb_wrapper.cpp:388) Opening databases...
-INFO - 2017:02:05 22:33:55.790
: (BlockUtils.cpp:1550) Executing: doInitialSyncOnLoad_RescanBalance
-INFO - 2017:02:05 22:33:55.791
: (BitcoinP2P.cpp:790) Connected to Bitcoin node
-INFO - 2017:02:05 22:33:55.891
: (DatabaseBuilder.cpp:169) Reading headers from db
-INFO - 2017:02:05 22:33:59.121
: (DatabaseBuilder.cpp:208) Found 451824 headers in db
-INFO - 2017:02:05 22:34:01.002
: (DatabaseBuilder.cpp:51) updating HEADERS db
-INFO - 2017:02:05 22:34:01.225
: (DatabaseBuilder.cpp:477) Found next block after skipping 519551bytes
-INFO - 2017:02:05 22:34:01.604
: (DatabaseBuilder.cpp:268) parsed block file #763
-DEBUG - 2017:02:05 22:34:01.604
: (Blockchain.cpp:242) Organizing chain
-INFO - 2017:02:05 22:34:01.631
: (DatabaseBuilder.cpp:56) updated HEADERS db in 0.466526s
-INFO - 2017:02:05 22:34:01.863
: (DatabaseBuilder.cpp:106) scanning new blocks from #451727 to #451727
-INFO - 2017:02:05 22:34:02.185
: (BlockchainScanner.cpp:650) scanned from height #451727 to #451727
-INFO - 2017:02:05 22:39:12.418
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:39:12.419
: (BlockchainScanner.cpp:1616) 0 blocks hit by tx filters
-INFO - 2017:02:05 22:39:12.419
: (BlockchainScanner.cpp:1705) found 0 missing hashes
-INFO - 2017:02:05 22:39:12.494
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 1.0901s
-INFO - 2017:02:05 22:39:12.556
: (DatabaseBuilder.cpp:156) scanned new blocks in 253.967s
-INFO - 2017:02:05 22:39:12.556
: (DatabaseBuilder.cpp:160) init db in 259.027s
-INFO - 2017:02:05 22:39:12.556
: (BlockUtils.cpp:1636) Enabling zero-conf tracking
-INFO - 2017:02:05 22:43:39.213
: (BlockchainScanner.cpp:650) scanned from height #451728 to #451728
-INFO - 2017:02:05 22:43:39.375
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:43:41.579
: (BlockchainScanner.cpp:1616) 34 blocks hit by tx filters
-INFO - 2017:02:05 22:43:45.949
: (BlockchainScanner.cpp:1705) found 35 missing hashes
-INFO - 2017:02:05 22:43:45.950
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 19.4586s
-INFO - 2017:02:05 22:47:07.784
: (BlockchainScanner.cpp:650) scanned from height #451729 to #451729
-INFO - 2017:02:05 22:47:07.936
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:47:08.069
: (BlockchainScanner.cpp:1616) 2 blocks hit by tx filters
-INFO - 2017:02:05 22:47:08.141
: (BlockchainScanner.cpp:1705) found 2 missing hashes
-INFO - 2017:02:05 22:47:08.141
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 1.0778s
-INFO - 2017:02:05 22:48:02.252
: (BlockchainScanner.cpp:650) scanned from height #451730 to #451730
-INFO - 2017:02:05 22:48:02.400
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:48:02.528
: (BlockchainScanner.cpp:1616) 1 blocks hit by tx filters
-INFO - 2017:02:05 22:48:02.531
: (BlockchainScanner.cpp:1705) found 2 missing hashes
-INFO - 2017:02:05 22:48:02.537
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 1.00526s
-INFO - 2017:02:05 22:53:23.735
: (BDM_Server.cpp:803) registered bdv: 7ff0fb39bf3ed354c041
-INFO - 2017:02:05 22:53:23.778
: (BlockchainScanner.cpp:650) scanned from height #451731 to #451731
-INFO - 2017:02:05 22:58:25.093
: (BlockchainScanner.cpp:1560) resolving txhashes
-INFO - 2017:02:05 22:58:25.228
: (BlockchainScanner.cpp:1616) 2 blocks hit by tx filters
-INFO - 2017:02:05 22:58:25.233
: (BlockchainScanner.cpp:1705) found 2 missing hashes
-INFO - 2017:02:05 22:58:25.240
: (BlockchainScanner.cpp:1750) Resolved missing hashes in 1.05478s
-INFO - 2017:02:05 22:58:26.245
Still missing that one tx hash. Any other ideas how to resolve it?
Maybe there is a way of asking DB to process specified hash?
Quote
There is a piece of code called 'resolve missing tx hashes ' - could it be that in case tx hash is missing above function can return unexpected return value? Why resolving missing hashes is needed anyway?
Because keeping an exhaustive list of resolved hashes is too demanding for full nodes. What you need is the hash of your utxos (covered by DB_BARE), and potentially the hashes of the supporting transactions (supported in DB_FULL). That's a narrow dataset that can be processed easily on its own (hence the targeted tx hash resolver).
Quote
Second thing to consider is transaction size - tx am missing has 26 inputs and thirteen outputs, tx hash is 1cc0cd0ed909dc5f7fa75e34821c7909a6daac7db47687a41c4dfea7d4f17972
any ideas why armorydb could have problems with serializing/unserializing/storing it?
any ideas why armorydb could have problems with serializing/unserializing/storing it?
The DB isn't missing the transaction (you wouldn't see its effect on your tx history otherwise). What it is missing if the hash resolution. Transactions are not keyed by their hashes. You need to convert tx hashes in outpoints to actual block height and tx id. This is the piece the DB is missing.
Try to run the DB with --rescanSSH and see if that fixes it. If it fails, post dbLog.txt.
hi guys,
I'm running pretty active and quite big (30k addresses) wallet using armory 0.95.1 - in some cases when I'm trying to create unsigned transaction I'm getting error from Transaction.py, lines around 2169:
I'm not getting InvalidHashError, but below:
My first guess was that somehow this transaction is missing in db - and that's why its not being returned.
There is a piece of code called 'resolve missing tx hashes ' - could it be that in case tx hash is missing above function can return unexpected return value? Why resolving missing hashes is needed anyway?
Second thing to consider is transaction size - tx am missing has 26 inputs and thirteen outputs, tx hash is 1cc0cd0ed909dc5f7fa75e34821c7909a6daac7db47687a41c4dfea7d4f17972
any ideas why armorydb could have problems with serializing/unserializing/storing it?
Any help greatly appreciated
I'm running pretty active and quite big (30k addresses) wallet using armory 0.95.1 - in some cases when I'm trying to create unsigned transaction I'm getting error from Transaction.py, lines around 2169:
Code:
cppPrevTx = TheBDM.bdv().getTxByHash(txhash)
if not cppPrevTx:
raise InvalidHashError, 'Could not find the referenced tx'
I'm not getting InvalidHashError, but below:
Code:
File "/home/armory/BitcoinArmory-0.95.1/armoryengine/Transaction.py", line 2259, in createFromTxOutSelection
return self.createFromPyTx(thePyTx, pubKeyMap, txMap, p2shMap)
File "/home/armory/BitcoinArmory-0.95.1/armoryengine/Transaction.py", line 2169, in createFromPyTx
cppPrevTx = TheBDM.bdv().getTxByHash(txhash)
File "/home/armory/BitcoinArmory-0.95.1/CppBlockUtils.py", line 1508, in getTxByHash
def getTxByHash(self, *args): return _CppBlockUtils.BlockDataViewer_getTxByHash(self, *args)
RuntimeError: unexpected return value
My first guess was that somehow this transaction is missing in db - and that's why its not being returned.
There is a piece of code called 'resolve missing tx hashes ' - could it be that in case tx hash is missing above function can return unexpected return value? Why resolving missing hashes is needed anyway?
Second thing to consider is transaction size - tx am missing has 26 inputs and thirteen outputs, tx hash is 1cc0cd0ed909dc5f7fa75e34821c7909a6daac7db47687a41c4dfea7d4f17972
any ideas why armorydb could have problems with serializing/unserializing/storing it?
Any help greatly appreciated
Jump to: