Topic: Armory 0.96 is not syncing new blocks from Bitcoin Core 0.14.1 - page 2. (Read 2539 times)
For some reason the project is locked on the wrong compiler version. You need to set the compiler to v120. Right click the project, in the General tab, pick it on the platform toolset row.
1) Actually there is only one file to open. Close everything and do File -> Open Project and pick BitcoinArmory.sln in cppForSwig.
2) Once you have that done, make sure to pick Release and x64 next to the green play button named "Local Windows Debugger"
3) Now you want to introduce the artificial delay checking on disk blocks. Add this line here:
https://github.com/goatpig/BitcoinArmory/blob/master/cppForSwig/BDM_mainthread.cpp#L219
4) Now build cryptopp: in the solution explorer, pick the cryptopp project then Right Click -> Build
5) Build BlockDataManager.
This should give an ArmoryDB.exe binary in the root folder.
2) Once you have that done, make sure to pick Release and x64 next to the green play button named "Local Windows Debugger"
3) Now you want to introduce the artificial delay checking on disk blocks. Add this line here:
https://github.com/goatpig/BitcoinArmory/blob/master/cppForSwig/BDM_mainthread.cpp#L219
Quote
auto updateChainLambda = [bdm, this]()->bool
{
this_thread::sleep_for(chrono::seconds(5));
auto reorgState = bdm->readBlkFileUpdate();
if (reorgState.hasNewTop)
{
...
{
this_thread::sleep_for(chrono::seconds(5));
auto reorgState = bdm->readBlkFileUpdate();
if (reorgState.hasNewTop)
{
...
4) Now build cryptopp: in the solution explorer, pick the cryptopp project then Right Click -> Build
5) Build BlockDataManager.
This should give an ArmoryDB.exe binary in the root folder.
I've got the following error messages at step 4:
Code:
Fehler 1 error MSB8020: The build tools for v140 (Platform Toolset = 'v140') cannot be found. To build using the v140 build tools, please install v140 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Upgrade Solution...". C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.Cpp.Platform.targets 64 5 cryptopp
2 IntelliSense: Die Datei "Quelle" kann nicht geöffnet werden: "./fcgi/include/fcgiapp.h". c:\Users\..\Downloads\BitcoinArmory-0.96\cppForSwig\BDM_Server.h 19 1 BitcoinArmory_CppTests
3 IntelliSense: Die Datei "Quelle" kann nicht geöffnet werden: "./fcgi/include/fcgios.h". c:\Users\..\Downloads\BitcoinArmory-0.96\cppForSwig\BDM_Server.h 20 1 BitcoinArmory_CppTests
4 IntelliSense: Die Datei "Quelle" kann nicht geöffnet werden: "./fcgi/include/fastcgi.h". c:\Users\..\Downloads\BitcoinArmory-0.96\cppForSwig\FcgiMessage.h 16 1 BitcoinArmory_CppTests
It seems I would need some type of tool upgrade?
1) Actually there is only one file to open. Close everything and do File -> Open Project and pick BitcoinArmory.sln in cppForSwig.
2) Once you have that done, make sure to pick Release and x64 next to the green play button named "Local Windows Debugger"
3) Now you want to introduce the artificial delay checking on disk blocks. Add this line here:
https://github.com/goatpig/BitcoinArmory/blob/master/cppForSwig/BDM_mainthread.cpp#L219
4) Now build cryptopp: in the solution explorer, pick the cryptopp project then Right Click -> Build
5) Build BlockDataManager.
This should give an ArmoryDB.exe binary in the root folder.
2) Once you have that done, make sure to pick Release and x64 next to the green play button named "Local Windows Debugger"
3) Now you want to introduce the artificial delay checking on disk blocks. Add this line here:
https://github.com/goatpig/BitcoinArmory/blob/master/cppForSwig/BDM_mainthread.cpp#L219
Quote
auto updateChainLambda = [bdm, this]()->bool
{
this_thread::sleep_for(chrono::seconds(5));
auto reorgState = bdm->readBlkFileUpdate();
if (reorgState.hasNewTop)
{
...
{
this_thread::sleep_for(chrono::seconds(5));
auto reorgState = bdm->readBlkFileUpdate();
if (reorgState.hasNewTop)
{
...
4) Now build cryptopp: in the solution explorer, pick the cryptopp project then Right Click -> Build
5) Build BlockDataManager.
This should give an ArmoryDB.exe binary in the root folder.
oic. Building just the DB on Windows is easy enough. You need one piece of software:
Easy enough to find on search engines. Then you want to clone the repo. Once you have that done, I'll walk you through the rest.
Code:
Microsoft Visual Studio Express 2013 for Windows Desktop
Easy enough to find on search engines. Then you want to clone the repo. Once you have that done, I'll walk you through the rest.
Okay, I will install that and inform you when I have it. Thx!
Now I've installed Visual Studio Express 2013 WD and opened every sheet file in the 0.96 repo. What should I do next?
But I've actually no clue how I can import these sheet files into a project map and compile them to run... so it would be great if you could tell me how to do that :-)
I got one problem with version 0.95, then I was advised to let bitcoin download blockchain, instead of leaving Armory to do it. After a week of downloading I then opened Armory to let it build databases and scan transactions and so on, but I forgot to go to settings and have checked that Armory let run bitcoin in the background. This of course, made armory be "online" but with a node offline. I then procceded to let bitcoin sync again, the checked armory to run bitcoin in the background and my issue was solved. Hope that helps you.
oic. Building just the DB on Windows is easy enough. You need one piece of software:
Easy enough to find on search engines. Then you want to clone the repo. Once you have that done, I'll walk you through the rest.
Code:
Microsoft Visual Studio Express 2013 for Windows Desktop
Easy enough to find on search engines. Then you want to clone the repo. Once you have that done, I'll walk you through the rest.
Okay, I will install that and inform you when I have it. Thx!
FWIW, I'm having the same sort of problem on Ubuntu- I'm not receiving TXs that show up on the blockchain. Only thing that seems to work is to wipe out and rebuild the database, and even then it is back to the same issue again in short order.
oic. Building just the DB on Windows is easy enough. You need one piece of software:
Easy enough to find on search engines. Then you want to clone the repo. Once you have that done, I'll walk you through the rest.
Code:
Microsoft Visual Studio Express 2013 for Windows Desktop
Easy enough to find on search engines. Then you want to clone the repo. Once you have that done, I'll walk you through the rest.
Why are you running the code on Windows when you are using a Linux host? It's easier to build on Linux.
No I'm using Windows and activated the Linux subsystem as a feature in Win10 Pro.
Why are you running the code on Windows when you are using a Linux host? It's easier to build on Linux.
If you can build from source on Windows, I can give you a one liner to test my theory. It's fairly easy since you only want to build the DB.
Unfortunatly I don't know how to build from source. I wanted to do so with UASF on Bitcoin Core, but never finished because I was sceptical about the safety of an UASF. But I have activated Windows Subsystem for Linux (beta) by following this guide https://msdn.microsoft.com/en-us/commandline/wsl/install_guide.
Maybe if you could give me a link to an instruction guide I will be able to do so...
If you can build from source on Windows, I can give you a one liner to test my theory. It's fairly easy since you only want to build the DB.
This is my guess at the issue:
Armory gets its new blocks signal from the node, then reads the block data off of disk. Chances are the node pushes the block inv packet before the data is flushed on disk, and the DB ends up just reading a partial block (or only the header).
I'll try to build a unit test with a forced delay to reproduce the issue. I'll need you to test that stuff out when I push the testing builds for .1
Armory gets its new blocks signal from the node, then reads the block data off of disk. Chances are the node pushes the block inv packet before the data is flushed on disk, and the DB ends up just reading a partial block (or only the header).
I'll try to build a unit test with a forced delay to reproduce the issue. I'll need you to test that stuff out when I push the testing builds for .1
Thanks goatpig for your effort, I really appreciate it! I had joined the testing build of 0.96 but this issue occured afterwards, so I was not able to report on that before release - sorry for that.
Of course I will test your mentioned testing build as soon as it's available. This makes absolutely sense - these 256bit AES encyptions can really slow down a HDD. To compensate that I have also activated under my hardware properties the "don't let windows empty your hdd cache" policy to speed things up, but it won't get as fast as an unencrypted HDD.
Kind regards,
Mr.Vice
This is my guess at the issue:
Armory gets its new blocks signal from the node, then reads the block data off of disk. Chances are the node pushes the block inv packet before the data is flushed on disk, and the DB ends up just reading a partial block (or only the header).
I'll try to build a unit test with a forced delay to reproduce the issue. I'll need you to test that stuff out when I push the testing builds for .1
Armory gets its new blocks signal from the node, then reads the block data off of disk. Chances are the node pushes the block inv packet before the data is flushed on disk, and the DB ends up just reading a partial block (or only the header).
I'll try to build a unit test with a forced delay to reproduce the issue. I'll need you to test that stuff out when I push the testing builds for .1
This is fairly odd, I've had the code run for weeks on end. You using a HDD?
I'm using Win10 Pro x64 with the manufacturer's buildt in HDD. I also use Microsoft's BitLocker with a TPM module if that's relavant?
Actually I have a block deser exception again after rebuild & rescan today. :-S
Armory's dbLog.txt
Code:
Log file opened at 1494439626: C:\Users\..\AppData\Roaming\Armory\dbLog.txt
-INFO - 1494439626: (..\main.cpp:23) Running on 4 threads
-INFO - 1494439626: (..\main.cpp:24) Ram usage level: 4
-INFO - 1494439626: (..\BlockUtils.cpp:907) blkfile dir: C:\Users\..\AppData\Roaming\Bitcoin\blocks
-INFO - 1494439626: (..\BlockUtils.cpp:908) lmdb dir: C:\Users\..\AppData\Roaming\Armory\databases
-INFO - 1494439626: (..\lmdb_wrapper.cpp:388) Opening databases...
-INFO - 1494439628: (..\BlockUtils.cpp:1091) Executing: doInitialSyncOnLoad
-INFO - 1494439628: (..\BDM_Server.cpp:996) registered bdv: 0947bfabf926df473789
-INFO - 1494439628: (..\DatabaseBuilder.cpp:169) Reading headers from db
-INFO - 1494439650: (..\DatabaseBuilder.cpp:208) Found 465771 headers in db
-INFO - 1494439652: (..\DatabaseBuilder.cpp:51) updating HEADERS db
-INFO - 1494439653: (..\DatabaseBuilder.cpp:477) Found next block after skipping 392700bytes
-INFO - 1494439653: (..\DatabaseBuilder.cpp:268) parsed block file #863
-DEBUG - 1494439653: (..\Blockchain.cpp:242) Organizing chain
-INFO - 1494439654: (..\DatabaseBuilder.cpp:56) updated HEADERS db in 1.547s
-INFO - 1494439654: (..\DatabaseBuilder.cpp:106) scanning new blocks from #465765 to #465764
-INFO - 1494439654: (..\BlockchainScanner.cpp:52) no history to scan
-INFO - 1494439654: (..\BlockchainScanner.cpp:813) no SSH to scan
-INFO - 1494439654: (..\DatabaseBuilder.cpp:156) scanned new blocks in 0.009s
-INFO - 1494439654: (..\DatabaseBuilder.cpp:160) init db in 26.057s
-INFO - 1494439654: (..\BlockUtils.cpp:1206) Enabling zero-conf tracking
-INFO - 1494439729: (..\BitcoinP2P.cpp:947) Connected to Bitcoin node
-INFO - 1494439785: (..\BlockchainScanner.cpp:650) scanned from height #465765 to #465766
-ERROR - 1494439803: (..\DatabaseBuilder.cpp:353) block deser except:
-INFO - 1494439898: (..\BlockchainScanner.cpp:650) scanned from height #465767 to #465768
-INFO - 1494439898: (..\BlockchainScanner.cpp:650) scanned from height #465769 to #465769
-ERROR - 1494439911: (..\BlockDataMap.cpp:79) merkle root mismatch!
-ERROR - 1494439911: (..\BlockDataMap.cpp:80) header has: c7d7e4297efaef06d038403f5f4102aeaf66f453ba34750898683dd153def247
-ERROR - 1494439911: (..\BlockDataMap.cpp:81) block yields: e0db443cd839aa1c07a3525a4c77c6c803e51083b23a5112aef2c77c061314bf
-ERROR - 1494439911: (..\DatabaseBuilder.cpp:353) block deser except: invalid merkle root
The strange thing is that the merkle root's mismatch blocks are always different (the hash values are never the same).
I have shut down Armory normally about 2 hours ago and set my laptop into idle mode - don't know if it's called idle mode in English but I hope you know what I mean. After restarting my Laptop and openening Armory everything loaded like it should but as you see in the log the same issue encounters every time.
I see exactly the same problem with 0.96, similar logs running with params:
"C:\Program Files (x86)\Armory\ArmoryQt.exe" --datadir=f:\armory096 --dbdir=f:\armory096
"C:\Program Files (x86)\Armory\ArmoryQt.exe" --datadir=f:\armory096 --dbdir=f:\armory096
This is fairly odd, I've had the code run for weeks on end. You using a HDD?
How long do you let Armory run for this to happen?
About 4-5 days (not in one rush). I use Armory multiple hours a day. It actually works pretty fine after rebuild and rescan, but when the block deser exception and ivalid merkle root occours it does not take too lang until Armory can not sync anymore. At this stage armorydb often can not close normally any longer (by pressing File>Close).
Recently I have deinstalled Armory from the Program Files directory and installed it in Program Files (x86) instead and I've done a rebuild and rescan again. Let's see how long it may last from now on, but when I see something unusual in the log files I will post it in this thread.
Thanks for helping me out with this!
How long do you let Armory run for this to happen?
Now I've got a block deser exception...
Code:
-WARN - 1494405602: (..\BDM_supportClasses.cpp:1891) running 10 zc parser threads
-WARN - 1494405602: (..\BDM_supportClasses.cpp:1891) running 15 zc parser threads
-WARN - 1494405602: (..\BDM_supportClasses.cpp:1891) running 20 zc parser threads
-WARN - 1494405602: (..\BDM_supportClasses.cpp:1891) running 25 zc parser threads
-WARN - 1494406900: (..\BDM_supportClasses.cpp:1891) running 30 zc parser threads
-WARN - 1494406900: (..\BDM_supportClasses.cpp:1891) running 35 zc parser threads
-ERROR - 1494406941: (..\DatabaseBuilder.cpp:353) block deser except:
-ERROR - 1494406967: (..\BlockDataMap.cpp:79) merkle root mismatch!
-ERROR - 1494406967: (..\BlockDataMap.cpp:80) header has: cf96929443598958bbef1e94bcce1c810bda9d9d0dc95d3f901c228f284f1e24
-ERROR - 1494406967: (..\BlockDataMap.cpp:81) block yields: 1c997a4b021a3675498850375b3596f79f44999a72a49db41e15ed9743927433
-ERROR - 1494406967: (..\DatabaseBuilder.cpp:353) block deser except: invalid merkle root
-ERROR - 1494407004: (..\DatabaseBuilder.cpp:353) block deser except:
-ERROR - 1494407149: (..\BitcoinP2P.cpp:1037) caught StopBlockingLoop in processDataStackThread
-INFO - 1494407149: (..\BitcoinP2P.cpp:969) Disconnected from Bitcoin node
-INFO - 1494407174: (..\BitcoinP2P.cpp:947) Connected to Bitcoin node
-WARN - 1494407330: (..\BDM_supportClasses.cpp:1891) running 40 zc parser threads
-WARN - 1494407330: (..\BDM_supportClasses.cpp:1891) running 45 zc parser threads
-WARN - 1494407330: (..\BDM_supportClasses.cpp:1891) running 50 zc parser threads
-WARN - 1494407330: (..\BDM_supportClasses.cpp:1891) running 55 zc parser threads
-WARN - 1494407349: (..\BDM_supportClasses.cpp:1891) running 60 zc parser threads
-WARN - 1494407349: (..\BDM_supportClasses.cpp:1891) running 65 zc parser threads
-WARN - 1494407349: (..\BDM_supportClasses.cpp:1891) running 70 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 75 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 80 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 85 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 90 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 95 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 100 zc parser threads
-WARN - 1494407358: (..\BDM_supportClasses.cpp:1891) running 105 zc parser threads
Jump to: