Starting preliminary 0.94 testing - "Headless fullnode" - page 5.

fallinglantern

sr. member

Activity: 260

Merit: 251

current revision of ffreeze (hash b627160) won't build on debian 7.8 32-bit:

Quote

g++ -Icryptopp -Imdb -DUSE_CRYPTOPP -D__STDC_LIMIT_MACROS -I/usr/include/python2.7 -I/usr/include/python2.7 -std=c++11 -O2 -pipe -fPIC -c ScrAddrObj.cpp
ScrAddrObj.cpp: In member function 'void ScrAddrObj::purgeZC(const std::set&)':
ScrAddrObj.cpp:160:45: error: invalid initialization of non-const reference of type 'TxRef&' from an rvalue of type 'TxRef'
ScrAddrObj.cpp:175:46: error: invalid initialization of non-const reference of type 'TxRef&' from an rvalue of type 'TxRef'
make[1]: *** [ScrAddrObj.o] Error 1

I've reverted to 90586da for the time being which does still compile.

joel_

full member

Activity: 190

Merit: 100

Quote from: doug_armory on July 07, 2015, 04:15:47 PM

I don't think so. I believe that will come when the v2.0 wallets are ready. (We're still working on them! I'm guessing they'll be in the post-0.94 release. Sorry for the wait.)

Thank you very much for reply. Never mind, it is certainly not one of most important features. Just that yesterday I was trying to import Multibit PK's to Armory wallet, and failed miserably. And have found that I am not alone, when searching this forum for solution

Anyway, I have started to use Armory recently and ditched all those SPV/ dumbed down web wallets and so far I am loving Armory advanced features! The only thing I hate, but seems fixed with 0.94 is that it eats double the blockchain space on hdd

So let me wish to all devs of Armory good luck with making Armory the best bitcoin wallet.

doug_armory

sr. member

Activity: 255

Merit: 250

Senior Developer - Armory

Quote from: joel_ on July 07, 2015, 02:33:54 PM

Just a quick question for devs - will Armory 0.94 finally support import of compressed private keys ?
Thank you very much for reply.

I don't think so. I believe that will come when the v2.0 wallets are ready. (We're still working on them! I'm guessing they'll be in the post-0.94 release. Sorry for the wait.)

joel_

full member

Activity: 190

Merit: 100

Just a quick question for devs - will Armory 0.94 finally support import of compressed private keys ?
Thank you very much for reply.

And good luck with testing, hope I and other - less tech savvy people - might put our hands on 0.94.x binary soon! Or is it already out there ? Thanks.

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

Thanks for debug logs guys, will get to it soon.

Roy Badami

hero member

Activity: 563

Merit: 500

Just to say that 5c0497f fixes the db build/scan issue for me. Correct balances and no errors in the logs.

I did get one crash though (not during db build/scan):

Code:

Crashed Thread:  5

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000011981d001

Code:

Thread 5 Crashed:
0   _CppBlockUtils.so             	0x000000010417b006 BtcUtils::readVarInt(unsigned char const*, unsigned long, unsigned int*) + 22
1   _CppBlockUtils.so             	0x000000010417ac58 BinaryRefReader::get_var_int(unsigned char*) + 40
2   _CppBlockUtils.so             	0x000000010419951e BtcUtils::TxCalcLength(unsigned char const*, unsigned long, std::__1::vector >*, std::__1::vector >*) + 78
3   _CppBlockUtils.so             	0x00000001041bd443 Tx::unserialize(unsigned char const*, unsigned long) + 51
4   _CppBlockUtils.so             	0x00000001041bd7b0 Tx::unserialize(BinaryRefReader&) + 32
5   _CppBlockUtils.so             	0x000000010419db30 Tx::Tx(BinaryRefReader&) + 144
6   _CppBlockUtils.so             	0x00000001041a71e5 StoredHeader::unserializeFullBlock(BinaryRefReader, bool, bool) + 1237
7   _CppBlockUtils.so             	0x00000001041a8a66 StoredHeader::unserializeFullBlock(BinaryDataRef, bool, bool) + 54
8   _CppBlockUtils.so             	0x00000001041ce52e std::__1::__function::__func const&, bool)::$_1, std::__1::allocator const&, bool)::$_1>, void (BinaryDataRef const&, std::__1::pair const&, unsigned int)>::operator()(BinaryDataRef const&, std::__1::pair const&, unsigned int&&) + 510
9   _CppBlockUtils.so             	0x00000001041d279b BlockDataManager_LevelDB::BitcoinQtBlockFiles::readRawBlocksFromFile(BlockFileAccessor&, unsigned int const&, unsigned long long, unsigned long long, std::__1::function const&, unsigned int)> const&) const + 1611
10  _CppBlockUtils.so             	0x00000001041cf197 BlockDataManager_LevelDB::BitcoinQtBlockFiles::readHeaders(std::__1::pair, std::__1::function const&, unsigned int)> const&, bool) const + 423
11  _CppBlockUtils.so             	0x00000001041c6bdb BlockDataManager_LevelDB::loadBlockHeadersStartingAt(ProgressReporter&, std::__1::pair const&, bool) + 283
12  _CppBlockUtils.so             	0x00000001041cb25e BlockDataManager_LevelDB::readBlkFileUpdate(BlockDataManager_LevelDB::BlkFileUpdateCallbacks const&) + 110
13  _CppBlockUtils.so             	0x0000000104235956 BlockDataManagerThread::run() + 3798
14  _CppBlockUtils.so             	0x00000001042349c9 BlockDataManagerThread::thrun(void*) + 9
15  libsystem_pthread.dylib       	0x00007fff8af3f899 _pthread_body + 138
16  libsystem_pthread.dylib       	0x00007fff8af3f72a _pthread_start + 137
17  libsystem_pthread.dylib       	0x00007fff8af43fc9 thread_start + 13

m.fridge

newbie

Activity: 7

Merit: 0

Yes, I tried it 3 times. Although only the last time in gdb, where I got the backtrace.

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

is it happening every time?

m.fridge

newbie

Activity: 7

Merit: 0

I get a segmentation fault during the "Organizing Blockchain" step.

The last messages:

Code:

[New Thread 0xac9f1b40 (LWP 12492)]
[Thread 0xac9f1b40 (LWP 12492) exited]
[New Thread 0xac9f1b40 (LWP 12493)]
[Thread 0xac9f1b40 (LWP 12493) exited]
[New Thread 0xac9f1b40 (LWP 12494)]
[Thread 0xac9f1b40 (LWP 12494) exited]
-INFO  - 1436209883: (BlockUtils.cpp:1651) Loading block data... file 0 offset 0
[New Thread 0xac9f1b40 (LWP 12495)]
-INFO  - 1436209883: (BlockUtils.cpp:395) reading blocks from file 0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xab9b5b40 (LWP 12174)]
0xb7d87062 in ?? () from /lib/libc.so.6

And the backtrace:

Code:

#0  0xb7d87062 in ?? () from /lib/libc.so.6
#1  0xb7d852b5 in ?? () from /lib/libc.so.6
#2  0xb4bf23b0 in BlockDataManager_LevelDB::BitcoinQtBlockFiles::readRawBlocksFromFile(BlockFileAccessor&, unsigned int const&, unsigned long long, unsigned long long, std::function const&, unsigned int)> const&) const (this=0x8cba778, bfa=..., f=@0xab9b4edc: 0, blockFileOffset=0, stopBefore=2097361271, blockDataCallback=...) at BlockUtils.cpp:506
#3  0xb4bf1bd9 in BlockDataManager_LevelDB::BitcoinQtBlockFiles::readRawBlocks(LMDBBlockDatabase*, std::pair, std::pair, std::function const&, unsigned int)> const&) (this=0x8cba778, db=0x83829d0, startAt=..., stopAt=..., blockDataCallback=...) at BlockUtils.cpp:401
#4  0xb4bea795 in BlockDataManager_LevelDB::loadBlockData (this=0x8344ef0, prog=..., stopAt=..., updateDupID=true) at BlockUtils.cpp:1653
#5  0xb4be8403 in BlockDataManager_LevelDB::loadDiskState(std::function const&, bool) (this=0x8344ef0, progress=..., forceRescan=false) at BlockUtils.cpp:1395
#6  0xb4be70df in BlockDataManager_LevelDB::doInitialSyncOnLoad(std::function const&) (this=0x8344ef0, progress=...) at BlockUtils.cpp:1224
#7  0xb4c9243f in BlockDataManagerThread::run (this=0x838e7f0) at BDM_mainthread.cpp:289
#8  0xb4c9349f in BlockDataManagerThread::thrun (_self=0x838e7f0) at BDM_mainthread.cpp:446
#9  0xb7e00ffb in start_thread () from /lib/libpthread.so.0
#10 0xb7d48fee in clone () from /lib/libc.so.6

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

I have not had time to deal with your bug reports on supernode yet, working a few issues with parallel address scanning and zero conf balance. I expect I'll have time to go over your issues before the testing builds are released (that's the last item on my bug list btw).

Also, it is expected supernode users will manage their own Core instance, and I've personally never tested it with auto managed bitcoind.

Carlton Banks

legendary

Activity: 3430

Merit: 3080

Quote from: goatpig on June 29, 2015, 04:29:43 PM

Looks like some false positive being blown out of proportion. Sadly, I have some short term assignment till Tuesday so I have to put 0.94 on hold until then.

Following more testing, I think I may have been misinterpreting. Armory appears to deadlock upon std:exception file not found, but this actually seems to be attributable to some interaction between SDM mode and the supernode DB build code; running bitcoind manually will generate the exception, but not the apparent deadlock. I suspect if I'd walked away for 10 minutes then it may have continued correctly, and it's already been said that SDM needs an overhaul so I didn't try testing it with supernode again (I figured testing multi-threaded scans with SDM would be worthwhile seeing as it's the default Armory setting that new users will encounter, I can confirm that it works as well with multi-threaded scans as it did with the previous scanning logic)

Supernode DB builds fine (given the many hours that takes), that code seems robust to hours of testing, including several deliberate quit/restarts. Segfaults ensue during tx scanning.

Also tested 5c0497f03b today (rebuild+scan), no problems so far.

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

Looks like some false positive being blown out of proportion. Sadly, I have some short term assignment till Tuesday so I have to put 0.94 on hold until then.

Carlton Banks

legendary

Activity: 3430

Merit: 3080

Same issue again with supernode, some console output from a restart after a successful quit:

Code:

Do wallet checking    : True
(ERROR) ArmoryQt.py:1311 - 4 attempts to load blockchain failed.  Remove mempool.bin.
(ERROR) ArmoryQt.py:1316 - File mempool.bin does not exist. Nothing deleted.
-INFO  - 1435590364: (BlockUtils.cpp:873) blkfile dir: /home/user/.bitcoin/blocks
-INFO  - 1435590364: (BlockUtils.cpp:874) lmdb dir: /home/user/.armory/databases
-INFO  - 1435590364: (lmdb_wrapper.cpp:439) Opening databases...
-INFO  - 1435590364: (BlockUtils.cpp:1223) Executing: doInitialSyncOnLoad
-INFO  - 1435590364: (BlockUtils.cpp:1295) Total number of blk*.dat files: 291
-INFO  - 1435590364: (BlockUtils.cpp:1296) Total blockchain bytes: 38,901,027,826
-INFO  - 1435590364: (BlockUtils.cpp:1789) Reading headers from db
-INFO  - 1435590370: (BlockUtils.cpp:1815) Found 363050 headers in db
-DEBUG - 1435590370: (Blockchain.cpp:214) Organizing chain w/ rebuild
-INFO  - 1435590374: (BlockUtils.cpp:1337) Left off at file 290, offset 5090655
-INFO  - 1435590374: (BlockUtils.cpp:1340) Reading headers and building chain...
-INFO  - 1435590374: (BlockUtils.cpp:1341) Starting at block file 290 offset 5090655
-INFO  - 1435590374: (BlockUtils.cpp:1343) Block height 363035
-INFO  - 1435590374: (BlockUtils.cpp:345) parsing headers in file 290
-DEBUG - 1435590374: (Blockchain.cpp:214) Organizing chain w/ rebuild
-ERROR - 1435590454: (BlockUtils.cpp:559) std::exception (error encountered processing block at byte 99610185 file /home/user/.bitcoin/blocks/blk00143.dat, blocksize 707891)
(ERROR) Networking.py:366 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times
-INFO  - 1435590523: (BlockUtils.cpp:1380) Looking for first unrecognized block
-INFO  - 1435590523: (BlockUtils.cpp:1384) Updating Headers in DB
-INFO  - 1435590523: (BDM_mainthread.cpp:273) Stop requested detected
-INFO  - 1435590523: (BDM_mainthread.cpp:317) UI asked build/scan thread to finish
(WARNING) SDM.py:769 - Overriding not-available state. This should happen 0-5 times

Carlton Banks

legendary

Activity: 3430

Merit: 3080

Supernode

Just giving it a test. It didn't tolerate being interrupted and restarted; I gave it 2 hours to complete yesterday, and it only got ~40% finished. Starting again today produced a std:error exception relating to reading a block file. Otherwise seemingly ok, will mess around with the fully built Db later today.

Carlton Banks

legendary

Activity: 3430

Merit: 3080

Forget it completely lol, caught it again. I'm tempted to say that random instances of this warning can occur say 4-5 times consecutively for no apparent reason, likewise the inverse (no warning several times in a row).

Carlton Banks

legendary

Activity: 3430

Merit: 3080

Quote from: goatpig on June 26, 2015, 06:03:32 AM

That's really weird because the change affects code that takes place after scans, and the Gtk error message shows up early on startup (most likely when a build or scan operation is underway).

That pattern didn't apply to my later experience of the error (see above), but then again, it didn't cause any crashes or otherwise aberrant behaviour. And now it's not appearing at all. Make of it what you will (if the long term goal is still to ditch Qt 4, then it's possibly not worth pursuing).

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

That's really weird because the change affects code that takes place after scans, and the Gtk error message shows up early on startup (most likely when a build or scan operation is underway).

Carlton Banks

legendary

Activity: 3430

Merit: 3080

What was added for 875ac898db seems to have killed off the GTK+ CRITICAL assertion fail warning, for me at least. It appeared before only in runs after the initial build and scan was shutdown and restarted, but now it doesn't appear even then. Not that it was causing any problems at that point, just thought it might help improve your understand of what's causing it.

goatpig

legendary

Activity: 3766

Merit: 1364

Armory Developer

Quote from: doug_armory on June 25, 2015, 03:24:11 PM

I don't think it's that OpenSSL is painfully slow at ECDSA - I've found it to be okay, and probably a bit better than Crypto++ - as much as libsecp256k1 is specifically designed to optimize the hell out of working with the secp256k1 ECDSA curve (i.e., the curve Bitcoin uses). I believe Wladimir & Pieter have both reported 6-8x speed increases in verifications. They're just concerned about the possibility of a specially crafted Bitcoin transaction being validated in libsecp256k1 and not in OpenSSL or vice versa (i.e., a fork would occur). So, for now, libsecp256k1 is used only for signing, where it's apparently easier to test all the various use cases.

Anyway, yes, once the switch is thrown and libsecp256k1 is used for verification, BC Core should speed up quite a bit. The main contributors are doing some pretty cool things if you pay attention to the commits and the related conversations.

I should rephrase. What I meant is OpenSSL is painfully slow at secp256k1 because it has none of the specific optimizations for that curve. It is coded to sign any curve, so that's understandable.

doug_armory

sr. member

Activity: 255

Merit: 250

Senior Developer - Armory

Quote from: goatpig on June 24, 2015, 07:08:07 PM

0.94 is the first version to reduce hardware requirements (120MB disk space from 30GB+, <2GB RAM from 8+) while increasing performance AND increasing scalability.

Seriously everyone, 0.94 is shaping up nicely. I've been flying by the seat of my pants and using 0.94 regularly since the first major chances were committed. After some initial crashes (totally understandable given the nature of software development), it worked really well for me. I don't doubt that there are bugs here and there - Carlton Banks inadvertently showed us quite a few serious bugs - and I encourage everybody to bang on the test builds when they come out. I'm just saying that, for my purposes, 0.94 is quite nice and does go a bit easier on the system.

Quote from: goatpig on June 25, 2015, 11:43:08 AM

Quote from: Carlton Banks on June 25, 2015, 03:28:18 AM

Cutting the demand for CPU cycles by 6 or 8 as is claimed could cut the intial sync time by a similar factor.

I expect it will be that much of an improvement, OpenSSL is painfully slow at ECDSA.

I don't think it's that OpenSSL is painfully slow at ECDSA - I've found it to be okay, and probably a bit better than Crypto++ - as much as libsecp256k1 is specifically designed to optimize the hell out of working with the secp256k1 ECDSA curve (i.e., the curve Bitcoin uses). I believe Wladimir & Pieter have both reported 6-8x speed increases in verifications. They're just concerned about the possibility of a specially crafted Bitcoin transaction being validated in libsecp256k1 and not in OpenSSL or vice versa (i.e., a fork would occur). So, for now, libsecp256k1 is used only for signing, where it's apparently easier to test all the various use cases.

Anyway, yes, once the switch is thrown and libsecp256k1 is used for verification, BC Core should speed up quite a bit. The main contributors are doing some pretty cool things if you pay attention to the commits and the related conversations.

Topic: Starting preliminary 0.94 testing - "Headless fullnode" - page 5. (Read 15240 times)