Topic: Starting preliminary 0.94 testing - "Headless fullnode" - page 10. (Read 15098 times)

Having trouble getting past this stage in the DB build:


CPU cores hit the floor, that block never gets found.

New version out in ffreeze, please test away. There's a wealth of robustness changes and it should also be a bit faster. However no one at Armory has experienced the issues Carlton Banks and btcchris ran into, so most of these changes were speculative. It's possible these bugs will still be there partially or in full.

As usual, looking forward to your bug reports and logs, and again thank you for the time and effort you put in helping us to test things out =)

I have to give up at about block 290000. It takes ~20s to process a block. The worst thing is it freezes the VM

My VM's config:

Host: Hyper-V 2012 R2
OS: Debian jessie
RAM: 29GB
CPU: Intel i5 Quad-core
HD: RAID 1 (non-SSD)

That's a good idea, I'll do that.

Thanks. I hope it could be logged by fixed time interval, not by block interval. It would be very helpful for slow machine like mine.

It became very slow again and I restarted the VM. However, it is redoing ~4000 completed blocks


-WARN  - 1432179524: (BlockUtils.cpp:1071) Scanning from 272867 to 357353
-WARN  - 1432183241: (BlockWriteBatcher.cpp:355) Finished applying blocks up to 275000
-WARN  - 1432190441: (BlockWriteBatcher.cpp:355) Finished applying blocks up to 277500
-WARN  - 1432212681: (BlockWriteBatcher.cpp:355) Finished applying blocks up to 280000

(I killed it here with Control-C)

Log file opened at 1432217549: /home/xxx/.armory/armorycpplog.txt
-INFO  - 1432217554: (BlockUtils.cpp:850) blkfile dir: /home/xxx/.bitcoin/blocks
-INFO  - 1432217554: (BlockUtils.cpp:851) lmdb dir: /home/xxx/.armory/databases
-INFO  - 1432217554: (lmdb_wrapper.cpp:439) Opening databases...
-INFO  - 1432217554: (BlockUtils.cpp:1181) Executing: doInitialSyncOnLoad
-INFO  - 1432217554: (BlockUtils.cpp:1253) Total number of blk*.dat files: 272
-INFO  - 1432217554: (BlockUtils.cpp:1254) Total blockchain bytes: 36,422,961,275
-INFO  - 1432217554: (BlockUtils.cpp:1628) Reading headers from db
-INFO  - 1432217567: (BlockUtils.cpp:1654) Found 357398 headers in db
-DEBUG - 1432217567: (Blockchain.cpp:214) Organizing chain w/ rebuild
-INFO  - 1432217574: (BlockUtils.cpp:1295) Left off at file 271, offset 65665089
-INFO  - 1432217574: (BlockUtils.cpp:1298) Reading headers and building chain...
-INFO  - 1432217574: (BlockUtils.cpp:1299) Starting at block file 271 offset 65665089
-INFO  - 1432217574: (BlockUtils.cpp:1301) Block height 357353
-DEBUG - 1432217574: (Blockchain.cpp:214) Organizing chain w/ rebuild
-INFO  - 1432217575: (BlockUtils.cpp:1337) Looking for first unrecognized block
-INFO  - 1432217575: (BlockUtils.cpp:1489) Loading block data... file 271 offset 65665081
-ERROR - 1432217575: (BlockUtils.cpp:516) Next block header found at offset 65665089
-INFO  - 1432217888: (BlockUtils.cpp:544) Reading raw blocks finished at file 271 offset 87382506
-INFO  - 1432217888: (BlockUtils.cpp:1354) Wrote blocks to DB in 7.70157s
-INFO  - 1432217888: (BlockUtils.cpp:1371) Checking dupIDs from 276324 onward
-WARN  - 1432217900: (BlockWriteBatcher.cpp:1556) Starting with:
-WARN  - 1432217900: (BlockWriteBatcher.cpp:1557) 1 workers
-WARN  - 1432217900: (BlockWriteBatcher.cpp:1558) 1 writers
-WARN  - 1432217900: (BlockUtils.cpp:1071) Scanning from 276325 to 357427

The armoryd suddenly runs very slowly and makes my VM very unresponsive

It took 2089s to complete 265000-267500
3848s for 267500-270000

everything was fine up to this point

it died with "segmentation fault" at 270396 and I restarted

It took 3291s to complete 270396-272500
11847s to complete 272500-275000

the VM becomes very unresponsive. I renice the armoryd to 19 but seems not very helpful. Few minutes after I renice it, it died showing "Killed" but I don't think I did anything to kill it manually. The VM becomes normal after armoryd is dead.

I know later blocks will take longer to process but I don't think it would suddenly take 3x longer.

EDIT: after restarting armoryd it took only 3717s to complete 272867-275000. So there must be some problem before

Yes


I got some changes coming in soon, I'd rather you wait on those

How do I know if it's running properly? Wait until it finishes 272500?

EDIT: It passed 272500. Can I assume my database is not corrupted?

Would it help if I try building a Supernode Db? I'm retiring the 0.9.x bitcoin blockchain from my system disk, so I'll have the space and the patience to let it build.

Well then I guess both fullnode and supernode suffer from the same symptoms. As for the resume height, that looks correct to me.

It died around block 270000 with "segmentation fault". Nothing special in log

EDIT: I restarted and it shows "Scanning from 270396 to 357270"

On a VM with 4 cores and 29GB RAM, it took 8.4 hours to build to block 270000, about 1/3 of the total blockchain

Unless something goes stupid, I don't intent to modify the format of the DB ever again. I don't expect I'll have to change it to fix the current bug Carlton Banks and btchris are experiencing.

On the other hand I'd like some feedback on stability, since a lot of changes were to improve supernode stability.

I'm building a supernode database with 0.93.99.1. Do I need to do it again when 0.94 is finalized and officially released?

Unless you choose to use, say, an armoryd instance to power your blockchain explorer or web store (these are not my words...).

All I would add is that people re-using your code or stealing it wholesale doesn't matter.

Someone passing off others ideas as their own only behave that way because they're not sufficiently creative to do it themself. They'll always end up with a magpie's nest of other people's work, so it's all self correcting. If they want new ideas, they will have to go again to those people who actually produce original stuff, you can't fool anyone behaving like that. OTOH, keeping your work proprietary and close to your chest encourages any non-contributors who are privy to the code to manipulate the situation. Proving attribution of unpublished, unsigned work in that case would be far more difficult.