Author

Topic: [ANN][XCN] Cryptonite - NEW Thread | 1st mini-blockchain coin | Bounties! - page 115. (Read 215807 times)

full member
Activity: 243
Merit: 105

Thanks, I will try this.
Maybe the faster AcceptBlock (which is actually a faster Activate), since it should speedup getslice as well, may make the trie sync issue much less common.
Thus, when the network is updated, it may go away by itself.

So people please test the krnlx patch to death, and also we need to set a much higher hard limit for block number or make it dynamic (without this we can't push the improvement to production).

A few words about my patch:

I. What is doing TrieView::Activate ?

We have two chains with blocks:
Code:
1,2,3,4,5,6,7,8,....,99,100,103,104
Code:
1,2,3,4,5,6,7,8,....,99,100,101,102,105

blocks get listed backwards (pindex->pprev) into two lists:
Code:
104,103,100,....3,2,1
Code:
105,102,101,....3,2,1
then same blocks removing from lists(starting from 1), we getting:
Code:
104,103
Code:
105,102,101
when we have a lot of blocks in chain, there is involved a lot of slow c++ boost staff (linked lists, memory allocs,etc)

II. What I've done?
1.allocated two arrays, and filled it with pointers to blocks.
2. Then scan them backwards until index reached SAFE_UNIQ or the blocks become different. So we get:
Code:
94,95,96,97,98,99,100,103,104
Code:
94,95,96,97,98,99,100,101,102,105
3. fill two linked lists with this blocks
4. then the original code works with those linked lists, which are short.

III.
I have never seen a situation where these two chains differ from each other by more than one block, but I did not rule out such an opportunity (for example 3 or more blocks in a row are orphan). If you rely on the fact that they will always differ by one block, then the code can be greatly simplified.

IV. Possible problems
1. Size for arrays: must be a bit more(number?), than block height. How correctly determine actual blockheight from TrieView::Activate?
2. Check my code for errors: it is small, but errors can exist : wrong indexes, which can cause illegal memory access, or incorrect results.
sr. member
Activity: 266
Merit: 250
This is greatly appreciated! If possible (unless bitfreak has responded to Pallas and said yes already), it'd be great if you could eventually compile the qt wallet as well. I use both cryptonited and qt. I use the qt wallet on my PC because it's easier to look through transactions etc with the GUI.

Regarding that, is it possible to use both the new cryptonited and old qt for the same wallet on different computers? As in, can I use the new cryptonited wallet on my rigs and the old qt wallet on my PC without messing anything up?

Edit: Also, will the new wallet have to connect to only other nodes with the same wallet version in the config file?

Think it's safe, pallas avoided syncloop by limiting number of getslice from one peer, it shouldn't affect your local copy of chain and you're free to connect to any node (anyway, nodes with new wallet preferred)

Stuck with qt build, environment doesn't recognise half of static qt libs so can't compile qt bin, will investigate further. Some patches have already been needed because of newer dependant libs versions. As I see wallet is based on bitcoin-qt 0.9.x with modifications.
sr. member
Activity: 378
Merit: 250
0.1.3.0 built for Windows:

https://github.com/palginpav/Cryptonite/releases/tag/0.1.3.0

With latest pallas commits, krnlx patches not included (this time, need someone to test my build first) think I didn't forget to statically link somethng  Grin

Qt wallet will be added later, setting up build environment for this is pure pain (at least for me).
This is greatly appreciated! If possible (unless bitfreak has responded to Pallas and said yes already), it'd be great if you could eventually compile the qt wallet as well. I use both cryptonited and qt. I use the qt wallet on my PC because it's easier to look through transactions etc with the GUI.

Regarding that, is it possible to use both the new cryptonited and old qt for the same wallet on different computers? As in, can I use the new cryptonited wallet on my rigs and the old qt wallet on my PC without messing anything up?

Edit: Also, will the new wallet have to connect to only other nodes with the same wallet version in the config file?
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
0.1.3.0 built for Windows:

https://github.com/palginpav/Cryptonite/releases/tag/0.1.3.0

With latest pallas commits, krnlx patches not included (this time, need someone to test my build first) think I didn't forget to statically link somethng  Grin

Qt wallet will be added later, setting up build environment for this is pure pain (at least for me).

Test if you want and report.

And everybody have a nice weekend!



Thanks for your build!
As soon as we get enough good reports, I will put it into the first post and notify btc38!
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
Reducing CPU usage while accepting new block.

It is indeed a lot faster when accepting blocks, that's really what was needed, great! :-)
But it will need to be tested and hard limits removed.
Currently my test wallet hanged on trie sync, not necessarily related to the patch.

Trie sync issue is another problem, but I never seen it, when I force wallet to sync from one node (connect=xcn.suprnova.cc).

Thanks, I will try this.
Maybe the faster AcceptBlock (which is actually a faster Activate), since it should speedup getslice as well, may make the trie sync issue much less common.
Thus, when the network is updated, it may go away by itself.

So people please test the krnlx patch to death, and also we need to set a much higher hard limit for block number or make it dynamic (without this we can't push the improvement to production).
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
GUIDE: Compiling xcn wallet with TCMALLOC allocator.

Thanks for this guide! I linked it on the first post, if you don't mind.
sr. member
Activity: 266
Merit: 250
0.1.3.0 built for Windows:

https://github.com/palginpav/Cryptonite/releases/tag/0.1.3.0

With latest pallas commits, krnlx patches not included (this time, need someone to test my build first) think I didn't forget to statically link somethng  Grin

Qt wallet will be added later, setting up build environment for this is pure pain (at least for me).

Test if you want and report.

And everybody have a nice weekend!

full member
Activity: 243
Merit: 105
GUIDE: Compiling xcn wallet with TCMALLOC allocator.

Quote
The fastest malloc we’ve seen; works particularly well with threads
and STL.

This fix eventual memory leaks and improve overal stability and performance.

1. Download and compile TCMALLOC

Code:
sudo apt-get install libunwind-dev
git clone https://github.com/gperftools/gperftools
cd gperftools/
./autogen.sh
./configure CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native" --prefix=/usr
make -j4
make install

2. Compile xcn wallet with TCMALLOC library

Code:
git clone https://github.com/pallas1/Cryptonite
cd Cryptonite
./autogen.sh
./configure CFLAGS="-O3 -march=native -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free"  CXXFLAGS="-O3 -march=native -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free" CRYPTO_LIBS="-lcrypto -ltcmalloc"
make -j4

Then check, that cryptonited linked with TCMALLOC :
Code:
ldd src/cryptonited

Code:
linux-vdso.so.1 =>  (0x00007ffd6abe4000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6164a87000)
libboost_system.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.58.0 (0x00007f6164883000)
libboost_filesystem.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.58.0 (0x00007f616466a000)
libboost_program_options.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.58.0 (0x00007f61643ec000)
libboost_thread.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0 (0x00007f61641c6000)
libdb_cxx-4.8.so => /usr/lib/libdb_cxx-4.8.so (0x00007f6163e21000)
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f6163ba1000)
libssl.so.1.0.2 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.2 (0x00007f6163939000)
libcrypto.so.1.0.2 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2 (0x00007f61634ec000)
libtcmalloc.so.4 => /usr/lib/libtcmalloc.so.4 (0x00007f61630f2000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6162d70000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6162a66000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6162850000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6162486000)
/lib64/ld-linux-x86-64.so.2 (0x000056455c3f5000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f616227d000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6162079000)
libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007f6161e5d000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f6161c3b000)

This string must exist :
Code:
libtcmalloc.so.4 => /usr/lib/libtcmalloc.so.4

I checked CPU usage of new allocator with valgrind:
hero member
Activity: 2114
Merit: 619
Launch of Masternodecoin in approximately 23 hours, it is offering 1000 users the amount 52,000 coins to implement a masternode completely free, if certain requirements that can see in this thread are met
https://bitcointalksearch.org/topic/ann-mtnc-masternodecoin-multiple-privacy-center-platform-2056867

You can also win coins by keeping the currency signature on your profile,
By re-twiter
full member
Activity: 243
Merit: 105
Reducing CPU usage while accepting new block.

It is indeed a lot faster when accepting blocks, that's really what was needed, great! :-)
But it will need to be tested and hard limits removed.
Currently my test wallet hanged on trie sync, not necessarily related to the patch.

Trie sync issue is another problem, but I never seen it, when I force wallet to sync from one node (connect=xcn.suprnova.cc).

There is another bottleneck in wallet - to many malloc() (c++ new()) calls

I have one old trick from 2006, which seems stable, I am testing it now.
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
Hi,is it possible to solo mine? suprnova disconnect too often.
my conf is
" rpcuser=rpcuser
rpcpassword=rpcpass
rpcport=14443
listen=1
server=1"
after I started ccminer,It shows "No payout address provided,switchin to network",is it ok?
thanks.


Getwork, yes it's correct.
newbie
Activity: 22
Merit: 0
Hi,is it possible to solo mine? suprnova disconnect too often.
my conf is
" rpcuser=rpcuser
rpcpassword=rpcpass
rpcport=14443
listen=1
server=1"
after I started ccminer,It shows "No payout address provided,switchin to network",is it ok?
thanks.
full member
Activity: 237
Merit: 100
WTF??? I just rerun miner and ocminer`s scripts ban me again!
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
Reducing CPU usage while accepting new block.

It is indeed a lot faster when accepting blocks, that's really what was needed, great! :-)
But it will need to be tested and hard limits removed.
Currently my test wallet hanged on trie sync, not necessarily related to the patch.
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
wallet takes 10 min to start, after sync it takes 6+GB RAMM + CPU load very huge Huh


are you talking about the standard wallet or the new krnlx patch?
the standard wallet takes less than 1GB on my machine.
newbie
Activity: 4
Merit: 0
wallet takes 10 min to start, after sync it takes 6+GB RAMM + CPU load very huge Huh
full member
Activity: 243
Merit: 105
Reducing CPU usage while accepting new block.

I mean did you measure the time difference?

OLD


With patch




This CPU load spikes appear when new block accepted
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
Reducing CPU usage while accepting new block.

I mean did you measure the time difference?
full member
Activity: 243
Merit: 105
Reducing CPU usage while accepting new block.
legendary
Activity: 2716
Merit: 1094
Black Belt Developer
With Valgrind/callgrind I found bottlenecks, related to slow block accept in cryptonite wallet.

THIS IS ONLY FOR TESTING. IT IS DIRTY, AND WALLET WILL CRASH AFTER BLOCK 2097152

https://github.com/krnlx/Cryptonite/commit/d0363d10ffa8d8cf1fa9e3d1749408827aed8961

//TODO

1. Test everything many times : sync, sync from scratch, from snapshot, etc, mining, transactions

2. Automatic memory size adjustment of MAX_BLOCK

3. Reducing SAFE_UNIQ

4. Test again.

Thanks very much! What kind of improvement are you getting with this patch?
I will test it ASAP! :-)
Jump to: