Topic: Bitcoin Core 0.14.x Mod (Read 1302 times)

You can run whatever code you want to run. However if other nodes think that your node is malicious, they might not connect to you or blacklist you.

It is very important to use bitcoin as the source code, we are not allowed to use source code with harmful changes. I think we should use the most popular source code released by the publishers, which is safer and more convenient. You need to have a deeper look at the source code for best insight, besides, you also have parallel processing and data synchronization to enhance the core. Most of the changes occur in the files and the command line, so if the change will cause unintended consequences.

Maybe one day I will understand the code in depth and will produce a github fork for code review. It may or may not require a complete ground up re-work. It is possible the current code was designed to solve a cryptographic solution, and did not take into account for efficient data flow and highly asynchonous parallel processing considerations. I am not a cryptographic expert, so don't expect to see one any time soon. I have a lot to learn.

gmaxwell
Great thanx and RESPECT!!!

-O2 is really safe and some dependencies has base setiings like -O3 and so on, actually i did not see incorrect results, but when to complile with mingw 5.3.0+ there are bugs with linking and etc. there are no posibility to build. Probably it is better when one process takes a little less CPU time to get more for another one.
I think it is worsting of time because of linking problems, but i like that "fine" tuning and have idea to compile deps with latest compiler like 6.3.0 and Core with 4.8.2 or in another sequence - Perfectionism is our all  

You are very right that there are a lot of possibilities to get ban-score, but i argue with you that my version will sync faster - will try to bench both Official and my compilation on weekend and measure time.

Fee theme:
I did not get to the source [Since this began here, then I feel that I will study the mechanisms very deeply, or maybe not, maybe it's just a waste of time and the satisfaction of ego] but as understand if pool get transaction:
it will make command like a prioritize_transaction depending on fee, so if all pools get that transaction to their mempool in one time-frame window(i assume that other transactions will come in next frame, because of less broadcasting) then there are possibility that pool which first mine the block will have your transaction little above others in the list. Another pool probably will get more transactions with bigger fees at the same window and the same transaction has less priority and if first pool does not mined the second will mined it on next block.. [sorry for English, it is really hard to suggest/explain].


AngryDwarf
It would be great if we will see your optimization of parallel processing.

Interesting information. Whether a code implementation improvement could optimise this parallel processing capability I am not currently in the position to be able to tell you (I don't know the core code in-depth). If the rest of the core software is not blocked from operation during this process, it is probably not a high priority improvement at this moment in time anyway.
A better default would by a maximum corresponding to the number of cores on the system or 16 depending on whichever is lower. But perhaps the core code already does this anyway.

This is not useful when you have more than 16 cores: it will be considerably slower, even when you do have many cores. The numbers were set based on actual measurement on 48 core systems. Overheads dominate by the point of 16 script check threads.

Similarly, increasing outbound connections just wastes resources, severely _hurts_ your latency, and will end up getting you banned by many people (same thing for the addnode maximum).  (Several people correlate hosts that are mass connecting and add them to shared ban-lists -- once your address is on these lists it won't come off for a long time).

It will also not improve your sync speed on practically any system-- sync is verification/database bound and when it isn't, it's usually reorder buffer bound.

The outbound connection increase is especially bad for miners because it increases your connection service loop time, and keeps your block in the linear regime longer where  it's just your node replicating the block rather than an exponentially growing cone.  An optimized mining setup has mining nodes with a minimal number of connections connecting to multiple border nodes and relay networks (e.g. two local border nodes, one remote border node, two reliable external nodes, and one relay network). At least one of the border nodes should  be outbound only, and at least one should accept inbound (usually best to do with remote nodes, in case it attracts dos attacks).


Most of the other settings changed can just be changed with the config file and command-line, anything with "DEFAULT" in the name.


The download window changes will likely cause database corruption if you use them with pruning. The checkblocks/checklevel changes will do effectively nothing (also controllable from the commandline).  The MAX_BLOCKS_TO_ANNOUNCE will just waste your bandwidth in the event of a huge reorg.

The timeout interval change will make you easier to DOS attack, causing you to wait longer on messages that will never be answered.

This is absolutely untrue. What you broadcast to has nothing to do with what fees you pay.  


I hope you have at least 32 GB of ram, because that is the peak memory usage these (commandline accessible) options imply on top of the nearly 2.5GB of other usage implied by  other settings.

Which permits optimizations which may produce incorrect results... mostly which shouldn't matter for Bitcoin Core, but O3 should also be just as fast and would only suffer from compiler bugs, and not directly unsafe optimizations.

Virtually all these settings which can be safely tinkered without much danger of screwing things up and without musch understanding are command-line options.

Changing things further is great-- but you should probably take some more time to understand what you're changing, especially before inviting others to use your changes!

Work on making the rescan faster would be very welcome. The largest likely improvement there would be bypassing hash validation when reading blocks from disk.

sgbett

Do not be so agressive :-)
Will you agree with two defenitions ? :
#1: There is pretty high probability not to see transaction in mempool of big nodes/pools fast (even in unconfirmed status, for example on https://blockchain.info/ ) if to broadcast transaction via 2 nodes
#2: You will sync faster if you will have more connections in case if computer is pretty good(CPU/mem/SSD)

Probably the main reason, i experimented with all these stuff because of situation like case #1 - there were no my transaction for 1 day, it was nowhere.

Thank you.

Go ahead and do what you like, but I don't think it will do what you think it will... I'd urge others to read this other post (by Pieter Wuille) before making up their own mind about whether its good/necessary:

http://bitcoin.stackexchange.com/questions/8109/how-does-one-attain-1-000-connections-like-blockchain-info/8140#8140

AngryDwarf
Thank you. It is always matter to see really dep to source...

I understand, what does it mean when one thread has to wait result from thread#2 and thread #2 from #N and so on. I wrote CPU benchmark in 2002-2003 :-)
To my mind bottleneck is hard drive perfarmance, i will try to install faster SSD and later PCI-E SSD with 3000MB/s perfomance :-)

Probably the best choose for MAX_OUTBOUND_CONNECTIONS is number of mining pools ( https://blockchain.info/pools ) + C, approximately 24..32.
I will make benchmarks of sync process on new SSD with diff. params.

I have been reading and trying to understand the lines clearly but I was hesitant to download the file. There are many thoughts that comes into my mind since we are using a third party client. But I will check and read reviews on this one so I could get the advantages and disadvantages of using this kind of system. I cannot make a god comment on this since I havent tried it yet, I am hoping for somebody who tried this system and has a good experience.

You really need to look deeper at the code to have an understanding of what each parameter does to see if any optimisations are good for your equipment. Miners may very well do this already as they want to validate blocks to build on ASAP and create them and push them out of the door with the most transaction fees included on them before another mine wins the network propagation race. It's all wasted time for the asics proof of work.

You also need to understand more about asynchronous parallel processing to improve core. This along with data storage optimisations is one of the areas where there is a potential to make many improvements (they are cryptographic specialists after all).


In this case the number of script verification threads appears to already default to the number of cores you have. I can only assume reducing it may have a positive effect by leaving a core free to perform other work to reduce context switching.

If you don't see CPU-load exceed 70%, then it may be due to other bottlenecks in the software or other parts of your system. I might have more physical cores than you, or faster disk I/O than you so any simple comparison like that is meaningless. Note also, that writing multi-threaded software just so that you can see 100% load does not mean it is faster. It could be horrendously slower due to cache line ping-pong (thread synchronisation effects).

MAX_OUTBOUND_CONNECTIONS = 8 - This is an arbitrary number decided by the devs to be a good connection rate, whilst not overloading the capacity of other nodes incoming connections. If you allow inbound connections then you can service those who can only perform outbound connections. Each connection type is equal in its P2P effect as far as I am aware.

Note: DASH defaults to 16 outbound connections, and I've never seen anyone make an inbound connection. Presumably this is because it has more masternodes than actual users.  

Sorry i was not able to upload 3.2Gb of Ubuntu VM Vare image, but now - it is done - everybody can check everything or build/compile his own core for Windows / Linux / Mac.
To get latest Bitcoin Core sources use command:
git clone https://github.com/bitcoin/bitcoin.git

AngryDwarf
Did you see CPU-load more than 70% even if you launch with -par=16 on i7 quadcore-cpu (i did not on i7-2630QM)

sgbett
I may agree a little with you but on your link there were no tech explanation why MAX_OUTBOUND_CONNECTIONS >8 is very bad.
To the end user, especially if he launches the wallet with interval a week/month - it is very good, because he may sync very fast, also it is good because his transaction will be BROADCASTED via a lot of nodes, sometimes it helps to pay less fee to be earlier confirmed.

Thanks for critics.

14/5000
I thinks, only official source and true wallet and Recommended from https://bitcoin.org/en/choose-your-wallet

There is no such thing as "official" source for bitcoin.

There is a source that is called "Bitcoin Core" by it's developers, and that source is probably the most popular source at the moment, but there isn't anything "official" about it, and it may not always be the most popular.


What is important when it comes to running bitcoin software is to use OPEN source (source code that is available for EVERYONE to read) which is well reviewed (not brand new source that nobody has every seen before or which very few people have looked over).  And to be certain that any binary you download was actually built from that exact source and not some variation.

I like all development for BTC, but only wallet i like official source

Several people have suggested increasing outbound connections over the years, the answer has always been "don't do this it's bad!" or words to that effect.

This is one from recently https://github.com/bitcoin/bitcoin/issues/9217

Many others if you google around.

I would not risk to download anything that is related to bitcoins that does not come from the official source.

Download a third party pre-compiled binary at your own risk.

Validation.h
static const int MAX_SCRIPTCHECK_THREADS = 32; /16/

Useful for those who have PC's with greater than 32 physical cores, otherwise increased context switching will make it slower.

Hello guys and girls.

I like Bitcoin Core client but there are some limits for outbound connections ( 8 ) and etc, some parameters can be overriden via command-line, some  not.
If you wanna setup a real big node and help Bitcoin infrastacture or just wanna faster blockchain sync speed, then it is very build for you.
It is compiled in Linux(Ubuntu 14) for Windwows 64-bit target.
VM image included - if you do not trust unknown authors or just wanna check everything and build your own client with BlackJack and.. :-))
Core is compiled with -Ofast flag instead of default -O2: more compiler optimizations, depends by default settings.
If you want to make your own Win32 or Linux build - everything is in or ask me :-)

Download:
https://drive.google.com/drive/folders/0Bz-d5qySiqGhNzhSODRWZUhMUXc

Bitcoin Core 14 mod by Kostia Minin
bitcoin-0.14.99-mod-win64.zip
SHA256: F92B603C4A5A3817DC0AA90DBB246112DAE5D6CCC2381D4DC0BAA2E4D4293324
MD5: 3F2A7A4A5CDED4A67CED18DA5852E06D


VM Ware Ubuntu image(root password is 'test', all sources with dependencies are in src folder)
Ubuntu140405.7z
SHA256: 3BC76AFA629B30D57BB9F2AB2DCC947425AA67D9B209C224E548BFA53BD55E83
MD5: 6E50714EF260BA29F73F5AD46A1B79A4


---
Changes:

Net.h
static const int PING_INTERVAL = 3 * 60; / was 2*60/
static const int TIMEOUT_INTERVAL = 30 * 60; /20*60/
static const int FEELER_INTERVAL = 180; /120/
static const int MAX_OUTBOUND_CONNECTIONS = 128; /8/
static const int MAX_ADDNODE_CONNECTIONS = 128; /8/
static const bool DEFAULT_UPNP = true; /false/
static const unsigned int DEFAULT_MAX_PEER_CONNECTIONS = 512; /125/
static const size_t DEFAULT_MAXRECEIVEBUFFER = 32 * 1024; /5*1000/
static const size_t DEFAULT_MAXSENDBUFFER    = 32 * 1024; /1*1000/

Validation.h
static const int MAX_SCRIPTCHECK_THREADS = 32; /16/
static const int DEFAULT_SCRIPTCHECK_THREADS = 0; /0/
static const int MAX_BLOCKS_IN_TRANSIT_PER_PEER = 64; /16/
static const unsigned int BLOCK_DOWNLOAD_WINDOW = 1536; /1024/
static const unsigned int MAX_BLOCKS_TO_ANNOUNCE = 16; /8/
static const signed int DEFAULT_CHECKBLOCKS = 3; /6/
static const unsigned int DEFAULT_CHECKLEVEL = 2; /3/

txdb.h
static const int64_t nDefaultDbCache = 768; /300/
static const int64_t nMinDbCache = 8; /4/
static const int64_t nMaxBlockDBCache = 4; /2/
static const int64_t nMaxBlockDBAndTxIndexCache = 2048; /1024/
static const int64_t nMaxCoinsDBCache = 16; /8/
---

Future plans: make -rescan option more faster - now as understand it works in 1 thread, recompile dependencies with more optimizations.
I will be glad to see your suggestions/proposals/advices, also i will be glad to get your donations here:

BTC: 1M6X1oMH9sz4TBtBjp1ViVzVkPd2kuLmZr
ETH: 0x1182726e4bf8e5483e6006be8bd6ea6a78d94196