Topic: A bit of criticism on how the bitcoin client does it - page 3. (Read 2797 times)

Not sure what you mean by "as deep as possible". We always send getdata starting at whatever block we already know. The reason for starting from early blocks and moving forward is because validation is done is stages, and at each point as much as possible is already validated (as a means to prevent DoS attacks, mostly). As most checks can only be done when you have the entire chain of blocks from genesis to the one being verified, you need them more or less in order.


That's not true, we only ask for each block once (and retry after a timeout), but it is done to a single peer (not to all, and not balanced across nodes). That's a known badness, but changing isn't trivial, because of how validation is done.

There is one strategy however that's pretty much accepted as the way to go, but of course someone still has to implement it, test it, ... and it's a pretty large change. The basic idea is that downloading happens in stages as well, where first only headers are fetched (using getheaders) in a process similar to how getblocks is done now, only much faster of course. However, instead of immediately fetching blocks, wait until a long chain of headers is available and verified. Then you can start fetching individual blocks from individual peers, assemble them, and validate as they are connected to the chain. The advantage is that you already know which chain to fetch blocks from, and don't need to infer that from what others tell you.


The Bitcoin Foundation has only been around for a year or so, and they don't control development. They pay Gavin's salary, but other developers are volunteers that work on Bitcoin in their free time.


BIP37 actually introduced a way to fetch parts of blocks, and it can be used to fetch a block with just the transactions you haven't heard about, so it avoids resending those that have already been transferred as separate transactions (though I don't know of any software that uses this mechanism of block fetching yet; once BIP37 is available on more nodes, I expect it will be). Any other system which requires negotiating which transactions to send adds latency to block propagation time, so it's not necessarily the improvement you want.


There are many ideas about how to improve historic block download. I've been arguing for a separation between archive storage and fresh block relaying, so nodes could be fully verifying active nodes on the network without being required to provide any ancient block to anyone who asks. Regarding moving to other protocols, there is the bootstrap.dat torrent, and there's recently been talk about other mechanism on the bitcoin-development mailinglist.

Sorry, but I think I can only help by implementing any of these new ideas in my client.
I just don't like to have a boss and in my code I am the boss - hope you understand

Anyway, if anyone wants to test a solution against a different client - just let me know.
I can implement it, if it's not really crazy - and then we can test each other, benefiting everyone

These are all pretty common topics in IRC.  I recall some of them from the mailing list too, but I don't read that daily any more.  People are working on various parts.  For one example, see here.  The bootstrap torrent is another example.

Pitch in and help if you can.

It is simple and efficient and it surely performs well.
I would even risk a statement that it performs better than the current satoshi client, though it takes much more RAM, so we are talking about different abstraction layers here.
Crypto operation are likely less efficient in Go, then what openssl gives, but my Go client only does them after checking a new highest block against the expected difficulty - so less often...

But the only way to be sure how fast it actually is, is to check it out by yourself

I thought the moto of the Go language is simple and efficient, will it even perform well?

I have implemented it, in Go language: https://bitcointalksearch.org/topic/gocoin-totally-different-bitcoin-client-with-deterministic-cold-wallet-199306

In what language are you implementing the node? I've been having an idea of also writing one, perhaps in C, but who knows, I don't usually finish projects.

As some of you may have noticed, I have worked on my own implementation of a bitcoin node. During this process I found things, which I would like to point out, so maybe they could be improved in a future...
Basically, they are all network bandwidth wasting related.

1. [EDIT: never mind]
Maybe I have not verified it well enough, but I have an impression that the original client, whenever it sends "getblocks", it always asks as deep as possible - why to do this? It just creates unnecessary traffic. You can surely optimize it, without much effort.

2. [EDIT: never mind]
IMO, you should also optimize the way you do "getdata".
Don't just send getdata for all the block that you don't know, to all the peers at the same time - it's crazy.
Instead, try to i.e. ask each node for a different block - one at a time, until you collect them all...

3. [EDIT: please, do mind]
Last, but not least.
The blocks are getting bigger, but there have been no improvements to the protocol, whatsoever.
You cannot i.e. ask a peer for a part of a block - you just need to download the whole 1MB of it from a single IP.
Moreover, each block has an exact hash, so it is just stupid that in times when even MtGox API goes through CloudFlare to save bandwidth, there is no solution that would allow a node to just download a block from an HTTP server, and so it is forced to leech it from the poor, mostly DSL armed, peers.
The way I see it, the solution would be very simple: every mining pool can easily use CloudFlare (or any other cloud service) to serve blocks via HTTP.
So if my node says "getdata ...", I do not necessarily mean that I want to have this megabyte of data from the poor node and its thin DSL connection. I would be more than happy to just get a URL, where I can download the block from - it surely would be faster, and would not drain the peer's uplink bandwidth that much.


That's about it, but if I recall something more, I will surely append it to this topic.

Also, please don't take my criticism personally - it's only meant as a feedback, pointing out areas that I think are important to improve, becasue the official bitcoin client is already eating up my internet connection more efficiently than BitTorrent, and that is very not cool