Topic: BIP: Increasing the Network Hashing Power by reducing block propagation time - page 2. (Read 6201 times)

You have to be careful with transmitting transaction hash lists rather than the transactions themselves. While it definitely makes propagation faster in the average case, it also means that the worst-case, a block entirely composed of transactions that have not been previous broadcast on the network, is significantly worse. For the purpose of just reducing orphans, as is done by P2Pool as gmaxwell pointed out, the worst case isn't a problem, but don't make assumptions that have security implications. In particular by the mechanism I pointed out here you'll actually create a market for large hashpower miners where they can offer lower fees, paid for by the lower effective competition, provided the sender promises not to send the transaction to any other miner. I can't see such markets developing for 1MiB blocks, but they might develop for much larger block sizes.

P2Pool is interesting because miners on it have an incentive for any P2Pool block to be propagated to the network as a whole as fast as possible. In addition the perverse propagation incentives for shares within P2Pool are probably less of an issue given that the higher the hash power of P2Pool as a whole, the lower the variance for any individual miner - miners on P2Pool aren't playing a zero-sum game. P2Pool miners also have very little control over propagation because P2Pool shares are all identical.

I'm not sure where you're getting 10% from— but orphaning rates are not that high, even extrapolating on block size. Your analysis is also ignoring RTT, and changing the data size transmitted doesn't change the contribution to latency from the speed of light.

P2Pool already implements basically the protocol you're describing. Works reasonably well.  I don't know that it's especially interesting except for miners, however. Objectively, it doesn't seem that miners care that much since they haven't all switched to p2pool.

The parent must be fetched or the network will not converge.
The time does not tell you this, its not monotonic (and can't be monotonic without creating minor vulnerabilities.)
The amount of work assigned to a header is a strict function of the prior chain. There is no permitted variation _at all_

If you're going to go crazy with compression you could actually transmit only half the transaction IDs (16 bytes rather than 32) and half that, I suppose.

Hi!
I'm sending this BIP for the community to discuss. I hope I get good feedback so I can proceed to implementing it, publish it in the BIPs wiki, and make it into the protocol.

BIP:
Title: Increasing the Network Hashing Power by reducing block propagation time
Author: Sergio Demian Lerner
Status: Draft (incomplete)
Created 19-Feb-2013

Abstract

When a block get orphaned, it means that some of the available network hashing power has been wasted. This wasted hashing power does not benefit anyone in the Bitcoin network.
Orphan blocks are created mostly because of the network propagation time of new blocks. A 1 Mbyte block, sent over a 50 Kbytes/sec connection, takes 20 seconds. Assuming an average network graph diameter of 4 hops, it takes more than 1 minute for such a block to propagate throughout the network. Current block average size is 150 Kbytes, so this is not an issue today. As we expect the Bitcoin network to grow upto its maximum capacity, it's reasonable to expect a 13% waste in network hashing power in the near future because of orphaned blocks.

One of the ways to reduce block propagation time is by sending only the block header and transactions hashes in a first command payload ("header"), and then sending the remaining data (the transactions) in a second command payload ("block"). Since the coinbase transaction cannot be sent to peers without the block enclosure, the "header" command must also send this special transaction.

Specification

"header" command format is:

- Block header
- transactions hash list
- Coinbase transaction (maximum 10 Kbytes in size)

An average "header" command size (for an 1 Mbyte block, considering an average 400 bytes tx) is 80 kbytes, that takes 1.5 seconds per hop.

The "header" command should only be sent for newly created blocks and not for historical ones.

Semantics

Just after the first command "header" is received, a node should:

1. Verify that the block parent is known. If it's not, then discard the command.
2. Verify that the block is the last block of a chain (by the time field)
3. Compute the PoW and check if it's similar to the parent block PoW (+/- the possible expected variation). If it's not, then the command is ignored and the sender is blacklisted (DoS prevention).
4. Resent the "header" command to all peers.
5. Check if transaction hashes contained in the "header" command are already in the Tx pool.
6a. If all hashes are in the pool, reconstruct the block referred by the "header" and announce to peers.
6b. If very few are not in the pool, then the missing txs can be requested from peers.
6c. If almost all of them are not, ignore the "header" command and request the block from the peer having sent the "header" command by using "getblocks".
7. After the complete block X is reconstructed/received, proceed as normal (announce the block by "inv" to peers and forward when requested)

Miners should, after doing the previous steps 1-6, do:

1. Wait until the full block is reconstructed or received (while mining as normal without interruption)
2a. If the block X is received before our own block is solved, start over trying to solve a block whose parent is X.
2b. If a block is solved locally before the complete block X is reconstructed, discard X and try to win the block chain race against X.

This BIP does not require any soft-fork or hard-fork and is 100% backward compatible because it only involves changes in the p2p communication protocol. Peers that do not understand the "header" command will process the "block" command instead.

Reference Implementation



Further related improvements

With a maximum 10 seconds propagation time, a 2 minute block confirmation interval should be stable, so it might be possible in the future (with the community acceptance) to reduce the block interval to 2 minutes or less with a hardfork (reducing also the fees and the block size proportionally to keep the same economic properties of Bitcoin intact and keep the contract with the community).

Who benefits from this BIP?

Miners: miners that implement this protocol (even if privately between them) get a 10% advantage over the remaining ones.

Users: They get 10% additional security for almost no extra cost (overhead should be negligible). Also if blocks are reconstructed using existent transaction in the pool, and not sent, bandwidth usage is reduced to a half.

This is a win-win situation.

Best regards,
 Sergio.