Author

Topic: Ideas for a new block header format (Read 1600 times)

legendary
Activity: 1232
Merit: 1094
August 18, 2015, 07:07:35 AM
#9
Considering all the recent block size drama, it's probably not a good idea to be overly optimistic.

They are hopefully not that much of a major issue, if there is near total agreement.

The header will have the following fields:
1. version (2): unsigned integer. Voting is moved to coinbase. 65536 versions should be enough forever.
2. prev_block (32): hash of the previous block
3. merkle_root (32): Root of merkle-sum-tree for fee, tx size, sigop counts, etc, so compact proofs could be produced for these parameters.
4. timestamp (5): unsigned integer. Enough for >34,000 years.
5. nonce (9): While a 1TH/s miner will exhaust 5 bytes of nonce in 1 second, we won't need longer nonce until miners become 4 billion times faster. (Current ASIC is only 1 million times faster than CPU, so we are taking about >1018 times faster than current CPU. This may happen only if SHA256 is partially cracked but then we need another algorithm)
6. coinbase (varint): the coinbase will become part of the header, which includes:
6a. height (4 currently): same as BIP34, consume 4 bytes for the following 300 years.
6b. bits (4): target in compact form
6c. arbitrary data: voting for fork proposals, merge mining, etc.

Moving the data to the Merkle tree means that it is completely transparent to mining hardware.

Code:
Primary Header

int(4)      version:         The version number
int256(32)  prev_block:      Hash of the previous block's header
int256(32)  aux_header:      Hash of the auxiliary header
int(4)      timestamp:       Lower 4 bytes of timestamp
int(4)      bits:            Difficulty target (compact)
int(4)      main_nonce:      Primary nonce

Auxiliary Header
int256(32)  merkle_root:     The merkle root
var_str(1+) aux_data:        The auxiliary data

Auxiliary Data (initial version)

short(2)    timestamp_upper: Upper 2 bytes of timestamp
long(8)     aux_nonce:       Secondary nonce
int(4)      height:          Block height
int(4)      vote_start:      Voting data starts at this byte position
var_str(1+) votes:           Voting data


aux_header = Hash256(merkle_root | Hash(aux_data))

block_hash = Hash256(version | prev_block | aux_header | timestamp | bits | main_nonce)

The advantage of arranging it this way is that it is fully backward compatible with current mining hardware.  It even supports current merkle blocks, since the new auxiliary data is added into the Merkle tree, rather than as new fields.

Additionally, the aux_data field is defined as a variable length string.  This means that if new fields are added, they can be handled by legacy clients.

The voting vector can specify a start position to skip obsolete vote positions.

Alternatively, the vote data could be a series of var_ints.  The first int would be the vote_id and the 2nd would be the miner's choice.
legendary
Activity: 1792
Merit: 1111
August 17, 2015, 09:25:20 PM
#8
3. More space for miner voting. Currently, there are only 28 bits available in the version field for this purpose.

+1. It seems we are heading into the future where major features will be "voted with a fork", similar to XT.

So it would be nice to have more complex voting than a simple version number.

4. Recently there is some discussion on the mailing list that whether we should use the timestamp, median time of the past 11 blocks, or block height to determine the activation of a fork, etc.

Block size might also be useful, if we ever want a dynamic block size limit (dreams, sweet dreams Smiley ).


Yes, I'm also thinking of block size, which should be a var_int, taking 4 bytes currently.

Also UTXO committment, taking 32 bytes.
hero member
Activity: 840
Merit: 1002
Simcoin Developer
August 17, 2015, 06:20:07 PM
#7
3. More space for miner voting. Currently, there are only 28 bits available in the version field for this purpose.

+1. It seems we are heading into the future where major features will be "voted with a fork", similar to XT.

So it would be nice to have more complex voting than a simple version number.

4. Recently there is some discussion on the mailing list that whether we should use the timestamp, median time of the past 11 blocks, or block height to determine the activation of a fork, etc.

Block size might also be useful, if we ever want a dynamic block size limit (dreams, sweet dreams Smiley ).
full member
Activity: 219
Merit: 102
August 14, 2015, 08:34:44 PM
#6
Why not use the first byte of the version as a header length indicator to give you up to an additional 95 bytes of extended header for clients that are aware? If the version is FF, then the version is in the extended header and clients should look there for it because it is bigger than 1 byte.-> Fully backwards compatible.

If you need more, just keep leapfrogging adding 255 bytes or less whenever you need a bigger header and always maintaining backwards compatibility.

How would you hash the "extended header"? If it is not hashed, it is not a part of the header. If it is hashed, it is not backward compatible
The hash is kept in txn_count.

I am making some pretty broad assumptions here though (like there is no check for zero in txn_count) . So please do pick holes or tell me I'm an arse and shut up.
legendary
Activity: 1792
Merit: 1111
August 13, 2015, 08:31:08 AM
#5
Why not use the first byte of the version as a header length indicator to give you up to an additional 95 bytes of extended header for clients that are aware? If the version is FF, then the version is in the extended header and clients should look there for it because it is bigger than 1 byte.-> Fully backwards compatible.

If you need more, just keep leapfrogging adding 255 bytes or less whenever you need a bigger header and always maintaining backwards compatibility.

How would you hash the "extended header"? If it is not hashed, it is not a part of the header. If it is hashed, it is not backward compatible
legendary
Activity: 1792
Merit: 1111
August 13, 2015, 08:28:24 AM
#4
Targets:
1. Existing ASICs must survive the new format. Otherwise, miners won't agree to upgrade
2. To allow more information included in the header
3. Include height in the header so we could have a monotonic indicator in the header.
4. Headers not being too big for SPV clients
What did you have in mind with 2 and 3? Why is a monotonic indicator necessary at all? (besides you could also derive that from the current latest block, if necessary)

1 and 4 are only of concern if we actually need to change the header format, which I seriously doubt. Exactly what is the problem you're trying to solve?


Problems to solve:

1. To introduce merkle-sum-tree: https://en.bitcoin.it/wiki/User:Gmaxwell/alt_ideas

2. More nonce space for ASIC: 4 bytes are not enough already.

3. More space for miner voting. Currently, there are only 28 bits available in the version field for this purpose. More complicated ideas, e.g. BIP100, require casting votes in coinbase. However, coinbase is not part of the header and is difficult for SPV nodes to follow

4. Recently there is some discussion on the mailing list that whether we should use the timestamp, median time of the past 11 blocks, or block height to determine the activation of a fork, etc. Timestamp is the easiest but it is not monotonic. The rest 2 are monotonic but can't be determined purely but the header of one block. We could solve the problem by putting height in the header.

5. Some space for future expansion

I think the first one is the most important as blocks become larger and more people run SPV nodes. The second one is also a common complaint. The rest are beneficial side effects.
full member
Activity: 219
Merit: 102
August 13, 2015, 06:39:18 AM
#3
Why not use the first byte of the version as a header length indicator to give you up to an additional 95 bytes of extended header for clients that are aware? If the version is FF, then the version is in the extended header and clients should look there for it because it is bigger than 1 byte.-> Fully backwards compatible.

If you need more, just keep leapfrogging adding 255 bytes or less whenever you need a bigger header and always maintaining backwards compatibility.
legendary
Activity: 1176
Merit: 1011
August 13, 2015, 06:22:17 AM
#2
Targets:
1. Existing ASICs must survive the new format. Otherwise, miners won't agree to upgrade
2. To allow more information included in the header
3. Include height in the header so we could have a monotonic indicator in the header.
4. Headers not being too big for SPV clients
What did you have in mind with 2 and 3? Why is a monotonic indicator necessary at all? (besides you could also derive that from the current latest block, if necessary)

1 and 4 are only of concern if we actually need to change the header format, which I seriously doubt. Exactly what is the problem you're trying to solve?
legendary
Activity: 1792
Merit: 1111
August 13, 2015, 12:07:07 AM
#1
I just have some ideas for a new block header format.

Targets:
1. Existing ASICs must survive the new format. Otherwise, miners won't agree to upgrade
2. To allow more information included in the header
3. Include height in the header so we could have a monotonic indicator in the header.
4. Headers not being too big for SPV clients

To archive all these targets a hardfork is needed

The header will have the following fields:
1. version (2): unsigned integer. Voting is moved to coinbase. 65536 versions should be enough forever.
2. prev_block (32): hash of the previous block
3. merkle_root (32): Root of merkle-sum-tree for fee, tx size, sigop counts, etc, so compact proofs could be produced for these parameters.
4. timestamp (5): unsigned integer. Enough for >34,000 years.
5. nonce (9): While a 1TH/s miner will exhaust 5 bytes of nonce in 1 second, we won't need longer nonce until miners become 4 billion times faster. (Current ASIC is only 1 million times faster than CPU, so we are taking about >1018 times faster than current CPU. This may happen only if SHA256 is partially cracked but then we need another algorithm)
6. coinbase (varint): the coinbase will become part of the header, which includes:
6a. height (4 currently): same as BIP34, consume 4 bytes for the following 300 years.
6b. bits (4): target in compact form
6c. arbitrary data: voting for fork proposals, merge mining, etc.

The max header size at the beginning is 160 bytes, which will allow 160-2-32-32-5-9-1#-4-4=71 bytes of arbitrary data.
(#To calculate the header size, the length indicator for coinbase is also counted)
The max header size is allowed to double every 4 years so more info could be included in the future.

Block hash is calculated as follow:

d-SHA256(version|prev_block|d-SHA256(merkle_root|coinbase*)|timestamp|nonce)
(* without length indicator)

In this way, the change is transparent to existing ASICs because they will still think they are hashing a 80-bytes header.
Jump to: