Author

Topic: Correct header data for block 1 (Read 1345 times)

Peter Todd

legendary

Activity: 1120

Merit: 1152

FWIW I wrote some Python code that recreates the raw hex block header, the part that is hashed to create a block hash, from the output of the RPC interface getblock:

Code:

import struct
import hashlib

bitcoin_header_format = struct.Struct("
def serialize_block_header(block):
"""Serialize a block header from the RPC interface"""
return bitcoin_header_format.pack(
block['version'],
unhexlify(block['previousblockhash'])[::-1],
unhexlify(block['merkleroot'])[::-1],
block['time'],
unhexlify(block['bits'])[::-1],
block['nonce'])

def calculate_block_hash(raw_block_header):
hexlify(hashlib.sha256(hashlib.sha256(raw_block_header).digest()).digest()[::-1])

Note the [::-1] to the byte order to change the endianness in a few places.

MatthewLM

legendary

Activity: 1190

Merit: 1004

Re: Correct header data for block 1

December 19, 2012, 10:33:07 AM

#9

I managed to get a hex dump of the raw block data. The blockchain data storage is more simple than I thought it would be. The blocks are stored with the network identifier ("magic bytes") followed by the block length and then the block. This is the result:

Code:
(null):cbitcoin matt$ hexdump -C -n 80 -s 301 /Users/matt/Library/Application\ Support/Bitcoin/blk0001.dat
0000012d 01 00 00 00 6f e2 8c 0a b6 f1 b3 72 c1 a6 a2 46 |....o......r...F|
0000013d ae 63 f7 4f 93 1e 83 65 e1 5a 08 9c 68 d6 19 00 |.c.O...e.Z..h...|
0000014d 00 00 00 00 98 20 51 fd 1e 4b a7 44 bb be 68 0e |..... Q..K.D..h.|
0000015d 1f ee 14 67 7b a1 a3 c3 54 0b f7 b1 cd b6 06 e8 |...g{...T.......|
0000016d 57 23 3e 0e 61 bc 66 49 ff ff 00 1d 01 e3 62 99 |W#>.a.fI......b.|

And indeed when I compared the data I saw there is one incorrect byte in the merkle root. I think I remember what I did. I replaced all the lowercase hexadecimal literals with uppercase ones and must have made this mistake and forgot about it. It works now.

Quote from: kjj on December 19, 2012, 10:25:02 AM
Quote from: MatthewLM on December 19, 2012, 09:57:12 AM
Does anyone know how to dump block byte data from the block database for the satoshi client?

https://bitcointalksearch.org/topic/m.1291088

Thanks, though I managed to figure it out.

kjj

legendary

Activity: 1302

Merit: 1026

Re: Correct header data for block 1

December 19, 2012, 10:25:02 AM

#8

Quote from: MatthewLM on December 19, 2012, 09:57:12 AM
Does anyone know how to dump block byte data from the block database for the satoshi client?

https://bitcointalksearch.org/topic/m.1291088

MatthewLM

legendary

Activity: 1190

Merit: 1004

Re: Correct header data for block 1

December 19, 2012, 09:57:12 AM

#7

Thanks for the link Alan, as it's interesting.

But I did not obtain the block data from the block database from the satoshi client. I've created the data manually. It was working before, so at some point I accidentally changed something, I just do not know what. Does anyone know how to dump block byte data from the block database for the satoshi client? Doing that would be very useful to compare the data.

As for bit errors, I've received mixed messages as to whether or not error correction is important at the application level. To be safe, in any case, I plan to implement some error correction to my database software. Before I do that I want to get the full validator unit tests working. The error correction will probably be something simple at first, such as hamming code.

etotheipi

legendary

Activity: 1428

Merit: 1093

Core Armory Developer

Re: Correct header data for block 1

December 18, 2012, 06:54:54 PM

#6

Perhaps it's a single-bit error! I documented my experience with this here. It was maddening, to be able to confirm 1.5 million tx hashes, and have exactly one fail. It took me a while, but I eventually narrowed it down. I was able to manually compare the tx from a different source to the one in my blkfile.

MatthewLM

legendary

Activity: 1190

Merit: 1004

Re: Correct header data for block 1

December 18, 2012, 05:33:33 PM

#5

Quote from: kjj on December 18, 2012, 05:31:49 PM
Double check the endianness of your prevhash. In the genesis block, it is all zeros, meaning that you still get the right result even if you have it backwards.

Just a guess.

That's a good guess kjj. Unfortunately I tried reversing the previous block hash and it still failed.

kjj

legendary

Activity: 1302

Merit: 1026

Re: Correct header data for block 1

December 18, 2012, 05:31:49 PM

#4

Double check the endianness of your prevhash. In the genesis block, it is all zeros, meaning that you still get the right result even if you have it backwards.

Just a guess.

MatthewLM

legendary

Activity: 1190

Merit: 1004

Re: Correct header data for block 1

December 18, 2012, 05:28:22 PM

#3

Hi koin. I am referring to the block with the index 1, aka. the 2nd block.

http://blockexplorer.com/b/1

This is driving me slightly insane. :-)

koin

legendary

Activity: 873

Merit: 1000

Re: Correct header data for block 1

December 18, 2012, 05:24:48 PM

#2

Quote from: MatthewLM on December 18, 2012, 05:11:06 PM
for some reason I cannot get a correct hash for block 1..

for block 1 or for the genesis block (block 0)? https://bitcointalksearch.org/topic/decoding-block-index-0-119530

MatthewLM

legendary

Activity: 1190

Merit: 1004

Re: Correct header data for block 1

December 18, 2012, 05:11:06 PM

#1

Hello. I have some code that calculates a hash for a block. It was working perfectly sometime before, but now for some reason I cannot get a correct hash for block 1. I can calculate the correct hash for the genesis block, but not block 1. I've taken what I've thought is the correct header data for block 1 and I've tried to hash it manually using a technique that works for the genesis block and it seems that the header data is incorrect. Can anyone find a problem with this data? I might be going crazy as I can't find it:

Code:
// VERSION
0x1, 0x0, 0x0, 0x0,
// PREV BLOCK HASH
0x6f, 0xe2, 0x8c, 0xa, 0xb6, 0xf1, 0xb3, 0x72, 0xc1, 0xa6, 0xa2, 0x46, 0xae, 0x63, 0xf7, 0x4f, 0x93, 0x1e, 0x83, 0x65, 0xe1, 0x5a, 0x8, 0x9c, 0x68, 0xd6, 0x19, 0x0, 0x0, 0x0, 0x0, 0x0,
// MERKLE ROOT
0x98, 0x20, 0x51, 0xfd, 0x1e, 0x4b, 0xa7, 0x44, 0xbb, 0xcb, 0x68, 0xe, 0x1f, 0xee, 0x14, 0x67, 0x7b, 0xa1, 0xa3, 0xc3, 0x54, 0xb, 0xf7, 0xb1, 0xcd, 0xb6, 0x6, 0xe8, 0x57, 0x23, 0x3e, 0xe,
// TIME
0x61, 0xbc, 0x66, 0x49,
// TARGET
0xff, 0xff, 0x0, 0x1d,
// NONCE
0x1, 0xe3, 0x62, 0x99

Thanks.