Blockchain Test Data | Bitcointalksearch.org

piotr_n

legendary

Activity: 2053

Merit: 1356

aka tonikt

Quote from: pmlyon on June 18, 2013, 06:54:41 AM

Does there exist a set of blocks that can be used for unit testing? I'm thinking something like a few thousand blocks that is representative of everything a node would need to implement.

testnet3 chain is very good for this.
it does not have everything, but I believe it has enough to keep you busy for awhile...

etotheipi

legendary

Activity: 1428

Merit: 1093

Core Armory Developer

Quote from: pmlyon on June 18, 2013, 09:18:00 AM

I think you'd want a one-click repeatable way to generate the test data as well. Some kind of program that would interact with bitcoind, for example, to generate the transactions and blocks. I'm not sure how much invalid data you could introduce directly using bitcoind, but you could also twiddle the bits after the fact to introduce errors.

This still gives the raw blocks, so other implementations don't need to depend on bitcoind at all for the test suite. But it means you could roll out updates to the test suite much more easily.

A long time ago I created a re-org unit test:

https://bitcointalksearch.org/topic/m.577556

I created a completely valid blockchain, with valid signatures and proof of work, an orphan chain and a double-spend. It's valid as long as you set your COINBASE_MATURITY to 1 (so coinbase blocks are immediately spendable). This has been wildly invaluable for me to test all kinds of different logic. The only thing I regret doing was not adding a couple more blocks to re-org back onto the original chain. That would've been even more stressful...

pmlyon

member

Activity: 72

Merit: 10

I think you'd want a one-click repeatable way to generate the test data as well. Some kind of program that would interact with bitcoind, for example, to generate the transactions and blocks. I'm not sure how much invalid data you could introduce directly using bitcoind, but you could also twiddle the bits after the fact to introduce errors.

This still gives the raw blocks, so other implementations don't need to depend on bitcoind at all for the test suite. But it means you could roll out updates to the test suite much more easily.

Sukrim

legendary

Activity: 2618

Merit: 1007

Ideally this data would be starting with the Bitcoin genesis block and be structured in a way that blocks can be added from the outside too (e.g. a new block is announced on the network but it builds on a block 10 blocks behind, then the next 10 blocks also build on this fork and your client needs to be able to reorg).

Also some blocks with strange transactions, invalid headers (that have difficulty just off by one - as you start with diff=1 it shouldn't be too hard to find this), difficulty rising/falling more than 4-fold after a retarget...

I guess it would be interesting to have such a data set.

pmlyon

member

Activity: 72

Merit: 10

Ahh, that makes more sense.

What I'm thinking is a bunch of blocks, some valid, some not. You'd have transactions, some valid, some not. Things like orphan blocks would be present. I'm still really new to this, so I can't really give a full list of what all would comprise a representative sample.

You could feed this data into a parser, yes. You could also feed it into a system that validates blocks, another that deals with constructing the blockchain and handling orphans, etc. You could make a mini unit test network consisting of a couple of nodes broadcasting the test data blocks, and use that to validate node functionality.

The expected results could even be described in some kind of format so that different implementations written in different languages could leverage this same set of test data, creating a validation suite of sorts.

jackjack

legendary

Activity: 1176

Merit: 1280

May Bitcoin be touched by his Noodly Appendage

I was suggesting using only two blocks, the first and the last one.
I'm not sure I really understand your question though: are you looking for data to test a blockchain parser?

pmlyon

member

Activity: 72

Merit: 10

Using the real blockchain would be too slow and large for unit testing purposes, I think.

jackjack

legendary

Activity: 1176

Merit: 1280

May Bitcoin be touched by his Noodly Appendage

All blocks have the same, fixed structure so using the genesis block and the last block should be enough
It's reading transaction that can be trickier

pmlyon

member

Activity: 72

Merit: 10

Does there exist a set of blocks that can be used for unit testing? I'm thinking something like a few thousand blocks that is representative of everything a node would need to implement.

Topic: Blockchain Test Data (Read 734 times)