This may have already been created, but in addition to each client software having its own test suite, it would be possible to test clients by connecting to them.
If a client can be ordered to connect to a single node and/or only accept incoming connections, then the testing software would be able to feed the client a completely false chain. This chain could be kept at difficulty 1.
For example, to test that message CRCs are calculated correctly, the test software could send a false CRC on an otherwise correct message. Acceptable results would be that the client disconnects or that it drops the message. There could be 1 test for each message type.
The only hiccup is checkpoints. Clients would need a way to disable them. From looking at the code, the standard client can do it with -checkpoints false.
Checkpoints could be checked too, but they may different between clients.
Some of the BIP stuff has blocks where the rule changes. With a difficulty of 1, that is 4 billion hashes. At 100 MH/s, that is 40 seconds per block. To generate 300k blocks would require 138 computer days to generate a difficulty 1 chain. This would be much faster if more than 1 computer was used and obviously faster for an FPGA or ASIC miner.
The chain could be a chain with just block headers paying to random addresses. It would just be so that it could be fed to a client to test breakpoints in the rules.
The advantage of this system is that before releasing a new version, any client author could verify that their client passes all the standard tests without having to integrate them into their build environment. It would also give consistent testing between all clients.
With luck forks would be detected before a release of any client.
Topic: Generic client tester (Read 500 times)
Jump to: