But how is this all reconciled among everyone who uses the network? Clients that aren't peers can't know all that is going on. How does everyone know what the fittest block is unless they have all the blocks? How can a client be sure that blocks XYZ were rejected for a valid reason unless you give them the full, signed copy? And even then they can't be sure unless they have the whole block chain as well.
I really need to draw something. I'll probably screw this up explaining in text. I'm going to explain the issue for anyone else who might stumble in. I know you already understand it.
So the issue is, if we have (N) peers all validating every broadcasted transaction. And every ten minutes we want all valid transactions batched up into an immutable block. And we want every node to agree on the same block of transactions, so we all share the same history. How do we reach consensus?
In Bitcoin, every node works redundantly, and POW randomness blesses one node's block. This blessing is based on time. Everyone agrees to use the first block they hear about.
GEM's doesn't have this timing randomness as a starting point. Everyone's 10 minutes happens at the same time. So to replace the above process randomness, I'm suggesting a "fitness" function.
---
Say we have lettered nodes and numbered blocks that increase with time. And say (via magic) that Nodes A, B, C & D are all on the same block chain.
1) Contains announcement transactions from A, B, C, D
2) A, B, C, D
3) A, B, C, D
4) A, B, C, D (this is the announcement pattern everyone expects to see if the network is not partitioned.)
----
So, the first thing that happens is, nodes A, B, C & D broadcast announcement(4) transactions to tell everyone they are building upon block (4) in the chain.
Every node receives these announcements, validates the transactions and places them in the next block they are building.
Every node also receive normal transactions, validates them and keep adding them to their blocks.
[time passes, 10 minute timer elapses]
----
Now we have four independent potential next blocks. (5a, 5b, 5c, 5d)
We need a bit of randomness that can pick one so it can be broadcast to the others.
Before we can pick one, we need to know the set we are picking from.
Fortunately block (4) contains announcements from all the nodes who were around 20 minutes ago.
Our currently building block (5) contains announcements from all the nodes around in the past 10 minutes.
Let's assume that if a node was validating transactions 20 minutes ago and they are in our building block, then they are a candidate to choose from for block (5).
So as a trivial example fitness function, let's use the
hamming distance between block (4)'s hash, and the hashes of announcement transactions (A-D) inside of block (4). All nodes have that information already. (Hamming distance means XOR two hashes and count the ones digits in the result. So the result will be a number between 0 and 256 for Sha256 hashes.)
Say we get the following results:
sha(4) hamming sha(4A) = 128
sha(4) hamming sha(4B) = 125
sha(4) hamming sha(4C) = 127
sha(4) hamming sha(4D) = 130
That means the nodes in fitness order are: B, C, A, D
-- Node A --
Since Node (B) is the "fittest" (B) broadcasts his transaction block (5b) first.
Every node receives block (5b), revalidates it, diffs it with their personal block (5n) looking for transaction DOS and double spends.
If block (5b) fails,
the node removes node B's trust.
And broadcasts a warning transaction, "node(A) failed node(B, 5b) for DOS"
Then asks the next fittest node (C) for their transaction block (5c)
If block (5c) passes, and the node broadcasts an announcement(5c) transaction.
----
Since this all happens in parallel, node (A) receives announcements as the other nodes make decisions.
If the other nodes agree with (A) about (B's) transaction DOS, then node (A) will receive:
A's Announce(5c)
B's Announce(5b)
C's Announce(5c)
D's Announce(5c)
"node(A) failed node(B, 5b) for DOS"
"node(C) failed node(B, 5b) for DOS"
"node(D) failed node(B, 5b) for DOS"
If the other nodes disagree, then node (A) will receive:
A's Announce(5c)
"node(A) failed node(B, 5b) for DOS"
B's Announce(5b)
C's Announce(5b)
D's Announce(5b)
If A receives the latter it can change its mind, switch to 5b and re-announce to preserve consensus.
----
Anyway I was right. It needs a diagram. But that's the gist of the process.
This is a terrible, terrible thing though. Surely you can see that.
Yes, its almost a nuclear option. That's why I'm so sure nobody will let it happen. And why I want everyone to have advance warning is somebody even tries.
I asked in another thread what the exchanges do if a block chain fork attempts to erase/revert a "confirmed" transaction block they had already payed out on. The reply was "they shut everything down until everyone figures out what the fuck is happening!" I'm not sure it was an authoritative answer, but it made perfect sense.
That was basically what happened when they forked to bitcoin chain to remove the extra 184 billion coin. Absolutely everything stopped while everyone emailed everybody they knew to stop the old clients and start new ones. Fortunately, few were doing business or trading at that point. But if they were, those trading on the original client would have gotten a hell of a shock when 53 blocks of their transactions magically disappeared!