Hi slush!
I'm fully aware that those requirements are contradictory. It's pretty common that some solution needs to find some compromises.
After waiting a while and reading the replies and comments to your proposal I now understand your approach and my mistakes. Basically I only took a helicopter tour of Klondike, but you are the one who has a ground floor experience of Klondike. I'm kinda living a sheltered life: I'm never on the first line when talking to my customers: before they talk to me they need to jump a trench or two in my organization. I'm not avoiding my customers, it just that the ones I interact with were already filtered. This is completely unlike you: are the one who speaks to every customer.
I now understand that there's no point of trying to require advanced features like pair-of-FSMs design if the potential users are only familiar and comfortable with RPC-style designs. I searched this board for any sign of anyone writing about state-of-the-art financial services technology. There wasn't anyone who would mention things like CICS,Tuxedo or even no-cost MTS & MSDTC.
If the residents of Klondike demand lighter pickaxes there's no point of offering hydraulicking.
Although I personally dislike XML, I'm open to change my mind at this point.
I wouldn't recommend switching to XML. What I was trying to suggest is to try re-host some features of XML-based protocols on top of JSON or whatever else you choose. I think you already doing that. The main advantage of SOAP is that first-line tech support person can say things like "Parasoft SOAtest agrees with our diagnosis, the errors are on your side" and avoid the cost of escalated support calls.
Isn't TCP checksumming and TCP retransmissions on both ends enough to "fix" corrupted information?
Well, NAT breaks the end-to-end principle of IP networking. But the real culprits are usually various marketing-motivated enhancements like SPI (stateful packet inspection); "gaming mode"; "multimedia prioritization", etc.
I feel like overlay network should be "stateless", which mean that it will "forget" transaction once transaction has been succesfully broadcasted to Bitcoin network.
The "stateless" discussions remind me of "zero configuration" discussions in other areas. After all the late-discovered problems get fixed the "stateless" design is way more complex and fragile than the competing "stateful" designs.
My general suggestions can be summarized as follows:
1) store coins with as key pairs plus the block hash and height in chain
2) client connecting to server send a chain length and tip block-hash of the last connection
3) server responds with positive difference in height and new tip block-hash
4) in case of the reorg server responds with negative difference in height and hash of last block pre-fork followed by the same information as in (3) for the new branch.
5) client can recalculate the wallet and retry the transactions that still have valid inputs after the reorg
6) server takes a proactive retrying approach to the transactions it originated. This is particularly important in case of the reorg and when the originating client already disconnected.
I didn't do a complete
http://en.wikipedia.org/wiki/Transaction_processing analysis, but there is a number of possible compromises in the spectrum between a fully ACID-compliant design and a ball-of-rubberbands design.