Replying to some development-related stuff from the other thread . . .
Paid or not Peter this is extremely helpful, thank you (though it will take multiple readings for me to even remotely try to understand it all!).
JR, without wanting to be badgering I feel this is the most critical issue for mastercoin right now. How we store data in the blockchain is fundamental and should be locked away as early as possible (and ideally written into the spec). As you made that the number one goal of the first coding contest I'm guessing you feel the same
Tachikoma's work is great but perhaps we could refine it somewhat - in development we're spitting out invalid ECDSA points in multi-sigs some of the time and as Jeff has raised above, pubkey checking is on the way regardless of mastercoin.
As I've mentioned before I view the miners as our critical issue and if some of the major pools are doing things like looking for obvious compressable/text data as Peter notes, perhaps some further investigation into steganography (an apt analogy I think) would be in order - obfuscating the data we're storing in the pubkey (IMO) seems quite achievable and spotting our padding ("000000000" etc) isn't going to be hard.
Thoughts?
I agree that Peter's comments are extremely helpful. I think you guys are on the right track (massaging Tachikoma's multisig method output until it is valid). I can offer some general guidelines here:
- We don't want to to anything that can be easily blocked without specifically targeting MasterCoin. For instance anything which could be screened out by a validity check, or a "too much ASCII" check
- We can't really worry about blacklists. If something like that happens, we have a few options. The Exodus Address can always be changed. We could even get rid of it altogether, although that would make finding MasterCoins transactions MUCH more difficult (we'd have to watch the transactions from any address which holds MasterCoins).
- We also have the "nuclear option" of returning to the original spec (spending to fake addresses), but I don't think that will be needed.
- All clients should be generous with fees to bitcoin miners by default. We want to create a situation where an anti-mastercoin stance by a miner should be very expensive for them!
Found the problem, the official spec is still not updated. This makes it a lot harder for new developers (and developers who forget stuff...)
J.R. Could you please update the spec with all the new data packet order stuff and the new validation rules? (broken sequence, ambiguous sequence etc.)
Rolling out fix in the next few minutes.
Edit:
Transaction should be fixed.
I somehow missed part of the sequence discussion.
If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)
If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!
If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!
Is this really necessary? We have no control over the sequence number for a change address, is it fair to disregard a transaction based on random luck? I would like to propose to do a sanity check before invalidating a transaction based on sequence. You could try to decode the address and see if the transaction_type and the other options based on that transaction_type make sense. If one of the addresses with the same sequence for instance does that, but an other does not you know which one is the data address and which one the target. If that also fails it seems safe to flag it as invalid but I think we should try harder before falling back to invalid.
Sorry for the out-of-date spec. Fixing this (and adding some new stuff) is at the top of my list.
Note that the sequence number of the change address only needs to be considered when the change amount is exactly the same as all the other outputs. Invalidating a MasterCoin transaction because the change amount just happens to be 0.00006 BTC AND a bad sequence number should be exceedingly rare.
Multisig outputs are currently hard to spend, the reference client has no Gui support for it. It will need to be build into a Mastercoin wallet to make it easier. I think we better wait until there are some stable-ish wallets out before doing this.
Simple sends transactions would have plenty of space in the output for a target address, other messages however might not have enough space. Perhaps we can support both? If there is room in the multisig use that as target address, if not use an other output.
I have showed already in
https://github.com/grazcoin/mastercoin-tools/blob/master/NOTES how to spend multisig using the sx, and my wallet will support (eventually) also multisig as input, so:
It's not hard for us to do, but for somebody who needs a GUI to make payments it will be. Until we have a user friendly way to do it I rather not support it yet. But let's see what J.R. has to say about the topic :}
Spending multisig bitcoins is painful right now, and I'd rather avoid that extra complexity until it isn't painful. Let's keep that extra output to the destination address for now. I think that supporting two different encoding methods will end up increasing the testing burden dramatically, and I'd rather not do that yet.
Let me see if I understand the current proposal:
We are (if I understand it correctly) taking Tachikoma's proposed multisig, then for each key after the first one we change the sequence number until we get a valid ECDSA point? Can anybody explain why this problem only affects keys after the first one (sorry if I missed this somewhere)?
Current valid transactions (not using multisig) have the last data packet sequence number before the target address be n-1. Theoretically, earlier data packets would have had n-2, n-3, all the way back to the first data packet (although simple send never has more than one data packet). Perhaps for multi-sig we should change this paradigm. It might be easier to start at the sequence number of the destination/target address, then increment up from there. If a given sequence number does not pass our "valid ECDSA" test, we just go on to the next one. Is that what you guys are doing already?
It sounds like this method significantly reduces how much data we can potentially store per transaction. Can anybody give a guess as to what a reasonable upper bound for how much data we should try to store using this method in a single transaction?