....
So, the default P2SH scriptPubKey would become:
scriptPubKey: OP_P2SH OP_HASH160 [20-byte-hash of {[pubkey] OP_CHECKSIG} ] OP_EQUAL
I think that this would address Luke's (somewhat valid) complaint about the magic transaction special case code path, while still being exactly P2SH.
So what happens if I put two OP_P2SH's in a scriptPubKey?
Either ignore it and leave the flag set, or reject the transaction as invalid. Ignoring it should be safe enough, but setting a limit (like 1) will prevent people from spamming up the chain with tons of pointless repeated NOPs.
Bounce the transaction, at least for now. Maybe some day in the future we will be able to safely handle a full blown OP_EVAL, but we don't have to treat it like one right now.
You could either allow it to be set conditionally when encountered inside the script, or ignore placement and set the flag either way based on finding the opcode during static analysis. Conditionally setting the flag might actually be useful. I need to study up on the conditionals, but it could end up being a pretty short path to (N of M) or K, with transactions that could be triggered by either the P2SH script, or by the ordinary key.
We could even require that it be the first opcode for now, with the desire that the restriction be relaxed in the future, if safe to do so.
Well, not really. There wouldn't be any magic scriptPubKey. Any valid scriptPubKey could become magic by including the NOP. In practice, this means that most P2SH scripts would be identical to NOP + the current magic script, but down the road when scripting is better developed, people would be able to trigger P2SH from an arbitrary script, maybe.
It could possibly turn out that there are no useful cases for combining ordinary scripts with P2SH, in which case it would turn out that all we've done is made the magic codes one byte longer. Even then, having an explicit marker for "hey, I'm a special transaction" could turn out to be worth the ~5% bloat.