The point of OP_CODESEPARATOR is not a mystery. It is dead code left over from a time when Bitcoin did not execute scriptSig+scriptPubKey by running them separately with a shared stack but actually concatenated the programs together. This was a major security hole as you could just set scriptSig to OP_RETURN and make it satisfy any output. It is by far the worst security problem Bitcoin ever had. When he fixed it, Satoshi didn't bother to clean up all the code that supported the old method and thus we are left with OP_CODESEPARATOR. It has no utility today.
(in other words Peter is right - it's leftovers from a design mistake).
Actually I was thinking that even when Bitcoin worked that way OP_CODESEPARATOR would have served no purpose. However I was thinking about it more, and realized that if you didn't insert OP_CODESEPARATOR between the scriptSig and scriptPubKey before calling EvalScript() you could do some interesting things.
Basically I would be able to write a scriptPubKey like this:
OP_CHECKSIG OP_VERIFY OP_CHECKSIG
This is spendable by the following scriptSig:
OP_CODESEPARATOR
sig1 is a signature made by pubkey1 and what sig1 signs is the transaction; nothing exciting yet. But because the scriptSig and scriptPubkey are combined, and Bitcoin happily executes op's in scriptSigs, pubkey1 can also sign new opcodes that have to be present in the scriptSig for the signature to be valid. For instance pubkey1 could make a signature only spendable if the scriptSig includes this:
OP_CHECKSIG OP_VERIFY
Essentially pubkey1 has delegated the ability to sign the transaction to pubkey3 after the fact. Now if pubkey2 wants to spend the transaction they create a signature of their own over the transaction including those new opcodes, and create the following scriptSig:
OP_CODESEPARATOR OP_CHECKSIG OP_VERIFY
sig3, sig1 and then sig2 are all checked, and they are all hash a transaction including the following combined scriptSig+scriptPubkey:
OP_CHECKSIG OP_VERIFY OP_CHECKSIG OP_VERIFY OP_CHECKSIG
These days the scriptSig and scriptPubKey evaluated separately, and critically that means that when the scriptPubKey is evaluated pbegincodehash is set to the beginning of the scriptPubKey - there is no way to have a signature check any part of the scriptSig. Secondly there is no way for the scriptPubKey to determine what is in the scriptSig other than the contents of the data stack - there's no way to access the scripts themselves. Even though opcodes can be executed in the scriptSig there is no way that the scriptPubKey can make sure the right ones are executed.
Interestingly Luke's
CHECKHASHVERIFY BIP is really proposing something very similar to what I'm talking about above, except in how it relies on a strict hash verification rather than signatures; Luke's implementation may have even enabled the above, although I can't seem to find the code to check. (it'd be bad if it did though because it's a hard-fork) The right design would probably be something like OP_GETCODE to put the serialized code on the stack, followed by OP_HASH/OP_EQUAL/OP_VERIFY for the P2SH case, and OP_HASH/OP_SIGCHECK/OP_VERIFY. Add in some more opcodes to be able to get the other parts of a transaction, and match that against templates, and you'd have a lot of interesting capabilities.
Really Satoshi's one mistake was allowing OP_RETURN to return true in the scriptSig; if he hadn't put in OP_RETURN, or had the current behavior where OP_RETURN can only make the script fail, we could have kept combined scriptSig and scriptPubKey processing. Worth thinking about for the future if a script v2.0 ever happens.
edit: Note how all of the above works even better if there is some kind of OP_CODESEPARATOR "stack", or if OP_CHECKSIG can accept a number for how many OP_CODESEPARATORS "back" it counts; consider what happens when you try to do the delegation a second time.