Here are some reasons that this hash shouldn't be some (case insensitive) hash of the root public payment key itself, but rather, as you suggested, a hash of a public key (ECDSA or not) that is used to sign the root public payment key.
You want to put as little additional information as possible into the cert that you pay the CA for and keep that information as static as possible (never update). So let's just place one single hash in the form hash.main-domain.com, which is the hash of a bitcoin-specific cert that we define. The bitcoin-specific cert can contain
- more than one root key (e.g. different types, singlesig and multisig)
- keys for purposes other than payment, like
- signing derived weaker certs (e.g. for agents)
- signing payment root keys valid for certain periods of time
- keys from other cryptosystems than ECDSA
Yes, it would be technically usable right now. I made an attempt (https://github.com/bcpki/bitcoin) at defining a bitcoin-specific cert (protobuf) and implementing the payment address derivation (among other things), which is usable. Of course, the final link from the bitcoin-specific cert to the CA cert is missing. Not sure if BIP32 can be used because it does significantly more than what we need here. Not sure if we need a "compatible wallet format" at all. We can just handle this as an import into the existing wallet (an RPC doing exactly that is implemented in the above mentioned project). It seems the only things we need to agree on is the derivation function (two choices: additive or multiplicative) and endianess of "InvoiceID". And probably also on the format of "Invoice" (text or binary) and the hash function that produces "InvoiceID" (probably Hash160).
For compatibility with the currently developed payment requests I suggest to sign (this happens offline) the current root public payment key with some key that's defined for that purpose in the bitcoin-specific cert, and then store this root public payment key (in explicit form, not its address) online and ship it with every PaymentRequest message.
[EDIT] I just realized that with "Invoice" and "InvoiceID" I had something different in mind than the OP, which led to some confusion on my part and to my scepticism that BIP32 could be used as is.
What if InvoiceID is entirely customer generated (unlike in the OP)? Then it doens't fit in BIP32 anymore. Since there are strong use cases for customer generated InvoiceIDs, we should definitely allow those.
[/EDIT]