Pre-signed but unbroadcast or unconfirmed transactions seem to be a tough problem.
I disagree on the "tough" part. In my opinion this is less difficult than DOSbox/Wine on Linux or DOS subsystem in Windows 32 (and Itanium editions of Windows 64). It is more of the problem how much energy to spend on scoping the required area of backward compatibility and preparing/verifying test cases.
I don't think it is the same thing at all.
Support for legacy files and programs in newer relases of an OS is similar to the "clean fork" approach that I described. namely, the new software is aware of the old sematics and can use it when required. Any hard fork must have such backwards compatibilty, because it must recognize as valid all blocks and transactions that were confirmed before the fork.
Backwards compatibility in general is feasible as long as there is a feasible mapping of old semantics to the new infrastructure, and there is no technical or other reason to deny the conversion. However, that sometimes is impossible; e.g. if an old program tries to access hardware functions that are not accessible in newer hardware, or if the mapping would require decrypting and re-encrypting data without access to the keys.
Similar difficulties exist in handling an old transaction that was created before a soft fork but was broadcast only after it, and became invalid under new rules. The rules must have changed for a reason, so the transaction cannot simply be included in the blockchain as such. For example, suppose that the change consisted in imposing a strict limit to the complexity of signatures, to prevent "costly transaction" attacks. The miners cannot continue to accept old transactions according to old rules, because that would frustrate the goal of the fork.
(Note that there is no way for a miner to determine when a transaction T1 was signed. Even if it spends an UTXO in a transaction T2 that was confirmed only yesterday, it is possible that both T1 and T2 were signed a long time ago.)
maybe I am being a bit simplistic about this, but "unconfirmed" to me means that it hasnt been confirmed. So to require that all unconfirmed transactions must be confirmed contradicts the fundamental meaning of unconfirmed. What is the meaning of the word 'unconfirmed'?
I don't think that anyone is proposing to change the definition. Transactions that have not been broadcast yet
and transactions that are in the queue (mempool) of some nodes or miners, but are not safely buried into the blockchain, are equally
unconfirmed.
I meant to point out that there is no way that a client can make sure that an unconfirmed transaction will ever be confirmed, even if it seems to be valid by current rules. Everybody agrees on that.?
In fact, there is no way to put a probability value on that event, even with the usual assumptions of well-distributed mining etc. Everybody still agrees?
But then it follows that clients who hold signed transactions for broadcast at a later date cannot trust that they will be confirmed, even if they seem to be valid at the time of igning. Everybody OK with this?
Thus, there is no weight in the argument "we cannot do X because it would invalidate all pre-signed transactions that people are holding".