But, in order to have a reference to B, it means that B is an unsigned transaction, ready to be signed and broadcasted.
Well, the fact that A1 has a reference to B means that B exists in somebodies wallet, which means it's output B1 has not been spent yet and has no reference, which means it hasn't been signed yet. And that's why it's unpublished.
the way bitcoin works is it has to sign first and then a txid is created, i'm pretty sure. so you're doing it differently in that respect. you create the txid before signing. hmm. ok fine. and as you said in your OP:
This may permit the owner to generate two equally valid signatures in a transaction but the reference in the previous output means that only one transaction is valid so either signature can be used within it.
but the reference is the txid, no? and that txid is the same for both signatures. no?
keep in mind you don't have a consensus mechanism in your scheme. so you have no way of having the network come to an agreement on which transaction to go with and which one to ignore.
actually i think i see what you mean you're talking about signature malleability not changing the transaction in any way, such as changing the outputs. still though, you using the word "publish" when you admit to not needing or using any consensus mechanism seems problematic.
In step 1, A1 needs a reference to B to be published but B doesn't need any references at that point.
In step 2, B1 needs a reference to C to be published but C doesn't need any references at that point.
So the transaction process is infinitely repeatable but it is not an infinite loop.
So I take it that the txid of B does not take into account any references that B would later contain before it gets signed and published. So you are generating the txid of B just based on its outputs, not taking into account the signature or txid of C. ok. how are you going to maintain consensus? i still think it needs a consensus mechanism, at the very least. since without that, you could create multiple "B" transactions and whats stopping different nodes from having different txid references in their "A1" without a consensus mechanism?