What you describe will remove a transaction from Bitcoin. This removes the "already spent" status of particular unspent txouts (individual bitcoin payments you've received from someone else and change inputs from previous transactions, called "inputs" when you look at block explorer sites) and returns them to your balance sheet. However, this alone neither prevents the transaction from later being included in the blockchain or guarantees that the previous transaction will be invalidated with a new spend from your wallet.
Picture this scenario:
-My wallet has a 1BTC and a 2BTC payment I've received and no other transactions.
-I send seller 1BTC with no fee (this transaction uses the wallet's unspent 1btc txout)
-which doesn't confirm in a timely manner.
Then:
-I follow this procedure to remove the transaction from my Bitcoin client and return the balance to Bitcoins I can spend.
Now:
-I resend 1BTC to the merchant with a 0.01 voluntary fee
-The wallet picks the most sensible txouts to construct the transaction of 1.01BTC, which is to use the 2BTC payment instead. 1.01BTC is sent to the sender and the wallet gets 0.99BTC change back.
Result:
-Merchant quickly gets new 1BTC payment
-Original transaction is not invalidated by any of it's txouts being double-spent
-But... surprise, some miner includes the original transaction in a block, now you have paid twice.
You must document the actual txouts spent by a payment before removing it, and use a coin-control type wallet to re-spend some of the same txouts with appropriate fee; only when some of the same txouts are confirmed in the blockchain could the original payment be considered invalid.
There was a previous discussion about adding a re-send type feature to double-spend with more fee, with code for miners to recognize the new transaction as legitimate as it did not alter the original payment:
I'm working on a patch that will create the basic infrastructure to add fees to transactions that have been sent but not yet included in a block. It will only let you add additional inputs and outputs, but never make any output have a lower value than it did before. Nor will it let you modify a transaction that has already been spent.
I've been considering what criteria nodes should have to allow discarding a pending in-memory transaction and replace it with an increased-fee transaction. Your conclusion is logical.
Since the output of a transaction that is change can't be identified by miners or nodes, to validate what constitutes a legitimate non-double-spend transaction, it appears that a retransmitted transaction must not alter any inputs or outputs in the original transaction. If one of the outputs were allowed to be reduced to increase the fee, this could mean a 0 confirmation payment to someone could be changed from the expected value, which is the definition of a malevolent double-spend. If any output could be modified in the original, this means that 0 confirm payments could not be trustable, because output values could be moved from one recipient to another or reduced.
Instead, fee must solely be added by adding a new input, and a separate change payment must be sent. In the example below, I show where we add another complete input and change (where fee = input - change).
OutputX, another payment to another recipient (or a third change payment to enhance anonymity even more), could be optional and shouldn't be a "blocker" if it exists. Change from adding a fee demands some new outputs. However, being able to add such "mini-transaction" chunks with new inputs and outputs and retransmit them, could mean that a service that issues many regular payments could just keep adding to an existing 0 confirmation transaction and retransmitting, instead of creating a new transaction. I don't know if this would be a novel feature or undesirable.
Another strange situation is if the original tx has spent all the available inputs, the user can't increase the fee. Explaining why the wallet can't add more fee even though it shows a balance will be one for the UI designers.