Topic: Cancelling unconfirmed transactions (Read 21155 times)

Here is the status of my work on tx replacement: http://www.mail-archive.com/[email protected]/msg03126.html

That particular version is the "zeroconf safe" version that only lets you replace a transaction if all outputs are paid in the replacement with equal or greater value. I also wrote a pure replace-by-fee version months ago. What's left to be done is the hard and risky work of fixing the wallet code.

Note that John Dillon has offered 4BTC to whomever gets it done - if anyone wants to take on the wallet side of things it'd fix some outstanding issues and I think the effort would deserve 95% of the funds.

Unconfirmed wallet transactions are automatically rebroadcast by bitcoind/Bitcoin-QT until they are confirmed.

That's all well and good. But what about the user with zero IT knowledge? Problems have to be automatically taken care of for them by the wallet software as much as possible.


Are either of these closer to implementation?

- http://en.bitcoin.it/wiki/Raw_Transactions#Re-broadcast_a_transaction

Yes a Rebroadcast feature should b e added. There's no way to verify at least from what Im seeing if a QT is doing this.

I appreciate your reply it gives better sense. But still there is bad interface design. Compare with Electrum -- all connected peers failed to follow my transaction (rare thing but it happens). In 5 seconds client reports error and releases funds, allowing me to try to send again or use another wallet. Though "official" client keep funds in hostage for hours driving me mad as well as fund receiver. And it happened to me 3 or 4 times already.

I don't believe that during that hours in the background client trying hard to rebroadcast pseudo-randomly. If it doesn't go from the first attempt it will go on the second.

There is different behavior based on the client.

With Bitcoin-Qt/bitcoind, the client (if left running) will sporadically attempt to re-broadcast the transaction indefinitely.   If the reason it doesn't end up in a block is because it is a low-priority transaction and a fee wasn't paid then there is no solution other than to perform wallet surgery (using a tool like jack jack's pywallet utility) to delete the unusable transaction.  

The hybrid E-Wallet from Blockchain.info behaves differently.  It will eventually stop retrying (in about a day) and the transaction removed from the wallet, allowing the user to try to spend the fund again.

I'm not familiar enough with Multibit, Electrum and other clients to know their behavior.

As far as the ability to bump an existing transaction there have been two approaches, ... "parent pays for child" and "replace by fee", however neither is targeted for inclusion in any specific upcoming release.

I know there are good reasons but to the casual observer and user it does seem that Bitcoin is a bit rough-and-ready if transactions can get "stuck" in cyberspace and the sender can't do anything about it.

If transactions can't be easily cancelled while pending, could they have a user-defined expiry date-time at origination, with a default of, say, +24 hours? Then, any unconfirmed transactions beyond expiry are deleted from the pool. Wallets can also then re-credit the BTC to the sender, knowing the amount is not going to be processed, and the transaction could be done again with a higher fee.

There is no preset time limit.  The node psuedo-randomly rebroadcasts transactions.  Your node has no method of knowing if other nodes are still retaining a copy of your unconfirmed tx, only that a tx is self created and has been broadcast.  It broadcasts the tx, and then wait, if it sees the tx in a block it removes it from its memory pool.  When a self generated tx in its memory pool becomes stale it broadcasts it again.  It is possible peers already know of that tx however it doesn't check it simply broadcasts the tx and refreshes its status in its memory pool.

Spamming a tx to all nodes continually until they appear in a block would use up a lot of network resources and that could be exploited by an attacker as camouflage to create a denial of service attack.  In other words if "well behaved" nodes are spammy then it becomes harder to identify and ban truly malicious nodes.  As it is right now actual malicious are easy to spot and ban.

Bitcoin is an open source project.  The rebroadcast algorithm can be improved so feel free to do a pull (or bounty for a pull).
Keep in mind however that:
a) an node has no method of knowing if an arbitrary unconfirmed tx is in the memory pool of peers without asking
b) when a node drops a tx from its memory pool either on receipt (invalid tx or insufficient fee) or when memory pool is full it does not notify the sending node.
c) a node should rebroadcast tx to peers on random intervals to avoid giving away information (i.e. your node sends the same tx every 1 minute to all 8 peers an a malicious entity is two of your peers you have essentially identified yourself as the originator)
d) the volume of communication should not make it possible for a malicious node to use that as cover to spam the network.

One attempt per 45 minutes to avoid DDOS? You were kidding, right?

Actually the reason for the slow broadcasts is to not give away information on what coins are in your wallet.

Replace-by-fee  helps with privacy here actually as re-broadcasts would generally have fee increases and thus any node could have sent them.

It does rebroadcast just not at a rate which would be seen as a denial of service attack.

Bloody client program holds my funds neither bouncing them back nor pushing to the network acting as The Dog in the Manger. Which part of this is good for the network?

Don't ever confuse "good for the network" with "good for me".

I wonder to whom else (except you) it could be normal and desired to wait hours before program deigns to broadcast a transaction? Attempt on every other minute must be backbreaking effort for the said program.

That's normal (and desired) behavior.

If you want to force a re-broadcast to occur on-demand:
 - http://en.bitcoin.it/wiki/Raw_Transactions#Re-broadcast_a_transaction

In my case it happened several times even the client was online. Either network was too busy or whatever else but transaction stuck for hours. Receiver was going mad because he though I was lying. And it was not petty money. Even I would like to re-send from another place I was bound to that hung transaction which can dispatch any moment or stay frozen for longer.

There is good technical logic behind asynchronous behavior in many programs. But here it is proven fault of this pattern. Developers should take action.

Update:  the transaction finally arrived while i was typing the above post.  Still, it took a good 45 minutes instead of the expected < 5 seconds to appear in the destination wallet.  And the messaging in the client is useless at that point as it gives no indication of retries or what is going on with the transaction.

This just happened to me.  ( Using primecoin-qt, but I expect it applies to bitcoin-qt also. )   My network went offline and I didn't realize it before I made a payment to another wallet of mine.  When the network came back, the wallet synced with the network, but the status stayed 0/offline.  After 10 minutes or so, I restarted the QT wallet with -rescan option.  Now it says status is Unconfirmed, which suggests to me that the client thinks it has been broadcast.  However, my receiving wallet does not see the transaction when I use listtransactions.  and it has been at least 30 minutes now.

Fortunately I was sending a small amount, and I know I can export/import if need be...  and I guess there is some chance it will fix itself with time, but I could see how this could really bite a non-technical person hard.

And it is not an uncommon situation.... really easy to get into actually.

If "cancel transaction" is not an option, then it seems to me that the client should not allow offline sending.

Or a "rebroadcast transaction" option.   Or at the very least a "will retry again in 60 seconds" tip text.

Right now, the messaging is useless, and the transaction is very clearly stuck.

It is not necessary that transaction will be broadcasted at first attempt even if you press send button. Easy to reproduce. Take Satoshi client. Disconnect network. Send some bitcoins. See it appear in transaction list. It is unbroadcasted transaction. And it has same grey "?" sign as 0 conf transaction.

This behavior is client dependent. Other clients (i.e. Electrum) do not queue transactions but instead bounce it if broadcasting was unsuccessful.

I'm sure none of it's "double spending". AFAIK real double spending is when you get your transactions fraudulently confirmed in 2 different places in a block because of a continuous 51% attack. Or when you pay for something, get the transaction confirmed X times, but then the blocks get orphaned because of ad-hoc network crappyness/man-in-the-middle exploits/51% attack so you're able to "re-spend" the coins elsewhere.

My view is that the memory pool is just a queue with zero guarantees, and it should be treated that way! It should not be used as a checking/IOU system. Having pending transactions stuck indefinitely and being forced to use 3rd party software sucks balls. The system should be improved so that:

• Sending the same transaction multiple times but with different fees is standard practice. No need to treat the first attempt like it's sacred .
• Harden the memory queue against spamming, and do it properly. How do website or email systems do it? How about: for each request, send a small proof-of-work problem for the wallet software to solve? That way it's like a small fee.
• Get some pricing visibility. Confirmation delay vs fee generosity needs to be graphed and put on a website somewhere, or even embedded in the wallet software.

You can delete transactions from your wallet
You can also export all your private keys to a CSV file, then import them all in a clean wallet
Both can be done by pywallet

There is no "unbroadcasted" transaction, except when you haven't press the send button yet. However a transaction that does not include at least the minimum relay fee (currently 0.0001 BTC/kB) will not be communicated through the network to others by 99% of the Bitcoin nodes or recognized by miners, making it's timely inclusion in the blockchain even less likely. A replacement transaction that does include the required fee will be the one that is used, because it is not considered a double-spend by everybody that ignored the first one.

One should determine between unconfirmed transaction and unbroadcasted transaction (what blablahblah was trying to say). Both look similar in the transaction list of Bitcoin client. But first has information in the chain already and latter doesn't.

It is up to client implementation. Satoshi client allows unbroadcasted transactions in it's internal pool (and by the way takes long time before attempt to re-broadcast it again). Other clients just don't even hold such transaction. They either broadcast them successfully or reject.

So unconfirmed transaction are not to be taken out of the chain. But unbroadcasted transaction are removable from the client queue (though with some third party software, Satoshi client doesn't have such functionality out of the box).

If the transaction was sent with no fee when a minimum was required, the rebroadcast one with adequate fees (something more than the minimum) will probably go right through. There is no guarantee that it will use the same input payments and actually double-spend, unless you have a very minimal receiving history or send your entire balance. Note that this will make your Bitcoin-qt wallet your new wallet, as it will be the only one that has the change transaction, Bitcoin contains multiple addresses and doesn't reuse them.

I just did that same thing, because Im an idiot. I'm using blockchain.info's online wallet, but I can get the private key and run it off the bitcoin client locally. Can someone explain to me what would stop me from "double spending" the coins that are stuck in limbo, and would it actually be "double spending"?

Most of this topic is beyond the scope of my understanding. Is there any simple instructions how to get my unconfirmed transaction confirmed?

I'm a pragmatist. If I wrote a patch that replaced transactions with any newer one if the total fees were increased, it would never get into the reference client.


As an aside you must require a limited resource to be expended to broadcast a transaction on the network for DoS protection. This also implies that the network bandwidth required to run a full node will now be an even further higher multiple of the "blockchain bandwidth" than it already is. How to price that properly is an open question.

On one hand bitcoin's network structure + code is not the smartest one around and relies on flooding, so sending the same thing in different versions might cause even more problems.

On the other hand memory is even more expensive and probably limited amongst nodes than storage capacity for blocks. Bitcoin-qt already takes up ~half a GB of RAM on my machine, if it then has to store multiple versions of each transaction someone decides to publish it might get out of hand quickly. This stuff is not specified by the way and doesn't really influence the way blocks are built, so you can create your own client with a smarter netcode + multiple versions of the same transaction. Bitcoin-qt will probably not like your client, as it will be viewed as potential double-spender (or at least trying to cheat somehow), but as long as you manage to get these transactions to miners, I don't really see a problem that this cannot be done.

I would advise against it however, as there is little benefit to having multiple versions vs. updating one version, especially since in the end only one single version will be included anyways.

I've been considering what criteria nodes should have to allow discarding a pending in-memory transaction and replace it with an increased-fee transaction. Your conclusion is logical.

Since the output of a transaction that is change can't be identified by miners or nodes, to validate what constitutes a legitimate non-double-spend transaction, it appears that a retransmitted transaction must not alter any inputs or outputs in the original transaction. If one of the outputs were allowed to be reduced to increase the fee, this could mean a 0 confirmation payment to someone could be changed from the expected value, which is the definition of a malevolent double-spend. If any output could be modified in the original, this means that 0 confirm payments could not be trustable, because output values could be moved from one recipient to another or reduced.

Instead, fee must solely be added by adding a new input, and a separate change payment must be sent. In the example below, I show where we add another complete input and change (where fee = input - change).



OutputX, another payment to another recipient (or a third change payment to enhance anonymity even more), could be optional and shouldn't be a "blocker" if it exists. Change from adding a fee demands some new outputs. However, being able to add such "mini-transaction" chunks with new inputs and outputs and retransmit them, could mean that a service that issues many regular payments could just keep adding to an existing 0 confirmation transaction and retransmitting, instead of creating a new transaction. I don't know if this would be a novel feature or undesirable.

Another strange situation is if the original tx has spent all the available inputs, the user can't increase the fee. Explaining why the wallet can't add more fee even though it shows a balance will be one for the UI designers.

I'm working on a patch that will create the basic infrastructure to add fees to transactions that have been sent but not yet included in a block. It will only let you add additional inputs and outputs, but never make any output have a lower value than it did before. Nor will it let you modify a transaction that has already been spent.

Unfortunately this kind of thing takes quite awhile to get implemented due to all the careful testing required. Expect a few months before it is usable.

No a transaction is just that a transaction.
A confirmed transaction is one which is recorded in a block.
An unconfirmed transaction is one which is in the memory pool.

Bitcoin was never intended to allow cancelling transactions.  It can be done by hacking the wallet file and waiting for the transaction to be removed from the memory pool but that isn't made easy due to the likelihood that it would cause even more user confusion/chaos.  For example you could "cancel" a transaction and it could end up in a a block as your "cancellation" is only local.  You can't control what other nodes do.  Also your client may not be up to date due to a delay or protocol issue.  The client may report a transaction as cancellable when in reality it was confirmed in a block hours (or even years ago).

Yes.

Since the wallet has no provision to delete transactions, what you can do is export the keys, create a new wallet and import the keys.  Voila, your funds are spendable once again.

It was an accidental double-spend attempt. But because most of it had already been spent, it won't verify... leaving the remainder in limbo.

If it is a low priority tx the "anti-spam" fee is mandatory.  Without the fee most nodes won't even relay the tx much less include it in a block.  It is a denial of service prevention mechanism.

Now how can that be?

Can you resend it with a fee?

Even without a fee, surely it should have gone through by now?

I have an unconfirmed transaction from October. 

It isn't just the duration, it is space in the memory pool.  As that space fills up, older transactions get bumped out.

How long time does bitcoind remember a transaction?

Amazing that this hasn't been discussed in detail before.

Such a basic feature. I guess it must be because fees have never been much of an issue before.

I was thinking about Git... Is it possible to backup / restore (effectively cancelling unconfirmed transactions) your wallet by transforming the ~/.bitcoin into git local repository?

So, after each transaction (sending or receiving), a git commit occurs... Or something like that... It would be possible to "undo" transactions easily (for git users at least)... Going back and forward into transaction history...

http://en.wikipedia.org/wiki/Git_(software)

I'm going to try that, do I need to shutdown bitcoin-qt before running git-stuff to commit wallet changes?

Best,
Thiago

If there was a significant demand for non-standard transactions they could set up a separate node, perhaps only available to subscribers, in order to accept them. Of course the people wanting to send these non-standard transactions would need to pay for the additional risk of orphaned blocks, since they wouldn't be in the signature cache of other nodes.

They could but for it to be useful it would need to be integrated into the client.  If you need to hack the wallet.dat to force a change then you might as well just delete the offending transaction and wait for the network to forget about it.

Also most mining pools limit incoming connections as a pool wants to get a block to other miners (namely other large pools) quickly.  Having a bitcoind bogged down with hundreds of connections doesn't help orphan rates.  In theory a miner's optimal peer configuration would be direct connections to all other mining pools and then a few connections to major tx hubs (exchanges, blockchain.info, major merchants, etc).

Mining pools could offer this as a service by setting up a node that allows for lots of direct connections and bypasses the normal relaying rules, and which immediately includes qualifying transactions in the next block.

It is possible to cancel a transaction although it is intentionally not user friendly.

The first thing to know is that all nodes will forget about transactions after a few hours.  This is intentional otherwise someone could bring the network to its knees by spawning a huge number of low/no fee transactions when the blocks are close to max size.  Memory Pool spam vs blockchain spam.

The problem is your client is very diligent.  It continually looks to see if a transaction it sent has been including in a block.  If it hasn't then periodically it will notify its peers and the transaction will propogate the network again.  Any node which has forgotten about the tx will be informed of it again and the process will continuing infinitely.


To cancel a transaction you must delete it from your wallet.dat.  This will prevent your client from keeping it alive. Eventually all nodes will drop it from their memory pool automatically.

It would be useful to include a "special exception" to the double spend rules.  Nodes will drop double spends of outputs currently in their memory pool.  A double spend which consists of the exact same inputs and same outputs and only differs in the amount of the fee to miner by a preset amount (say up to 5x min fee) could be allowed to "pass through" the double spend shield.  This would allow users to "upgrade" a transaction from low/no fee to standard/higher fee and get it to miners.

It's not a hard rule (in the sense of "a client that does this will create an invalid fork") that the newer transaction HAS to be ignored, no matter what, but currently this is the case.

Hey,

wouldn't it be a realistic scenario, that the transaction without fee gets just ignored by the miners. I mean, if you have 1 BTC and try to make a transaction without fee. Than you wait, and wait, wait and nothing happens. Than you try the same transaction with 0.001 BTC fee and the miners include that transaction into a block and the other transaction never will, because it is invalide? Or is there currently a hard rule, that the later transaction will always be ignored?

greetings

I thought I have seen Gavin somewhere talking about adding fees to already existing transactions... am I mistaken ?

There was a feature that was written but never released in which the child could pay for parent.  What this means is that the code that determines if the fee is appropriate will also consider any other transactions that depend on this transaction and what are the fees for those transactions.  That way the miner can receive the fee for processing both the parent and the child. 

 - https://github.com/bitcoin/bitcoin/pull/1647

That would give the recipient of the payment a method to pay an additional fee after the sender has broadcast the transaction.

There is no cancellation available at the protocol level. The best one could do would be to double-spend a duplicate transaction. Clients consider spent = spent, the spent funds being removed from your balance and unavailable, even if the transaction never will confirm.

The way a client could do this would be to offer a "modify and retransmit" feature, allowing one to re-spend the identical inputs and outputs of a transaction with more fees (adding more inputs if needed to increase the fee). Implementing a double-spend feature with this method shouldn't make an unconfirmed payment less trustable if the resent payment was recognized as a replacement.

The reference client doesn't let you create inadequately-fee'd transactions, so this is a feature unnecessary in any client that understands the rules. Only if larger-than-minimum fees become required for timely processing in the future, by everyone else paying more than standard fees for preferential treatment on an extremely busy network, would this be useful. I would expect new clients which start automatically recommending higher fees based on network activity if Bitcoin becomes this busy.

As far as I know something like this does not yet exist and yes, imho this is a far more pressing issue than MAX_BLOCK_SIZE.

There might be problems for services that allow 0-confirmation transactions to already gain some kind of access/benefit if you implement a "currently pending transactions" screen with a "cancel" buttin next to each one - and also there might be some flooding issues if you can just create and withdraw transactions at will...

A solution might be to drop unconfirmed transactions out of the memorypool after 1008 blocks (~1 week) automatically and after 6 blocks (~1 hour) upon explicit request. Until then other clients will see other transactions with the same input(s) as a double spend and not relay them. I don't know if this should be a hard rule though, seems rather like a "nice thing to do" rule of thumb towards people who get a transaction stuck in limbo.

Hi

As far as I am aware it is not possible to cancel unconfirmed transactions currently.  If the block limit is maintained, this will likely lead to a scenario where first transactions without fees will go through in calmer moments, then not at all anymore. 
I was now thinking of the following scenario: somebody tries to be a cheapass by not adding a fee to his transaction, transaction volume picks up forever, and the transaction is stuck in limbo forever.  He might try a new doublespend with fee, but apparently that is not trivial either? 
Would it be good to implement a "cancel unconfirmed transaction" option in the clients?  Or does this already exist and is my information outdated?