Topic: Initial replace-by-fee implementation is now available on testnet (Read 16173 times)

Horrible idea... it sounds great to have "undo" button until you think about double spending. If I could I will not accept this idea.

Here's a version suitable for mainnet:

https://github.com/petertodd/bitcoin/tree/replace-by-fee

It's got DoS attack resistance by simply punting on cases where it matters, so running a node with the patch shouldn't cause any problems. The big "punt" is it doesn't do child-pays-for-parent, so to resist DoS attacks via long unconfirmed chains it simply won't do a replacement if any outputs are spent.This has the advantage of being dead simple; the code is quite similar to satoshi's original replacement code. I've done some testing on testnet, but if you want to mine on mainnet, keep in mind that you may be risking an invalid block if there's something I missed. You can -addnode=replace-by-fee.bitcoin.petertodd.org to peer with a node with that patch.

I haven't written any of the useful infrastructure like an "undo" button or way to change fees automatically, and it will leave unconfirmed tx's in your wallet. (fixable with -salvagewallet) I'm not going to get a chance to fix any of that immediately; I've got a child-pays-for-parent mempool rewrite I want to finish first. Patches for that stuff are very welcome.


jdillon: change the title if you get a chance...

Gavin is correct, this patch is a ridiculous idea. It doesn't matter that anyone could write it at any time, no rational miner will use it.

There are some very odd ideas at work here about economic rationality. Rational does not mean "motivated by short term profit at any cost". The most rational choice for a miner is to do whatever maximises the value of the Bitcoins they're mining. Making it hard to use unconfirmed transactions would very clearly reduce the value of Bitcoin for large numbers of people, meaning fewer merchants, a smaller ecosystem and less demand, which suppresses the exchange rate. As miners have costs in dollars but income in Bitcoins they're very much motivated to maximise the value (and thus utility) of the whole system.

Bitcoin is at heart a system that manages risk. Being in a block doesn't mean "safe" vs "unsafe", re-orgs can cause transactions to become double spent at any time. So to make blanket statements about the safety of transactions is unsound: you cannot justify an assertion like "unconfirmed transactions are unsafe" without providing proof of what you say.

I assert that unconfirmed transactions are safe enough for many users today, and can be made safer in future. As evidence I cite the lack of complaints about double spending from merchants, easily measured by the lack of media coverage of such events. I also cite the wide acceptance of such transactions and the ability to build double spend alerts to increase the trustworthiness of the broadcast channel. I would need to see equally compelling evidence there's a problem to change my mind on that, and if you have to put in large efforts and use arguments like "it's inevitable" then you aren't showing evidence there's a problem, you're deliberately trying to create one from scratch. Not the same thing at all.

John Dillon is indeed a terrible diplomat, I've had him on my ignore list for a while. There are plenty of productive things he could be doing to increase the utility of Bitcoin for everyone, but instead has chosen to focus on controversial changes that if successful would lower Bitcoins overall utility. He thinks he is doing this for the greater good, but this idea is at best controversial and at worst flatly incorrect. If it's truly inevitable he doesn't need to do anything, as it will happen anyway, and he'd avoid annoying a lot of people he might want to work with in future. If it's not inevitable then he's still better off doing nothing. Regardless, I am not worried about this overly much because as I said, I don't believe miners will use it, nor (given the failure rate in the presence of <100% adoption) will end users.

It could be. Or it could be that you're making wrong assumptions about how human society and economic incentives work.

jgarzik: So on the blocksize issue we do roughly agree on the problem, if not as much the solution, and we do agree that I do not make a good diplomat. That is a good start.


Nubarius: The likes of you will be in for a rude awakening about how little Bitcoin has to offer as a payment systems as mints around the world create lovely little chip-to-chip secure hardware dongles as a means to combat Bitcoin. That, or they will combat Bitcoin directly by destroying its decentralization. One or the other.

The problem I see with this is that the growing adoption of Bitcoin is likely to follow an exponential rather than linear trend during the next few years. A linear growth in the maximum block size would mean that blocks keep getting smaller relative to the demand of transactions. An approach I find more sensible would be:


But then, after a few years this would be tantamount to an infinite size (which is fine with me; my basic assumption being that Bitcoin should essentially be a payment system, its store-of-value properties being a consequence of its usefulness for payments, and not the other way around. I don't believe in the idea of Bitcoin as some sort of digital gold for a minority of Tor-connected übernerds).

Now back to topic...

While I find it perfectly legitimate to work on a pay-to-edit-transactions system, and I think it is good that such possibilities are explored and tested by the core developers, I don't think this is in line with what users expect Bitcoin to be, and I think it will never make it as a basic part of Bitcoin functionality. In fact, I don't even see how such a feature would be attractive to miners. Users would understand a feature like 'this transaction can still be cancelled during the next 24 hours', which can be implemented as a third-party system that simply schedules the actual Bitcoin transactions, but a mechanism that's only available before the transaction gets into a block offers no such time-window guarantee, which would put off potential users. Because of that, I think that there's no real demand from users for that 'feature' and that miners won't have any incentive to support it. But for the sake of argument let's suppose some miners feel tempted by the idea that they can profit from the odd user that wants to pay a fee to edit/undo an already published transaction. My feeling is that a reputation system would naturally grow where people would know which miners are known to have reneged on transactions they've relayed and which ones stick to their original transactions. I haven't thought much about this, but I suppose services that list unconfirmed transactions like blockchain.info could devise some sort of reliability metric for unconfirmed transactions based on statistics about the past behaviour of known IP's.

Oh dear.

The userbase will keep using them until you force them not too. This can either happen in the future, when a lot more money is at stake, or it can happen now.


The majority of actual payments, as opposed to internal Bitcoin activity like selling mining equipment, is probably conducted with off-chain transactions right now: The Silk Road, The Black Market Reloaded and similar services all use 100% off-chain transactions. Sure, it makes for nice PR to point to people selling socks and other niches, but the reality is most of that business is from Bitcoin enthusiasts and investors; it's not genuine economic activity. It's no surprise to me that the only Bitcoin transactions anyone I know of has told me about, or asked me help to perform, is buying from the silk road and buying proxy services. For that genuine economic activity anonymity is key, particularly the former.

The other genuine economic activity, and what I suspect is where the majority of Bitcoin's market cap is derived, is investors seeking a decentralized asset that is free from regulation. Jon Matonis has written about the issue better than me, but what I hear from the people who have been donating to my efforts, in particular my video project, is they are pissed. They own thousands, and for some tens of thousands, of dollars worth of Bitcoins and/or mining equipment yet they see Gavin and The Bitcoin Foundation as not caring about privacy and decentralization. What they do see them caring about is internet startups with shoddy business models that would be better served by something like MintChip. These are the people who saw Cypress as a turning point for Bitcoin, and they want the degree of privacy they enjoy to be their choice, not someone elses.

In any case, why should I personally work on any off-chain tx systems? That just means I get attacked even more by all the people, Gavin included, constantly pushing conspiracy theories that I'm only pushing this issue because I want people to be forced to use my off-chain tx systems. Indeed, it's why I'm talking at the conference: to raise the issue and make sure as many talented developers as possible understand what ideas are out there, so they can build their *own* systems. Diversity is a good thing.


It's more than that: it makes privacy more difficult for everyone. SPV mode leaks information for instance about what is in your wallet; if you can't afford to run a node yourself you have to rely on someone else to do so. With large blocks you will be paying for your Bitcoin network access in the future, either with cash or with your privacy, or even both: https://bitcointalksearch.org/topic/m.2051415


The moment you do that, literally on year one, you are making the decision to make it far harder to use Bitcoin anonymously for the benefit of the people using blockchain space for the least important, lowest value transactions; right now that's getting paid for clicking on ads on the internet.

But the real issue is what values do we have? The people advocating the raise the blocksize better be really clear about what they think is or is not important, and that includes what they think of anonymity, privacy and freedom from regulation and authority.

+1

Ultimately the change is too aggressive and anti-social right now.  Absolutely the miners market may change and make double-spending trivial, but that does not seem to justify removing hope of using a feature that businesses do rely on in the field today.

The userbase uses them, for better or worse.



This is full of hyperbole and exaggeration, much like the recent thread on dust.  However, it is fair to address the subject on a more reasoned level, because there is a valid point buried in there.

There is definitely a contingent of companies that seemingly want to change the blocksize limit immediately, from 1MB to infinity, to fix "this awful scaling problem" they see.  They want to sell the message that bitcoin can scale to Visa/MC levels tomorrow.  There should be absolutely no impediment to sending millions of transactions, for fractions of a penny apiece.

We absolutely do want to grow the network and encourage as many bitcoin users to use bitcoin as possible (well, I do at least, and I think Gavin does too)...  but one cannot ignore a key attribute conferring by a limit like the 1MB limit:  it encourages engineering efficiencies to be sought.  Programmers have an incentive to actively seek ways to reduce the number of transactions, or reduce transaction size, when faced with a limited resource.

Some business models simply don't care about that part of the equation.  It's not a conspiracy by Gavin and the Bitcoin Foundation funders, it is simply one facet of some bitcoin businesses.  They make money with increased transaction volume.  That's fine, but a key economic counter-point is that these businesses are not bearing the costs of the mining/blockchain impact of a million-TX-per-day policy.

Off-chain transactions should handle a resource limitation in a scalable fashion, but no one has good working prototypes.  Gavin has rightly pointed out, in Gavin/retep exchanges, that off-chain transactions remain a scalability solution in theory, while we know for certain that increasing the blocksize limit will enable increased transaction volume.

The danger in following that logic too far is that you de facto eliminate most incentives towards creating an off-chain transaction system/network and related businesses, or really as mentioned above, any need for transactional efficiency.  It also has impact on who is a bitcoin miner, selecting out anonymous miners as jdillon predicts.

I don't think anybody has The Answer right now, and my main preference is to avoid making decisions that dramatically and immediately change bitcoin's economics.  $Topic might do that, hurting payment companies for no good reason.  Removing the blocksize limit also injects chaos for unclear value.

Ultimately it is a complicated, zen balance of factors:  too-harsh limits, and you potentially choke off bitcoin utility just when it is being adopted, and eliminate some business models.  too-lax limits, and you choke off certain technologies, encourage spam, and eliminate some other business models.

My current personal preference for a blocksize limit solution is

As that's something that cannot be gamed by miners or payment companies.  But most payment companies do indeed react in horror at any impediment to "send as many transactions as possible."

Definitely in the middle here too.  Not as aggressive as Gavin or Mike Hearn, but not as conservative as "1MB forever" people either.  1MB was clearly a temporary solution.  (that does not imply simply removing it is without negative impact)

If the answer isn't clear, and the system isn't broken right now, err on the side of doing nothing.  The answer was clear, with the recent data spam / dust changes.  Not clear at all, with blocksize limits and the impact on fees thereof.

The issue of zero conf transaction security is not a problem - it's a business opportunity. Here's one way to solve it:

Mining pools start offering double spend protection on a subscription-only basis. The way it works is the customer submits a transaction to the pool, and the pool either immediately rejects it because there is already a conflicting transaction in the in-progress block, or it accepts it, includes in the next block, and refuses to include conflicting transactions. Now a merchant can get instant notification about whether a transaction is valid or not, with a degree of assurance equal to that pool's fraction of the network hashing power.

Individual merchants who want to accept zero conf transactions don't even need to set this up individually. They could subscribe to a service that negotiates bulk rates with enough pools to account for X% of the network's hashing power.

I'm in the middle about this.

One the one hand, I do understand the reason for a change like this. If miners become (short-term) selfish/rational, and start implementing behaviour like the one implemented by this patch, I will completely be in favor of making it the default - as anything else would indeed just be a false sense of security. It would remove the "best-effort attempt" by the network to secure 0-conf transactions, and force people to find real solutions for fast transactions, instead of relying on behaviour that cannot be guaranteed. And perhaps that is inevitable anyway. And looking at Bitcoin as an experiment, it's so much more interesting to aim for a system that doesn't require specific parties to act in the best interest (especially non-rationally) of the community anyway...

On the other hand, it's not true that doing double-spending 0-conf transactions is trivial now. Not particularly hard, but not as easy as whistling in a telephone either. So in that sense, the network right now does provide some 0-conf security. Not much, and nothing guaranteed, but it works, and there is economic infrastructure that relies on it. I don't think they're even wrong about relying on it, if they're willing to eat the costs from damage caused by it (on the other hand, perhaps few can actually estimate the risk correctly, and will get hurt when their business becomes profitable enough to attack...). Anyway, what I'm getting at is that this is a drastic change if we'd put it in mainline, and drastic changes may be more damaging than the actual problem we're solving. Perhaps we need time to outgrow relying on 0-conf security.

So: I don't know. I like the idea of aligning miner's short-term self interest (even if they don't exploit that right now) with the policy implemented by the reference client. But perhaps not immediately...

Another thing that just popped into my head:
Why stop at unconfirmed transactions?

If its all about miners getting the maximum fees, why not make a replace-by fee possible for transaction already in blocks?
You can't really implement a cancel-button into an end-user-app that potentially only works for 10 seconds.

If the replacement fee is higher than the last blocks combined fees it would make sense to orphan the last block and mine a new one at height-1 .

First of all thanks for taking the time.

As I understand it it is not that simple. If it were there would be a satoshi-dice bot that does it. Please correct me if I'm off here:
To successfully double-spend you need to

A) send two transactions spending to different parts of the network and hope that transaction A reaches your target earlier and transaction b reaches the miner of the next block, which is way below 50%.

or B) send transaction A to your target and mine the next block yourself, which makes the success rate your percentage of hash rate (if you go this route you may as well be doing confirmed block double spends)

or C) mine away and if you find a block hold it back, do your transaction and then publish the block, which has a 100% success rate minus the risk that someone finds a block while you hold one back. This however requires good timing of you purchase.

So you need to be either a big miner or very lucky. So its not just downloading an auto-doublespend-tool (2600Hz whistle).


While the funding part may be true I heavily doubt that Gavins primary concern is the profitability of bitcoin businesses. Thats not his job.

I have never been lied to. I knew exactly how everything worked and assumed the risk to create a viable service. I just would have never guessed that a pay-to-doublespend feature would ever be included. If it does, bad for me.

Besides: This also concerns you as an investor. Bitcoin value heavily depends on its usability.

You have to understand the position Gavin is in right now. It's kinda like the position the phone companies found themselves in, starting in the 60's. They had built a completely insecure phone system, one so insecure that a blind seven year old boy figured out how to activate phone switches and get calls for free by accident. Turned out all you had to do was whistle a 2600Hz tone into your phone and the local switch would activate.

When it comes to zero-conf transactions Bitcoin is just like those 60's era phone companies, and all you have to do to double-spend a site like yours is just broadcast two different transactions at the same time to different parts of the network. You'll see one or the other, and there's a 50:50 chance that your payment is the one that gets mined.

When hackers, known as "phreaking" back then, started seriously exploiting the tremendously insecure phone system, what do you think the phone companies did? Obviously they fixed the problem right? Fuck no, engineers are expensive, but lawyers and politicians are cheap!

So they got laws passed to make sure phreaking was illegal, even going as far as to make it illegal to tell others how to do it, or sell equipment to help the process. (not many people can whistle 2600Hz exactly) Of course that didn't solve the problem at all and phreaking continued to be a problem for decades. Eventually the phone system was fixed so signalling happened out-of-band and phreaking become impossible, with nearly all telephone switches upgraded by the 1990's.

Gavin wants to go down the same stupid path that the phone systems did. It's understandable why, his salary is paid by the Bitcoin Foundation, which in turn is heavily funded by payment-using non-anonymous companies like bitcoinstore, bitinstant, zipbit, eCardOne, CoinLab etc. etc. Most of these companies are still startups that are burning capital fast, and they need Bitcoin payments to become big NOW NOW NOW. Anything that might delay that could easily put these companies out of business and make their investors lose a lot of money.

Unfortunately the real work required to actually make zero-conf payments secure is going to take time that those startups just don't have. So from Gavin we get trite, insubstantial comments like "+1" rather than actually dealing with real technical issues.

The danger is if we don't succeed in getting replace-by-fee accepted and zero-conf transactions made safe with real security. Just like the phone companies, we'll see this issue become one of regulation both within Bitcoin and from outside authorities. Gavin's other big goal is to get the blocksize limit removed, which makes it impossible to mine on a small scale or anonymously, which means it's easy for regulators to start tracking down mining operations and punishing them for mining "illegal" transactions. One form of illegal transaction will of course be double-spends, but the only way to avoid accidentally mining a double-spend is to use a central authority to determine what transactions are or are valid. (remember that the whole point of Bitcoin mining is to answer that question!)

The sad thing is investors like myself get excluded from all these decisions. I don't belong to the Bitcoin Foundation because they require accurate identity information from every one of their members. Why would I want to be on a list of Bitcoin owners that may be given or leaked to governments around the world? There may be lots of "John Dillons" out there, but only one is living at my address. As Gregory Maxwell pointed out elsewhere how it's hard for decentralized software to get funding for development, which means centralization gets pushed instead. My $1000 reward to make replace-by-fee happen is something myself and my partners have decided to do as a way to start changing that and we intend further efforts in the future.

As for you, I'm sorry you've been lied to, but Bitcoin just isn't as fast as PayPal yet.

My site also accepts 0-confirmation transactions, not because I'm ignorant, but because it needs to in order to compete with paypal-backed ones.

At the moment it is too much hassle to do a double-spend on the unconfirmed transaction for small amounts up to 100$. I ran it this way for 2 years without a single successful double spend (or noticeable attempt for that matter).

Putting this in the code will force minimum 1 confirmation for every 1.50$ transaction which is not uncommon to be 20-30 minutes, which makes bitcoin unusable for brick-and-mortar-stores, bitcoin ATMs as well as instant-delivery online stores.
No one wants to wait 20 minutes for a small transaction and no one needs to as long as the effort needed to do the attack outweighs the gains. It works well, why change this?

Please explain to me why this is inevitable? Every miner that implements this will make bitcoins more useless and thus less valuable, which in my opinion will cost far more than a little extra fees gained.

I guess I had thought it obvious that none of this discussion applies when a merchant is actively managing their risks through other means.  I have no objection to people making the choice to be unsafe.  Hell, I don't even like seatbelt laws.

What I don't like is people thinking that the network is providing them some safety, when it isn't.  Your quote clearly indicates that you think that the network is providing "some protection for zero-conf transactions", but the network isn't doing that.  The network is providing absolutely no protection until the transaction is confirmed, because confirmation is the only protection that the network is capable of providing.  If someone wants to operate on hope, they can.  But we need to stop telling them that hope and safety are the same thing.

Of course not! I fully expect a small percentage of miners to deploy this, and that's OK! That does not mean that we should encourage this behavior as a community by building it into the main client by default, however.

Code-wise, it is indeed an illusion. But that doesn't automatically mean that the human element in this is also an illusion.

Huh? They don't even need to run this code for that to be the case right now. All you need is for a miner to restart their node and a double-spend can get through just as easily. Remember that fork we had?

For the hundredth time, this has nothing to do with safety, but with playing the odds. Fast food restaurants realized that they can save several seconds per transaction by not having people sign their credit card receipt, meaning that they would automatically lose any chargeback. And yet, that savings more than made up for the additional losses that policy brought about. This issue is not just black and white, nor is it even 50 shades of grey.

I'm sorry dude, but you are wrong about this.  Most blocks are already assembled by software other than bitcoind.  Also, we have to accept that people using the stock client for mining are certainly capable of getting and applying this patch.  We can discourage patches like this from the main client, but we can't keep them off the network.

The some protection that you and other see is an illusion.  If even a single miner is running this code, or similar code in some other block assembly software, then no transaction is safe until confirmed.  The bulk of them may continue to get confirmed as everyone expects, but that is luck, not safety.

Why do you keep assuming this? What's wrong with simply rejecting the block discouraging mechanism as a bad idea and keeping the situation as it is? Just because we may currently have some protection for zero-conf transactions, doesn't mean we have to add more. It's already good enough!

0 confirmation transactions are not legitimate, never have been, never will be.  People need to stop pretending that they are.