Topic: Blocking the creation of outputs that don't make economic sense to spend - page 2. (Read 3525 times)

I hope you are wrong. If we suppose that Bitcoin is successful enough for UXTO volume to become a problem it may very well be when/if Bitcoin reaches Paypal-like levels : the 1000 tx/s you mentioned earlier.

There's no way all enthusiasts will be able to host a node relaying that much. I'm not sure about the bandwidth people have but with ADSL the upload capacity isn't nearly enough. Block transfers already stress my connection (which can reliably move <100 kB/s). Assuming a mean tx size of 500 bytes including network overhead (probably very generous), I can't put more than 200 out each second.
The P2P network is based on broadcasts: each TX can be transfered multiple times. Assuming there's no optimization on average a node sends a transaction to at least half of the nodes it is connected to (the other half of them send first). My node is using a standard configuration and has ~20 connections. This means it would use all my ADSL upload BW when there's more than 20 tx/s on the network. 

For 1000 tx/s with the current broadcast system you'd need 5MB/s upload bandwidth: ~ 50Mbps upload link. This is the very best a consumer can get today in my country and for that you need to switch from landlines to fiber... I'm not against a switch to fiber myself, but I wouldn't do it just to run a full Bitcoin node.

Unfortunately consumer-grade connections capacity doesn't evolve at the same rate than RAM/CPU/disk. It jumps when infrastructure is updated, eventually we will all benefit from a 10x BW increase in the future, but we don't know when.

What can be done today and the foreseeable future is renting a VPS with a 2GB of RAM and a 100Mbps BW link. I don't expect all enthusiast to do so.

In my opinion, if full nodes are too expensive to run for enthusiasts and require funding by the miners, bitcoin will die.

Hence the 100,000 nodes, which would be government agencies, universities, big companies, big non profit organizations and enthusiasts (wealthy individuals).

There's no way bitcoin can survive with 1000 full nodes.

Why so many? I'm not sure why the Bitcoin P2P network would need so many nodes to be secure. A good geographical distribution may be desirable to avoid latency and government censorship but can be achieved with far less nodes.


Agreed but you can use several tricks to avoid looking up the storage to protect against this kind of attack. For example Bloom Filters or preferably counting Bloom Filters (to make them more resilient as UXTOs can be volatile) can be used to avoid hitting the disk by detecting invalid outputs early. This is a bit costly to setup (generating the filters is I/O intensive and they must be regenerated when they aren't big enough to handle the growth of the UTXO database and their efficiency plumets).
Developers are smart and know about them (see BIP 0037).


A bloomfilter is quite efficient and can use a few bits per entries (unspent TX) it needs to filter (10 bits give you a ~99.9% efficient filter). The RAM cost in this case is negligible vs the SSD cost. You can store a billion unspent TX on a system with a couple of GB today and have to hit the disk during an attack one time out of one thousand transactions.


I believe you can ignore the RAM cost with a software update.
But why would you run a full node if you aren't a miner? The only reason I can see is because a SPV client security wouldn't be enough for you. Why would it be and why would it be for hundred of thousands of people?

We may see the situation from the opposite angle: if the costs of storing UTXO rise and there's only a market for one thousand full node instead of 100 000 (being paid by miners using them), then some people will run at a loss and/or full nodes will shut down until there is only 1000 of them.

And it boils down to: what is the minimum size of the Bitcoin P2P network considered healthy/secure when it's mainly composed of nodes being paid by miners.

Have you seen my post on how miners have perverse incentives to create blocks large enough that not all the hashing power on the network can validate them?

So, what you can do as an evil miner, is pad the blocks you produce with transactions that use up UTXO space. You can create them deterministicly, so you don't have to waste your own resources storing them. As you say, the only cost to the miner is decreased propagation of your blocks, which as I pointed out is actually a good thing is small quantities, and in any case the overall orphan rate and bandwidth costs to you is strongly dependent on how much infrastructure you have.

For instance Google would be able to easily run a very successful and extremely large mining pool/validation node network for very little money because they can re-use their existing infrastructure to broadcast data around the world. Smaller players have costs hundreds or thousands of times what Google does to send a marginal byte out over the network - it pays to own your own fiber.

As you fill up your competitors UTXO space they'll be able to afford less and less IO bandwidth. The problem is as they fall behind on IO bandwidth, it takes them longer to validate blocks, and thus their orphan rate increases. Eventually they'll be forced out of business.

What's really nasty, is you don't even have to do this in an obvious way. Remember that a miner can pay fees to themselves only limited by the risk that another miner will take the fee. However, if you are the biggest miner selling blockspace any "false-fee" that you make that's less than the fees other miners would accept has no chance at all of getting picked up by them; they can't afford to. So the fee has cost you absolutely nothing. You can then get a large quantity of BTC from someone else, maybe you know someone operating a hedge fund investing in BTC, and spread that large quantity into a pile of phony transaction outputs. (tell your hedge fund clients this is to ensure the privacy of their funds) You gain because Bitcoin looks more popular, you gain because you push your competitors orphan rates up, and you gain because they're forced out of business in the long run.

What about Meni's  Output upkeep costs https://bitcointalksearch.org/topic/output-upkeep-costs-80435 proposal ?

I rather liked that.

Yup. It's easy to forget the meta risk that we'll get our predictions wrong, either because technology doesn't advance the way we thought it would (where's my flying car?) or because it turns out the usage pattern didn't happen the way we though it would. (Satoshi completely missed mining pools for instance) UTXO set bloat is something that has real security risks in that it can make running a validating node prohibitively expensive, and thus breaking the security assumption underlying Bitcoin that information is easy to spread, and tough to stifle.

Anyway, all this focus lately on trying to come up with hard cost figures kinda makes me laugh, because essentially it's a whole bunch of software guys taking an opportunity to muck about as hardware designers/futurists, and then base the whole security of Bitcoin on their estimates. It doesn't really inspire confidence to me.

I mean, heck, hardware people are bad enough at making forward looking predictions: I've been working at the same cutting edge hardware startup for nearly 5 years now, and on the day of my interview they had a big schedule up on a wall... which said we were supposed to be finished the year before that interview. Whoops.


In any case d'aniel's analysis looks reasonable to me, but the big thing that stands out is he came up with the number $0.13 which frankly is relatively expensive, and for just 10 years of storage. With current 0.0005BTC/KB fees, that means my marginal cost to create a UTXO is just the data required to represent it in the transaction, 34 bytes for a standard tx. So 5mBTC/KB * 34 bytes=17uBTC or about $0.000765

So that wild guess has us undercharging for UTXO space by a factor of 170. It also shows how the cost to the creator of that UTXO is wildly dependent on the current market for blockchain space and the current exchange rate. Finally d'aniel's analysis shows how the cost of that UTXO space is wildly dependent on how many validating nodes there are, by orders of magnitude.

Assumptions:
1. Bitcoin is fully adopted tomorrow, the network is 100,000 full nodes. The network is also crunching 1000+ transactions/second, requiring UTXO (unspent tx outputs) lookup to be fast.
2. UTXO need to be stored in fast access storage. https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures#CVE-2013-2293 . You could maybe make a system setup of multiple HDD systems, but that wouldn't be cheap, at least not as cheap as "HDD space is cheap". Not if you're using 100 disks to do fast seeks.
3. 1 unspent dust output takes 100 bytes of memory.
   Prattler: should be 41.5 bytes for each transactions which is not entirely spent (excluding any of its outputs)
   plus 23 bytes for each unspent output
   in-memory it's a lot more, but that's just the lowest-level cache
4. RAM costs $0.008/MB, SSD costs $0.001/MB.

For example http://blockchain.info/tx/0fda0a78210e2913ba269b3a5ff9385d29a004873333f44e56eb9efb1a488f72
Assuming the dust output is never spent and the network has to store it on:
1) RAM – the network UTXO storage hardware cost is $0.08.
2) SSD – the network UTXO storage hardware cost is $0.01. However, if SSD speed is not enough, costs will go up, as more complex systems will be required.

Ok. $0.08 might not sound like much, but remember the fees go to the miner, the network full nodes get nothing! A miner would probably be content with 0.00008 BTC (See https://gist.github.com/gavinandresen/5044482) fee to include it in a block. A malicious party could pay 0.00008 BTC to put $0.08 cost on the network. This is UTXO storage alone! In the case of SD, the miners are paid directly by the users, costing SD nothing. The only factor limiting the network costs is the block size limit.

Will correct the post if someone points out any errors in the calculations.

In five years will you really install the Satoshi client if you are a simple Bitcoin user? Even today, lightweight clients like Electrum may be better suited to users (and it was expected IIRC).

If you install a full-fledge Bitcoin node, it should be because you want to have a say in which transactions are given priority (ie: you are propagating them or mining without delegating the choice of transactions included in a block to a pool).

If I'm not mistaken Bitcoin users don't have to pay for the blockchain size, only miners and servers used by lightweight clients. The cost of the blockchain maintenance may hit users in the end (servers for lightweight clients selling their services for example) but would probably be far less than what it would be if they all ran a full node.

Sigh.

Ok, fine, so do a back-of-the-envelope for what THAT cost is.

Big +1 to d'aniel for doing the work of actually calculating a number, that is very helpful.

Yes, the thing you didn't consider is that if I am a new user and fire up my brand new install of Bitcoin in five years, I'm being paid nothing for having to download 10GB of blockchain even with pruned transactions, and might just give up. Blockchain bloat has a higher price.

I'll assume RAM is replaced every 5 years, and that Moore's law for it holds for at least 5 more years.

16GB costs $115 today and consumes about 5W of power. *  Take $0.10/kWh for electricity, and it costs

(5/1000 kW)*(5*365*24 hours)*$0.10/kWh + $115 = $137

to buy and run 16GB of RAM for 5 years.  Assuming a factor of 1/4 reduction in this price after 5 years (doubling size/unit cost every 2.5 years), we have $171 for 16GB for 10 years.

Assuming 125 bytes per utxo on average, this is 128M utxos per 16GB of RAM, giving

$0.0000013 to store a utxo for 10 years.

But we have to scale this by the number of full nodes, say 100K: $0.13 for the whole network to store a utxo for 10 years.

I only went to 10 years cause I don't know if Moore's law holds for longer.  But we can safely say it's roughly $0.13 for a network of 100,000 nodes to hold a utxo for as long as Moore's law for memory holds.

*
http://www.newegg.com/Product/Product.aspx?Item=N82E16820104302
http://www.kingston.com/dataSheets/KHX1600C10D3B1K2_16G.pdf

I did this quickly so please correct me if I did something stupid...

Edit: I overheard sipa say that the average utxo size is a bit more than 100 bytes, so I changed the number from 50 to 125 bytes.

I forgot to be recursive - if these miner rules are enforced, clearing the spams also must follow rules; the examples I gave above to spend dust can't be used:

5 inputs: 976-980 bytes = minimum input .00010001 -> sends .00000005
5 inputs: 976-980 bytes = minimum input .00012000 -> sends .00020000

6 inputs: 1157-9 bytes = minimum input .00016668 -> sends .00000008
6 inputs: 1157-9 bytes = minimum input .00020000 -> sends .00020000

I would say the minimum output should be 0.0002 BTC (40% of min per-kb fee). That only costs SD 40% more to spam the blockchain with spendable outputs.

Why did you pick 0.0005 BTC ?  That is a mostly arbitrary number.

I estimate a current-worst-case "orphan cost" of an average 250-byte transaction is 0.00008 BTC. See https://gist.github.com/gavinandresen/5044482 (more accurate analyses welcome, I don't pretend to have a monopoly on the "right" answer).

That number will drop as CPUs get faster/cheaper, or bitcoin value rises. So you could argue that even though dust is not economical to spend today, in 20 years it will be.

So I guess I'll rephrase my question again:  Rough, back-of-the-envelope: how much does it cost to keep a dust-like transaction output in the unspent outputs set for 20 years?

If it is a lot, then we should set the "expected time when it will be economical to spend" to either "right now" or "very soon."

If it is tiny, then we shouldn't worry so much about optimizing unspent txout size, and concentrate on other things.

I have no idea what the answer is.

Bitcoin didn't anticipate that people would pay fees to spam the blockchain - we required a fee for anything below .01, and if you have a profit model you can pay that...

Payments need to be self-funding, that the recipient will have a net gain by receiving the payment after including retransmittal costs.

So then what is the definition of not making "economic sense to spend"?

A single 0.0005 payment meets the most simple definition of this - received alone, it can't be spent.

However, if my wallet had 1000 BTC in it, I can currently send someone or myself 1000.0005 for free immediately. I can also spend lots of dust. This is why free transactions should be removed - if you are wealthy, Bitcoin works differently for you.

Without changing any other Bitcoin rules though, I would say it is a payment that as an input, when evaluated by minimum fee rules in aggregate with like inputs, would cost more to send in fees than the value of the payment. This is a payment likely to become an unspent TXO - for most it is cheaper to discard than to spend. This is the scenario with Satoshi Dice, a gambler will be getting hundreds of like payments of small value.

Grabbing some sample transactions with two outputs (payment + change), and determining the minimum per-input amount that would result in a post-fee value greater than zero:

1 input: 257-259 bytes = minimum payment .00050001
2 inputs: 436-439 bytes = minimum payment .00025001
5 inputs: 976-980 bytes = minimum payment .00010001
6 inputs: 1157-9 bytes = minimum payment .00016668
8 inputs: 1514 bytes = minimum payment 0.00012501
39 inputs: 5848 bytes = minimum payment 0.00008975

So it looks like a good "receiving this payment will cost the recipient" threshold is any output that is less than 1/5 of minimum fee, from the examples, 20%-40% of minimum fee should be required.

I don't know what the numbers are, but the way I would calculate it would be to ratio the size of an unspent output to the price of RAM, integrated out to a suitable time in the future to get the net present value.

I agree.

That is why I ask "what is the external cost, with reasonable assumptions."

So I'll ask again: what is the cost-per-(pick-your-favorite-time-unit) to the network of an extra unspent transaction output?

More of them would do this if it wasn't for the distorting effect of the block subsidy. Miners won't have a large incentive to optimize their transaction fee policies until fees are a significant portion of their revenue, which won't happen until the transaction rate starts to approach PayPal's.

I think that, above all, we must maintain bitcoins decentralized nature. That means we should let miners price transactions.

At the end of the day, if a miner accepts a transaction in a block, he agrees to basically store it forever on his hardware. A transaction would therefore have a storage cost. It also has a computational cost as validation might be required in the future when spending outputs of this transaction.

Therefore, I strongly believe that, instead of finding static solutions to transaction costs, we must enable miners with tools to price them correctly. This is the only way we will see the light in a bumpy, uncertain and highly dynamic market environment. Good pricing competition should help us find an equilibrium.

If they don't do it already, miners need to start operating more like businesses. They need to crunch the number and understand the relationship between various transaction parameters and their cost. They need to do that for their own sake as they will need to host these transactions pretty much forever. They also need tools to tweak pricing quickly and dynamically as market forces change.

Under such assumptions, the current transaction dust we are experiencing should be reduced. My prediction is that if miners acted more as rational market players, they would require much higher fees for very small and young transactions. If the transaction is so small that it makes no economical sense to spend it, it should be priced accordingly. It will most likely be impossible to prune it and will require storage in UTXO memory. Some miners might also decide to drop them entirely. This is entirely up to them.

So how does the landscape look today? Are miners sufficiently empowered to dynamically change their pricing policies?

Did no one notice that there are corner cases where you have no other way than creating very low volume outputs (dust)?

Suppose you just created your wallet and got a single 1 BTC input.

Then you go shopping and the (automatically computed) price for what you want is 0.99959 BTC.

As the only Bitcoins available to you come from a single input of 1 BTC when you pay for the goods at this price, you create a transaction that pays 0.99959 BTC to the merchant, the usual 0.0005 BTC fee and 0.00001 BTC back to yourself.

Obviously this is an extreme case chosen for illustration. That said there were 70000+ transactions during the last 24h. Unless the bitcoin clients aggressively try to prevent dust from being created I guess tens or hundreds of them created dust (meaning sub 0.0005 BTC outputs). Even clever algorithm can't prevent the occasional freak transaction (and the Bitcoin user won't understand why his/her payment doesn't go through if dust is blocked).

I believe trying to prevent dust is simply not compatible with the protocol: with the standard Satoshi client you can't even be sure if the 0.00001 BTC are paid back to the original buyer (they are paid to a brand new address).

In fact today's dust may be probably tomorrow's gold. I'm not sure exactly when 0.0005 BTC fees were made the default/kB but at the time they were certainly less valuable than today. If the BTC value keeps rising in the following years, fees will have to be reduced if small payment/transfers are to remain cheap.

I've seen developers discussing a way to track the confirmation delays of transactions to compute a recommended fee and it makes sense to me. Users shouldn't have to think twice about fees for every transaction : the recommended one should reflect the true "market value".
This means than dust will not necessarily remain dust: it will depend on the current transaction rate and miner efficiency at putting transactions in blocks.

Transactions with zero-value outputs are already considered non-standard.  For practical purposes, 0.00000001 BTC is the same as zero (it is less than one ten-thousandth of a US cent).  The fact that bitcoin supports 8 decimal places is somewhat arbitrary -- if it supported 100 decimal places, would we permit outputs of 10^-100 BTC?  Perhaps sufficiently small outputs should be treated the same as zero-value outputs.

Another solution is a fee policy where spending an output is paid for by a fee in the transaction that created it rather than the transaction that redeems it.  See https://bitcointalksearch.org/topic/fee-policy-proposal-charge-for-outputs-not-inputs-150749.