Shower thoughts about the growth of the utxo set (with stats inside)

laurentmt

sr. member

Activity: 384

Merit: 258

I guess it's time to draw a few "conclusions" from this journey into the blockchain

A few numbers for 2014:
- 78,669,554 utxos were created
- 70,806,194 utxos were consumed
- as a direct consequence 7,863,360 utxos were added into the blockchain (about 10% of #utxos created)

Looking for patterns in the number of utxos created per transaction, we can detect 5 specific fingerprints (transactions with 101, 135, 200, 2501, 2935)

Considered together, transactions with these 5 patterns:
- are about 0.09% of #txs created in 2014
- have created 5,753,098 utxos (about 7,3% of #utxos created in 2014)
- have consumed 29,582 utxos (about 0,04% of #utxos consumed in 2014)
- have added 5,723,516 utxos into the blockchain (about 99,4% of #utxos created by themselves and about 72,8% of #utxos added in 2014)

While a more detailed analysis would be necessary to validate the point, it's likely that all these transactions are created by a small number of legit services doing recurring micropayments (faucets, mining pools, ...)

Charts of fees vs #txos and #txs seem to confirm that current model used for fees doesn't help to "regulate" the growth of the utxo set.
(Actually, nothing new here, it's straightforward to reach this conclusion if you know the model used to compute fees)

My very personal conclusions about all this stuff:

While the memes "blockchain" & "micropayments" often walk together (especially in media and discourses of some startups), it seems obvious that the current architecture of bitcoin isn't very adequate for this use case. I guess it's something all bitcoin experts already know for long but I fear the public perception is very different...

While many services (exchanges, tipping services, ...) have already took a alternative path with centralized off-chain transactions, this little journey into the blockchain also tells me that even a small number of services doing micropayments may have a "big" impact on bitcoin (see how the growth of the utxo set has come up into the discussions about the blocksize).

I like the fact that some core devs, like Jeff Garzik, are more and more vocal about the need for a decentralized Layer2 built upon bitcoin. Except if we consider that services doing micropayments are spammers (which I don't), I think we (developers) should be more supportive of Layer2 projects (payment channels, hub & spoke, lightning, ...) and that we should try to better explain to the public the importance of these projects.

My 2 satoshis.

laurentmt

sr. member

Activity: 384

Merit: 258

Out of curiosity, I've tried to find which entities have sent transactions with 138 and 101 outputs.

It's likely that transactions with 138 outputs are related to a faucet called Microwallet (see 1MiCRoEgje2a8U1FthQGDAwaWabDbcBsTe)

Transactions with 101 outputs are a funny case that I'll call 'Crazy10073'.
They might be related to long peeling chains, splitting an utxo in 100 outputs of 10073 satoshis + 1 output used at the next step (example 112LJ7ZjaHwhsXWcq44tgG6KX6qmjdKrhY)
And if you try to find the origin of these coins, you'll find coins coming from...SatoshiDice.
I guess that Crazy10073 is pure spam or a kind of postmodern experiment... Cheesy

laurentmt

sr. member

Activity: 384

Merit: 258

A few additional charts which aggregate data for 2014. That should help to confirm/refute the first intuitions.
Comments/suggestions are welcome ! Don't hesitate to send me a PM if you're interested by the raw data used for this charts (csv files).

Disclaimer:
"#UTXOs created" is the number of utxos created in the blockchain (including op_return...) which is different from the concept of UTXO set in bitcoin core.

Weekly cycle

The two following charts confirm the existence of a weekly cycle since Q2 2014, with peaks occuring every 7 days.

Ratio (#UTXOs created / #UTXOs consumed) per day

#UTXOs added per day (i.e. #UTXOs created - #UTXOs consumed)

Sundays VS others days of the week

#UTXOs added per day of the week

#Transactions per day of the week

Fees per day of the week

Transactions fingerprints

#UTXOs added per #UTXOs created per transaction (sundays only)

Transactions with 200 utxos created per tx have added 1,635,768 utxos (45.6% of utxos added on sundays)
Transactions with 2501 utxos created per tx have added 1,112,202 utxos (31% of utxos added on sundays)

These two patterns of transactions represent around 76% of all utxos added on sundays.

#UTXOs added per #UTXOs created per transaction (all days)

Transactions with 200 utxos created per tx have added 2,120,703 utxos (around 26.96% of utxos added in 2014)
Transactions with 2501 utxos created per tx have added 1,561,468 utxos (around 19.85% of utxos added in 2014)
Transactions with 138 utxos created per tx have added 834,686 utxos (around 10.6% of utxos added in 2014)
Transactions with 2935 utxos created per tx have added 651,348 utxos (around 8.28% of utxos added in 2014)
Transactions with 101 utxos created per tx have added 555,311 utxos (around 7% of utxos added in 2014)

These five patterns of transactions represent around 72% of all utxos added in 2014.

Fees & incentives

These charts are for Solex.

Notes:
- each point stands for a day
- fees are expressed in satoshis

Fees vs #Transactions

Fees vs #UTXOs consumed

Fees vs #UTXOs created

Fees vs ratio (#UTXOs created / #UTXOs consumed)

laurentmt

sr. member

Activity: 384

Merit: 258

Here are a few additional stats. Later this week, I'll check if I can produce new ones.

Number of utxos consumed & produced per day

While the values remain close during the week, a gap can be observed on sunday: #utxos consumed decreases while we have peaks for #utxos produced.
Nothing new here, just another way to state previous observations.

Address reuse

Address reuse is the ratio (#utxos created which have an address already seen in the blockchain) / (#utxos created)

Whatever the services, this chart may confirm the intuition that the growth of the utxo set on sunday is related to services doing periodic automated payments and reusing the same address for the payee.

laurentmt

sr. member

Activity: 384

Merit: 258

Quote from: solex on May 10, 2015, 08:51:14 PM

This is really good research, as it is clear that these types of tx are "underpriced" and there is a clear mismatch with cost in storage.
Watching the thread with interest.

I'm far to be an expert on these matters but my feeling is the subject is hard.

For example, these services could send a lot of txs with 2 or 3 utxos instead of a single tx with so many utxos. That may raise the fees received by the miners, but on the other side, that would increase the number of tx inputs (which embed signatures).

As a result, we would have the same growth for the utxo set but with a bigger blockchain. That may be a good compromise for miners and pruned nodes. It's not so good for the services and for full nodes which don't want to prune the blockchain and don't receive fees.

solex

legendary

Activity: 1078

Merit: 1006

100 satoshis -> ISO code

This is really good research, as it is clear that these types of tx are "underpriced" and there is a clear mismatch with cost in storage.
Watching the thread with interest.

laurentmt

sr. member

Activity: 384

Merit: 258

Quote from: virtualx on May 09, 2015, 12:00:57 PM

The growth is from something else, I don't know what but perhaps salaries or automated payments.

laurentmt

sr. member

Activity: 384

Merit: 258

Quote from: virtualx on May 09, 2015, 12:00:57 PM

This is very interesting. Any idea why the transactions rise so much on those days?

To be honest the answer is no because my tool needs some more work before allowing to identify the source of these steps.

A few months ago, I've done a quick manual check in the blockchain and I was able to spot transactions which obviously participate in the growth of the utxo set. These transactions seem associated to faucets doing weekly payments but nothing says that these faucets are the unique cause.

Actually, it's likely this growth is caused by a small number of services and not by a unique service. If it's the case, it's not easy for these services to detect that their individual usage of the blockchain might cause a problem. Especially if you think that public discussion about the growth of the utxo set is very recent.

On the positive side:
- For now, nothing proves that we face malicious intent. That might be just another good example of the tragedy of the commons.
- I'm convinced that startups like coinalytics or chainalysis might help here to get more detailed information. It's up to them to decide if they want to help the community on that matter but I think they have a financial incentive to do it, since the success of their business is linked to the success of bitcoin.
- If these steps are caused by a small number of services, I want to think that they have an incentive to improve things since their success is linked too to the success of bitcoin. It might happen that some of these actors cannot change their behavior without breaking their business and it's by no means my intent to ask them to sacrify their business for the sake of bitcoin.

It's far to give us a long term solution but, as I wrote in the OP, that might be an easy way to buy more time.

virtualx

hero member

Activity: 672

Merit: 508

LOTEO

This is very interesting. Any idea why the transactions rise so much on those days?

Quote

If the current growth of the utxo set is caused by a small number of actors, it's possible that these actors may change a few things in their model without disrupting their business. For example, if it appears that faucets doing weekly payments are the main cause of the actual growth, may be they could replace weekly payments by monthly or quaterly payments. A payment done every 2 or 3 months may decrease the growth by one order of magnitude. It's by no means a solution to scalability issues but it may be an "easy" way to buy more time and that sounds like a good news.

The growth is from something else, I don't know what but perhaps salaries or automated payments.

laurentmt

sr. member

Activity: 384

Merit: 258

FWIW, here are a few observations related to the growth of the utxo set.

EDIT: For a TL/DR, you can jump to the conclusion

The growth rate of the utxo set has increased since late 2013 / early 2014.

(please note this chart counts the cumulated number of utxos in the blockchain (different from the utxo set of bitcoin core) but this observation remains valid for the utxo set in bitcoin core (can be checked with statoshi tool)).

More interestingly, a repeated pattern has appeared since early 2014, showing steps every sunday (around 100k utxos added to the set every sunday).

This pattern is interesting because activity in the bitcoin network is low on sunday in term of number of transactions

or in term of volume

We can also observe huge peaks every sunday in the average number of utxos created per transaction

(max value is the average number of utxos created per transaction at blocks level and is the max value among all blocks created on a given day)

A few months ago, my intuition was that these peaks are "caused" by a small number of actors (bitcoin services ?) reorganizing their utxos or doing some weekly payment. A very quick manual check in the blockchain led me to think that faucets doing weekly payments may play a role here but others actors may also have a role. A more formal analysis would be welcome.

So, please, don't get me wrong. I don't call to a new witch hunt targeting some actors from the ecosystem. Even if these observations are correct, there's no hint of a malicious intent. Actually, it's quite the opposite, if we consider that these transactions occur during low activity periods in the network.

Here's my point:
If the current growth of the utxo set is caused by a small number of actors, it's possible that these actors may change a few things in their model without disrupting their business. For example, if it appears that faucets doing weekly payments are the main cause of the actual growth, may be they could replace weekly payments by monthly or quaterly payments. A payment done every 2 or 3 months may decrease the growth by one order of magnitude. It's by no means a solution to scalability issues but it may be an "easy" way to buy more time and that sounds like a good news.

My 2 satoshis

Topic: Shower thoughts about the growth of the utxo set (with stats inside) (Read 2677 times)