Topic: Huge transactions dependency graphs in mempool (Read 184 times)

Well, maybe it's not laziness, One must have priorities, I've posted this just in case somebody find it useful/interesting.


I think something like this is the most plausible answer, nevertheless I felt the urge to share it with the community.


Yes, I made a special effort in having a mining queue which place CPFP Transactions in it's correct place, if that weren't the case, the mining queue wouldn't be accurate. Glad to see somebody has noticed it.
I'm proud of the hierarchical visual representation in https://mempoolexplorer.com/mempool and in the graphical representation of dependencies between transactions, but the algorithm I've used it's the same as in bitcoind getBlockTemplate but generalized for the entire mempool, not only the first block. You can check it at https://github.com/mempoolexplorer/txMemPool/blob/master/src/main/java/com/mempoolexplorer/txmempool/entites/miningqueue/MiningQueue.java

As I've stated in https://mempoolexplorer.com/faq#aims this begun as a side project to learn microservices in java using spring, be aware of the amount of services needed to make it work (i.e. administration server, configuration server, gateway service... etc). I think you can ignore all that paraphernalia and focus on the code I've pointed you to. I'm planning to remove the microservices architecture sooner than later anyway...

It's been a few years since I've been involved in this sort of stuff, but to me it looks like a high-volume service that is sending out bitcoin (e.g. an exchange or casino). If I wasn't so lazy, I'd try analyze the time-of-propagation for the transactions and see if it matches my hunch.

Most coinselection algorithms (including bitcoin cores) are kind of designed for user-level wallets, so they are massively consolidatory in regards to wallet utxos. This naturally means when you start sending out a lot of transactions, you're going to often generate huge unconfirmed transaction chains. This easily results in problems because of the mempool limitations (e.g maxdepth type thing).

With bustabit, I spent a few months and wrote my own purpose-built coinselection system to avoid these problems, but for most people that's a lot too much work and it's easier to just do a hack like make transactions occasionally include multiple-change outputs. That would likely explain some of the of the DAG like graphs you see?

---

Edit: Looks like your website is one of the few that actually calculate cpfp fee rate correctly I'm curious if you have the code you use for that calculation available? I (personally) really struggled with this calculation, and the solution I have feels like it's 10x as complicated as what someone smarter than me would come up with.

Seeing if you could get away with somehow scamming them.
Although in theory 1 unconfirmed is the same as 50, if you have a way of just screwing with 1 unconfirmed input in the chain you have to do something to that one to make the entire transaction invalid.

If there are 50 then and you can screw with a bunch of them your odds are that much better of doing it.

Other than that I have no idea.

-Dave

Yes, I know, The distinction on the webpage is because "linear" chains are pretty common, at first I thought they were CPFP to the extreme, so I colored the transactions from red to green to distinguish this case. But I found a lot of chains of this kind with transactions having two outputs: one being a OP_RETURN and the other the utxo dependency of another transaction. Maybe you can get some of them just now, they are pretty common.


Yes, something more to the todo list. But not sure if worthy. Maybe just for fun.

Indeed, Txs with red color has the most sat/Vbyte and the green the less.

Thanks a lot for your guesswork. Not really sure for what could this be useful anyway.

Even if a transaction chain appears to be "linear", all transaction chains are formed via a DAG by bitcoin's nature (assuming the transaction is valid).

I can only speculate why someone would chain so many unconfirmed transactions together. I would suggest you monitor any such transactions and capture the graph immediately prior to the transaction chain getting confirmed.

If immediately prior to when the graph of transactions are confirmed, there is a transaction at the "tail" of the graph that has an outsized transaction fee, most likely someone was trying to minimize their tx fees, and was using a strategy similar to CPFP.

If the above is not the case, there could be a number of other reasons why someone is engaging in the above practice. The first thing that comes to mind to me is that someone might be trying to test the limits as to when certain entities are willing to accept unconfirmed transactions.

Two months ago, while developing my webpage about mempool statistics, I found that there was a huge dependency graph having more than 700 transactions in (my) mempool.
I could capture some minor examples having around 470 transactions (see below)

Of course no mempool rule is broken, i.e. max mempool ancestors for a transaction, but I'm wondering about what can be doing this kind of dependency graphs.

I posted a question about it in bitcoin.stackexchange without luck:
https://bitcoin.stackexchange.com/questions/110723/huge-dependency-graphs-for-transactions-in-mempool

The best answer given to me by https://b10c.me/projects/ was that maybe there was an agent running out of utxos in its wallet, and then started using its non-confirmed utxos in the mempool, forming the dependency graphs.
If that is the case, the graphs seemed not having a structure to minimize its depth for not exceeding the max transactions ancestors rule in the mempool (i.e. a tree), and seemed pretty unstructured?. (but investigating about the best strategy in this case and recognize its visual shape is something I haven't done. Take that with a grain of salt)
So although the former answer is the most plausible, I feel it is worthy to share this with you.

What can be creating these dependency graphs?
Are they worthy to investigate or has been investigated before?

Captured transaction graphs:
https://gist.github.com/dev7ba/c144c68127b97082652bc860cc95edf6
https://gist.github.com/dev7ba/50167d6e336100698ab2599123444efc
These .json are offered by the backend of my web page you can call it here: https://mempoolexplorer.com/txmempool/miningQueueAPI/tx/{txIdHere} to obtain a transaction statistics currently in mempool.
See txDependenciesInfo.nodes and txDependenciesInfo.edges data to obtain the dependency info (if any) for queried transaction.

You can visually see the transactions graphs currently in (my) mempool here: https://mempoolexplorer.com/txsGraphs.
These graphs can be linear: forming a long chain of dependencies, or non linear: forming a Direct Acyclic Graph.
When clicked on the number of transactions on a graph, a list of the transactions ids contained in that graph is shown. You can go to the graph by clicking any of these transactions id. The result is shown in the main page along with its position in the mining queue (from my node's perspective) and other useful data.
To avoid the annoying constant refreshing please check "lock mempool" in the upper right corner.

Be aware that I currently don't have any historic section of dependencies graphs or something like that in the webpage. The results show are in real time, so if mempool is almost empty maybe no graphs or very small ones are shown.

Note that the following graphs are minor examples compared to the huge ones I'm talking about. (sorry, I didn't take a screenshot for them, only the .json returned by my backend)
https://imgur.com/a/hfj2k1s.png
https://i.imgur.com/9nnJwWY.png

Bonus Points: This webpage has other sections discussed in https://mempoolexplorer.com/faq and also here https://bitcointalksearch.org/topic/a-way-to-know-how-good-a-miner-is-choosing-its-transanctions-5383112 that may be of interest to you. I'd be glad receiving feedback.