Topic: Reliability of sites such as mempool.space for estimating fees. (Read 429 times)

I agree with you but let me explain why I prefer Jochen-hoenicke.de site again.

Even the red bands in Jochen-hoenicke.de don't always mean overpaying but also can means expensive, sometimes they are fee rates you have to pay to get a confirmation quickly (next few blocks), in my view, it is still a red flag (warning) for people. If you are not in emergency to move coins, it's good to sit back, think more about red-banded fee rates and you can wait a few days, weeks to move your coins. With this view, that site and its color design are very helpful.

My guess only but likely the person behind that site, actually has clear idea about what colors to use and for what purposes. It's different than mempool.space and how they use colors.

Thank you, I don't have skills to do that but your post is damn helpful, I like it.  

More specifically, you can read how the suggestions are calculated here: https://github.com/mempool/mempool/blob/827b0f6ad1c419e79d1a5cfa55b74b9c993a4748/frontend/src/app/docs/api-docs/api-docs.component.html#L199

Snipped and reformatted for easier reading:


As I said earlier in the thread, I think those estimates are bit on the high side. At the moment, the "No Priority" fee of 16 sats/vbyte still puts the transaction less than 1.5 MvB from the tip, which is not what I would call "No Priority".

According to the image you posted, mempool.space is saying that if no new transaction is made, the next block will include transactions using the fee rate of at least 14 sat/vbyte and the fifth block will include tranactions paying 11-12 sat/vbyte.
Mempool.space isn't saying that if you use the fee rate of 12 sat/vbyte, your transaction will be included in the fifth block.

If you want to use mempool.space for fee estimation, you should use the fee rates displayed under "High Priority", "Medium Priortiy" and "Low Priortiy"

Mempool.space suggests using the high priority fee rate for placing your transaction in the first mempool block, the medium priority fee rate for placing your transaction between first and second mempool blocks and the low priority fee rate for placing your transaction between second and third mempool blocks.


See the image I just took from mempool.space.



According to the image, Mempool.space is saying that if no new transaction is made, the next block will include transactions paying the fee rate of at least 15 sat/vbyte and the second block will include transactions paying 14-15 sat/vbyte.
Mempool.space is suggesting the fee rate of 18 sat/vbyte for placing your transaction in the next block and the fee rate of 17 sat/vbyte for placing your transaction in the next two blocks.

I would extrapolate the transactions made in the past few hours, and use that as an estimate of transactions in the coming hours. It's not always accurate and there can be exceptions, but most of the time it's pretty accurate.
That's why I like Johoe's site: it quickly shows fees are currently slowly going up, which means if you pay the minimum and it doesn't get confirmed quickly, it's likely to take a lot longer. If it would have been the opposite and fees are slowly dropping, paying the minimum is likely to get your transaction confirmed quickly.
I can't get this information from mempool.space's estimate.

To be fair, there isn't a way to figure out how many new transactions will be produced in advance.

Maybe a machine learning model could be made, that looks as previous mempool activity over the past 2 or 3 years and then tries to estimate how many new blocks will be created at any given time, and then estimate the median fee for all of these transactions, and then "push" the stated transaction fee right now closer towards this median. But that would be resource intensive.

I never really looked at it before until now:

It looks like their estimate in confirmation time is based on the assumption that no new transactions are being made. So, if you pay 12 sat/vbyte now, your transaction will be in the fifth block. That sounds acceptable to me, except for the fact that there's about 1 block worth of new transactions every 10 minutes, so it can take days to clear just those 5 blocks.

You can reverse the colors used on the chart because it's implemented as a gradient, and then run your own frontend - since this stuff is defined inside Angular source which is then compiled down into not-so-easy-to-edit JS.

First go to this file: https://github.com/mempool/mempool/blob/master/frontend/src/app/components/svg-images/svg-images.component.html

Clone the repository, and change all the instances of this:


to this:


Then follow the instructions to self-host Mempool.space on your own hardware.

That's not what the colors on Jochen's site mean. The fee bands are simply different colors. There is no color that means "overpaying". Go back to May this year or go back to the end of 2017 and there were plenty of times when you had to pay fees in the red bands to have a reasonable chance of being confirmed within the next few hours. This was not an overpayment, but rather completely appropriate.

Nope.

I did neither mean about accuracy of mempool.space fee recommendations nor the color in your screenshot. I meant that band colors in their chart and about red flag, overpaying, it is exactly what I meant.

Like on Jochen-hoenicke.de, when fee rate band color hits hot bands like red, it means overpaying. For lower fee rates, even you are overpaying, it is still fine because not too much expensive.

Compares it to mempool.space, I hope you see difference in their color design. Let me emphasize, I only dislike colors used in their chart.

Mempool.space is open source and you can run it yourself as self-hosted, so you don't have to depend on any third party.
If you don't want to do that than you can check mempool state from several sources and get median mid value.
You can check my BTC Mempool Fees Extension/add-on for Firefox/Chrome browser that includes mempool.space with two more source (blockchain.com and bitcoinber.live).
Default settings in mempool.space but you can do manual change quickly.

PS
Note that nothing is 100% precise since all of this is just estimated fees.

If the change is smaller than the dust limit, that's the only option. The difference could even be more than the dust limit, as adding another output would increase the fee further.

I've made mistakes with fees, especially when I try to rush things instead of taking my time to review the transaction. It happens.

I don't think I emphasised it well before. Not just the fee. I know that in every block there are people who pay high fees well above what would be reasonable to include the transaction in the next block. I mean the fee in relation to the amount you are sending. If someone sends 2 bitcoins I am not surprised that he overpays the fees. But in the transaction I have posted, there is one input and two outputs of about 0.001 bitcoin and he is paying a third in fees. I don't know how someone who is sending 25 euros can be in such a hurry to pay 8 to get into the next block. That's not what bitcoin is about as far as I know. It's not done for privacy, or to avoid an unspendable change or anything else. That's why I say I think he's someone who I think has no idea what he's doing.


I did not say that.


It has happened to me sometimes that it is the wallet itself that rounds up the fees.

Sometimes mempool.space was recommending me to send transaction with 6 sat/vByte and within minutes fee was increased up to 26 sat/vByte, that happens and mempool.space has nothing to do with this.
By the way, you can manually check some recent blocks, upcoming ones too and decide yourself what fee will be enough to include your transaction in next few blocks. You have to keep in mind that sometimes miners manually add transactions in their blocks and you shouldn't include added transactions when you calculate fees.

With the privacy implication of linking whatever payment you were making to your favorite VPN or account at some website. Sometimes it's simply not worth it for the sake of 1000 sats.

Alternatively, send it to the public donation address for something like Tor or EFF.

I take it as a challenge to waste as few sats as possible. This small output for instance could have added a few days to my favourite VPN, or topped up the balance at some website.

Look at any block and you will see dozens of such transactions.

Here's the most recent block, for example: https://mempool.space/block/000000000000000000001d63a0516748160bb6b01741ab14b41277416fd5f54d

mempool.space lists transactions in a block in the order the miner includes them. Since miners preferentially include higher fee paying transactions, then usually all the highest fee transactions are listed first. I count 19 transactions which pay a fee of 100 sats/vbyte or more, when actually 8 sats/vbyte was all which was needed to get in to that block. Click on any other block and you will see the same thing.

You can't say that every single one of these transactions is simply because someone doesn't know what they are doing. Some will be, sure, but there will also be people who absolutely must get their transaction in to the next block for whatever reason, centralized exchanges with their poorly optimized algorithms, people who simply don't care about the extra few bucks they are spending in fees, some poorly configured wallets or software, and so on. I for one have created transactions with excess fees when paying a regular fee would leave me with a change output of a few hundred or a few thousand sats. Rather than be left with an output which would cost 50% of itself in fees just to spend, I'll just add it to the fee I'm already paying and donate it to the miners.

Right. But it's not really difficult to use the website.

I just took a screenshot of jochen-hoenicke website.



Acording to the image:
There are around 143 vMB of transactions paying at least 1 sat/vbyte.
There are around 125 vMB transactions paying at least 2 sat/vbyte.
..
..
..
There are around 22 vMB of transaction paying at least 6 sat/vbyte.
There are only 0.37 VMB of transaction paying at least 7 sat/vbyte.


Considering that each block can contain up to 1 vMB of transactions, if I want my transaction to be confirmed fast, I would set the fee rate to at least 7 sat/vbyte.
If I set the fee rate to less than 6 sat/vbyte, my transaction would be at least 22 vMB from the tip and considering that more and more transactions will enter the mempool, it will take a very long time until my transaction is confirmed.

The image clearly shows that how increasing the fee rate from 6 sat/vbyte to 7 sat/vbyte can increase the chance of getting confirmation.

I was just looking and I see that in the last block, transactions with a little under 7 sats/vB have been mined, but one transaction has caught my attention, because I understand that there are people who pay very high fees if they send a large amount of money, but don't you think the fees paid by this transaction for the amount they send is very unreasonable? I understand that this is someone who does not know very well what he is doing.

The fees required for transactions are always changing, if you use a slightly lower fee and your transactions still get  confirmed, It is possible that within a very short time while you were not monitoring the fees, that the fee dropped to that amount of fee you used which made your transaction to be confirmed before it went back up. I personally don't like to wait so long for transactions to be completed, so I always use a medium or high priority fees so I have the assurance of a quick transaction.

For newbies: https://mempool.space/
For advanced users: https://jochen-hoenicke.de/queue/#BTC,24h,weight

I always prefer to recommend mempool.space for news. Advanced users that understand very well how to use jochen-hoenicke.de will prefer jochen-hoenicke.de.

It is still one of the best mempool site that newbies can use, although jochen-hoenicke.de can be more accurate. I have used mempool.space times without number with good fee rate estimation.