Pages:
Author

Topic: Discussion: optimization of mempool fee levels (Read 576 times)

legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
Decimal Places.

People start at the lowest and work up. Normal non-spam users would linger in the lowest levels. Why not.

What would happen if - just hypothetically - Bitcoin had 16 decimal places and not 8 ?

1 satoshini was 1*10-16 BTC.. transaction fee priced in satoshini / byte.

I think fees would come down, but not by as much as 8 decimal places.


Why not? The idea of adding more decimal / allow smaller fraction of Bitcoin has been suggested for years, even though most people would wait until Bitcoin price is very high.

But since it requires hard-fork, there are better & easier alternatives such as make default minrelayfee on Bitcoin full-node client become 1 satoshi.
legendary
Activity: 3906
Merit: 6249
Decentralization Maximalist
What would happen if - just hypothetically - Bitcoin had 16 decimal places and not 8 ?

1 satoshini was 1*10-16 BTC.. transaction fee priced in satoshini / byte.

I think fees would come down, but not by as much as 8 decimal places.
Exactly, I think psychology and our limited brain resources would prevent us to use an "optimal" fee strategy (which would be, in this case, to always chose a fee exactly 1 satoshini per byte higher than what we think our competitors would chose). But it could have a small effect.

Apart from that, 8 more decimal places would enlarge transactions - not by that much, but significantly. Simply chosing "sat/kB" instead of "sat/Byte" seems a better strategy.
hero member
Activity: 718
Merit: 545
Decimal Places.

People start at the lowest and work up. Normal non-spam users would linger in the lowest levels. Why not.

What would happen if - just hypothetically - Bitcoin had 16 decimal places and not 8 ?

1 satoshini was 1*10-16 BTC.. transaction fee priced in satoshini / byte.

I think fees would come down, but not by as much as 8 decimal places.
legendary
Activity: 3906
Merit: 6249
Decentralization Maximalist
I like pooya87's idea of an "increased granularity" (being able to set fractions of a satoshi per byte). However, Core afaik already is using BTC/kB with a granularity up to a single satoshi (and not sat/B) as the default unit. And afaik you also can always select an absolute fee amount.

But lots of wallets and websites continue to use the "sat/B" unit. The reason is probably that this is the unit "most easily digested", with numbers currently of 1-10 in low-fee periods, 10-200 in mild congested times and 200-1000 in highly congested times. If in the last block the average fee of confirmed transactions was, say, 5 sat/Byte, it's reasonable to pay 6 sat/Byte if you want the tx to be confirmed soon, and not 5001 sat/kB.

So increased "fee granularity" in wallets with sat/B as the default option would certainly be desirable, but the impact would be probably not too high. But as there are no risks associated, I certainly would support this kind of measure.

The OP's proposal, however, would probably not work. It over-estimates the "tx relaying power" of non-mining nodes. It gives incentives, instead, to simply connect directly to nodes that you know that they're mining. This will probably become the default behavior if such a mechanism was implemented, as gmaxwell writes.

@franky1: A "priority-based" tx inclusion by miners like in early bitcoind versions would be certainly interesting, but I struggle to understand how this could really be enforced. Maybe like the "weight" calculation in Segwit transactions? However, I've read some time ago that some miners are sometimes selectively preferring to include non-Segwit transactions over Segwit-type ones. If this is possible (and not punishable by the protocol), then it's probably not reliabily possible to really enforce priority-based tx confirmation. (And I'm sure the reason why the priority mechanism was abolished in newer Core versions has to do with that problem.)

If it's possible to enforce, priority-based calculation would certainly lead to lower fees, because it would incentive people which aren't in a hurry to confirm to pay only a very low fee.
legendary
Activity: 4396
Merit: 4755
this topics idea is no better than current state of affairs.

firstly if using the scenario of say 20mb(WU)(5mblegacy) mempool size where
5sat is priority(0-4mbWU(0-1mb leg)) next block,
4sat is less priority(4-8mbWU(1-2mb leg)) second block,
3sat is less priority(8-12mbWU(2-3mb leg)) third block,

a spammer pays 5sat, gets confirmed then instantly throws that utxo into a new tx of 5 sat and is instantly in the priority nxt block listing.
meaning those that paid 4sat are not guaranteed to be in the second block as the spammer is in the second block too

this, like current status just leads to games of everyone just paying ever increasing amounts, until people give up and the price then eventually resettles until people realise it settled down to then play another round of ever increasing amounts

what needs to be done is bring back a fee formulae that is a consensus rule (have to abide by) which actually has options.

EG if a tx is only 1confirm deep its 'score' is lower than someone with 144 confirms. so the spammer(1confirm) has to pay more to raise its score to be given priority.
this way it actually makes spammers pay the price, where as occassional spenders are not affected by th price. and by using the score based system its not a everyone is hit by a premium due to a few users.

this way it actually make spammers who multispend daily having to pay more, then become more incentivised to spam less or use offchain solutions. while the occassional non abusive user is not paying out alot but getting their transactions noticed by miningpools collating transactions

to explain better
this is what i see as the logical punishment for bloating/rspending spammers. whilst rewarding moral normal transactors

one which includes a CLTV voluntary option. where users gain priority points if they voluntarily agree to put their funds into a 1-day maturity. but those avoiding the one day before respend or have bloated transactions pay more to get into a block sooner.

EG
if you really need priority you agree that once confirmed you cant respend for a day.
it also means you can be selective of priority. by only putting a 142block wait if your happy to wait a couple blocks because it wont be priority for a couple blocks by not paying quite enough fee. allowing the age/maturity/fee variables to give a better flag of desire.

obviously those moral users that actually need to spend more than once a day could see the niche of LN as a way to transact often and cheaper.
and those that dont spend every day get priority and not need LN or to CLTV mature funds, because they are not spending everyday, anyway.

here is one example of a formulae that does not care about how much people are spending (not a rich gets priority, poor are victimised old formula), but rewards people willing to wait a day, have lean transactions. and penalises those that want to respend often or have bloated transactions



basically
if your transaction is 2x a lean tx. you pay twice as much. if 44x a lean tx you pay 44x
if you dont want to mature your funds for 144 blocks and only want to wait 1 block you pay 144x.
if the tx is both 44x bloated and wants to respend the very next block after getting confirmed then it costs 44x*144X


though my formulae is not finalised or perfect for every utility. i see how changing the priority formulae can cause more benefits for good people and penalise the bad, without making it used just to be snobby about rich vs poor. due to it no longer rewarding the rich with points just for being rich, which the old formulae done
legendary
Activity: 3430
Merit: 3080
Aside: the reason that the total transactions fell was due to the price bubble burst, and segwit's design to ignore most transactions.

Segwit's not doing a good job of "ignoring most transactions", because most transactions are not segwit. That was how this little debate started. you saying segwit failed due to low uptake.


You're a big fan of non-sensical arguments:

  • segwit ignores most transactions
  • most transactions ignore segwit

both those things are true, depending on which version of "everything is segwit's fault" you're trying to argue
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
segwit was implemented in 2017 ... BIP91 ...

and in Nov 2017 to Jan 2018 it failed to manage the mempool such that txn fees were VERY high

that's unreasonable

From segwit activation (September '17 IIRC) to Jan 2018 was just a few months. Segwit couldn't possibly be used to react to big hike ups in tx backlogs straight after it activated
Well any block size change based on any bitcoin transactions (not just some new transactions that you want to force everyone to use) would have worked.

Nov block reward Avg: 118.27%
Dec Avg: 129.11%
Jan Avg: 132.04%

Then total txns fell - some of the blame resides with segwit for that.

so....

  • mempool full: because segwit
  • mempool empty: because segwit

uh, ok
Total transaction counts fell - seriously you know nothing about the number of transactions in bitcoin over time?
Ignorance is bliss I guess. Helps you make up false arguments.

Aside: the reason that the total transactions fell was due to the price bubble burst, and segwit's design to ignore most transactions.
legendary
Activity: 3430
Merit: 3080
segwit was implemented in 2017 ... BIP91 ...

and in Nov 2017 to Jan 2018 it failed to manage the mempool such that txn fees were VERY high

that's unreasonable

From segwit activation (September '17 IIRC) to Jan 2018 was just a few months. Segwit couldn't possibly be used to react to big hike ups in tx backlogs straight after it activated, that wasn't quite enough time for the whole of Bitcoin to move to segwit addresses, and then start spending from them Cheesy. Good thing too, blocks stayed effectively capped at ~ 1.1MB until Jan '18, compromise: achieved Smiley

seems you forgot the reasoning why BIP100 got rejected too, but whatever


Nov block reward Avg: 118.27%
Dec Avg: 129.11%
Jan Avg: 132.04%

Then total txns fell - some of the blame resides with segwit for that.

so....

  • mempool full: because segwit
  • mempool empty: because segwit

uh, ok
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
if you continue this excuse for years (it has been 'years')

Huh e.g. the entirety of the year 2018,  during which mempools were half empty?
Oh, you forgot did you ... memory loss - advanced stage?
segwit was implemented in 2017 ... BIP91 ...

and in Nov 2017 to Jan 2018 it failed to manage the mempool such that txn fees were VERY high
Nov block reward Avg: 118.27%
Dec Avg: 129.11%
Jan Avg: 132.04%

Then total txns fell - some of the blame resides with segwit for that.

the 'solution' is clearly NOT a solution if it still fails to solve the problem"

which problem? Wink

blocks are too big? or blocks are too small? By choosing to use P2PKH addresses, blocks stay smaller. If you use segwit addresses, blocks can be made bigger.

Compromise: achieved
Lulz - we are already talking about yet another recent failure of segwit here ...
I guess you consider it's multiple failures an achievement ... most people consider that simply a failure.

Giving excuses for it's continued failure still makes it a failure.
legendary
Activity: 3430
Merit: 3080
if you continue this excuse for years (it has been 'years')

Huh e.g. the entirety of the year 2018,  during which mempools were half empty?


the 'solution' is clearly NOT a solution if it still fails to solve the problem"

which problem? Wink

blocks are too big? or blocks are too small? By choosing to use P2PKH addresses, blocks stay smaller. If you use segwit addresses, blocks can be made bigger.

Compromise: achieved

legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...

If you are complaining about what happened last week with fees sky-rocketing, then discuss that with Core who implemented Segwit which was "SUPPOSED" to solve such problems ... but failed to as usual.

Roll Eyes didn't take you long to abandon "the free and uncontrolled market", lol


Segwit's working as the compromise it's supposed to be. Only 2% of addresses are even using segwit addresses, because many people just aren't aware, and also since the industry as a whole has only really now begun to support it (probably a good idea, taking risks with new tech in the real world could be an expensive mistake).

So until more users and businesses switch, 1.2MB average blocks is as big as they'll get. If you want bigger blocks and cheaper fees, the incentive is there, use it. Or don't. It's a f(r)ee market
I wonder how many years you keep saying this Smiley

One could re-word it as:
"People don't use SegWit ... so it's not SegWit's fault"

Then reply with
"People don't do many things, if your solution is worthwhile, then get them to use it ... but if you continue this excuse for years (it has been 'years') then the 'solution' is clearly NOT a solution if it still fails to solve the problem"

Though I could throw into this the fact that BIP100 would have been an immediate solution and would be working today, and had/has a longer expected solution timeframe than SegWit.
P.S. BIP100 had a 70% coinbase vote at it's peak without anyone knowing if it would get implemented - but alas, no one paid core the money to implement it, core implements 'major' changes that they are paid to implement.
newbie
Activity: 14
Merit: 16
If you don't like the way core estimates your fee, then estimate it yourself ... which you can do.
Of course I do set the fee manually for my transactions.

But if there is a bad algorithm for fee estimation and everyone uses it, for the network, it doesn't really matter what one or 2 users do.

The algorithm used by bitcoin core is likely used by many big players that are not going to set the estimate manually, they have to rely on an algorithm and they trust bitcoin core.

To be more clear:
Right now I'm not advocating for any central planning or stuff like that.
I'm just trying to understand why the vast majority of the network does fee estimation poorly.
It's something everyone can see by looking at the last few days and hours.
legendary
Activity: 3430
Merit: 3080
The free and uncontrolled market is one of the biggest reasons why people want bitcoin.

indeed


If you are complaining about what happened last week with fees sky-rocketing, then discuss that with Core who implemented Segwit which was "SUPPOSED" to solve such problems ... but failed to as usual.

Roll Eyes didn't take you long to abandon "the free and uncontrolled market", lol


Segwit's working as the compromise it's supposed to be. Only 2% of addresses are even using segwit addresses, because many people just aren't aware, and also since the industry as a whole has only really now begun to support it (probably a good idea, taking risks with new tech in the real world could be an expensive mistake).

So until more users and businesses switch, 1.2MB average blocks is as big as they'll get. If you want bigger blocks and cheaper fees, the incentive is there, use it. Or don't. It's a f(r)ee market
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
If you don't like the way core estimates your fee, then estimate it yourself ... which you can do.
(... personally I write my own transactions and choose my own fees I want)

As for mining transactions, miners will choose higher fees per byte over lower fees per byte.
So these so-called 'overpaid' transactions will be confirmed first.

Implementing rules in bitcoin to enforce some group's ideals is not going to happen.
The free and uncontrolled market is one of the biggest reasons why people want bitcoin.
Central control (which is what you are effectively suggesting) goes against that ideal and wont be tolerated.
Miners will just ignore it also.

If you are complaining about what happened last week with fees sky-rocketing, then discuss that with Core who implemented Segwit which was "SUPPOSED" to solve such problems ... but failed to as usual.
hero member
Activity: 718
Merit: 545
https://gist.github.com/morcos/d3637f015bc4e607e1fd10d8351e9f41

Bitcoin Core Fee Estimation Algorithm.

---------

Hmm.. I've skimmed it twice now..
newbie
Activity: 14
Merit: 16
Is there any issue in bitcoin core fee estimation algorithm right now?
Looking at the mempool:
https://i.ibb.co/vsFLTC8/Screenshot-2019-04-12-at-11-50-34.png
While the mempool looks empty, I get +110 sat/b fee estimation from bitcoin core to get into the next block and +90 to get into the next 6 blocks.
Am I the only one seeing these numbers?
staff
Activity: 4242
Merit: 8672
The Copay wallet (and Bitcoin.com's which is based on it) is very problematic because it's a sleek wallet with easy UI for beginners -- lots of newbies use it. Users have no manual control over fees. If they choose to the lowest fee option -- which means they specifically don't need next-block confirmation and are willing to wait several hours -- they are paying up to 5x-15x what it actually costs to get in the next block.

Fair point but to counter: At one point copay leaked users private keys, there is little that can be done about sufficiently severe incompetence much less malice except encourage people to not use that software/service.

Quote
Coming from Bitpay/Bitcoin.com, it feels like a continuing political attack. I think they've been deliberately charging users absurdly high fees, which has had an unfortunate spillover effect on the fee market.
Agreed there too, but at least the subject of this thread can't really do anything about maliciously bad wallets.   There are 1001 ways to have the same effect-- they could still overpay and mine them themselves (overpayment for profit hurrah), they could simply make the wallets also spend additional long unconfirmed chain inputs to slow them down etc.

I do agree that things like these adversarial parties masquerading as supporting bitcoin is a problem, but trying to do somethin about overpayment only addresses a symptom at best, it if could work.

Quote
I agree there's no real way to fix this and the OP's idea won't really work. But I'm willing to acknowledge that it's a shitty problem for the network in spite of my appreciation for the design of the fee market. Users will keep overpaying because of shite fee estimation and ignorance and now that fees are strongly trending up again, they'll start blaming the protocol and complaining more loudly about it like in 2016 and 2017. I sort of wonder if we're gearing up for another political attack.

+1 Agreed here, sorry for perhaps overstating my case.
legendary
Activity: 1666
Merit: 1196
STOP SNITCHIN'
If you'd like to make a case that some people are paying higher than you when they'd really rather wait longer than you for confirmation-- that might be interesting.

In some cases, users obviously are paying higher than me when they'd rather wait longer.

The Copay wallet (and Bitcoin.com's which is based on it) is very problematic because it's a sleek wallet with easy UI for beginners -- lots of newbies use it. Users have no manual control over fees. If they choose to the lowest fee option -- which means they specifically don't need next-block confirmation and are willing to wait several hours -- they are paying up to 5x-15x what it actually costs to get in the next block. A couple examples:

https://www.reddit.com/r/Bitcoin/comments/7ngbng/how_to_avoid_fee_sending_btc_from_copay_wallet/
https://www.reddit.com/r/Bitcoin/comments/89xx9d/copay_is_the_most_broken_bitcoin_wallet_137/
https://www.reddit.com/r/Bitcoin/comments/6fyxs9/copay_super_economy_confirmed_in_11_minutes_are/

Coming from Bitpay/Bitcoin.com, it feels like a continuing political attack. I think they've been deliberately charging users absurdly high fees, which has had an unfortunate spillover effect on the fee market. This is what newbies see when they choose the lowest fee option (4 hour estimated confirmation time) and are still being overcharged for next block confirmation:



I'm sure there are many users who are happy to pay more than they need to. But I can't imagine all users looking at earn.com's estimator and sending 160 satoshis/byte for the "fastest and cheapest transaction" when there's 1,400 transactions in my mempool down to 100 satoshis/byte want to overpay. They just don't know any better, and unfortunately the Copay wallet and earn.com have been highly recommended for years on social media and this forum.

I agree there's no real way to fix this and the OP's idea won't really work. But I'm willing to acknowledge that it's a shitty problem for the network in spite of my appreciation for the design of the fee market. Users will keep overpaying because of shite fee estimation and ignorance and now that fees are strongly trending up again, they'll start blaming the protocol and complaining more loudly about it like in 2016 and 2017. I sort of wonder if we're gearing up for another political attack.
staff
Activity: 4242
Merit: 8672
Other people overpaying is none of your business. Only the fact that they paid more than you has any influence on you at all (they get confirmed ahead of you).

Sometimes people make mistakes, sure, but you shouldn't assume that everyone acting differently from you is making a mistake... or if they are, that its any of your concern.

Nothing I've seen discussed in this thread seems interesting.  If you'd like to make a case that some people are paying higher than you when they'd really rather wait longer than you for confirmation-- that might be interesting. Regardless, trying to fix prices on nodes will just make the network work less well and encourage centeralization when people then realize they need to give their transactions directly to miners.
newbie
Activity: 14
Merit: 16
I think it is necessary to punish wallets that do a bad fee estimation because they damage the user experience and ultimately bitcoin in general.

Indeed. I've always hoped these wallets would lose users due to higher-than-normal fees -- Copay is one example, and they don't even offer Segwit to boot. But the reality is that they drastically inflate the entire fee market so everybody is forced to similarly overpay to get any reasonable guarantee of confirmation.

I do seriously resent these assholes who immediately jump up 10-20 satoshis/byte just because there's 2,000 transactions in their mempool. This type of deliberate overpayment really screws us all over and it's really rampant. It snowballs when there is any spike in demand for block space. People are cramming $100 bills into a coin slot. It's just a tough philosophical problem because people should be able to compete in the fee market. It just sucks that the tendency of many wallets -- and people who are manually setting fees and rounding up to the nearest 10 sat/byte -- is to drastically overpay to maximize chances of getting in the next block. When everyone is overpaying so much, we all have no choice but to do the same.

You bring up an interesting point, that's for sure. These users are hurting my interests, driving up my costs, and relaying their transactions only reinforces their bad behavior. Miners have financial incentive to reinforce this bad behavior, but what's my incentive as a user and node operator?


I couldn't have said it better myself.
Pages:
Jump to: