Zero fee for bitcoin transaction [Historical data]

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: tranthidung on August 11, 2020, 05:01:13 AM

The zero-fee transactions time-series chart somewhat looks similar to empty blocks charts (daily, monthly basis). They have all been in a long downward trend.

i think the situation with empty blocks is a lot more complicated than zero fee transactions though. the tx situation is part fuller mempool and part default behavior of 90% of the nodes being minrelayfee>0.
but when it comes to empty blocks the reasons are:
- more usage which translates into more transactions in mempool for miners to pick up (eg. compared to 2009 with no one using bitcoin)
- little to no spy mining or at least a hugely improved way of doing it to avoid having an empty block
- increasing price which means even 1s/b fee transactions in a full block could be worth at least $350 or it could be a lot higher for example a block found an hour ago (643308) had $13,995.60 total fee.
- improved connections and internet speed and lower latency of mining pool nodes
- much smaller data to download from a new block thanks to BIP-152 (node basically downloads the header alone not the transactions since it already has them in its mempool).

tranthidung

legendary

Activity: 2310

Merit: 4085

Farewell o_e_l_e_o

Quote from: pooya87 on August 11, 2020, 12:51:29 AM

< ... >

Thank you for sharing some API skills here and the chart. Your chart demonstrates that there are more non-empty zero-fee transactions and proves that my approach is not correct.

The zero-fee transactions time-series chart somewhat looks similar to empty blocks charts (daily, monthly basis). They have all been in a long downward trend.

Quote from: tranthidung on May 04, 2020, 03:49:23 AM

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: pooya87 on August 11, 2020, 12:51:29 AM

Code:

https://api.blockchair.com/bitcoin/transactions?q=fee(0),is_coinbase(false)&a=date,count()&export=csv

You have some cool API skills

Quote

the result looks interesting though:

It looks like a spam flood on 2015-08-01 with 26429 zero-fee transactions instead of (about) 1000 per day on other days.

I'm also surprised to see the number of zero-fee transactions drop for about 3 years. I remember my mobile wallet at the time had a minimum fee of 1000 satoshi, while Bitcoin Core could make zero-fee transactions. So I'm guessing this is caused by wallet software default settings.

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: LoyceV on August 11, 2020, 12:15:59 AM

Quote from: pooya87 on August 10, 2020, 11:47:46 PM

i don't know what the blockchain API limitations are

It only shows 10 results at a time, the data dumps are much easier to use.

you can always play around with the "limit" and it seems to be maxed at 100 for transaction endpoint which that much better

Code:

https://api.blockchair.com/bitcoin/transactions?q=fee(0),is_coinbase(false)&limit=100

if you have the API key you can also export it to .csv file by adding some more filters (for example ID and hash)

Code:

https://api.blockchair.com/bitcoin/transactions?q=fee(0),is_coinbase(false)&fields=id,hash&export=csv

with a limited functionality you can export .csv by aggregating data using ?a=
for example the following link will you then umber of zero fee transactions on each date in tiny a .csv file

Code:

https://api.blockchair.com/bitcoin/transactions?q=fee(0),is_coinbase(false)&a=date,count()&export=csv

the result looks interesting though:

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: pooya87 on August 10, 2020, 11:47:46 PM

i don't know what the blockchain API limitations are

It only shows 10 results at a time, the data dumps are much easier to use.

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: LoyceV on August 09, 2020, 11:58:22 PM

Quote from: pooya87 on August 09, 2020, 08:50:27 PM

you are thinking of standard behavior that the "wallets" let you do.

I was thinking about those, but I'm aware about the current cases which are most likely pushed by miners. I've seen a transactions that looks like mining pools paying their miners with zero fee. I'm kinda curious now how often that happens, and I kinda want to download all transaction data again. I can't really make a 100 GB CSV-file though, and I'm running low on VPS disk space.
It would be a waste to download all this data just to check the fees.

i don't know what the blockchain API limitations are but you could always use their API to fetch the data you need with some effort. for example all zero fee transactions (excluding coinbase txs) from start to today are fetched like this:

Code:

https://api.blockchair.com/bitcoin/transactions?q=fee(0),is_coinbase(false)

you'll have to change the limit and increment the offset in multiple calls. according to the explorer they are ~6 million total. which is a lot less than the entire transaction database.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: pooya87 on August 09, 2020, 08:50:27 PM

you are thinking of standard behavior that the "wallets" let you do.

I was thinking about those, but I'm aware about the current cases which are most likely pushed by miners. I've seen a transactions that looks like mining pools paying their miners with zero fee. I'm kinda curious now how often that happens, and I kinda want to download all transaction data again. I can't really make a 100 GB CSV-file though, and I'm running low on VPS disk space.
It would be a waste to download all this data just to check the fees.

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: LoyceV on August 09, 2020, 12:43:48 PM

Quote

i personally have sent transactions with 0 fee

I remember the days that transactions with Medium priority could be sent without fee, as long as they were no more than 1000 bytes and at least one of the inputs wasn't dust (I don't remember the exact details). I often used that to consolidate small inputs without losing a part, all I needed was to wait for enough coin days to be destroyed to get high enough priority. Those were the days Cheesy

you are thinking of standard behavior that the "wallets" let you do. i used to create my own zero fee transactions with any size to consolidate outputs and then sign them with Electrum and broadcast it there (Electrum standard behavior prevents you from setting fee to zero but it signs it fine). last one i had is surprisingly on November 2017 and was around 1.4 kB.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: pooya87 on August 08, 2020, 10:10:44 PM

you can't find zero fee transactions by only looking at the blocks' total fee though.

Correct. If you're looking for all zero-fee transactions, you should use data from https://gz.blockchair.com/bitcoin/transactions/. I estimate it's about 40 GB to download.
I checked one day's data, and I'm surprised that most transactions paid a fee in 2014. That wasn't necessary!

Quote

i personally have sent transactions with 0 fee

I remember the days that transactions with Medium priority could be sent without fee, as long as they were no more than 1000 bytes and at least one of the inputs wasn't dust (I don't remember the exact details). I often used that to consolidate small inputs without losing a part, all I needed was to wait for enough coin days to be destroyed to get high enough priority. Those were the days Cheesy

pooya87

legendary

Activity: 3472

Merit: 10611

you can't find zero fee transactions by only looking at the blocks' total fee though. there can be a couple of thousands of transactions in a block and very early on the "default" behavior changed to setting fee for every transaction and as a result majority of transactions started paying a fee even though it was possible to send 0 fee transactions and get them confirmed. i personally have sent transactions with 0 fee in 2017 which were mined in blocks containing all non-zero-fee transactions.

you can also find many 0-fee transactions today which mainly belong to mining pools paying their miners in a self-made self-mined transaction which makes no sense to have a fee.

tranthidung

legendary

Activity: 2310

Merit: 4085

Farewell o_e_l_e_o

Quote from: nc50lc on August 08, 2020, 05:16:55 AM

Seems like it's done to prove that it's possible, an example or something a publicity stunt. The two zero-fee transactions were included by them obviously.
I thought those are like full blocks where the miner didn't claim the transaction fees (I'm a bit disappointed, just a bit ).

I have BTC empty blocks (2009 - 5 May 2020): miners, size, daily, monthly,yearly stats. According to this (data is up to 5 May 2020 only), Poolin has not yet found 8+ empty blocks from 2011 to 5 May 2020. I will include miners for the analysis and will update it when I use the computer where I store data again. Wink

Edit: Update is here.

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

Quote from: tranthidung on August 08, 2020, 05:04:44 AM

Thanks!
Seems like it's done to prove that it's possible, an example or something like a publicity stunt. The two zero-fee transactions were included by them obviously.
I thought those are like full blocks where the miner didn't claim the transaction fees (I'm a bit disappointed, just a bit :D).

tranthidung

legendary

Activity: 2310

Merit: 4085

Farewell o_e_l_e_o

Quote from: nc50lc on August 08, 2020, 04:40:53 AM

So, there were two non-empty-zero-fee blocks in 2020, can you specify the height of those blocks?
I'm a bit curious on the included transactions (just a bit). TIA

Here you go. I gave you zero-free blocks since 2016. Within two years 2017, and 2018, there is no zero-fee blocks.
Links to two zero-fee blocks have been surprisingly found in 2020, and one block was found in 2019:

2020
- Block #617750
- Block #626507
2019
- Block #600590
Poolin pool is the miner in 2019 and 2020.
BTCC Pool is the only pool who did non-empty zero-fee transactions in 2016. What a surprise!

Code:

+------------------------------------------------------------------------------------------------------------+
   | blockheight blocktime year guessed_miner transaction_count fee_per_kb empty zerofee |
   |------------------------------------------------------------------------------------------------------------|
391384. | 391383 02jan2016 11:46:11 2016 BTCC Pool 2 0 No Yes |
391409. | 391408 02jan2016 16:38:55 2016 BTCC Pool 2 0 No Yes |
391474. | 391473 03jan2016 01:14:05 2016 BTCC Pool 2 0 No Yes |
391641. | 391640 04jan2016 04:32:33 2016 BTCC Pool 2 0 No Yes |
391664. | 391663 04jan2016 07:13:51 2016 BTCC Pool 2 0 No Yes |
391926. | 391925 05jan2016 21:15:34 2016 BTCC Pool 2 0 No Yes |
392011. | 392010 06jan2016 13:12:26 2016 BTCC Pool 2 0 No Yes |
392094. | 392093 07jan2016 00:57:43 2016 BTCC Pool 2 0 No Yes |
392264. | 392263 08jan2016 03:20:38 2016 BTCC Pool 2 0 No Yes |
392289. | 392288 08jan2016 07:30:34 2016 BTCC Pool 2 0 No Yes |
392350. | 392349 08jan2016 16:32:58 2016 BTCC Pool 2 0 No Yes |
392504. | 392503 09jan2016 15:55:26 2016 BTCC Pool 2 0 No Yes |
392507. | 392506 09jan2016 16:16:26 2016 BTCC Pool 2 0 No Yes |
392545. | 392544 09jan2016 22:19:02 2016 BTCC Pool 2 0 No Yes |
392586. | 392585 10jan2016 03:51:34 2016 BTCC Pool 2 0 No Yes |
392642. | 392641 10jan2016 12:31:14 2016 BTCC Pool 2 0 No Yes |
392842. | 392841 11jan2016 14:54:13 2016 BTCC Pool 2 0 No Yes |
392872. | 392871 11jan2016 19:41:02 2016 BTCC Pool 2 0 No Yes |
393169. | 393168 13jan2016 13:42:56 2016 BTCC Pool 2 0 No Yes |
393260. | 393259 14jan2016 07:48:15 2016 BTCC Pool 2 0 No Yes |
393360. | 393359 15jan2016 01:47:15 2016 BTCC Pool 2 0 No Yes |
393407. | 393406 15jan2016 10:36:37 2016 BTCC Pool 2 0 No Yes |
393448. | 393447 15jan2016 18:22:46 2016 BTCC Pool 2 0 No Yes |
393460. | 393459 15jan2016 20:46:46 2016 BTCC Pool 2 0 No Yes |
393557. | 393556 16jan2016 13:42:03 2016 BTCC Pool 2 0 No Yes |
393755. | 393754 17jan2016 22:31:29 2016 BTCC Pool 2 0 No Yes |
393764. | 393763 17jan2016 23:37:11 2016 BTCC Pool 2 0 No Yes |
393927. | 393926 18jan2016 22:02:06 2016 BTCC Pool 2 0 No Yes |
394307. | 394306 21jan2016 13:58:52 2016 BTCC Pool 2 0 No Yes |
394619. | 394618 23jan2016 11:24:44 2016 BTCC Pool 2 0 No Yes |
394663. | 394662 23jan2016 16:45:41 2016 BTCC Pool 2 0 No Yes |
395289. | 395288 27jan2016 12:11:24 2016 BTCC Pool 2 0 No Yes |
395573. | 395572 29jan2016 06:51:36 2016 BTCC Pool 2 0 No Yes |
396262. | 396261 02feb2016 07:58:03 2016 BTCC Pool 2 0 No Yes |
397275. | 397274 07feb2016 20:13:36 2016 BTCC Pool 2 0 No Yes |
398078. | 398077 12feb2016 13:42:58 2016 BTCC Pool 2 0 No Yes |
399231. | 399230 19feb2016 23:26:50 2016 BTCC Pool 2 0 No Yes |
399480. | 399479 21feb2016 18:18:38 2016 BTCC Pool 2 0 No Yes |
399946. | 399945 25feb2016 07:45:17 2016 BTCC Pool 2 0 No Yes |
399963. | 399962 25feb2016 11:18:25 2016 BTCC Pool 2 0 No Yes |
399982. | 399981 25feb2016 13:41:14 2016 BTCC Pool 2 0 No Yes |
400064. | 400063 26feb2016 02:48:30 2016 BTCC Pool 2 0 No Yes |
400530. | 400529 29feb2016 09:06:52 2016 BTCC Pool 2 0 No Yes |
401003. | 401002 03mar2016 19:09:49 2016 BTCC Pool 2 0 No Yes |
   |------------------------------------------------------------------------------------------------------------|
600591. | 600590 22oct2019 21:30:44 2019 Poolin 3 0 No Yes |
   |------------------------------------------------------------------------------------------------------------|
617751. | 617750 17feb2020 08:05:11 2020 Poolin 3 0 No Yes |
626508. | 626507 18apr2020 06:39:47 2020 Poolin 3 0 No Yes |
   +------------------------------------------------------------------------------------------------------------+

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

So, there were two non-empty-zero-fee blocks in 2020, can you specify the height of those blocks?
I'm a bit curious on the included transactions (just a bit). TIA

Quote from: Charles-Tim on August 08, 2020, 01:49:25 AM

Zero fee blocks is possible but gone are the days, I do not even think there are wallets that can allow donor not to pay zero fee for their transactions.

AFAIK, some clients will let you set tx fee to '0' or lower than 1sat/B like Electrum... but most nodes wont relay it (including the servers) because of BitcoinCore's later version's default "minimum transaction relay fee".
0 fee isn't invalid and the relay setting is configurable so, it's possible to send a <1 or 0 sat/B TX by sending it directly to a "friend" solo miner or pool.

Charles-Tim

legendary

Activity: 1512

Merit: 4795

Leading Crypto Sports Betting & Casino Platform

Zero fee blocks is possible but gone are the days, I do not even think there are wallets that can allow donor not to pay zero fee for their transactions. I really like this topic as it contain details about zero fee blocks.

With the level of adoption of bitcoin, zero fee blocks is impossible again as it has become the thing of the past because such transactions can no even have 1 confirmation and can not be mined into recepient's address.

tranthidung

legendary

Activity: 2310

Merit: 4085

Farewell o_e_l_e_o

I saw the thread Zero Transaction Fees: What's the catch? which looks interesting but it is for altcoin. Therefore I run a simple analysis for bitcoin transaction and here you go:

Data:

https://loyce.club/blockdata
Observed blocks: Genesis - 642701

Results:

There are 125,228 blocks with fee_per_kb = 0. But not all of them are zero-fee transaction some of them are empty blocks.
In 125,228 blocks, there are only 36,266 (28,96%) blocks that are not empty ones and are true zero-fee blocks.
2010 and 2011 are two years with highest zero-fee blocks: 2010 (20,517 blocks ~ 56.57%, 2011 with 13,600 blocks ~ 37.5%. They all account for 94.55% of all zero-fee blocks so far.
There is no zero-fee blocks in 2017 and 2018.

Plots:

Raw results:

Code:

Blocks | Empty blocks
(zero fee) | No Yes | Total
-----------+----------------------+----------
   No | 517,474 0 | 517,474
   | 100.00 0.00 | 100.00
-----------+----------------------+----------
   Yes | 36,266 88,962 | 125,228
   | 28.96 71.04 | 100.00
-----------+----------------------+----------
   Total | 553,740 88,962 | 642,702
   | 86.16 13.84 | 100.00

Code:

year | Freq. Percent Cum.
------------+-----------------------------------
   2009 | 171 0.47 0.47
   2010 | 20,517 56.57 57.05
   2011 | 13,600 37.50 94.55
   2012 | 1,500 4.14 98.68
   2013 | 146 0.40 99.08
   2014 | 3 0.01 99.09
   2015 | 282 0.78 99.87
   2016 | 44 0.12 99.99
   2019 | 1 0.00 99.99
   2020 | 2 0.01 100.00
------------+-----------------------------------
   Total | 36,266 100.00

Topic: Zero fee for bitcoin transaction [Historical data] (Read 362 times)