Topic: Are High Disk Writes Expected? - page 2. (Read 438 times)

I'm getting the impression that adding more RAM mostly reduces disk writes before the node is fully synced, not after. But I have extra RAM and I can bump it up to 16GB so I can give it a shot. The Celeron supports up to 64 GB of RAM according to Intel's website. I could also put in 32 GB later (I can swap it out from another old computer), but I'll start with 16 for now.

From what I can tell, I should set this line in bitcoin.conf (12288 = 12 * 1024):


Is that the only thing I need to change? Or are there other settings I should try too?

Adding more RAM should largely reduce your disk writes, but that may not be possible considering it's a Celeron.

That makes sense: with enough RAM, chainstate can stay in memory until you shut down Bitcoin Core. So all that's written is the full blockchain, which must have been 561 GB back then.

At a glance, i find it's reasonable since you only have 8GB of RAM when current UTXO size is about 11.6GB according to https://statoshi.info/d/000000009/unspent-transaction-output-set and you need to share with both OS and ElectrumX. But for starter, you probably want to run iotop --accumulated and wait for few days to know which process perform most write.


In addition, according to Jameson Lopp benchmark, the write is much lower if you have much bigger RAM and configure Bitcoin Core to use lots of RAM. Although his benchmark doesn't enable txindex either.

The system has 8GB of RAM on an Intel 8th gen Celeron CPU (2 cores, 2 threads). The node isn't pruned. Xubuntu's built-in task manager is showing Memory at 25% (1.9 GiB / 7.6 GiB) and Swap at 34% (1.4 GiB / 4 GiB) at this exact moment, having had both Bitcoin Core and ElectrumX running 24/7 for about 3 or 4 days. I guess I don't really understand why the Swap is being used significantly given how much memory appears to be available, but maybe this is normal. If not, let me know.

I don't have anything in the Bitcoin.conf file specifically changing the database cache. However, from what I can see in the GUI using bitcoin-qt, under Settings -> Options -> Size of database cache, I see it is set to 450 MiB.

Also, I checked smartmontools again and confirmed the sector size is 512 bytes, so my math should be correct.

Can you share how much RAM you have, how much swap is in use, and Bitcoin Core's dbcache setting?

The 500 GB version can handle 300 TBW. Even at 150 GB per day, you're looking at 5 years.

I have been running Bitcoin Core 27.0 with txindex=1 and ElectrumX 1.16.0 on a Samsung 860 Evo SSD on Xubuntu. This is a dedicated machine doing nothing but running those two programs all day. I've also been using smartmontools to monitor the total disk writes once a day over time, and I've noticed the numbers are shockingly high. Obviously this is a concern since SSDs can only perform so many writes before they die.

Unfortunately, I only have a few data points right now, but all of these were recorded AFTER both the initial block download and after the transaction index was finished being built:

30 total gigabytes written to the SSD from June 24th to June 25th
108 total gigabytes written to the SSD from June 25th to June 26th
158 total gigabytes written to the SSD from June 26th to June 27th

To be clear, this is the difference in total GiB (512 bytes * Total LBAs Written /1024**3) written to the disk from day to day, not actual capacity being used. The amount of disk space being used is about what I would expect (increasing a few hundred megabytes per day).

I'm getting the LBAs written from smartmontools using:
sudo smartctl -a /dev/sda
and pulling the Total_LBAs_Written field, in case anybody else wants to try to monitor this for themselves and compare.

Obviously there's a lot more testing I need to do before I can determine what's causing this for sure but of course my intuition is that it's either related to Bitcoin Core or ElectrumX since the machine is dedicated to running them.

It is also possible I have the LBA size wrong for this drive and I'm inflating the gigabytes written without realizing it, but 512 bytes is standard from what I can tell.

Do these disk write numbers seem reasonable to you, or are these way too high?