issue: Bitcoin core require more than 16GB RAM

Cricktor

hero member

Activity: 714

Merit: 1010

Crypto Swap Exchange

I tested running Bitcoin-Qt on my daily Ubuntu machine with 8GB RAM instead of running Bitcoin Core as bitcoind without GUI. Memory consumption looks almost the same to me after nearly 24h continuous running of Bitcoin-Qt. I've installed atop package and run it as a service. Need to become more familiar with it, yet.

On startup the GUI needed considerable time to "load" my default watch-only wallet (21954 descriptors or so-called Patoshi blocks and some tenthousands of transactions) and show the wallet's overview/balance and be able to display the wallet's transactions. I left Bitcoin-Qt do its thing while browsing with Firefox and didn't measure after what time the GUI was usable.

In fact, from debug.log it takes only ~5.5s to load the wallet, but considerably longer if you ask for the final balance and display a list of all known transactions.

If of interest, here are some parts from debug.log from the Bitcoin-Qt startup:

Code:

2024-09-26T14:25:26Z Bitcoin Core version v27.1.0 (release build)
2024-09-26T14:25:26Z InitParameterInteraction: parameter interaction: -bind set -> setting -listen=1
2024-09-26T14:25:26Z InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
2024-09-26T14:25:26Z InitParameterInteraction: parameter interaction: -proxy set -> setting -natpmp=0
2024-09-26T14:25:26Z InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
2024-09-26T14:25:26Z InitParameterInteraction: parameter interaction: -onlynet excludes IPv4 and IPv6 -> setting -dnsseed=0
2024-09-26T14:25:26Z Qt 5.15.11 (static), plugin=xcb (static)
2024-09-26T14:25:26Z Static plugins:
2024-09-26T14:25:26Z  QXcbIntegrationPlugin, version 331520
2024-09-26T14:25:26Z Style: fusion / QFusionStyle
2024-09-26T14:25:26Z System: Ubuntu 22.04.5 LTS, x86_64-little_endian-lp64
...
2024-09-26T14:25:26Z scheduler thread start
...
2024-09-26T14:25:30Z Cache configuration:
2024-09-26T14:25:30Z * Using 2.0 MiB for block index database
2024-09-26T14:25:30Z * Using 56.0 MiB for transaction index database
2024-09-26T14:25:30Z * Using 49.0 MiB for basic block filter index database
2024-09-26T14:25:30Z * Using 8.0 MiB for chain state database
2024-09-26T14:25:30Z * Using 335.0 MiB for in-memory UTXO set (plus up to 976.6 MiB of unused mempool space)
2024-09-26T14:25:30Z init message: Loading block index…
2024-09-26T14:25:30Z Assuming ancestors of block 000000000000000000026811d149d4d261995ec5b3f64f439a0a10e1a464af9a have valid signatures.
2024-09-26T14:25:30Z Setting nMinimumChainWork=000000000000000000000000000000000000000063c4ebd298db40af57541800
...
2024-09-26T14:25:40Z Verification: No coin database inconsistencies in last 6 blocks (23319 transactions)
2024-09-26T14:25:40Z  block index            9954ms
...
2024-09-26T14:25:40Z init message: Loading wallet…
2024-09-26T14:25:40Z [watch_patoshi] Wallet file version = 10500, last client version = 270100
2024-09-26T14:25:42Z [watch_patoshi] Descriptors: 21954, Descriptor Keys: 0 plaintext, 0 encrypted, 0 total.
2024-09-26T14:25:46Z [watch_patoshi] Wallet completed loading in            5521ms
2024-09-26T14:25:46Z [watch_patoshi] setKeyPool.size() = 0
2024-09-26T14:25:46Z [watch_patoshi] mapWallet.size() = 60415
2024-09-26T14:25:46Z [watch_patoshi] m_address_book.size() = 21954
2024-09-26T14:25:46Z Setting NODE_NETWORK on non-prune mode
2024-09-26T14:25:46Z block tree size = 863006
2024-09-26T14:25:46Z nBestHeight = 862958
2024-09-26T14:25:46Z initload thread start
2024-09-26T14:25:46Z Bound to 127.0.0.1:8333
2024-09-26T14:25:46Z Bound to 127.0.0.1:8334
2024-09-26T14:25:46Z Loading 104767 mempool transactions from disk...
2024-09-26T14:25:46Z txindex thread start
2024-09-26T14:25:46Z txindex is enabled at height 862958
2024-09-26T14:25:46Z txindex thread exit
2024-09-26T14:25:46Z Loaded 2 addresses from "anchors.dat"
2024-09-26T14:25:46Z 2 block-relay-only anchors will be tried for connections.
2024-09-26T14:25:46Z init message: Starting network threads…
2024-09-26T14:25:46Z DNS seeding disabled
2024-09-26T14:25:46Z net thread start
2024-09-26T14:25:46Z opencon thread start
2024-09-26T14:25:46Z addcon thread start
2024-09-26T14:25:46Z msghand thread start
2024-09-26T14:25:46Z basic block filter index thread start
2024-09-26T14:25:46Z basic block filter index is enabled at height 862958
2024-09-26T14:25:46Z basic block filter index thread exit
2024-09-26T14:25:46Z coinstatsindex thread start
2024-09-26T14:25:46Z coinstatsindex is enabled at height 862958
2024-09-26T14:25:46Z coinstatsindex thread exit
2024-09-26T14:25:46Z torcontrol thread start
2024-09-26T14:25:46Z Leaving InitialBlockDownload (latching to false)
2024-09-26T14:25:46Z init message: Done loading
...
2024-09-26T14:43:09Z Imported mempool transactions from disk: 104257 succeeded, 84 failed, 0 expired, 426 already there, 0 waiting for initial broadcast
2024-09-26T14:43:09Z initload thread exit
...

For the GUI to be fully responsive, AFAIR it took longer than initload thread exit at 14:43 UTC. While the GUI was still crunching on "loading/preparing" the wallet's details, I could see in debug.log that block sync and wallet's transactions sync was already in normal progress. This node wasn't far from blockchain tip, anyway.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

You must be running Bitcoin Core on the GNOME desktop. GNOME is known to be very resource-intensive by itself and it wouldn't surprise me if it used 4 GB, about half of your RAM, by itself.

However, Bitcoin Core running with no desktop can sync with 8 GB just fine - I am doing it right now actually (well, with 16 GB, but that is because I am running about 4 other nodes simultaneously. At any rate, RAM usage is about 50%)

Cricktor

hero member

Activity: 714

Merit: 1010

Crypto Swap Exchange

Quote from: LoyceV on September 25, 2024, 12:42:55 AM

You're only 2 or 4 screws away from adding 8 GB more

Haha, yes I know, it's fairly easy and only 4 screws in total if you have to access both RAM slots. I only buy used business laptops that are easily serviceable on purpose. I'm cured from hassles with some consumer laptops I had to open and service...

IIRC, my T520 might have two 4GB SODIMMs, but I will check next time I dust off the internals. I'm not too keen to update this particular device as I have other more recent laptops which are likely more power efficient than this one and are scheduled to replace this one. But as long as it's running fine and doesn't cause me issues, I don't feel much pressure to act.

Quote from: LoyceV on September 25, 2024, 12:42:55 AM

I never liked that 2.2 GB swap when it has enough memory, and used to run without swap, but enabled it again just in case some process suddenly needs a lot of memory (like Firefox on Aliexpress). That gives me more time to respond instead of becoming dead slow until it kills something.

That's true and ABCbits hints to tweak swapiness accordingly. I remember having done this for one of my Raspi 4B nodes, but that was quite some time ago and I forgot about this tuning. It doesn't bother me enough, though. So much to tweak, so little time.

Getting back to topic, I highly doubt that Bitcoin Core 27.1 (or earlier versions) really needs 16GB RAM or more, even when performing IBD. If you have inappropriate option settings, it can demand too much. But then it's mostly your fault to ask for too much memory if e.g. your dbcache value is way too high. Memory pressure also depends on other software running on your system and we didn't read much about this so far from OP.

During IBD Bitcoin Core will of course benefit from more RAM if that RAM is actually available without causing other issues. Set your options wisely and with understanding.

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: Cricktor on September 24, 2024, 03:09:49 PM

--snip--

After an uptime of bitcoind of nearly three days (~220,000 seconds) and with Firefox open (I have quite some tabs open that are retained during sessions), this is the memory consumption output by free -h:

Code:

               total        used        free      shared  buff/cache   available
Mem:           7,6Gi       1,9Gi       159Mi       324Mi       5,5Gi       5,1Gi
Swap:           15Gi       2,2Gi        13Gi

I don't see much memory pressure when about 5+GiB are still allocated as buffer/cache by the kernel and ~5GiB are flagged as available.

I scanned through output of journalctl and couldn't find anything related to systemd-oomd in my logs. I guess the out-of-memory daemon is bored to death on my system. I take this as a good sign...

Thanks for sharing. Although in this case, more advance monitoring tools such as atop could show more data. For example, showing RAM usage over time and top 3 process which use most memory. Here's a snipped example from my VM.

Code:

$ atopsar -G

-------------------------- analysis date: 2024/09/25 --------------------------

04:26:43    pid command  mem% |   pid command  mem% |   pid command  mem%_top3_
04:36:43   1671 x-www-br  12% |  2233 firefox.  10% |  2097 Isolated   5%
04:46:43   1671 x-www-br  23% |  4017 firefox.  10% |  4503 Isolated   5%

$ atopsar -m

-------------------------- analysis date: 2024/09/25 --------------------------

04:26:43  memtotal memfree buffers cached dirty slabmem  swptotal swpfree _mem_
04:36:43     3914M    154M     52M  1512M    0M    117M      974M    974M
04:46:43     3914M    135M     34M  1036M    0M    117M      974M    965M

Quote from: LoyceV on September 25, 2024, 12:42:55 AM

Quote

Code:

               total        used        free      shared  buff/cache   available
Mem:           7,6Gi       1,9Gi       159Mi       324Mi       5,5Gi       5,1Gi
Swap:           15Gi       2,2Gi        13Gi

I don't see much memory pressure when about 5+GiB are still allocated as buffer/cache by the kernel and ~5GiB are flagged as available.

I never liked that 2.2 GB swap when it has enough memory, and used to run without swap, but enabled it again just in case some process suddenly needs a lot of memory (like Firefox on Aliexpress). That gives me more time to respond instead of becoming dead slow until it kills something.

You could change the behavior by modify swappiness value.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: Cricktor on September 24, 2024, 03:09:49 PM

8GB RAM; Lenovo Thinkpad T520

You're only 2 or 4 screws away from adding 8 GB more

Quote

Code:

               total        used        free      shared  buff/cache   available
Mem:           7,6Gi       1,9Gi       159Mi       324Mi       5,5Gi       5,1Gi
Swap:           15Gi       2,2Gi        13Gi

I don't see much memory pressure when about 5+GiB are still allocated as buffer/cache by the kernel and ~5GiB are flagged as available.

I never liked that 2.2 GB swap when it has enough memory, and used to run without swap, but enabled it again just in case some process suddenly needs a lot of memory (like Firefox on Aliexpress). That gives me more time to respond instead of becoming dead slow until it kills something.

Cricktor

hero member

Activity: 714

Merit: 1010

Crypto Swap Exchange

Sunday morning I updated my unpruned Bitcoin Core node that runs 24/7 on my home desk laptop to version 27.1 (Ubuntu 22.04.5 LTS; 8GB RAM; Lenovo Thinkpad T520). Core runs as server daemon in the background and monitors a loaded watch-only wallet with nearly 22k descriptors and connects via Tor to other nodes.

I'm used to have a terminal window open where Core's debug.log is displayed and updates new entries immediately. I have certain keywords of interest highlighted in that view, at minimum I see new blocks and any transactions related to my watch-only wallet.

Other memory related settings in my bitcoin.conf file for this node are (dbcache is default, ie. 450MB):

Code:

rpcworkqueue=128
maxmempool=1024

The Ubuntu installation isn't fat, filesystem on a 1TB SATA SSD is encrypted, I've installed different browsers, GIMP and a few other apps that I regularly need. Usually I have mostly Firefox open when I do my daily browsing or look clips on Youtube or watch a movie stream. I don't open a lot of apps simultanously.

After an uptime of bitcoind of nearly three days (~220,000 seconds) and with Firefox open (I have quite some tabs open that are retained during sessions), this is the memory consumption output by free -h:

Code:

               total        used        free      shared  buff/cache   available
Mem:           7,6Gi       1,9Gi       159Mi       324Mi       5,5Gi       5,1Gi
Swap:           15Gi       2,2Gi        13Gi

I don't see much memory pressure when about 5+GiB are still allocated as buffer/cache by the kernel and ~5GiB are flagged as available.

I scanned through output of journalctl and couldn't find anything related to systemd-oomd in my logs. I guess the out-of-memory daemon is bored to death on my system. I take this as a good sign...

ujiko

newbie

Activity: 20

Merit: 4

i agree.

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: ujiko on September 21, 2024, 08:36:43 AM

Thank you, the IBD is over, now i'm more quiet.
But i have just a kill now

Code:

sept. 21 15:41:41 localhost systemd-oomd[749]: Killed /user.slice/user-1000.slice/[email protected]/app.slice/app-org.gnome.Terminal.slice/vte-spawn-1827bfac-891d-426e-ad67-1746ca3f9b25.scope due to memory pressure for /user.slice/user-1000.slice/[email protected] being 85.02% > 50.00% for > 20s with reclaim activity
sept. 21 15:41:41 localhost systemd[2439]: vte-spawn-1827bfac-891d-426e-ad67-1746ca3f9b25.scope: systemd-oomd killed 24 process(es) in this unit.
sept. 21 15:41:41 localhost systemd[2439]: vte-spawn-1827bfac-891d-426e-ad67-1746ca3f9b25.scope: Consumed 6min 58.109s CPU time.
sept. 21 15:41:42 localhost gnome-shell[2614]: Unhandled promise rejection. To suppress this warning, add an error handler to your promise chain with .catch() or a try-catch block around your await expression. Stack trace of the failed promise:
                                                 _registerItem/<@/usr/share/gnome-shell/extensions/[email protected]/statusNotifierWatcher.js:106:59
                                                 _emit@resource:///org/gnome/gjs/modules/core/_signals.js:114:47
                                                 _nameOwnerChanged@/usr/share/gnome-shell/extensions/[email protected]/appIndicator.js:162:14
                                                 Async*_emit@resource:///org/gnome/gjs/modules/core/_signals.js:114:47
                                                 AppIndicatorsNameWatcher/this._watcherId<@/usr/share/gnome-shell/extensions/[email protected]/util.js:212:22

At this point, you might want to use advance monitoring software to find out how much RAM is used and which application use most RAM on specific date and time. After all, Bitcoin Core shouldn't use that RAM.

JohanM

member

Activity: 144

Merit: 38

You really should not try to download the entire blockchain using a HDD, it will take literally weeks.
Do it on an SSD. After that just copy the entire thing over to your HDD. You'll save weeks.

ujiko

newbie

Activity: 20

Merit: 4

Thank you, the IBD is over, now i'm more quiet.
But i have just a kill now

Code:

sept. 21 15:41:41 localhost systemd-oomd[749]: Killed /user.slice/user-1000.slice/[email protected]/app.slice/app-org.gnome.Terminal.slice/vte-spawn-1827bfac-891d-426e-ad67-1746ca3f9b25.scope due to memory pressure for /user.slice/user-1000.slice/[email protected] being 85.02% > 50.00% for > 20s with reclaim activity
sept. 21 15:41:41 localhost systemd[2439]: vte-spawn-1827bfac-891d-426e-ad67-1746ca3f9b25.scope: systemd-oomd killed 24 process(es) in this unit.
sept. 21 15:41:41 localhost systemd[2439]: vte-spawn-1827bfac-891d-426e-ad67-1746ca3f9b25.scope: Consumed 6min 58.109s CPU time.
sept. 21 15:41:42 localhost gnome-shell[2614]: Unhandled promise rejection. To suppress this warning, add an error handler to your promise chain with .catch() or a try-catch block around your await expression. Stack trace of the failed promise:
                                                 _registerItem/<@/usr/share/gnome-shell/extensions/[email protected]/statusNotifierWatcher.js:106:59
                                                 _emit@resource:///org/gnome/gjs/modules/core/_signals.js:114:47
                                                 _nameOwnerChanged@/usr/share/gnome-shell/extensions/[email protected]/appIndicator.js:162:14
                                                 Async*_emit@resource:///org/gnome/gjs/modules/core/_signals.js:114:47
                                                 AppIndicatorsNameWatcher/this._watcherId<@/usr/share/gnome-shell/extensions/[email protected]/util.js:212:22

BlackHatCoiner

legendary

Activity: 1512

Merit: 7340

Farewell, Leo

Try reducing your database cache size in bitcoin.conf:

Code:

dbcache=2048

Alternatively, you can swap space with the disk, but note that this is going to result in an extremely slow IBD, especially if your disk is HDD. BTW, have you confirmed it's RAM the problem? Check how much RAM is used and left. In linux:

Code:

free -h

ujiko

newbie

Activity: 20

Merit: 4

Quote from: LoyceV on September 20, 2024, 02:23:19 AM

Quote from: ujiko on September 20, 2024, 12:25:10 AM

To download the initial blockchain you must have more than 16GB of RAM, after my personal experience here : https://forum.ubuntu-fr.org/viewtopic.php?pid=22787150#p22787150
bitcoind is killled by the system, and it's not stable, until the download is complete i observe some kills randomly.

I don't read French, but I can tell you from personal experience that Bitcoin Core can do a full IBD on Ubuntu with 8 GB RAM. 16 GB is much better though, and in that case I'd use dbcache=8192 to reduce disk writes.
You're going to have to provide more log details to find why it's consuming so much RAM.

Here is the journalctl of the kill ;

Code:

journalctl --no-pager --since "2024-09-18 21:47" --until "2024-09-18 21:49"
sept. 18 21:47:01 localhost systemd-oomd[763]: Killed /user.slice/user-1000.slice/[email protected]/app.slice/app-org.gnome.Terminal.slice/vte-spawn-2b8e69c4-9a86-45f6-b1a5-90c523681b99.scope due to memory pressure for /user.slice/user-1000.slice/[email protected] being 71.38% > 50.00% for > 20s with reclaim activity
sept. 18 21:47:01 localhost systemd[8336]: vte-spawn-2b8e69c4-9a86-45f6-b1a5-90c523681b99.scope: systemd-oomd killed 25 process(es) in this unit.
sept. 18 21:47:01 localhost systemd[8336]: vte-spawn-2b8e69c4-9a86-45f6-b1a5-90c523681b99.scope: Consumed 28min 49.683s CPU time.
sept. 18 21:47:02 localhost  gnome-shell[8515]: Unhandled promise rejection. To suppress this warning, add an error handler to your promise chain with .catch() or a try-catch block around your await expression. Stack trace of the failed promise:
                                                 _registerItem/<@/usr/share/gnome-shell/extensions/[email protected]/statusNotifierWatcher.js:106:59
                                                 _emit@resource:///org/gnome/gjs/modules/core/_signals.js:114:47
                                                 _nameOwnerChanged@/usr/share/gnome-shell/extensions/[email protected]/appIndicator.js:162:14
                                                 Async*_emit@resource:///org/gnome/gjs/modules/core/_signals.js:114:47
                                                 AppIndicatorsNameWatcher/this._watcherId<@/usr/share/gnome-shell/extensions/[email protected]/util.js:212:22
sept. 18 21:47:09 localhost kernel: [UFW BLOCK] IN=enp42s0 OUT= MAC=01:00:5e:00:00:01:c0:3c:04:d1:ac:40:08:00 SRC=192.168.1.1 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0x80 TTL=1 ID=2094 DF PROTO=2 
sept. 18 21:47:11 localhost systemd[8336]: Started VTE child process 13249 launched by gnome-terminal-server process 11158.
sept. 18 21:47:40 localhost firefox_firefox.desktop[11272]: [Child 11272, Main Thread] WARNING: Failed to create /home/gilles/snap/firefox/4955/.config/glib-2.0/settings: Permission denied: 'glib warning', file /build/firefox/parts/firefox/build/toolkit/xre/nsSigHandlers.cpp:187
sept. 18 21:47:40 localhost firefox_firefox.desktop[11272]: (/snap/firefox/4955/usr/lib/firefox/firefox:11272): GLib-GIO-WARNING **: 21:47:40.958: Failed to create /home/gilles/snap/firefox/4955/.config/glib-2.0/settings: Permission denied
sept. 18 21:47:51 localhost kernel: [UFW BLOCK] IN=enp42s0 OUT= MAC=01:00:5e:00:00:01:c0:3c:04:d1:ac:40:08:00 SRC=192.168.1.1 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0x80 TTL=1 ID=3378 DF PROTO=2 
sept. 18 21:48:32 localhost kernel: [UFW BLOCK] IN=enp42s0 OUT= MAC=01:00:5e:00:00:01:c0:3c:04:d1:ac:40:08:00 SRC=192.168.1.1 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0x80 TTL=1 ID=32618 DF PROTO=2

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

As stated by other member on your previous thread, you could adjust dbcache value based on your total RAM and how much RAM used by OS and other application you run. For example, you have 16GB RAM where 6GB used by OS and other application you run. In this case, i could recommend dbcache=7168 leaving 3GB as reserve.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ujiko on September 20, 2024, 12:25:10 AM

To download the initial blockchain you must have more than 16GB of RAM, after my personal experience here : https://forum.ubuntu-fr.org/viewtopic.php?pid=22787150#p22787150
bitcoind is killled by the system, and it's not stable, until the download is complete i observe some kills randomly.

I don't read French, but I can tell you from personal experience that Bitcoin Core can do a full IBD on Ubuntu with 8 GB RAM. 16 GB is much better though, and in that case I'd use dbcache=8192 to reduce disk writes.
You're going to have to provide more log details to find why it's consuming so much RAM.

hd49728

legendary

Activity: 2044

Merit: 1018

Not your keys, not your coins!

Quote from: ujiko on September 20, 2024, 12:25:10 AM

To download the initial blockchain you must have more than 16GB of RAM

bitcoind is killled by the system

If you don't have good hardware resources to run a Bitcoin node, pruned node or full node and complete IBD, you can choose easier options with SPV wallets like Electrum wallet.

SPV wallets don't need to download a full blockchain like Bitcoin Core and these wallets are lighter to use.

Simplied Payment Verification Wallets (SPV wallets)

Don't try to use snapshot of Bitcoin blockchain, because it can be malicious file that is used by hackers to steal your bitcoin. It's why if you run a Bitcoin node, it's strongly advised to do IBD by yourself.

ujiko

newbie

Activity: 20

Merit: 4

To download the initial blockchain you must have more than 16GB of RAM, after my personal experience here : https://forum.ubuntu-fr.org/viewtopic.php?pid=22787150#p22787150
bitcoind is killled by the system, and it's not stable, until the download is complete i observe some kills randomly.

Topic: issue: Bitcoin core require more than 16GB RAM (Read 221 times)