Topic: Set up Bitcoin node for data analysis (Read 264 times)

I use mostly the passive cooling Geekworm P173 aluminum case with Gelid GP-UL Ultimate thermal pads (those particular thermal pads are not necessarily needed but I had them anyway, it's fine to use the case's provided ones, too).

Passively cooled with this P173 case my Raspi 4Bs rarely exceed 50-55°C under load.

To be safe in summer heat and generally have a lower temp even under high load I upgraded this case with some forced airflow by a Noctua NF-A5x25 5V fan which I run on 3.3V from appropriate GPIO pins. At 3.3V this fan is barely perceptible even on close distance and the Raspi doesn't go over ~45°C under stress test with full load on all cores.

Average temps of my Raspi nodes with this setup is somewhere around 33°C and maybe 36-38°C in summer heat.

On a fairly compact surface area and my aluminium heat sink was apparently not enough, the earlier versions were at 7-8Ws and were known to overheat. Might be better to get a bigger heatsink and a better thermal pad for it.

The thing burns 6.4W max. That shouldn't be too difficult to dissipate with passive cooling.

To be honest I have created a node as a series of tutorials for Bitcointalk here.

This node runs Raspbian Lite and to be honest the temperature went close to 65-67 degrees during the IBD.

The other node that I have runs Start9 and the IBD went very smoothly and much faster than the other one's.

I used to have Umbrel too and I don't remember any issues too. I just switched to Start9 because I loved its simplicity and the fact that it is much easier  to customize.

Raspbian. Mine was above 70 for most of the time, but again your cooling method makes a huge difference. Having the entire casing as a heatsink is undoubtedly better with what I've had on hand. RPi4s were notorious for overheating actually, might've been the issue with the earlier versions.

RAM is important, but RPis are pretty expensive so I had to skimp on it back then. RAM shouldn't affect temperature anyways.

What OS did you run? Mine is downloading the monero blockchain now and the temp is currently 43.8 celcius. I don't remember exactly with Bitcoin but I can tell you for sure that it never went over 58-60 celcius.

But RAM is very important for this. So 8GB work much better than 4GB.

I think I ran it with my RPi 4, 4GB. How were your temps looking during the synchronization? Mine was fairly hot throughout, granted that it was just a heatsink and not a fancy heat dispersing case.

Not to mention, it took weeks.

Out of curiousity, what where you running on the Raspberry and how much RAM did it have?

I run 2 Raspberry Pi 4b 8GB RAM and I never had overheating issues.

I use this case on both of them and the temperature never exceeds 58-60 celcius. It has passive cooling and there is no noise at all.

I second what is said above: Don't use Raspberry Pi. My Pi 4 was overheating constantly when running a node and I had to get active cooling on it.

It is okay to remove your data directory, provided that your Core has shutdown properly and is not writing any data on it, or it'll get corrupted and you'll have to do a reindex again. You might just want to run everything on a single computer instead; indexing the transaction data and parsing them to another format can take quite a while and external harddrive are slow, and are prone to corruption.

Probably yes. You can configure how much RAM used by your SQL database. And after you download all blockchain, you can setup Bitcoin Core to use less RAM as well without much problem. Just don't try to load all data to RAM at once.


Yes, assuming you use Bitcoin Core to download blockchain data.


It's the latter.


I expect Raspberry Pi will struggle for data analysis.

In my view you should try both options and select the most optimal one which depends on that how resource-avid  is your your data analysis.

If you current laptop   has a juice to do both jobs simultaneously, i.e. to support the copy of  blockchain on your drive in sync state and at same time perform data analysis then there is  no need in the second  laptop.

In any case don't forget   to have txindex=1 in your conf file so that you could analyse all transactions rather than your own ones.

Before doing all this for data analysis, have a look at these topics:

Bitcoin nodes can be shut down and then they will continue exactly from where they stopped. But of course they need to be terminated gracefully!

Why do you need to transfer the SSD from one computer to the other? You want one computer for the node and one computer for the "analysis"?

Thank you again very much for the information!

Right now, I am thinking about running two nodes on two devices:

1. Windows laptop (as of now, easier for me)
Is a setup including a 16GB RAM and Intel Core i5 (12th or 13th generation) sufficient to run a node and do some data analysis (Python, SQL) simultaneously? I know it is not easy to answer such a question, but I would appreciate your opinion.

Can someone please tell me if the following idea makes sense:
I will set up a node that I will only use for actions related to interacting with Bitcoin. Ocassionally, I will download some data from the blockchain and store the data on another external harddrive. Then I will use this harddrive to transfer the data to my current laptop (which has a good performance) and do the data analysis on my current laptop.
Or can I simply disconnect my SDD (where the blockchain data is stored) from my node for a few hours and download the blockchain data directly to my other laptop? Will it cause problems if my node is offline for several hours or does it just need some time to synch once it is online again?

2. Rasperry Pi
I am eager to run one node on the Rasperry Pi for the previously mentioned reasons and to get more familiar with Linux. Accumulating knowledge and skills will take time but I am willing to invest the time.

Since you only mention Python, Linux probably is better option since Python comes with many Linux distro and you could allocate more RAM and CPU for your Python script since Linux use less RAM/CPU.

I have multiple headless nodes on a few Raspberry Pi 4B with 8GB RAM (RaspiBlitz, RaspiBolt, Umbrel). All work fine and are nicely power efficient, especially when you run them 24/7/365. While such nodes like RaspiBlitz or especially Umbrel abstract the underlying Linux from the user, it's still beneficial to have or learn some Linux knowledge. You can do it all on Windows, too, but frankly I would rather choose a Linux base.

Another suggestion is to get a used business laptop with 16GB RAM (or option to upgrade to it) which are usually not very expensive. On my ThinkPad T540 I run Ubuntu and another Bitcoin node (no, I'm no Bitcoin node addict, I think). While idle (Bitcoin Core runs still in the background) and no active display this laptop consumes on average somewhere in the ballpark of 11-12W (about double of a Raspi Pi 4B).

Hi guys,

first of all thank you all for your feedback, including details and links with further information!

To answer your questions, I will break down my answer into several points:

1.) Data Analysis

-I want to run analyses to "explore" and "visualize" the Bitcoin ecosystem (some data providers do not provide the raw data that I intend to use).

-There are obviously data providers that I could use but I want to make sure that
1. I always have access to the data
2. I familiarize myself with interacting with Bitcoin on a "data-level"
3. I have access to data I need for my analysis and on a high level of granularity

-Initially, I only intend to download data, store it in tables, and use Python to run further analysis. This might change as time goes by.

2.) I will run the node and use the data for personal reasons (being part of the Bitcoin network, research interest)

3.) I was actually considering to use a Raspberry Pi as it is very compact in size and easy to store somewhere in a room. However, I am unsure about how well it performs when I want to do some data analysis. That is why I want to use a device with more performance so that I can host the node and conduct further activities on the device simultaneously. I will try to get a decent CPU and sufficient RAM.

As for now, I will use a 4TB SSD for storage. Regarding the computer, I have one more question: which OS is the "best" to use? I am a Microsoft user and I am aware of its vulnerability compared to other OS. Is it still a valid option to use, even if I also want to use my node to host my own wallet, or are the risks to high if I use Microsoft?

Honestly it comes down to which data analysis software you use and additional software you run (for example, self-hosted block explorer).


Current blockchain is size is about 533GB[1] and maximum possible growth per month is 17280MB (4MB * 144 blocks/day * 30 day). So 4TB should be sufficient.


I'd say CPU assuming data analysis software you use support using multiple core/thread.

[1] https://blockchair.com/bitcoin

May I ask what exactly do you want to do? For personal reasons?


Prefferably aim for:
8GB RAM
2TB SSD

Anything higher than that is good but not necessary / mandatory.

In that case, you might want to consider to "max-out" the motherboard of your choice. (for futureproofing purposes)

Ordinals haven't increased the pace since the maximum block weight is still at 4WU (1vMB) even after TapRoot upgrade.
Maybe a bit because of the more consistent full blocks that's caused by the large transaction volume that we're experiencing since then.
Refer to this chart: https://ycharts.com/indicators/bitcoin_blockchain_size
At that pace, your 4TB SSD should be sufficient enough.