Topic: Cheap Node Self Hosting: Just because you CAN does not mean you SHOULD (Read 817 times)

I didn't break the cards, but the data was lost. Symlink is, however, a cool idea I agree with. (Also having Bitcoin core on a smartphone sounds overly cool!).


From what I know it depends greatly on the usage. It's not the same if it sits in the pocket doing almost nothing or you play a game. It's not the same even if you have a lot of widgets refreshing all the time from the internet or not having them.
And I expect Bitcoin core will use some of that CPU...


You really make it hard for me to decide 


Of course, but I already have monitor for that.


Indeed, lately when I look for HDDs I also check for this (of course, in case of SH I don't get much detail, however, read below).

---

Another direction which you may like more: I may be able to get my hands for free onto an ASUS Pro57V (CPU = Intel T5800, 4GB RAM) and then I'll just buy a proper external HDD and insert it into this baby (which now has an old school 500GB WD Black in it). What you say? [PS. I will need tips on how to turn off and on the screen of that ASUS laptop]

There are devices that do this, but usually the surface temperature is a pretty good estimate. If it's not warm, it's not worth worrying about.

Ideally one with DVD drive (to replace for a HDD).

I only recently found out that many well known harddrive brands use something called SMR (Shingled Magnetic Recording):
My 2 TB HDD has this problem, which means that writing a lot of data leads to terrible performance. I've seen it down to writing just kilobytes per second:
Since I now know about it, I'd never buy an SMR disk again.

I don't know that good the prices. I have some sort of feeling that new hardware is cheaper in US and SH is a little bit cheaper here.
With IBD already done I don't find SSD an useful expense, hence I tried to prefer bigger HDD.
I've seen here too i3 and i5 refurb or SH, but they are more expensive and I expect them also be more power hungry. The configuration I've put is almost 110$ and it already has 1 TB HDD on it.


Yes, and it's difficult and expensive to get hands on one nowadays. The only one official reseller in my country is out of stock for maybe a year.


While I expect everybody calculates only the power drew by the Pi, and indeed, there's more components to actually buy (plus if I get a Pi, I get 1TB SSD to keep the system fanless/noiseless).
And yes, I expect to have more to work at it. Still, since I already have a (much smaller) Pi, I started to really like those little buggers 
And over years the difference in power consumption may (wishful thinking) pay the difference in acquisition price.


I've been eyeing Fulcrum before Electrs. I've been thinking to run it under Windows. But I gave it up. I didn't find anybody really using it, and it also seems to be primarily developed for BCH not BTC. And since those guys tend to say "Bitcoin" when they actually refer to their altcoin, it can get confusing and problematic, so I decided not to risk even more headaches.


Nice find! Still, I don't think that I'll go back to that path. My Pi Zero W has became print server in my home and if I ever get to the point I no longer need that.. it may become the hardware for a SeedSigner.
It is too weak for what Bitcoin Core needs.

A few thoughts.
1st and I don't know how it is where you are but here (NY) there tends to be a 'base' price of working PCs between $70 to $125 which puts them at about what you are paying for about those specs. However, there is a jump to $175 to $200 where they get a lot better.

Smaller then needed SSD but more ram and a faster processor and you can get a 1TB spinning drive for under $50 for the blockchain:
https://www.microcenter.com/product/622158/hp-elitedesk-800-g1-usff-desktop-computer-(refurbished)

How do prices look for stuff like that where you are?

As for the RPi, the 4 is the only one worth running at this point.
https://magpi.raspberrypi.com/articles/raspberry-pi-4-specs-benchmarks

Yes it would probably be fun to tinker with the smaller / slower / less power hungry ones a bit but it's going to take a ton of work to get it to work and then it will not work well.

But still keep in mind, although there is a higher cost (RPI4 + case + SATA adapter + m.2 SATA drive + SD card + wall power adapter) the power pull from the nodes in a box that I like to build is usually a rounding error in most peoples electric use:



https://bitcointalksearch.org/topic/nodes-in-a-box-5364113

-Dave

I wish it would be easy to measure. Now the electricity price is subsidized, until next spring at least. There are so many taxes and then reductions I cannot properly measure.
By not-so-strict calculations 4 months ago I paid around 15 cents/kWh, 2 months ago I paid some 7 cents/kWh. This month... I will find out in some 10 days.
Since I expect the 7 cents may be an error, I'll go to a middle 11 cents for my calculation, which (based on yours) means 2.42 EUR/month and 88 EUR for 4 years.
While you did scare me, since I feel like CISC processors tend to be more power hungry than RISC ones (eg ARM), hence a RPi4 would have been better for the job, this setup doesn't look like a bad business after all...
I will do the math again after the next invoice arrives.


microSD may not be best suited for such load. From my experience it tends to heat up and start failing (but maybe the adaptors I've been using were not top quality either).
And heavy duty microSD are overly expensive. Plus the 512 is just barely enough. I also guess that a more powerful smartphone may be more power hungry too, but I may be awfully wrong.
Interesting research though.
[Also, fyi: that SH system I consider buying, with that Intel G proc, 4GB RAM and 1TB HDD is just a tad over 100 EUR].


You should have no SIM at all in it. Then no risk to have data
And I would not take into account systems that have own screen. There's usually no reason to buy screen for a machine supposed to run in unattended mode. Remote desktop - in a way or another - should do.
But this is also because I was talking about buying hardware.

---

Fun fact: before this thread I was considering to play with Bitcoin core on a RPi zero W (32bit), just to see how bad it runs. Afterwards I decided I should not waste my time.

Measure it
Here, prices easily go up to 50 cent (euro/dollar, it's the same nowadays) per kWh. That means 30W costs 11 euro/dollar per month. If not for the disk space requirements, a VPS could be cheaper (but also more risky for running a node).
Back to the 11 dollar per month: that's 400 dollar in 3 years, and could very well justify buying a much more energy efficient second hand laptop to use as a node.

This got me thinking: how cool would it be to run a node on a phone? Replace Android by Linux, add a 512 GB microSD (around 50 bucks), and login remotely.
Or even without replacing Android: stackexchange mentions installing Bitcoin Core in Termux.
Phones take at most a few watt power, and modern ones have more RAM than low-end laptops. You may want to turn off data

Thank you both for the input. My conclusion is that since there's no IBD to take care of, the setup is not that bad, although it could use some more RAM.
The only really worrying part may the consumption.

I would add 4 GB (8 GB RAM total), and go for it! With SSD, it should be able to synchronize 450 GB blockchain within a day.

I wouldn't like the 35W TDP. In summer because of the heat, the rest of the year because electricity prices are outrageous here nowadays.

The laptop I'm using now has a slower CPU (but more RAM). I should really upgrade, but don't like the hassle.

IBD is - by far - not an issue, I've already done that with a much more powerful machine, the blockchain is already on the designated HDD.
This 6-7 years old system would just sit running 24/7, hopefully without creating too much useless heat.

Only Electrs (or ElectrumX?) will have to rebuild its indexes (Bitcoin Core will have his own, built on the main PC).
And I hope I'll do this without pre-made (and not as much configurable) things like Umbrel and so on.

Hate is a strong word. Only 2 cores is very little by today's standards as is 4GB of ram.
Remember it's a 7 year old CPU. A little more RAM for the IBD and an SSD will make it livable.

But, and this is an important but. You are still doing this on what I assume is a 6 or 7 year old system. What is the condition of the rest of the hardware.
Bitcoin sync does push some HW to it's limits, which by itself is not a bad thing. But it will bring what would be smaller issues to the front to become larger issues.

-Dave

Since I am not good in hardware stuff and since I somewhat eyeing a configuration to move my bitcoin node away from my main computer... how bad is this CPU: Intel Pentium G4400T 2.90GHz?
Would I hate myself if I start using a mini PC with that CPU and 4GB RAM?

I was once running the node with SSD(Kingston A400 SATA SSD ) + HDD(My Passport from WD) and it works quite fine. Though if I plug the HDD when the Pi 4 is already on, it will always restart the Pi, so it needed to be plugged in before turning it on. I'm sure it drains all the power as it can get since even if I either connect a USB keyboard or a display, the OS just won't booted up and the Pi keep restarting.

Interestingly enough, the blockchain never got corrupted when it regularly got power outages. But due to the outages keep coming, it only lasts roughly 10 months, and the HDD started to make some noise before it fails.

As of now, I run the node with 2 SSD, and sometimes with a USB keyboard and connected it to a display, it never got any issues.

That's good to know! With my 3B+ (last Pi I tried running a node on) and various HDDs, it wouldn't work. It surely also depends on the model of HDD used.

USB flash drive will be the same as the microSD - not suited as an OS drive - a second SSD would be an option; though then we might get into power issues again, or figure out booting from the one SSD that holds the blockchain. I believe the Pi 4 is the first model that can do that.

---

I'd point out another issue with SBC nodes: as described in my Futurebit Apollo BTC guide, compilation times on these single board computers are extremely long. Building electrs took three times as long as it did on my $50 node and probably 10 times as long as on a modern desktop PC. Therefore people probably more often resort to prebuilt binaries (like I did in that guide) which I'm not really a fan of. If we already have the source and aim to minimize trust, we should compile everything.

Cross-compilation is a thing, but even trickier to set up and get right, especially for newcomers. That's why I think sticking to old x86 hardware is the easiest and fastest way to get up and running cheaply and reliably. A while back my laptop 'home node' powered through a night-long power outage due to its internal battery. To be fair, a Raspberry Pi in the same network powered back up without issues, but in the past I've had power cuts negatively affect SBCs (i.e. corrupted microSD drive).

It's actually the Raspberry Pi's fault.. I'm not sure if it's the case on the latest models, but pretty confident model 3B only supplied max. of 0.9A, which is not enough to power an external 2.5" HDD. Even though these drives are basically made for expanding a laptop's storage, so they're made with portability in mind and reduced power draw compared to a 3.5" HDD, a Pi can't always reliably power it.

Edit: It appears the Model 4 can pull 1.2A. So that may work. But most people who make these Raspberry Pi node guides use SSDs, which draw much less power and remark like 'you can replace it with an HDD to save some money', while not having tried it and not realizing this problem. I noticed this problem way back in Raspberry Pi model 2 era.

Combined with a microSD card as an OS drive (more prone to memory corruptions & limited r/w cycles compared to an SSD or onboard flash), it doesn't make for the most reliable system - you know, something you'd like to run a Bitcoin node on..

Many new laptops are sold with 256 or 512 GB storage. Since SSDs took over HDDs, this hasn't gone up much. That would mean a brand new laptop can barely install Bitcoin Core without pruning, and I don't think that would be a good thing.

Most people tend to learn from their own mistakes. If somebody tells them, they think they know better.
This being said, people usually find out that pruned nodes don't worth it when they want to try out something and change the wallet.

That is part of the issue. As more RPi4 come back into stock and other embedded systems get retired I can see a lot of older slower RPi hitting the market.
Either RPi3 or the 1GB RPi4 that people bought during the shortage.
Even now you are seeing posts about people building a pruned node to fit it into a smaller drive because they don't want to spend the money for a used 1TB drive.

So telling people not to try is worth a mention.
Personally I would like to see them get rid of the pruned node option entirely. It's just not worth it.

-Dave

I don't visit this subforum a lot to be honest, so I didn't know this was a common question. Honestly, as you pointed out, with used parts and around 50 bucks of any currency you can get a decent node.

I honestly don't see it happening for anything under 50 as long as you don't have a lot of parts already.

Generally speaking though, used laptops and other used hardware will be better value than single board computers. I'm playing with the Futurebit Apollo's built-in Orange Pi 4 at the moment, with 6 ARM cores and whatnot and it feels slower than my '$50 node'.

It's worth noting I bought a 1TB SSD and 8GB of RAM which made it an excellent node, with an insane uptime due to its built-in battery.

---

I had tried with a Raspberry Pi 3B in the past (when it was recent). Even though it was many years ago with smaller IBD size, with a USB HDD, it was running for weeks. At one point there was a power issue and the HDD was corrupted. Reindexing would have taken just as long again, so I had to hook up the HDD to an actual PC for reindexing and finishing IBD and then plug it back into the Raspberry Pi.
So essentially I never finished initial block download on the Pi itself. I don't think this USB power issue is brought up often enough; since it's pretty common, yet there are tons of guides, 3D printed cases and whatnot for Raspberry nodes to be used with USB HDDs.

In the Raspberry Pi 'community', it seems a long and well-known fact that they struggle with USB power.

I would put a budget of around maybe $500 USD equivalent to run a full node full time on enterprise level rack server machines. You can get refurbished to save money, it does not have to be brand new. That amount of money buys you a Xeon dual processor running faster than 2 Ghz, with at least 16 GB of RAM and another $100 for whatever hard drive space or pair of SSDs you need for the near future. 1 TB should last awhile.

The point, in general does remain the same.
You can cram a lot of stuff on under-powered slower devices with I/O and other things that can't keep up.
However, in the end you are not doing yourself any favors.

And as the blockchain gets bigger and the other apps that use it need more and more storage and everything else, you will hit sooner or later hit a wall.
Sometimes the wall is soft and padded and all you have to do is dd your older slower smaller drive to another one. Other times the entire thing gets corrupted and you have to start from scratch.

-Dave

To be fair, that instruction meant to run blockexplorer for 6 (six) different cryptocurrency network. Mempool also mention this on their installation methods.

This post reminded me of the message in the README from the mempool.space production setup:


 

As far as I understand the Raspi 4 is the first Raspi that can be easily used to set up a node. So in 2-3 years we can expect to have a very strong increase in the number of nodes when all the current Raspi 4 are sold as second hand for 20 Dollars. So for me I will just wait for that and also don't put 200-300 USD in a decent raspi node setup.

I usually try to avoid investing into hardware, since its price keeps dropping sharply just after buying. I've bought my laptop in 2017 and I've already been using your strategy: I've bought the laptop with SSD and then added a HDD to it.
My data stays on a NAS that seems to be aging pretty well. I've upgraded there not long ago an 500 GB HDD to 4TB (I feared that bigger HDD won't work there) and now that 500 GB HDD I use for the blockchain, plugged into a thingy that I can connect to on USB3. Clearly suboptimal solution, but I just didn't have the hearth yet to throw away that HDD

But you're right. My laptop's HDD is indeed silent. All I can hear (when I take my headphones off, that is) is the cooler the laptop stands on.

---
I usually don't have the hearth to throw away old stuff, but this topic may be just the thing to convince me do it! I have yet to find a device I can leave running 24/7 without fearing it'll set the room on fire, but I find today's 150-200 EUR too much for a decent RasPi (board only).

I've had old server disks that make a loud KRRR KRRR sound, but I can barely hear my modern 2.5" disks.

In my experience, anything with 8 GB RAM or more will work just fine to run a full Bitcoin Core on Linux. If the hardware supports 8 GB, I assume it's modern enough to have sufficient processing power. I've used Bitcoin Core with 2 GB and HDD on an Atom netbook years ago (back then I liked to keep a backup blockchain ready), but I abandoned it because it took painfully long to update.
I'm a sucker for laptops nowadays, but unfortunately most of them can handle only one HDD. My preferred setup is adding a large HDD for data that doesn't need to be on the (smaller) SSD.

There are bunch of Rpi clones for years, some of them are RockPi, Odroid, Pine64, Banana Pi, Orange Pi and some others could be considered as alternative, for example ASUS Tinker Board.
I am sure there are other single board computers that are inspired by raspberry pi.

I'd guess that one in the list could be Orange Pi which I heard about not long ago and I don't know it it's indeed good, still, you can read some comparison here: https://versus.com/en/orange-pi-rk3399-vs-raspberry-pi-4-model-b
There may be others too, let's see what @dkbit98 meant.



Yeah, probably. I guess that I've just proven the point that old hardware is probably not worth it

Watt Raspi clones do you mean?

Some disks are creating more noise than others, but if the HDD noise is the problem, I found easy solution for that by purchasing silencing case enclosure that almost totally removed noise from my HDD.
It is made from solid metal from inside (I think copper) with combination of foam and other materials that makes drive cool and quiet.
I found this very cheap, but I guess you can easily make something similar if you like DIY projects.



I can confirm that Rpi400 has a huge heatsink that made massive cooling improvements, so you don't need to install any fans and you will have 100% silent computer.
Version with 4GB and 8GB is not slow at all, but there is option of using Rpi computing module, or some Rpi clones that are even stronger.

 

Also update the firmware of the Pi 4, there are few update which reduce the heat without sacrifice the performance.


WD Black isn't exactly average customer HDD, it's designed for best performance at high price. Besides, i find newer HDD has less noise than older ones.

Think weeks vs. days.

Not quite the same but, a RPi4 with a SSD can sync Umbrel in 3 or 4 days. That is over TOR and indexing the blockchain and the other stuff to run a lightning node.

A RPi3, not a prepacked node setup, just the bare RPi software and bitcoind with a 7200 RPM is still churning through just the IBD  after 2 1/2 weeks.
Both are behind the same router on the same cable modem so it's not bandwidth just purely a performance issue.

-Dave

I thought that this is obvious enough to not state it


Then maybe I'm wrong. The HDD I use for blockchain is more than 12 years old, it's a WD Black and it's pretty noisy.


The "painfully slow" part is indeed useful (I'd guess though that it happens in the initial sync, or when it's a couple days behind), but I was more curious if anybody had the (expectedly bad) experience with lower spec Pi (3,2,1, or even Zero).

I think given the price to performance as well as the longevity aspect, the energy efficiency of HDD vs SSD would be negligible or simply a dollar or two extra per year, maximum. The price of a HDD is ridiculously low compared to SSD of similar size. The noise produced by HDD is actually quite overstated most of the time and it is barely audible.

Regarding the cooling required for RPi4, I ran into some throttling with RPi4 during synchronization and slapped a heatsink, which more or less solved the issue. Active cooling wasn't really necessary in my case. Never really an issue with the previous revisions though (7w vs 5w). It is still painfully slow regardless.

It's a very informative post, thank you very much for it. I did intend to "cut corners" and see whether it would work (and also how badly!) with the intention to just use the (few) hardware I have at hand.
Luckily I didn't have the time for the setup, so, again, this helps me a lot.

I will also vouch for the 500GB HDD "approach" problem. Right now my setup works with an external 500GB HDD for the blockchain and I already got rid of txindex in order to make sure it doesn't suddenly stop because of the lack of space (and that hurts badly my local block explorer, obviously).

Of course, one of my problems was that I've thought late to make myself a local setup and RasPi are overly expensive for now, so I'll keep the current setup (that unfortunately doesn't run 24/7), but I still have to look for a HDD soon.


That's correct. Even more, if you have another high spec computer (with SSD), you can use that one for the initial sync and just use HDD (some temporary symlinks to SSD and copying of files will be needed though).
While a normal spec is great, I think that SSD can be seen a bit "overzealous" by those very careful with the costs.
But: SSD is silent, HDD is not. SSD is probably much more energy efficient. So one has to make his choice properly for the long run.


PS: is there any info available about how much heating a RasPi do if used as a node? I fear that anything under RasPi 4 may need active cooling too (again, noise...)

You can buy low space SSD for OS which is cheaper than buying 1TB SSD and you can split them with HDD for blockchains just to improve the syncing process.
We already have a guide here on how to split them if you want to speed up the syncing process you can check this guide "[Guide] Split Bitcoin blockchain between SSD & HDD and speed up the initial sync"

Raspberry Pi4 is much better option because they have much bigger option to choose more RAM, that is if yo can find them available in stock 

Unless you are running pruned node, that can run on even smaller hard disk without any issues.
I don't think larger hard drives are problem for anyone nowadays, old style HDD are very cheap now and you can find them anywhere.

If you look at ebay or craiglist you can find very cheap good laptops with more than enough RAM, processor and hard disk.
I would say that for around $100 you can get great used laptop with 3rd or 4th generation i5 or i7 intel with SSD and 8gb RAM, but better look models that can be upgraded in future, with socketed parts, not soldered.

I don't think they are so much expensive anymore, you can quickly find very good new 1TB for around $80 to $90, and even cheaper on some deals.
There are a bunch of used 100% health 2.5 inch SSD drives being sold online, so you can find good Samsung 860 evo for around $60 to $70.
However, you should always look for TBW and higher number means that disk will last much longer, for 860 evo that would be 600TB.

I believe the bitcoin.org website lists the minimum required PC specs for running Bitcoin core (with the exception of CPU), but I don't think it lists recommended specs. I'll fill these in from my own experience:

CPU: at least 2.5GHz (block verification is single-threaded. Additional cores won't speed this up.)
Memory: 4GB RAM (healthy buffer for dbcache)
Disk: 1TB HDD (7200RPM separate from OS - but only because 1TB SSDs are still fairly expensive.)
Network: 50Mbit/s (anything less than that and block & tx download/upload will affect your connection quality)

Since the same things keep coming up:

1) Yes you can force a RPi2 or 3 to run a node but it is going to be a slow miserable time and you will have lots of issues.

2) Yes you can download the entire blockchain and run a node on a 500GB drive. But you WILL run out of space and then probably have corruption and then have to buy a larger drive and start again. And you might not even be able to copy the data from the old drive if the corruption was bad enough.

3) Yes you can use older equipment to do it. You can even use 10 year old laptops with a 2nd gen core i5. But much older then that the pure performance issues are going to be a killer. And as time goes on it's only going to get worse.

4) You can even use an old 5400 RPM drive and store your OS and blockchain on it. But if for whatever reason you have to do a full re-scan of the blockchain I hope you have a couple of days to kill.

5) Yes you can do it with 2GB of RAM, but your IBD is going to take 2 days short of forever and general performance is going to be miserable.

n0nce made a nice post about doing a node for under $60. https://bitcointalksearch.org/topic/guide-how-to-run-a-bitcoin-core-full-node-for-under-50-bucks-5364742
You can do it for a bit more, you can do it for a bit less.

Don't try to cut that price in 1/2 unless you are technically inclined and can work around the issues. Otherwise you are going to have a miserable time.

This is not to stop people from trying, far from it. This is a post letting you know that it WILL WORK. But the time and effort you are going to put into it, although probably a learning experience is going to be a lot more then having SLIGHTLY better equipment and doing it more efficiently.

-Dave