Topic: Best Hard Drive for Bitcoin Node? (Read 573 times)

On one computer, I have a full node  bitcoin running on an old 2009 child disk WD6400AAKS 640 Gb.
Operating time of almost 50,000 hours.
The initial synchronization takes a long time, but then it works 24/7 without any problems, I will spit so as not to jinx it.
So you can run a full node on the old disk.

There are no moving mechanical parts in SSD drives, you can track condition of drive with good accuracy, and if you enable enough reserved free space for over provisioning, you would significantly increase lifespan of your drive.
SSD's are quiet, not producing any noise, and they are usually heating less than old HDD drives, this can also increase their life.

It probably depends on type of memory and controller used in SSD drives, but I heard that big data centers are using some combination of SSD and HDD drives.
HDD are using for files not moving a lot and moving files are usually stored on SSD drives.

So this issue is mostly related with higher temperatures, meaning you need to have proper cooling system and ventilation in your computer.
I liked conclusion in that extremetech article - no single storage type is perfect, and always back up important data

I didn't know that They should write a big warning on the packaging!
I searched a bit, and it turns out data retention gets a lot worse when the drive is either powered off, or stored at a high temperature. And, to my surprise, enterprise drives are much worse at data retention than consumer drives:

There is also the issue of data retention.
SSDs don't typically hold the data for as long HDDs do.
Writing the block data (blocks/ directory) is usually a one off write, never to be written again, so how long will that data last?
Wallet is possibly a similar issue, certainly areas of it and also if you are hodling.

From some technical reading I've done, since I use RAID1, 2 disks, different batch, on all my disks everywhere,
I feel the need to do a double resync of data about every 2 years or so on SSD, and about every 5 years or so on HDD.
i.e. resync one to the other, then the other back to the first one - to do a full rewrite of all the data on both disks.
YMMV

Yeah... aside from IBD, using an HDD (on both my PC-based and Pi-based nodes) isn't really an issue. It's certainly "cheaper".

I'm sure we'll get to a point where the price difference between HDD and SSD becomes so negligible that you may as well just get an SSD... but, for my budget (and location), we're not quite there yet SSDs are still around 2-2.5x more expensive than an equivalent sized HDD

It's a bit closer to 10x now. Basing on consumer 'retail' grade vs consumer 'retail' grade. And commercial vs commercial

The lifespan is an interesting question. With a lot of writes there is a finite lifespan to an SSD.
BUT in something for just holding node & wallet data. It's probably going to last a lot longer then a spinning drive. For the most part blockchain data once written is not written again and the little bit of data that other files rewrite is tiny.

In the general view, it's the proper tool for a job. Do you need speed or the ability to do a ton of data movement all day every day, do you need long term storage or

Saying one is better then the other is like saying car X is better then car Y without knowing what the driver needs it for.

-Dave

For a node, it doesn't matter too much what you use unless you care a lot about performance.
SSDs do not have a long life due to rewrite limitations vs HDDs.
SSDs are typically about 7 times faster than HDDs.
Adding a USB interface to either type of disk will of course slow it down.

However, a node without a wallet is simply a copy of data that is mirrored 10's of thousands of times around the internet.
So if the disk dies, go get a new one, then resync the node.

As mentioned already, 1TB should last you quite a while.

This is of course assuming you don't have a wallet on the node.

They served a purpose for a VERY short period of time. They gave a speed boost to spinning dives while keeping the cost down.
If SSD prices did not fall as quickly as they did then we would probably still be seeing them.

Because for the most part, I only cared about speed or size, so they did nothing for me, I did not look too closely. BUT, I remember seeing them mostly at the OEM level, not retail.

-Dave

I have supporting chipset and someone told me there is a way for making this work even on Linux OS.
Here is video review by Level1Techs explaining this better than me:
https://youtu.be/wbl2dYgjMQ4

You can combine and use StoreMI together with Intel Optane for full performance, as if that software was made for supporting Optane, maybe AMD did it to piss off Intel

Failed how, and what exactly failed? Seagate HDD or unified drive made by StoreMI?

Hybrid drives are junk, that is why nobody buy them anymore and they are not advertised.
StoreMI is creating unified layered drive with tiered storage that is much better solution and different tech compared to hybrid drives.

Yeah... I haven't really seen any full hardware "hybrid" drives for a while... The were a thing when SSD's were a lot more expensive, but as SSDs slowly came down in price, it seemed hybrid drives just kinda disappeared before they really became a thing.

Used to have one of these long ago, a seagate, that didn't require special code, the SSD cache was in the standard sata HDD interface.
This of course did speed it up.

... until it failed ... then decided that wiping such a HDD of all it's data could also be problematic ...

Never got another one due to both of those issues.

Only for Windows and, as noted, is actually AMD only... it requires a specific chipset:

Meanwhile, Intel have something similar with their "Optane" system, but that involves buying an extra bit of hardware. It too is Windows only.


Both systems effectively seem to be creating "Hybrid" drives... so you get large storage on the HDD, combined with faster speeds by caching the frequently accessed data on the SSD. There is a fairly decent comparison of the two here: https://appuals.com/intel-optane-vs-amd-storemi-what-is-the-difference/

I haven't checked voltages until now, but changing the cables and port worked very well and I don't have any disconnects after that.
One of the things I initially did was to disable hard drive going to sleep in power settings, and now I am monitoring voltage showing 12V correctly.

Getting back to shingled SMR slow drives, one of the interesting things I saw for improving speed for Hard Drives if you have supporting processor is using software called AMD StoreMI that combines SSD and HDD into drive, problem is that this software is only for Windows OS.

One thing which I'm fairly sure wasn't mentioned as yet... have you checked the actual voltages coming out of the PSU?

There are some utilities that might be able to display these values... but the most accurate way is to check using a multimeter and direct measuring the values.

A number of years ago, I had a PSU that was "dying" and was "occasionally" only providing ~10-11V on the the 12V rail... probably didn't help that I had a stupid number of HDDs installed in a stupid RAID array^[1]

But I digress, the "less than ideal" voltage meant that I was getting weird issues with a drive "disappearing" randomly and/or straight up data corruption and r/w errors etc.

At first I suspected it might be the drive, but it came up clean in all tests... I changed cables etc... still got weird stuff happening. It wasn't until I checked the voltages and noticed that the 12V rail was sometimes dropping down to values nowhere near 12V that I was able to finally figure it out and solve the issue.

New PSU... "proper" (and stable) voltages... and the drive worked perfectly after that.

---

As for the "best hard drive for a node" question... I'm fairly happy with the blocks on (1TB) HDD and "chainstate" symlinked to SSD solution that I've been using.

Not sure what it would be like with an IBD, but I've got 2 copies of the blockchain at the moment... so I don't anticipate it being an issue. Even when I set up my Pi node using one of the "cheap" USB3.0 1TB Seagate external HDDs (firstly with RaspiBlitz and then Umbrel), I was able to shortcut that process quite substantially by simply copying all the required data from my other node instead of having to "suffer" through a full sync.

---

[1] - RAID10 with a bunch of old NAS drives I got gifted by a friend... because I could (Mobo had a built in RAID controller)... I had never used RAID before and felt like experimenting with stuff... and then realised I had too much data installed to "undo" it all

I stopped worrying about my HDD breaking down a long time ago. The last time I replaced a 5 year old drive was because of a weird sound (which I later found out came from the fan). As long as I have my backups, and as long as I don't get SMART errors, I just keep it running. My newest disk (Seagate Barracude) has powered up only 40 times in 18 months, and those reboots had nothing to do with the drive.

Afaik WD black or blue are not designed for long term 24/7 use. Indeed WD black should have very good performance, it was my favorite 10 yeas ago, but for devices that usually sleep at night.
I do know though that it can easily go for many days without stop, since I also kept my computer running for months (torrents, folding@home, dabbling with altcoin mining) and the HDD is still pretty much fine. But it was months, not years and it was not receiving new data to write every couple of minutes. And I would not compare a WD black 2010 with a WD black 2021 (as reliability). So I don't know if it's a good choice. If you ask WD they'll probably say it's not.

Considering the increasing size of the blockchain, a 1TB WD Black or WD Blue (<$50) drive is fine if you don't want to spend $100 on an SSD. The more resilient the drive is, the better 😀

I have some good news and some bad news.
Good news is that three days passed after I replaced cables/port and my Seagate drive is working like a charm, so it looks like I don't have to purchase new drive after all.
I left it running 24/7, every parameters are looking very good and there are no issues with crashes or bad sectors, CrystalDiskInfo is showing healthy drive.
Bad news is that my drive is 100% shingled SMR drive and I am getting slower write speed, but I used this only for storing data and running a node, and my OS is on faster SSD drive.

Here is one list of all known SMR drives (including mine):
https://www.truenas.com/community/resources/list-of-known-smr-drives.141/

I decided to test it for 3 days before final confirmation so one more day left, but so far it looks like it was probably a faulty cable, I replaced it with brand new I had at home.
Just in case, I changed the port connection as well, but I doubt that was the issue in my case.

WD Red is on my list already, but I had to crashes with my HDD after I cleaned and replaced everything.
Temperature looks fine, I scanned my drive multiple times, defragmented it, tested for bad sectors, double checked bios settings... and crossed my fingers.

Especially at the non-pro drives, there's quite a problem with some of them (especially older) being CMR and many becoming SMR (cheaper and not that good).

I've bought a WD red (non-pro) for my NAS some 7 months ago and I've done some research back then, although I'm not that good with hardware. And I've seen back then a chart showing how much slower was a SMR compared to the "equivalent" CMR WD RED.
I've done a search now, I'm not 100% sure it was exactly this page, but will do (just check the EFRX vs EFAX comparison). Full story and images: https://www.servethehome.com/wd-red-smr-vs-cmr-tested-avoid-red-smr/

All in all, picking a non-pro WD Red (or Seagate Ironwolf - same story) may not be that easy.

And the RAM both for the cache and drive itself


Because good thermal paste / pads cost money and ADATA is cheap.
Sorry to say it but back in the day, over 5+ years ago, they were a good quality low end drive. Now they are just crap. Don't know if it's just greed or if something else changed.

@dkbit98 did changing the port / cable work?

-Dave

Thanks for info about that.
Before I buy anything I am usually doing deep research, reading reviews, and since I have new motherboard that support newest NVMe I would prefer Gen4 for fastest speed, but they are not cheap.

I really hope it was just a bad cable connection.
Last night I replaced cables with brand new and switched connection to different port on motherboard, so now I am testing to see if this crashes will continue to happen.

Thanks, I will check them out.

Here you go:

• Seagate IronWolf Pro 4TB = $146.99
  ^{- 5-year warranty + 3-year data recovery.}
• Western Digital Red Pro NAS Hard Drive 4TB= $174.99
  ^{- 5-year "limited warranty".}

Note: I haven't used any of the above HDDs but like what @ETFbitcoin mentioned, NAS HDDs are meant to work 24/7 and IIRC, you just have to reformat it then you're good to go.

I'll throw in my 2 sats too: In my experience, it's often the very cheap cable that's the culprit, and not the much more expensive drive. It always felt like a design flaw to rely on cheap connectors.

In my laptop, I use both SSD and HDD. A small SSD for cheap speed, a large HDD for cheap storage. I know the SSD wears from writing, but it doesn't make sense to use a much slower drive because of that. My HDD is 8 times bigger than the SSD, so I only have chainstate on SSD and blocks on HDD.
It really depends on your needs If your current 4TB HDD is full, you can't really replace it by a smaller SSD.

---

---

I haven't used warranty for a long time. Last time a (very new) 2 TB external drive failed, I didn't return it for privacy reasons. If a drive fails now, I'll just replace it myself.

Watch out for the non-pro version of the 980. It is a DRAM-less drive. Although, you should be fine as long as you are not going to make large file transfers. It will last you a long time anyway.


I bought some ADATA NVMe SSD and I returned it on the following day. It was overheating despite using the included metal heat spreader. I bought a different drive from Corsair and it's been working perfectly fine for over 3 years now.

970 or 980 series for sure are my main pick.
I didn't know that about Adata, but that is probably the reason they are much cheaper than Samsung.

Bios updated with latest version today and drive crashed some time after that, so I am excluding any bios issue, but I am going to triple check all connection again.

When you change the SATA cable also plug it in different SATA port on the Motherboard, also use another line with power connectors from the PSU,  check for bios updates. Check everything else before you consider the drive itself.

Sounds good. I am already doing it with my 1 TB drive since I have over 400 GB of storage left.


Buy an SDD from Samsung. 970 Evo series drives are solid. ADATA was one of the first companies to cheat by replacing the controllers.

It is cheaper to buy HDD but they also have maximum two year warranty now for a reason, that means they break much more after that period, like in my case, and I don't want to buy new HDD every two years  

My idea was to use this new drive for running a node and for storing some unimportant data, files, videos and music, if drive have larger capacity than 1 TB.

I have spare sata cable, but weird thing is that I can't pinpoint what the heck is wrong with him and I don't think it's cables that I checked several times.
Now doing deep scanning and checking for bad sectors again.

Please tell me one solid affordable HDD that has more than two year warranty and I may consider it.
I can get very good Samsung SSD with five year warranty easy.

Listen, I am doing regular disc scanning checks and everything was fine until recently, now I got that Error message and Crystal Disk Info showed me Caution yellow message yesterday, so there is something wrong with drive itself.
I also did a duckduckgo search and found that disconnecting like that may be a sign of dying drive...

Samsung and Adata SSD are my first options if I decide to buy one more SSD.

Check your cables & connectors on the drive. Disconnections / reconnections are not the way drives usually die.
Obviously, things happen and yours might be dying that way but after 25+ years doing field IT work I have not seen it happen more then once or twice.
Edit: ETFbitcoin beat me to saying the above but if possible don't just swap cables if you have a spare port on the MB try that one too.


If it was me I would split the difference and go with a 2TB Samsung SSD.
Well under $300 US I don't know where you live in the world or your financial situation, but they are all I have been using.

-Dave

If you have spare power/SATA cable, try replace existing power/SATA cable on your 4TB HDD.


If you plan to use the storage for various tasks (e.g. storing video and backup OS), HDD (for NAS, server or 24/7 usage) is better option. I use 3.5" HDD for running few full nodes without problem.

1 TB should be good for the next few years. I am running my Bitcoin node with txindex (~34 GB) and blockfilterindex (~6.5 GB) enabled, and its data folder weighs ~425 GB in total. I am also running ElectrumX which takes up around 70 GB. A Lightning Network node takes up a negligible amount of space. By the time you need more storage, 2 TB SSDs should have become more affordable.

I am currently using a 1 TB Sandisk Extreme USB SSD and it works well. If I hadn't had it from my old Raspberry Pi build, I would have bought an NVMe SSD. However, buying an SSD is a huge PIA. Less durable QLC SSDs are becoming more and more common. Some manufacturers quietly replace controllers in existing models which significantly degrades their performance or lifetime. You might even buy a DRAM-less SSD which again, hurts the performance.

You might find a good deal on some SATA SSD if you are willing to wait. I think it would be a good compromise between an HDD and an NVMe SSD if you are on a budget.

SSDs have fairly good load distribution and the longevity will be specified by the manufacturer or the specs sheet of the SSD. Generally, I don't find it that big of an issue or at least the proportion of rw load is fairly insignificant compared to the rated endurance by your manufacturer.

I'll probably recommend you to just get a HDD. It's 2X cheaper and speed doesn't really matter for running a node anyways. Your HDD will be fast enough, unless you're consistently synchronizing from scratch or reindexing where speed makes a huge difference.

What type of hard drive do you suggest for running full Bitcoin node, older technology HDD with big capacity or new technology SSD drives?

It seems that my 4TB HDD (not my main drive) may be dying soon as it started to disconnect and connect at random times, but health status is still good and I can't find any bad sectors or the source of problem except one software detecting there is one Uncorrectable ECC error.

My main SSD is used for operating system and software I use regulary, but I guess running a node have some effect on longevity of drives and I think they make them with lower warranty (mine expired) on purpose.

So my question: Is it better to buy new HDD with larger capacity or SSD with 1 or 2 TB, and what drives are you using for running your Bitcoin nodes?