Full Node on Raspberry Pi 3 | Bitcointalksearch.org

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: bitart on February 19, 2018, 04:24:28 PM

I'm not sure about the OS of Raspberry PI but if it's a linux, then it's not a good idea to run it on an SD card or USB flash drive. Or, I only need to move the swap partition to the HDD and it will solve the read/write cycle problem.

Do you really need the swap partition, or can you run without it? If at all possible, get enough RAM to run without swap.

Quote from: jackg on February 19, 2018, 07:07:57 PM

But off topic: I have a 3 year old smartphone with a class 10 32gb micro SD card that is still going strong and not dying.

My phone has a similar micro SD card, but it only writes those 32 GB once a year. After that, it's full with images and videos, and I clean up for another year. My point is: phone writes are nothing compared to what a swap partition can do to it.

Quote from: ranochigo on February 20, 2018, 04:50:38 AM

With SD cards, I doubt they are designed to function well under this kind of load.

Correct, SD cards don't have wear leveling.

jochemin

member

Activity: 133

Merit: 22

Quote from: bitart on February 20, 2018, 11:17:47 AM

Quote from: jochemin on February 20, 2018, 08:47:34 AM

I have a Bitcoin full node + lightning node running stable on a Raspberry Pi3. BTC blockchain is stored in an external usb drive, swap in a pendrive. Everything works fine.

Which LN node are you using? the C-lightning, the lnd or which one? The external USB drive you use to store the blockchain is an HDD with a USB connections to the PI? What is the size of the swap pendrive?
Could you please share the link you were using while you were working on the installation of the node?
Sorry for the lots of questions

Hi, i'm using c-lightning. I use the external pendrive for swap (2GB) and the external hard disk (500GB) to store Bitcoin data.

I have a step-by-step but it is in spanish and only BTC node --> https://medium.com/@jochemin/nodo-completo-bitcoin-en-raspberry-pi-3-3904c29d8ce1

To have c-lightning the difference with that document is to put swap on a pendrive (in the document I put it on the hard disk).

bitart

hero member

Activity: 1442

Merit: 629

Vires in Numeris

Quote from: jochemin on February 20, 2018, 08:47:34 AM

I have a Bitcoin full node + lightning node running stable on a Raspberry Pi3. BTC blockchain is stored in an external usb drive, swap in a pendrive. Everything works fine.

Which LN node are you using? the C-lightning, the lnd or which one? The external USB drive you use to store the blockchain is an HDD with a USB connections to the PI? What is the size of the swap pendrive?
Could you please share the link you were using while you were working on the installation of the node?
Sorry for the lots of questions

jochemin

member

Activity: 133

Merit: 22

Quote from: bitart on February 19, 2018, 04:24:28 PM

Quote from: jackg on February 19, 2018, 03:41:56 PM

Quote from: bitart on February 17, 2018, 05:38:22 AM

SD with 200 GB available... used from ebay... From shop, it would cost about $200, you can buy a decent USB HDD from this price and it would not have only 256 GB space but much more.
Anyway, how much space is needed to host a full node? I'm thinking about the same, using a Raspberry PI to run a full bitcoin node and I'm thinking about what size of USB HDD I need to buy.
Does SD card is a possible solution for storage, when running a node on PI? I mean the OS is writing on the swap partition continously, so SD can reach it's write limit in no time and after you can just trash your SD?

SD and SSD technologies are fairly similar in the way thy function and people run core on SSDs. Even though they're obviously more stable, they're still flash storage.

@OP, what does the SD card say on it? What brand it is, what size? The only FLASH producer I'd trust personally is Kingston (Samsung, Sony and Sandisk are alright also but not as good). Bitart, Kingston and Sandisk definitely go up to 256 gb micro sd cards (not entirely sure whether a raspberry pi is capable of supporting that in such a small and dense storage medium but...)

I've been considering using a raspberry pi also for the same operation but I always got my dependencies wrong and never had enough time to go back and correct myself from all the problems to reinstall everything.

Thanks for the advice. I won't think that the Raspberry Pi is able to handle the 256 Gb SD card...
My concern is the read/write cycles of the SD card, still.
I've found this thread, it's pretty old, but the problem is still valid:
https://www.raspberrypi.org/forums/viewtopic.php?t=21281
I'm not sure about the OS of Raspberry PI but if it's a linux, then it's not a good idea to run it on an SD card or USB flash drive. Or, I only need to move the swap partition to the HDD and it will solve the read/write cycle problem. Anyway, as soon as I'll have a working system (hopefully sometime this year

) I'll make a backup copy of the SD, just in case....

I have a Bitcoin full node + lightning node running stable on a Raspberry Pi3. BTC blockchain is stored in an external usb drive, swap in a pendrive. Everything works fine.

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

Quote from: ranochigo on February 20, 2018, 04:50:38 AM

Quote from: jackg on February 19, 2018, 07:07:57 PM

But off topic: I have a 3 year old smartphone with a class 10 32gb micro SD card that is still going strong and not dying.

Also, drives normally die when you plug in/unplug the drive, if the rpi stays on it'll probably make the SD live a bit longer.

In most cases, the number of times that you plug in a SD card isn't the problem. I've never had a card that failed on me due to that and I remove them pretty often for various of my projects.

However, as with all non-volatile memory chips, they have a limited number of read/write cycles. SD cards and SSDs are both susceptible to this problem but I wouldn't worry too much in the case of SSDs. With SD cards, I doubt they are designed to function well under this kind of load. Bitcoin nodes are pretty resource intensive with the client fetching information ever so often. If they aren't one of the "better" ones, I wouldn't expect them to last a very long time. The SD card in your smartphone is probably fine since it didn't go through that many read/write cycles.

To the point on Raspberry Pi nodes, I feel that it is more cost effective if you purchase a HDD to connect to it. It would be the best if you could salvage it from elsewhere. The sheer cost of a 128GB card is more than a 1TB WD HDD.

Cost per GB is definitely smaller, not actual cost in my country (though it is literally £5 difference).

I quite like Seagate for HDDs also but I guess that's just me as many like WD and others. Whenever I try to do this, I have a nice 8tb drive waiting to be used for the project. If you can salvage the drive from somewhere else then you could try to do that. I've seen companies here and regularly trying to ditch hard drives. If you're in the UK or US, they were selling 500GB internal drives for $10 second hand at CEX/webuy.com.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: jackg on February 19, 2018, 07:07:57 PM

But off topic: I have a 3 year old smartphone with a class 10 32gb micro SD card that is still going strong and not dying.

Also, drives normally die when you plug in/unplug the drive, if the rpi stays on it'll probably make the SD live a bit longer.

In most cases, the number of times that you plug in a SD card isn't the problem. I've never had a card that failed on me due to that and I remove them pretty often for various of my projects.

However, as with all non-volatile memory chips, they have a limited number of read/write cycles. SD cards and SSDs are both susceptible to this problem but I wouldn't worry too much in the case of SSDs. With SD cards, I doubt they are designed to function well under this kind of load. Bitcoin nodes are pretty resource intensive with the client fetching information ever so often. If they aren't one of the "better" ones, I wouldn't expect them to last a very long time. The SD card in your smartphone is probably fine since it didn't go through that many read/write cycles.

To the point on Raspberry Pi nodes, I feel that it is way more cost effective if you purchase a HDD to connect to it. It would be the best if you could salvage it from elsewhere. The sheer cost of a 128GB card is more than a 1TB WD HDD.

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

Quote from: bitart on February 19, 2018, 04:24:28 PM

Quote from: jackg on February 19, 2018, 03:41:56 PM

Quote from: bitart on February 17, 2018, 05:38:22 AM

SD with 200 GB available... used from ebay... From shop, it would cost about $200, you can buy a decent USB HDD from this price and it would not have only 256 GB space but much more.
Anyway, how much space is needed to host a full node? I'm thinking about the same, using a Raspberry PI to run a full bitcoin node and I'm thinking about what size of USB HDD I need to buy.
Does SD card is a possible solution for storage, when running a node on PI? I mean the OS is writing on the swap partition continously, so SD can reach it's write limit in no time and after you can just trash your SD?

SD and SSD technologies are fairly similar in the way thy function and people run core on SSDs. Even though they're obviously more stable, they're still flash storage.

@OP, what does the SD card say on it? What brand it is, what size? The only FLASH producer I'd trust personally is Kingston (Samsung, Sony and Sandisk are alright also but not as good). Bitart, Kingston and Sandisk definitely go up to 256 gb micro sd cards (not entirely sure whether a raspberry pi is capable of supporting that in such a small and dense storage medium but...)

I've been considering using a raspberry pi also for the same operation but I always got my dependencies wrong and never had enough time to go back and correct myself from all the problems to reinstall everything.

Thanks for the advice. I won't think that the Raspberry Pi is able to handle the 256 Gb SD card...
My concern is the read/write cycles of the SD card, still.
I've found this thread, it's pretty old, but the problem is still valid:
https://www.raspberrypi.org/forums/viewtopic.php?t=21281
I'm not sure about the OS of Raspberry PI but if it's a linux, then it's not a good idea to run it on an SD card or USB flash drive. Or, I only need to move the swap partition to the HDD and it will solve the read/write cycle problem. Anyway, as soon as I'll have a working system (hopefully sometime this year

) I'll make a backup copy of the SD, just in case....

But off topic: I have a 3 year old smartphone with a class 10 32gb micro SD card that is still going strong and not dying.

Also, drives normally die when you plug in/unplug the drive, if the rpi stays on it'll probably make the SD live a bit longer.

I think the pi2 claimed it could only handle 64gb max (I'll test it with my 128gb one tomorrow if I can find them both and see if it does as that might be helpful to both you and op - though you both have the 3 and not the 2).

Pi runs a lot of popular linux and what I assume is a windows-linux hybrid for one of the operating systems.

bitart

hero member

Activity: 1442

Merit: 629

Vires in Numeris

Quote from: jackg on February 19, 2018, 03:41:56 PM

Quote from: bitart on February 17, 2018, 05:38:22 AM

SD with 200 GB available... used from ebay... From shop, it would cost about $200, you can buy a decent USB HDD from this price and it would not have only 256 GB space but much more.
Anyway, how much space is needed to host a full node? I'm thinking about the same, using a Raspberry PI to run a full bitcoin node and I'm thinking about what size of USB HDD I need to buy.
Does SD card is a possible solution for storage, when running a node on PI? I mean the OS is writing on the swap partition continously, so SD can reach it's write limit in no time and after you can just trash your SD?

SD and SSD technologies are fairly similar in the way thy function and people run core on SSDs. Even though they're obviously more stable, they're still flash storage.

@OP, what does the SD card say on it? What brand it is, what size? The only FLASH producer I'd trust personally is Kingston (Samsung, Sony and Sandisk are alright also but not as good). Bitart, Kingston and Sandisk definitely go up to 256 gb micro sd cards (not entirely sure whether a raspberry pi is capable of supporting that in such a small and dense storage medium but...)

I've been considering using a raspberry pi also for the same operation but I always got my dependencies wrong and never had enough time to go back and correct myself from all the problems to reinstall everything.

Thanks for the advice. I won't think that the Raspberry Pi is able to handle the 256 Gb SD card...
My concern is the read/write cycles of the SD card, still.
I've found this thread, it's pretty old, but the problem is still valid:
https://www.raspberrypi.org/forums/viewtopic.php?t=21281
I'm not sure about the OS of Raspberry PI but if it's a linux, then it's not a good idea to run it on an SD card or USB flash drive. Or, I only need to move the swap partition to the HDD and it will solve the read/write cycle problem. Anyway, as soon as I'll have a working system (hopefully sometime this year

) I'll make a backup copy of the SD, just in case....

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

Quote from: bitart on February 17, 2018, 05:38:22 AM

SD with 200 GB available... used from ebay... From shop, it would cost about $200, you can buy a decent USB HDD from this price and it would not have only 256 GB space but much more.
Anyway, how much space is needed to host a full node? I'm thinking about the same, using a Raspberry PI to run a full bitcoin node and I'm thinking about what size of USB HDD I need to buy.
Does SD card is a possible solution for storage, when running a node on PI? I mean the OS is writing on the swap partition continously, so SD can reach it's write limit in no time and after you can just trash your SD?

SD and SSD technologies are fairly similar in the way thy function and people run core on SSDs. Even though they're obviously more stable, they're still flash storage.

@OP, what does the SD card say on it? What brand it is, what size? The only FLASH producer I'd trust personally is Kingston (Samsung, Sony and Sandisk are alright also but not as good). Bitart, Kingston and Sandisk definitely go up to 256 gb micro sd cards (not entirely sure whether a raspberry pi is capable of supporting that in such a small and dense storage medium but...)

I've been considering using a raspberry pi also for the same operation but I always got my dependencies wrong and never had enough time to go back and correct myself from all the problems to reinstall everything.

ntoken

newbie

Activity: 32

Merit: 0

Quote from: DeepSpace1 on January 05, 2018, 07:13:22 AM

Thanks for the help and sorry for the late reply.

The only unknown for me is this line:

Quote

git clone -b v0.15 https://github.com/bitcoin/bitcoin.git

Because in the tutorial it says "v0.11.2" and since its old I used the recent version "v0.15". Maybe thats the problem and I use the wrong repository?

Here is the output from my command line: https://pastebin.com/Hx8NHXCd

I faced similar issues on compiling. From the output above, there is some other dependency issues include libevent-dev. Here are the packages that I finally confirmed, may it help:


build-essential libssl-dev libdb++-dev libboost-all-dev libqrencode-dev libminiupnpc-dev
protobuf-compiler libprotobuf-dev libczmq-dev libczmq4 libevent-pthreads-2.0-5 libevent-dev

Jet Cash

legendary

Activity: 2814

Merit: 2472

https://JetCash.com

It was from personal observation, so I can't really quantify it. This article compares HDD and SSD though.
http://www.storagereview.com/ssd_vs_hdd

bitart

hero member

Activity: 1442

Merit: 629

Vires in Numeris

Quote from: Jet Cash on February 18, 2018, 11:13:33 AM

I ran a full node on a 256Gb SSD ( external), that was a few months ago, so I'd probably go for a 512 if I was starting now. I'm using a 2TB external drive ( Toshiba) for my new node , and that is fine, but it's a bit slow, and you probably ought to have an alternative power supply for it if you are going to use one.

What do you mean about 'a bit slow'? I have already a 512 Gb external 3.5" HDD with an external power supply, external HDD enclosure with fan, so I thought it will be OK to start with. As I've read somewhere if I run the OS of the Raspberry PI from the SD card, the OS will be relatively quick and I have to store only the blockchain on the external HDD. If the OS runs from the SD it should be as quick as an SSD? I'll give it a try as soon as the Raspberry arrives and I can start this little project. What I can do beforehand to install a node on a laptop, download the blockchain and to copy it to the external HDD.

Jet Cash

legendary

Activity: 2814

Merit: 2472

https://JetCash.com

I ran a full node on a 256Gb SSD ( external), that was a few months ago, so I'd probably go for a 512 if I was starting now. I'm using a 2TB external drive ( Toshiba) for my new node , and that is fine, but it's a bit slow, and you probably ought to have an alternative power supply for it if you are going to use one.

bitart

hero member

Activity: 1442

Merit: 629

Vires in Numeris

Quote from: LoyceV on January 21, 2018, 04:46:46 AM

Quote from: DeepSpace1 on January 20, 2018, 05:21:18 PM

I did not partition it.

Although you can create a filesystem without a partition, it's not recommended.

Quote

Yes, I bought it used...

You should start by testing the SD card it self. Write 100 GB, and see if you can read it back.

A simple test:

Code:

#!/bin/bash
mkdir ~/disktestbyLoyce
cd ~/disktestbyLoyce
wget http://ipv4.download.thinkbroadband.com/1GB.zip
i=1; while test $i -le 100
do cp 1GB.zip 1GB.zip$i
i=$((i+1))
done
md5sum 1GB.zip*
rm 1GB.zip*
cd; rmdir ~/disktestbyLoyce

If your SD card is okay, it will show the same checksum 101 times without errors.

SD with 200 GB available... used from ebay... From shop, it would cost about $200, you can buy a decent USB HDD from this price and it would not have only 256 GB space but much more.
Anyway, how much space is needed to host a full node? I'm thinking about the same, using a Raspberry PI to run a full bitcoin node and I'm thinking about what size of USB HDD I need to buy.
Does SD card is a possible solution for storage, when running a node on PI? I mean the OS is writing on the swap partition continously, so SD can reach it's write limit in no time and after you can just trash your SD?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: DeepSpace1 on January 20, 2018, 05:21:18 PM

I did not partition it.

Although you can create a filesystem without a partition, it's not recommended.

Quote

Yes, I bought it used...

You should start by testing the SD card it self. Write 100 GB, and see if you can read it back.

A simple test:

Code:

#!/bin/bash
mkdir ~/disktestbyLoyce
cd ~/disktestbyLoyce
wget http://ipv4.download.thinkbroadband.com/1GB.zip
i=1; while test $i -le 100
do cp 1GB.zip 1GB.zip$i
i=$((i+1))
done
md5sum 1GB.zip*
rm 1GB.zip*
cd; rmdir ~/disktestbyLoyce

If your SD card is okay, it will show the same checksum 101 times without errors.

DeepSpace1

newbie

Activity: 5

Merit: 0

I did not partition it.

Yes, I bought it used...

BTRIC

copper member

Activity: 98

Merit: 12

BTRIC: Innovate. Institute. Labs.

Quote from: DeepSpace1 on January 14, 2018, 02:47:05 PM

Thx ranochigo, that helped.

But there seems to be a problem with this SD card. I tried to copy the blockchain onto my Pi and after a few Gigs it started complaining that there is not enough space, even when Linux said that more than 200gb where available...

Did you partition it? Also, you didn't buy it on eBay did you?

DeepSpace1

newbie

Activity: 5

Merit: 0

Thx ranochigo, that helped.

But there seems to be a problem with this SD card. I tried to copy the blockchain onto my Pi and after a few Gigs it started complaining that there is not enough space, even when Linux said that more than 200gb where available...

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

It just means that the directory that you're in does not contain the make file. The ./configure script failed because you didn't have a dependency.

Try running this:

Quote

sudo apt-get update -y
sudo apt-get install libevent-dev -y

Try it and then run the configure command again.

DeepSpace1

newbie

Activity: 5

Merit: 0

Thanks for the help and sorry for the late reply.

The only unknown for me is this line:

Quote

git clone -b v0.15 https://github.com/bitcoin/bitcoin.git

Because in the tutorial it says "v0.11.2" and since its old I used the recent version "v0.15". Maybe thats the problem and I use the wrong repository?

Here is the output from my command line: https://pastebin.com/Hx8NHXCd

Topic: Full Node on Raspberry Pi 3 (Read 512 times)