Topic: EXT4 file system for usb node. (Read 388 times)

I've got quite a few charging options. I've got an extra heavy duty battery as the starter battery in the van, and I use that for small topups. I've also got a large leisure battery, and I plan to fit a split charger for that as soon as I have decided where to store the battery. I've got 3 small solar panels which are OK for topping up phones and lights etc. I plan to install a roof solar panel, but I've got a problem with that at the moment, as the roof is a high top plastic shell, and I need to find, or make, some long legs for a roof rack. The immediate updates are some body improvements, both for me and for the van. Also I bought a wood burning stove, and I need to install that.

I'm still wasting through problems from my past life, and these seem to prevent me from sorting out these updates. Every time I throw money at my situation to try to improve it, I seem to create more problems. This new phone is one example. My new solar powered battery bank is giving me problems as well. That has a wireless charger, and that doesn't charge the new phone.

[update]

Sending it back will be a massive nuisance for me, so I'm going to give it a final test. I've got a rat's nest of cables, so I'm going to buy a new solar backup with a new cable, and also Qi charging, and I'll experiment with that. I've ordered this one, and it arrives tomorrow.
https://www.amazon.co.uk/36000maAh-Wireless-Waterproof-Portable-Flashlight/dp/B08DF7G6Y6/ref=sr_1_15?dchild=1&keywords=solar+power+bank&qid=1604145211&sr=8-15

I’ll also add that you can get about three power banks for the phone and let it charge on one of them while the battery is half-finished, when that reaches 80% you can unplug it and let it run on battery until it drains back to 50% (this is so recharging will be quick) and repeat with another power bank, while your other power banks charge on your 230V sockets. The sockets will need to have USB-C or MicroUSB ports depending on what the power banks have.

You’ll be switching between power banks fairly quickly because the more traffic gets sent to the hotspot, the more battery usage it takes.

That's not bad for extending the battery's lifespan. As a digital nomad: have you considered mobile charging solutions? A car battery is probably the most likely: I've installed several 5V 2A USB charging ports in my car. Of course, all this was meant for a road trip that never happened because of the pandemic.

Well that would be the sensible and practical way to create the chain. However, I don't know anyone in real life that runs a full node, and certainly not amongst the few nomads I have met. Maybe |I should start a special interest group of Bitcoin nomads. The HDD has a copy of a previous node that I was running, and it developed a bad spot on the drive - it was the internal drive in the Notebook. This coupled with Covid led me to decide to postpone trying to sync the drive. I bought a Toshiba 2Tb HDD, and I copied the wallet and blockchain data onto it. When I started to sync it, it reported that it was 2.5 years behind. I've been running this intermittently over public WiFi, and it is now just over a year behind. I have to be selective in my choice of venue, as not all cafes allow the Bitcoin ports to be used, and the external HDD seems to bump up the power requirement. Core is reporting transfer rates of between 100 Kb/s and 10,000Kbs. Unfortunately around 200Kb/s is the most common. I'm in Sainsbury's right now, and this is one of the better locations. Speed is reported to be between 1,000Kb/s and 2,000Kb/s with 10 peers. I've also got 2x230 volt charging sockets, and 2 USB charging sockets, so I can recharge my angle grinder as well as run the computing stuff.

I'm mildly concerned that the core wallet is only reporting 2 receiving addresses, but I'll wait until it has finished syncing, and then I will check my backups for the others. I'm sure I will be able to import the missing addresses once I have read up on the process.

At the moment my greatest concern is the new phone. I was really pleased with it for the first few days, but I seem to have a problem with the battery charging at the moment. The battery looked as if it is new tech which gives it 4 times the life of my old phone, and it has a smart charging system. This seems to think that charging should be done between 20 and 80 percent of capacity, and over 80% should just be an intermittent trickle. Unfortunately it is at 23% at the moment, and the trickle is a drop ever 15 minutes of so. I've Googled the problem, and it seems that they are several things to check, and I'm working through those before I send it back.

There shouldn’t be a speed difference between tethering and using a hotspot now that WiFi and USB3 standards allow for much larger bandwidth speeds. The main problem will be your phone’s battery finishing quickly, because running a hotspot uses a lot of battery, from my experience. You could try making the hotspot while the phone is plugged in somewhere (no tethering internet with it) like to the notebook with a USB-C to USB-A cable, and it won’t matter that the phone isn’t recognized by OpenSuse because USB ports have a built-in standard for charging, so that hardware support is all you need.

If you have to rely on public Wifi, would it be possible to physically copy the blockchain from someone? Then, just to be sure, let your own Bitcoin Core check all blocks (I don't think --rescan is enough, but I'm not sure what would suffice).

I've done this before, and it works fine: just copy the blockchain, and change settings to pruning.

I didn't want to bump this thread with too many consecutive posts, but it looks as if there is still a bit of interest in the project, so I thought I would post an update.

The full node on the notebook that uses an external HDD is sync'ing nicely, but slowly. I'm only a year behind now. It's been slow because I have limited access to decent connections. On looking at the wallet, it only seems to have two addresses in it, so I'm assuming that it is an old backup, and I will need to import the additional addresses from another backup.

I was hoping to use the new Huawei phone for tethering, but I haven't managed to get that working. This is probably because I have disabled too many apps and options. I may try setting up a WiFi hotspot as an alternative. I'm no longer updating the Linux parallel node, and once the HDD note is up to date, I'll have a go at creating a pruned node for the netbook. I'm hoping that I can manage this from a copy of the blockchain, and I won't need to download the lot again. Once I have some positive results, and things are up to speed, then I will create a report page, and post that link for  comments.

Sainsbury's looks to be the winning location at the moment.  It's allowing for up,to 10 peers to connect, and traffic is spiking between !Kb/s and 10Kb/s. I'm 2 years behind, and it is telling me it will take 24 hours of continuous connection to catch up. I can probably manage tht if I spread the connection over the next week or so. They have installed some extra power sockets. so I can run the node with the full blockchain,and the pruned node, and keep them sync'd.

I has a bit of a wobbly this morning. I'm using an old USB mouse, and it seems to disconnect at odd intervals. It's easy to fix, I just pull the plug and reconnect it. Except that I unplugged the Bitcoin SSD as I was talking to somebody at the time. After this, Bitcoin wouldn't shut down normally, and wouldn't send or receive traffic. I had to power down the PC, and restart it. Core took a while to think about things when it restarted, but it seems to be stable and fast at the moment. I have 2 USB2 sockets and one USB3 socket, so I don't have any choice for the SSD. Probably the sensible thing to do is to use a wireless mouse,and scrap the faulty one.

Another thing that I may have to consider is my policy of using recycled or repaired components. It isn't as if things are expensive, it's just that I try to avoid waste. I did buy new RAM and SSDs though, and I gather many people use second hand products. The 320 GBb SSD is salvaged, and we will see how that stands up to daily use. Infrequent backups will allow me to recover the wallet if it fails, so I'm not too bothered about it.

Thanks for the nethogs suggestion - I'll have a look at that when I have the node sync'ed or maybe before.

I'm in Sainsbury's now, and I've got power. The sync is running at 3000 Kb at the moment with 4 peers, so I'll see if that improves as it settles down. The problem with using this cafe is that there is a £100 surcharge if I park for over 3 hours. I probably need to start a site about life as a digital nomad. Especially as so many people are turning to the lifestyle as a result of bankers and government power and wealth grabs.

It is now reporting that it will take 46 hours to complete the update. I'm not sure if I can drink that much coffee.

[FRIDAY]
I thought I would try Morrison's for today's morning coffee. There is a good source of power sockets, but the WiFi is a bit restrictive, and it looks as if theu block all non-standard ports. Core started with no problems, but with only the basic ports open, it can't find any peers, so the sync'ing will kave to wait. I could tether it to my phone, but I think that will burn too much bandwidth.

Dammit - SFTP won't work either. I am going to consolidate all my research into the crapspeak about viruses and vaccination onto my domain G8S.com. That will no doubt get me into trouble, as it sounds like Gates.com, and of course the supplier of the famous virus hospitality software wouldn't dream of doing the same thing with the human immune system.

goacess will only look at web server logs so I'm not sure how you'd make it monitor bitcoin core's bandwidth.

I personally use nethogs to monitor the network usage by process, it's a command-line tool that constantly updates statistics on the terminal. And it shows you how much sent and received traffic each process is doing in KB/s or MB/s, and a grand total, and it can also show running totals of the bandwidth sent by each process in KB and MB. Nethogs has a small memory footprint as well so it won't use a lot of battery. There is a package for it on OpenSUSE, I believe it can be installed with yast --install nethogs.

I''m OK for external drives. I bought 3 x 2TB drives to use via USB - one is an HDD and 2 are SSDs. I've also got an assortment of smaller SSDs, including the 320Gb Hitachi I mentioned in the other thread. I bought a drive docking station as well to check my 3.5" drives,but that needs to have its own power supply.

I habe been charring about openSUSE on a webmaster forum, and one guy suggested installing goaccess to analyse log files, and that looks like an interesting concept. If I can master its use, then I could use that to compare different WiFi providers. I'm not sure that comparing the synchronisation of different drive types would be useful on my system. So much seems to depend on the quality and speed of the WiFi provider.

[UPDATE]
I've updated the Linux software, and I've installed Core. I ran thjs with an external Toshiba "Tb HDD. This contains an existing wallet, and an out of date Blockchain. The last entry is for March 2018. core has verified the blocks, and stated to synchronise. It running at less than 50Kb, and I've only got 30 minutes battery left, and there is no power source here. It looks as if everything is working well, and is stable, so I'll shut it down, and find a better location to run the synchronisation.

[UPDATE 2]
I've shut down the Notebook, and switched to the Netbook. I'm finding this quite interesting, and I need to start loading applications to replace those that I was using under Windows. The extra 8Gb of RAM seems to have made a difference, and I'm pleased with that. Replacing the HDD with an SSD doesn't seem to have affected battery life, and I find that surprising. I need to do a bit more research into that. Runnung core with an external HDD seems to knock battery life as well, so I'm going to have to restrict use to restaurants that provide power sources. Once I am able to create a pruned node on the Hitachi SSD, then I'll try maintaining that node using the Netbook as a background job. I'm considering building a Raspberry Pi with a small display screen to run the node. I could use the solar battery back up to provide the power. That will have to wait until I have done some research into the current nodes that I am running, and until I am happy that I can replace Windows with openSUSE.

When you reach that stage, could you also make a test and copy it to an external SSD, while keeping the network speed the same as when you synced it on HDD? I'm interested in seeing the performance differences between using an external HDD and SSD, in terms of how many hours it takes to sync the blockchain.

You can get a 480GB SSD for around $55 to $85 on Amazon. This one's on the cheaper side: Pioneer 3D NAND External SSD (480 GB)-Portable Solid State Drive USB 3.1 Gen 1 (APS-XS03-480), though your laptop port will probably be limited to USB 3 or even 2 connectivity speeds, but that'll give you an opportunity to see how the USB port influences the speed.

As there didn't seem to be much interest in my pruned node project, I decided to change it a bit for my own use. I've doubled the memory in the Notebook to 16Gb, and replaced the HDD with a 2Tb SSD, and I've installed open SUSE on that. In fact I'm posting this on that updated machine, as there don't seem to be any problems here either. I haven't tested everything though. I've got a wallet with about 0.4 Bitcoin in it, and a blockchain that is about 3 years out of date. That is stored on an external HDD, and I'll update that with this machine. Once it is sync'd I'll copy it onto yet another drive, and I'll prune it. Once that is stable, then I'll see if I can copy the blockchain onto the 320Gb drive, and I'll associate it with an empty wallet.

I don't think that one should store more than half a Bitcoin online, any extra coins should be kept in more secure locations such as a hardware wallet.

happy to help LVM is very useful, and surprisingly old tech for Linux (early versions are from the early 2000's era)

I'm really grateful to you for suggesting this, and I have been reading this article.
https://opensource.com/business/16/9/linux-users-guide-lvm

I'm not sure that it is suitable for my node, but I think it will be a great help with my domains, web sites and images. I've got maybe a dozen or so external drives that I have salvaged, and the latest of these is this Hitachi one -
https://www.expertreviews.co.uk/storage/internal-hard-drives/50686/hitachi-z7k320-review
It came out of a scrap Xbox, and I'm waiting for the delivery of a USB case so that I can test it.

It really is time I got some order into my site structures, and maybe this will give me a chance to do this.

OK - things are becoming a bit clearer. I took the base cover off the netbook, and cleaned out the dust. I think the screren problem may have been caused by overheating. It has 8Gb of RAM, and a spare slot. An extra stick is about £35 if I buy a Crucial or Samsung one, so that is an eassy upgrade, and could well make quite a difference. The hard disk spot is persistent, but I will keep that disk as a historic backup, and hopefully the new SSD won't have any problems. I've more or less decided to go with openSUSE, as I don't want to spend too much time on experimenting any more. I've got some property and other issues that need to be resolved, and I want to move into the van full time. That is until the world economies stabilise, and I can see where I want to be in the structure. Actually, I probably want to be outside it, as that is where all the healthy fun will be in my opinion.

So back to the OP. The new SSD is just for Bitcoin, so I think I will set it up as 1 partition, and hope that this will extend its potential life. I take the point about the logical volumes, and that could be a nice idea for the web site disk, but that will have to wait for a month of so.

Many thanks for all the help and info, and I'll let you guys know if there are any problems that crop up in the future. I'm going to have a look at WiFi boosters to see if that will improve the network speed, as that seems to be another issue.

My suggestion would be to use Logical Volume Manager (LVM):

• setup LVM: one PV and one VG per physical disk
• setup several LV's inside your VG
• write your partitions to the LV's
• setup the file system you want for each LV partition

this method is very flexible, you can change the size of the partitions (impossible with the typical way of creating partitions) and add new partitions very easily. This will make it easy to experiment, to see what the best setup is for your hardware and how you want to use it, before you commit to the size of the partitions.

I'm not sure if the typical "Disks" gui application lets you use LVM, but I would steer you away from using that anyway. The CLI for lvm is pretty straightforward, and there's a good chance your base system is already using LVM by default

Well that is interesting.

A few months ago I bought another 2Tb EVO drive, and I replaced the internal drive on my Notebook with it. I installed Mint for evaluation and experimentation. I went back to the old Windows 10 HDD because I needed to use some of the software on it. I put the SSD into a slightly different Sabient case. This one has an aluminium case, and a protective sleeve, so it may have been a bit more expensive. I've just looked at that setup using files on the Netbook, and it reports it as EXT3/4, but it tells me that some files are unreadable. This is the first time that I have looked at it via USB. I don't think I am going to go back to Mint, as there were some problems with the WiFi in cafes.

It seems to be coming down to my acceptance of Carlton's advice to use btrfs. I'm going to repartition the Mint drive, and use that for my Bitcoin wallet. Is it best to set up a single partition, or should I split out the wallet info, and put it in a second partition. This could make things slightly easier if I am using multiple nodes on more than one machine. However, it may be better to have a single online node, and use hardware wallets for long term storage.

I'm a bit cautious about installing any more software on the Netbook, because of the limited internal disk space, but it would be handy to make Brackets available. I use this to maintain my websites.I'm not short of drives, so I could put a 512Gb drive into a case, and use that to store the web site pages and backups. This would make it easier to move machines as well as saving space on the system drive.

both XFS and btrfs are newer than ext4, and sometimes still considered beta software. As is bitcoin FWIW, I use btrfs and XFS (but not for the same length of time), and didn't encounter any issues other than the processing issues I described (the performance penalty of using either btrfs/XFS it quite noticeable if you copy some huge file, or the swap file is using btrfs/XFS and in heavy use). In general, I'm using btrfs/XFS for data disks, not for system disks (because of the performance penalty). Perhaps a good compromise would be to use btrfs/XFS for the root partition, then set up a separate partition for swap that uses ext3 or ext4, as the error correction is less essential for volatile data (and performance penalty would be reduced), but I have thus far not tried such a setup


it's possible that btrfs is either badly supported or not supported at all on alternative operating systems, I would check that if you intend to use anything else (obviously, most other people are using Microsoft or macOS). I'm pretty sure (not 100% though) that ext2-4 are well supported elsewhere.

Thanks for that helpful and informative post.

The keyboard issue seems to be pretty standard in the latest version of Ubuntu,and there are various suggestions as to how to fix it. I started to try them, and found that it was still an intermittent problem. I took the view that if Ubuntu released a version that was claimed to be stable, but contained such errors, then it was time to look for an alternative distro. So far, I haven't found any problems with the Gnome openSUSE installation, but I accept that there may be an overhead with Gnome. I can accept that on a backup machine, as long as it continues to work without my having to update the system. I may install oprnSUSE on the Netbook ifI decide not to replace it,and that has 8Gb of RAM.

I believe that the default file system is btrfs, and I am still considering that as an alternative to EXT4. There doesn't deem to be much in it, but I got the impression that EXT4 is slightly more stable, and may be less demanding on the cpu. This was why I considered it. However, using the default system might be a better option in the long run. I'm assuming that I won't have any problems copying files onto another hard disk if I decide to use an alternative file system on the solar powered system in the van - if I ever get round to building it.

yes, NTFS is not good (nor is it a "new technology" file system any longer)



XFS and btrfs are what you want if you're interested in handling data corruption. But I don't think the laptop suggested in the OP would handle it ideally, as they both use merkle-tree based checksumming to correct errors on disk (ext4 can detect, but not correct, errors on disk). These old budget CPUs probably don't have an in-built native processing unit corresponding to the hashing algorithm that XFS or btrfs use for the error detection/correction. Possibly the standard integer processing APUs will handle it ok, but you will be putting additional load on the CPU no matter what.

The linux kernel in your linux distribution (Suse/Ubuntu) might have XFS and/or btrfs enabled by default, and that would certainly be the most efficient implementation (you would have to check what config options the kernel uses). If not, you can enable them yourself by copying the standard config for your version of linux, changing the option for XFS/btrfs to 'yes', then compiling it (don't forget to install the userspace modules and regenerate the bootloader). You probably could've solved the issues with the keyboard and/or ethernet the same way (i.e. flipping some option in the standard kernel options, then re-compiling it)

I think I ignored the problems of premature disconnection, but that is probably unwise. If I am in a cafe, then there is a;ways the possibility that someone will close the netbook. Hopefully it will go through a closedown procedure, but there is probably no guarantee. of this. What could be more likely is that I am on battery, and thus am liable to run out of power, that gives an instant shutdown. I had a nightmare half day trying to fix the notebook when Windows started a mandatory update when I was running out of power. Not only did I have to waste time trying ( unsuccessfully ) to negate the effects of the so-called update, but I had to sort out the disk corruption as well.

Thasnks for reminding me of this risk. It isn't so serious with Bitcoin Core, as it seems to be fairly well written, and of course there are umpteen backups of the blockchain around the net.  It's still worth taking measures to avoid having to download the whole blockchain again though.

If the device is removed or the computer crashes without unmounting an EXT4/3/2 filesystem first, it gets corrupted and you have to run fsck on it to repair the partition.

NTFS on Linux using FUSE kernel driver also gets bad, unremovable, inodes (file handles) if you're copying files and you unplug it in the middle of the transfer, but it doesn't destroy the drive.

XFS is much more robust and doesn't get corrupted when things like that happen. I've been through, what, dozens of power outages and I never had a single error on my Red Hat's XFS drive.

I'm not sure how EXT4 would handle an accidental disconnect when you unplug the USB cable during writes. Other than that, it's a solid file system.

I've used a netbook (with slightly slower Atom CPU) with a backup blockchain in the past, but it took far too long to catch up when I turned it on, so I gave up.

I can put 8Gb in the notebook, but I really wanted to see if I could use ther netbook as a background machine. I can leave it running for 3 or 4 hours with just core active, and I was hooping I could tweak it to handle that.

The alternative would be a Raspberry Pi powered by a solar battery pack, but I was thinking of trying that as phase 4 or 5 rather than an immediate project.

You'll definitely need to upgrade the RAM on that netbook if you are going to use GNOME. On my system it regularly leaks an extra gigabyte or so if you leave it running for an extended period of time.

GNOME will run, but just barely, with 2GB. You need to set aside memory for bitcoin core. I would've suggested you add a second DDR2/3 2GB memory stick in it, but you mentioned the memory is non-expendable.

IMO, the RAM is a more important issue than the file system. 2GB will struggle running GNOME and bitcoin core at the same time (and Linux swap space is notoriously slow, at least from my experience).

If you don't mind editing config files to tweak the desktop, icewm is lighter than XFCE and LXDE and has a taskbar and menu at the bottom. Only ~256MB RAM usage. And the kernel uses a similar amount which puts the total usage at ~512MB.

I'm in a bit of a mess at the moment, and I've got a lot to sort out. The Kung 'flu isn't helping either - hence the move further towards off grid and battery power. I did tell it not to bother with Libre office, but it decided to install it anyway. Also, I use Thunderbird, but it installed Evolution as the default. I'll probably have a look at that for some of my most used addresses. I've got a load of hard drives, and I've put some of those into cases, so I can move a lot of my data between machines.

I wanted to put a bit of thought into the Blockchain storage, both for speed and longevity/reliability of the drive. I'm not bothered about Windows compatibility. Sometimes I run the netbook tethered to my mobile using a USB connection, but I might experiment with a blutooth connection to access the data between machines - but not this month.

I tried the KDE/plasma option in openSUSE, but I had issues with that, and gave up trying to correct it. Gnome worked straight out of the box, and as this is only a backup experiment, I thought I would run with that. I'll probably continue to do website building and image processing on the Notebook. If I continue to feel comfortable with openSUSE, then I'll double the RAM in the notebook, and replace the HDD with a SSD. The HDD on the notebook has a bad spot, and there may be a problem with the graphics controller. I'll do some testing, and at worst, I'll just buy another notebook if I have to.

I  bought it as a cheap backup about 3 years ago. It was new, but in a clearance because it is in a light blue case, the alternative was pink.
It lasts for about 9 hours on the battery, and came with Windows 10, and i promptly overwrote that with Ubuntu. The latest Ubuntu doesn't recognise the keyboard, hence the move to openSUSE. I'm reviewing all of my computing kit at the moment, and I decided to keep this as a backup - backup means an alternative after running out of battery on my main notebook. Also, I thought the netbook would be handy if I wanted to check a Bitcoin payment via a face to face meeting, and for smaller transactions into a current wallet. I will move to hardware wallets for long term storage, and I need to make a decision on my main ( more powerful ) notebook. I will be installing a comms centre in the van, and I will base this on one of the new single board computers with 12 volt or USB power..

The blockchain and wallet I want to use is over 2 years out of date, and I hoped to copy it onto the SSD and sync it as a background job whilst I am in a cafe. I'm really taken with this digital nomad life, and if I could power a pc from my wood burner, then I would love that.

There is really no "best" for this. It's going to be a "less bad" option. EXT4 will work fine.

With that being said:
There will probably be a lot of people who will disagree with me on this but the netbooks lack of RAM is going to be a big big issue.
Every time you are offline and it has to catch up that and with the initial sync 2GB RAM and the low power of the CPU Will really kill you.

Are you using it because of weight and you want to carry less or some other reason?
You might have mentioned it in some other post.

-Dave

I'm slowly putting together a system to run a Bitcoin node on an old HP Netbook. The machine spec is

Intel Celeron N3060 @ 1.60 GHz with 2Gb of Ram - the ram is not expandable, and the chip can't be changed.
The OS is the latest openSUSE Leap with Gnome. WiFi and Bluetooth are available, but no Ethernet
USB 2 and 3 and SD cards are available for extra storage. The internal drive only has about 28Gb available after software installation.
Proposed external drive - 2Tb Samsung EVO 860 in a Sabrient case to allow connection via USB 3

I propose to repartition the external drive and change the file system. At the moment I am considering a single EXT4 partition. The current OS uses BTRFS as its default, but I suspect that the older EXT4 system will place fewer demands on the netbook, and hopefully I can use it for surfing whilst sync'ing the blockchain. Net connection will be via public WiFi, and the system will often have to rely on the Netbook battery for power.

Whilst many may not consider this as an optimum setup, it is useful as part of my experiments as a digital nomad.
My question is - Do you think that a single EXT4 partition is the best choice under the circumstances?