Bitcoin + Electrum server + Block explorer under Windows (with WSL and Debian) - page 2.

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: MixMAx123 on October 25, 2021, 11:59:12 AM

I'm trying this on Ubuntu 18.04.3 LTS.

Do you have something older than Win 10? I'm asking because I see on the Windows store Ubuntu 20.04 LTS...
However, I'm not surprised, I had my own share of surprises with Ubuntu.

Unfortunately I am also far from good at Linux; this tutorial was made simply because I didn't find any reasonably good/complete one.
For your problems I have only one possible advice, if you didn't do this at start. Update your binaries:

Quote from: NeuroticFish on October 24, 2021, 05:35:12 AM

Code:

sudo apt update
sudo apt dist-upgrade

Also, I'm a bit surprised. If you are not great at Linux, why didn't you try with Debian as I did?

Quote from: MixMAx123 on October 25, 2021, 11:59:12 AM

Another note:
I do not know if that's normal, but if I want to install something with "sudo apt install xxx", this almost always fails with various errors.

I don't find it normal. But the environment variables may be different between your user and the super user (I hope that I didn't say something silly) and that could be one reason.
I'm not sure that it will help, but you can try to do sudo su, do the things without sudo in front and when you're done do exit.

MixMAx123

full member

Activity: 161

Merit: 168

As threatened, I try to imitate this now.
At this point, the first problems occur:

Code:

sudo apt install cargo

error: "Problems can not be corrected, they have retained defective packages "
https://de.share-your-photo.com/img/d80a1b2e3a.png

annotation
I'm trying this on Ubuntu 18.04.3 LTS.
I reinstalled Ubuntu yesterday. BitcoinCore now runs properly. No further software is installed.
I am an absolute Linux failure.
Things that you think for yourself, I have probably done wrong.
So put the worst case in advance.

Yesterday I also made a backup from the system. Every time something is defective, I simply set up the backup again.
And believe me, that happens very often!
So because of the security, I do not worry, I repeat and test as long as it goes.

Another note:
I do not know if that's normal, but if I want to install something with "sudo apt install xxx", this almost always fails with various errors.

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: ABCbits on October 25, 2021, 06:08:51 AM

This script will fail if you run it from different location (location when you run nano to create the script). You should replace it with absolute path.

In my defense

, if it's done as I expect, the .sh files are created in user home folder, the same folder the user gets in when he starts the Linux console, so he should not need to run that from somewhere else.
Also I tried to avoid having path containing the user name, but I've learned meanwhile about $HOME.

But you're right, users may do things in a way I don't expect Cheesy

Fixed it. Thank you.

...And if I started this, I've also ensured the paths for the .sh files are exactly where I expect them to be. So double thanks Wink

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: NeuroticFish on October 24, 2021, 06:11:05 AM

And inside nano editor:

Code:

cd ./electrs/target/release
./electrs

CTRL-X and Save

This script will fail if you run it from different location (location when you run nano to create the script). You should replace it with absolute path.

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: DaveF on October 24, 2021, 10:14:22 AM

In theory, Windows should cleanly save the linux subsystem to disk, and then reboot.
In reality, at times something locks and it's not done cleanly. And bad things can happen....

Thanks for the heads up. I try to always exit the consoles as gracefully as I can (CTRL-C then exit) before leaving windows do shutdown or restart. I hope that I'm on the safe side.

Quote from: DireWolfM14 on October 24, 2021, 10:04:06 PM

Good job, I hope you had fun. You're on the right path, and you'll pick up tricks that work for you as you go along. WSL is a great way for Windows users to immerse themselves in Linux.

Thank you for the kind words.

Quote from: DireWolfM14 on October 24, 2021, 10:04:06 PM

I prefer to run my node on dedicated hardware which I can leave running full time. All my services are started on boot by systemd, so I just need to make sure the machine is powered on. Unlike Linux on a physical or virtual machine, WSL doesn't have a system startup daemon like systemd. It shouldn't be confused with a virtual machine, it's a virtual environment.

I may get there too, but for now I cannot: even the oldest working laptop in the house is being used by the kids for online classes and when I bought a RasPi for my needs I was not smart enough and bought a weak one (Zero W), from what I've read it's too weak for this job.
Maybe I'll get to buy in the near future a RasPi 4 and a 1-2 TB HDD (I find SSD overpriced for the job at this size), maybe I open a new topic on that, since Black Friday is pretty close.

Quote from: DireWolfM14 on October 24, 2021, 10:04:06 PM

It's bugging me that you are unable to install Ubuntu.

I was able to install Ubuntu. I had other problems there:

it didn't find for me librocksdb-dev=6.11.4-3
it failed in creating files/folders onto my Windows partition for electrs data

Of course, it was my first install and the problems may have been caused by something I didn't do right or I didn't do at all. Or my antivirus blocking something I forgot to unblock.
Maybe I was just luckier with Debian. However, this worked, I've noted down this and that (for the case I have to do "backtracking" until I get it right) and.. this is what I have.
If people want to try it out on any other distro, be my guest, really Cheesy

Quote from: DireWolfM14 on October 24, 2021, 10:04:06 PM

I noticed that you used the Control Panel gui app to enable WSL, did also enable virtualization?

If you mean Hyper-V, no, it's not enabled and I don't intend to unless I really have to.
I've understood that it may interfere badly with VBox, which I still need now and then for my own tests (for simulating a fresh Windows or for running suspect apps) and my CPU is i7 so it has some in-built features, I think.

Quote from: DireWolfM14 on October 24, 2021, 10:04:06 PM

I've always installed WSL manually, you can find the installation instructions here (don't forget to start powershell as administrator): https://docs.microsoft.com/en-us/windows/wsl/install-manual

Also, you can use powershell to interact with your WSL instances and, run them with alternate options. For starters:

Code:

wsl --help

Although I work on Windows since Windows 95 (even 3.1, but that was for too short time), I have little experience with Power Shell. I find it too... Linux like and I didn't really need it; most of the time the normal command prompt or apps done by myself could do all I've needed.

Quote from: MixMAx123 on October 24, 2021, 11:39:16 AM

Great instructions! This is something that many people can use well, thank you for your effort!
I will try it with your instructions, sobal my system runs correctly again.

Thank you! Please let me know if I've missed or didn't explain good enough something.

DireWolfM14

copper member

Activity: 2296

Merit: 4460

Join the world-leading crypto sportsbook NOW!

Good job, I hope you had fun. You're on the right path, and you'll pick up tricks that work for you as you go along. WSL is a great way for Windows users to immerse themselves in Linux.

I prefer to run my node on dedicated hardware which I can leave running full time. All my services are started on boot by systemd, so I just need to make sure the machine is powered on. Unlike Linux on a physical or virtual machine, WSL doesn't have a system startup daemon like systemd. It shouldn't be confused with a virtual machine, it's a virtual environment.

It's bugging me that you are unable to install Ubuntu. I noticed that you used the Control Panel gui app to enable WSL, did also enable virtualization? I've always installed WSL manually, you can find the installation instructions here (don't forget to start powershell as administrator): https://docs.microsoft.com/en-us/windows/wsl/install-manual

Also, you can use powershell to interact with your WSL instances and, run them with alternate options. For starters:

Code:

wsl --help

MixMAx123

full member

Activity: 161

Merit: 168

Great instructions! This is something that many people can use well, thank you for your effort!
I will try it with your instructions, sobal my system runs correctly again.

DaveF

legendary

Activity: 3500

Merit: 6320

Crypto Swap Exchange

Side note.
Disable automatic updates on Windows, and install the Windows updates manually.
When done then shutdown Debian
Reboot Windows and bring up Debian.

In theory, Windows should cleanly save the linux subsystem to disk, and then reboot.
In reality, at times something locks and it's not done cleanly. And bad things can happen....

-Dave

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Fine tuning and other info.

Electrum

From this moment onwards, you'll be able to use Electrum with your own server.
Although I use Electrum on the same computer, I use the portable one (yeah, I'm lazy)
I've made a batch file which I run from now on instead of Electrum's exe and it looks like this:

Code: (run-electrum.bat)

electrum-4.1.5-portable.exe --oneserver --server 127.0.0.1:50001:t

Linux batch files and others

Open a new Debian console. This time we will create 2 batch files under Linux, because we are lazy. Just now we will make them directly under Linux:

Code:

nano $HOME/run_electrs.sh

And inside nano editor:

Code: (run_electrs.sh)

cd $HOME/electrs/target/release
./electrs

CTRL-X and Save

again

Code:

nano $HOME/run_explorer.sh

Code: (run_explorer.sh)

cd $HOME/btc-rpc-explorer
npm start

CTRL-X and Save

two more lines to execute:

Code:

chmod +x $HOME/run_electrs.sh
chmod +x $HOME/run_explorer.sh

Now, when you want to exit Electrs or BTC RPC Explorer, you press CTRL+C in those consoles, then exit.
When you want to run them, you start ./run_electrs.sh in one console then ./run_explorer.exe in another

Web page for block explorer

http://localhost:3002/exp/

Thank you

HCP for setting me on the right track
ETFbitcoin for hinting the correct block explorer
DaveF and NotATether for making me understand VirtualBox is not such a great idea
DireWolfM14 for trying to help me with auto-starting services, although I still don't know how to do that
everybody active in the Dev & Tech from whom I've learned a lot
and obviously the teams implementing all the great software I've installed and using now

It may be better to have a way to start everything from one click but I don't know how.
I've noticed that if I started bitcoind and immediately electrs, the second has failed, so now I just check when bitcoid is synced and then start electrs, I wait for electrs show it's ready and then start blockchain explorer and/or electrum. It's not optimal, but it's OK for my needs and... this is how much I've manged to do.

Edit 1: corrected/improved wording
Edit 2: corrected the .sh files per ETFbitcoin suggestion and ensured their paths too.

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

BTC RPC Explorer (from https://github.com/janoside/btc-rpc-explorer)

I expect this to be installed after Electrs is completely synced, and it's running.
Open a new Debian (R-Click on the Debian rectangle on the task bar and select Debian) and let's go.
Again some Linux commands to be ran:

Code:

sudo su
curl -fsSL https://deb.nodesource.com/setup_lts.x | bash -
apt-get install -y nodejs
exit
git clone https://github.com/janoside/btc-rpc-explorer
cd btc-rpc-explorer
npm install

Now I've made, again under Windows, and again on my E: drive a file called .env with the content based on the official sample:

Code: (.env)

BTCEXP_BASEURL=/exp/
BTCEXP_BITCOIND_HOST=127.0.0.1
BTCEXP_BITCOIND_PORT=8332
BTCEXP_BITCOIND_COOKIE=/mnt/X/BitcoinDataDir/.mycookie
BTCEXP_BITCOIND_RPC_TIMEOUT=5000
BTCEXP_ADDRESS_API=electrum
BTCEXP_ELECTRUM_SERVERS=tcp://127.0.0.1:50001
BTCEXP_ELECTRUM_TXINDEX=true
BTCEXP_SLOW_DEVICE_MODE=false
BTCEXP_PRIVACY_MODE=true
BTCEXP_NO_RATES=true

Now copy .env into the Linux folder and start

Code:

cp /mnt/e/.env .
npm start

Edit 2022-04-17: If, at a later point, you might want to update BTC RPC Explorer, you can take a look here: https://bitcointalksearch.org/topic/m.59888845

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Electrs (from https://github.com/romanz/electrs)

The steps are based on the official install page and my struggle.
Some of the tools (curl, git) were missing and I had to install them myself.

I had to do all the following commands, one by one:

Code:

sudo apt install curl
sudo curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

sudo apt install cargo

sudo apt update
sudo apt install clang cmake build-essential

sudo apt install librocksdb-dev=6.11.4-3

sudo apt install git
git clone https://github.com/romanz/electrs
cd electrs
ROCKSDB_INCLUDE_DIR=/usr/include ROCKSDB_LIB_DIR=/usr/lib cargo build --locked --release

[later note: if sudo apt install librocksdb-dev=6.11.4-3 fails on Debian you can try sudo apt install librocksdb-dev=6.11.4*]
[later note: if sudo apt install librocksdb-dev=6.11.4-3 still fails (probably on Ubuntu), read this post for workaround]

I've edited under Windows (as e:\electrs.toml, keep this path in mind, you'll have to copy it to Linux) the config file; it was based on the config from the official example and mine looks like this (I also kept only the useful lines)

Code: (electrs.toml)

cookie_file = "/mnt/x/BitcoinDataDir/.mycookie"
daemon_dir = "/mnt/x/BitcoinDataDir"
daemon_rpc_addr = "127.0.0.1:8332"
daemon_p2p_addr = "127.0.0.1:8333"
db_dir = "/mnt/y/ElectrsData"
network = "bitcoin"
electrum_rpc_addr = "127.0.0.1:50001"
#verbose = 2 -> obsolete, was working with version 0.9.0, no longer works in 0.9.3
#use log-filters for version 0.9.3+
log_filters = "INFO"
server_banner = "Electrs @ NeuroticFish, yay!"

[Edit: version 0.9.3 doesn't like verbose in the config, using log-filters instead.]

This means that the Electrs data (up to 70 GB, according to some docs, but right now that folder on my computer is only 30 GB) will be on drive Y:, as Y:\ElectrsData
Under Debian, please create that folder:

Code:

mkdir /mnt/y/ElectrsData

Go into the correct folder and copy the config (you've saved it, right?). And don't forget the ending space and point

Code:

cd target/release
cp /mnt/e/electrs.toml .

Start electrs and let it sync, it'll take some hours in the first run.

Code:

./electrs

At a later point it will show the sweet:

Code:

[2021-10-16T13:31:39.392Z INFO  electrs::server] serving Electrum RPC on 127.0.0.1:50001

Edit 2022-04-17: If, at a later point, you might want to update Electrs, you can take a look here: https://bitcointalksearch.org/topic/m.59888827

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

WSL on Windows, Debian

Under windows (10), go to Start menu and type Turn Windows features on or off
Near the end of the list there's Windows Subsystem for Linux. Set that checked, OK, let it install/restart and so on.

When that's done, Start menu and type Run, and in the run box type https://aka.ms/wslstore
There install Debian (I had troubles with Ubuntu, so Debian it is).
Keep in mind that Debian and what's on it will take some 4.5 GB on the Windows drive. [Edit 2022-02-17: if the size on windows partition is a problem like it was for me, here's the solution.]

Start Debian, create an user and password for yourself and we can continue the real deal.

If you have an aggressive antivirus like myself (e.g. Comodo), it may be a good idea to disable the auto containment component. And don't forget to re-renable it after all the install is done.
I didn't do this at start and I've lost a lot of time because this or that was blocked from running under WSL.

It worth mentioning (thank you HCP, you have spared me a lot of time) that for each Windows drive, under Debian you'll have /mnt//your path on that drive

Update the OS:

Code:

sudo apt update
sudo apt dist-upgrade

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

This may be useful to others too, since I've spent quite a lot of time figuring out how to have everything working on a machine that runs a lot, but it's on Windows.

This walkthough is meant to show how to install an Electrum server (electrs) and a block explorer (BTC RPC Explorer) on (more or less) Windows. It's "more or less" because I actually use WSL (windows subsystem for linux) to get a Debian run under Windows.

If you have only one wallet and you don't want your block explorer, Electrum Personal Server may be enough for you. Start reading the other thread from here, I have 2 posts on that topic.

Electrs is not restricted to one wallet and also has useful data for the block explorer, hence this more generic setup was what I was looking for.

I will post all the details I remember and I will post also my config files too. Improvements are welcome, since I'm not really a Linux guy and, as said, most of this setup is still under Linux.

Bitcoin

Bitcoin is easy to install under Windows, it just runs, make sure your data directory is set somewhere with plenty of space, I use an external drive (X:\BitcoinDataDir)
I've created in X:\BitcoinDataDir a file bitcoin.conf with the following content:

Code: (bitcoin.conf)

txindex=1
server=1
rpcbind=127.0.0.1
rpcallow=127.0.0.1
rpccookiefile=.mycookie

Some extras:
1. If you want to not be too generous with the network, you may be tempted to use maxconnections in your config. Although I've read you should put minimum value 12, it's not enough. When I've tried that the block explorer may not be able to connect. With a value like 25 it was OK.
2. Make sure that if you use maxuploadtarget in your config, you also add [email protected]

Of course, you can just keep it simple and don't add those extras.

I've also made 2 batch files because I'm lazy. Both are in the same folder ad bitcoind.exe and I've made for both shortcuts onto desktop.

Code: (start_bitcoin.bat )

bitcoind -datadir=X:\BitcoinDataDir

Code: (stop_bitcoin.bat)

bitcoin-cli.exe -datadir=X:\BitcoinDataDir stop

As a note, keep in mind that bitcoind will "discuss" with electrs and the block explorer only after the console shows progress=1.000000
And yes, you should leave at least bitcoind run as much as possible.

LE: If you prefer to use all this with RPC user/password instead of RPC cookie file, the configs are adapted in this later post.

Topic: Bitcoin + Electrum server + Block explorer under Windows (with WSL and Debian) - page 2. (Read 930 times)