Installing my own Electrum server | Bitcointalksearch.org

Cricktor

hero member

Activity: 714

Merit: 1010

Crypto Swap Exchange

Quote from: vv181 on April 26, 2022, 01:20:15 PM

Quote from: Cricktor on April 25, 2022, 07:43:39 AM

... Syncing Fulcrum took considerably longer than electrs but this is needed more or less only once after initial install of Fulcrum.

How long does your initial sync time process take? Do you also use the --fast-sync feature?

I'm also using RPI 4B 8GB + SSD, plainly running bitcoind and Fulcrum, the --fast-sync value of 5000 and 400 made the app OOM[1]. But with the value of 512, it works just fine. And the process took less than 48H.
...
[1] https://github.com/cculianu/Fulcrum/issues/96#issuecomment-1099346565

Sorry, I don't remember it exactly, but it took maybe around 4-7 days, likely on the longer than shorter side of my guestimate. I remember that I was a bit puzzled how long it took. AFAIR electrs synched in something like 24h on my RaspiBlitz node while with Fulcrum v1.6.0 it took something around 6x longer to build the index.
My RaspiBlitz node had running at least: Bitcoin Core, LND, Bitcoinexplorer, mempool.space, Fulcrum (so there were some memory eaters nibbling on the 8GB RAM)

Yes, I had --fast-sync in place, first with 2048 and likely changed this to 1024 as memory consumption seemed too high with the larger value for my RPi, though it didn't ever run OOM.

I used some of these settings in my fulcrum.conf:

Code:

bitcoind_timeout = 600
worker_threads = 1
db_mem=1024
# for 8GB RAM
#db_max_open_files=500
db_max_open_files=300
#fast-sync = 2048
fast-sync = 1024
...
# tweaks for huge address histories
max_history=500000
max_buffer=25000000
subnets_to_exclude_from_per_ip_limits = 127.0.0.1/32, ::1/128, 192.168.178.
txhash_cache = 256
#workqueue = 15000 (default)

vv181

legendary

Activity: 1932

Merit: 1273

Quote from: Cricktor on April 25, 2022, 07:43:39 AM

I run my Bitcoin Core and Lightning node and my personal Electrum Server on a Raspi 4B with 8GB RAM using RaspiBlitz (I'm not a fan of Umbrel).~ Syncing Fulcrum took considerably longer than electrs but this is needed more or less only once after initial install of Fulcrum.

How long does your initial sync time process take? Do you also use the --fast-sync feature?

I'm also using RPI 4B 8GB + SSD, plainly running bitcoind and Fulcrum, the --fast-sync value of 5000 and 400 made the app OOM[1]. But with the value of 512, it works just fine. And the process took less than 48H.

Code:

Apr 15 02:15:26 raspberrypi Fulcrum[110051]: Processed height: 1000, 0.1%, 1097.6 blocks/sec, 1118.4 txs/sec, 1146.9 addrs/sec
Apr 16 22:32:36 raspberrypi Fulcrum[110051]: Processed height: 731000, 99.9%, 1.77 blocks/sec, 3267.7 txs/sec, 16975.7 addrs/sec

[1] https://github.com/cculianu/Fulcrum/issues/96#issuecomment-1099346565

PawGo

legendary

Activity: 952

Merit: 1386

Quote from: Cricktor on April 26, 2022, 04:27:55 AM

My own wallets don't have large histories. From time to time I like to do some blockchain analysis or kind of "investigations", nothing professional. With electrs when you stumble over Bitcoin addresses with large histories you quickly run into trouble. You can tweak electrs to some extend, but to me it seems it's not well suited for this OR I never was able to figure it out how to do it with electrs.

I must agree, but as I did not investigate btc-rpc-explorer, I do not know whos fault it is.
Sometimes it does not display balance at all, sometimes not during first check (after page refresh it is ok).

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: Cricktor on April 26, 2022, 04:27:55 AM

My own wallets don't have large histories. From time to time I like to do some blockchain analysis or kind of "investigations", nothing professional. With electrs when you stumble over Bitcoin addresses with large histories you quickly run into trouble. You can tweak electrs to some extend, but to me it seems it's not well suited for this OR I never was able to figure it out how to do it with electrs.

Although I do look for transactions and history now and then, I usually^* have txindex=1 in Bitcoin Core and I look at a block explorer - BTC RPC Explorer (the only one I could make to work well on top of electrs), but all went well, with no surprises.

^* Unfortunately because of the rather small HDD I'm using, I go right now without txindex until I decide what to do next/go for bigger HDD.

Cricktor

hero member

Activity: 714

Merit: 1010

Crypto Swap Exchange

My own wallets don't have large histories. From time to time I like to do some blockchain analysis or kind of "investigations", nothing professional. With electrs when you stumble over Bitcoin addresses with large histories you quickly run into trouble. You can tweak electrs to some extend, but to me it seems it's not well suited for this OR I never was able to figure it out how to do it with electrs.

I favour data sovereignity, so all I do around Bitcoin and blockchain should be done with my own local data, therefore my own Bitcoin and Lightning node, my own Electrum server, my own local Bitcoinexplorer and local mempool.space instance, ... other Bitcoin related tools. I stumbled over Fulcrum when it was integrated as an experimental feature of RaspiBlitz. I studied the Github repo of Fulcrum and did some internet research on it. I like the fact that it addresses speed and that you can configure it to your needs even on such low-power platforms as a Raspi.

I don't have enough coding knowledge to do a proper audit, so I had to put some trust and faith into this project. Call me reckless...

I have not been disappointed so far, on the contrary!

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: Cricktor on April 25, 2022, 07:43:39 AM

I run my Bitcoin Core and Lightning node and my personal Electrum Server on a Raspi 4B with 8GB RAM using RaspiBlitz (I'm not a fan of Umbrel). I ditched electrs and switched to Fulcrum for its speed and ability to deal with large address histories far better than electrs could. Syncing Fulcrum took considerably longer than electrs but this is needed more or less only once after initial install of Fulcrum.

RaspiBlitz runs smoothly with Fulcrum, though you need to do some tweaking as Fulcrum Electrum server isn't yet nicely integrated into RaspiBlitz. On average the Raspi consumes only ~5-6W power. I'm very happy with Fulcrum!

It's the first actual-use review I see for Fulcrum, although I've searched for such posts ~6+ months ago.
Thank you for that!

The point is that's the only Electrum server that seems to have Windows binaries. Maybe Windows users will start looking into this one too.

Cricktor

hero member

Activity: 714

Merit: 1010

Crypto Swap Exchange

I run my Bitcoin Core and Lightning node and my personal Electrum Server on a Raspi 4B with 8GB RAM using RaspiBlitz (I'm not a fan of Umbrel). I ditched electrs and switched to Fulcrum for its speed and ability to deal with large address histories far better than electrs could. Syncing Fulcrum took considerably longer than electrs but this is needed more or less only once after initial install of Fulcrum.

RaspiBlitz runs smoothly with Fulcrum, though you need to do some tweaking as Fulcrum Electrum server isn't yet nicely integrated into RaspiBlitz. On average the Raspi consumes only ~5-6W power. I'm very happy with Fulcrum!

nemomaximus

newbie

Activity: 10

Merit: 18

I was able to figure out the problem

The default RPC port 8000 was already in use. Changed it to 8001 and then it started to work.

nemomaximus

newbie

Activity: 10

Merit: 18

I seem to have the ElectrumX service up and running and it is synchronising

I want to make a RPC call to see what is going on but as the documentation says that can only be done when synchronisation is finished except for LocalRPC.

So I try this (from the local server):

Code:

electrumx_rpc getinfo

but that gives me the following error:

Code:

Traceback (most recent call last):
File "/usr/local/bin/electrumx_rpc", line 120, in send_request
result = await session.send_request(method, args)
File "/usr/local/lib/python3.9/dist-packages/aiorpcx/session.py", line 540, in send_request
return await self._send_concurrent(message, future, 1)
File "/usr/local/lib/python3.9/dist-packages/aiorpcx/session.py", line 512, in _send_concurrent
return await future
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/electrumx_rpc", line 144, in
main()
File "/usr/local/bin/electrumx_rpc", line 139, in main
code = asyncio.run(send_request())
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
asyncio.exceptions.CancelledError

What is wrong?

Also, any suggestion of a good web based (or IOS) dashboard that I can use to review the status of my ElectrumX server?

PawGo

legendary

Activity: 952

Merit: 1386

Quote from: ABCbits on April 07, 2022, 06:37:44 AM

Quote from: NeuroticFish on April 07, 2022, 05:00:59 AM

Quote from: ABCbits on April 07, 2022, 04:49:21 AM

But currently ElectrumX is most popular version.

Imho ElectrumX is the most popular for "public" use. For local use it may be a bit of overkill. But actually that's only a feeling, I don't know that for sure.

I can understand why you think it's overkill for personal usage. But i'm not fan of lighter alternative Electrum Personal Server[1] since you need to enter each master public key (xpub, ypub or zpub) or address manually. electrs[2] might be best option for personal usage, but i never tried it.

I am now on electrs (with WSL/Debian), all is fine! DB synchronization took some time, for sure not 15 minutes - I cannot tell how much, because I left computer running and just checked a few hours later. There was no progress, no indication that sync is finished... I was only able to see what is the last processed block using Prometheus.
DB takes now around 35Gb.

But! finally you become independent and using rpc browser (like https://www.npmjs.com/package/btc-rpc-explorer) you may check any address you want.
EPS requires you to predefine list of addresses you want to browse/monitor (you must set static addresses or x/z/pub).

And, finally, I was able to connect Trezor Suite to electrs - I was not able to do the same with EPS.

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: ABCbits on April 07, 2022, 06:37:44 AM

I can understand why you think it's overkill for personal usage. But i'm not fan of lighter alternative Electrum Personal Server[1] since you need to enter each master public key (xpub, ypub or zpub) or address manually. electrs[2] might be best option for personal usage, but i never tried it.

I've used EPS for... one day or so. It convinced me fast to look for something else. So yes, we're basically on the same page with which server is for what.

Quote from: NeuroticFish on April 07, 2022, 05:00:59 AM

If you plan to try Docker, AFAIK you could just install Docker for Windows so you don't need to setup WSL.

Docker may interfere badly with VirtualBox, so, even if I no longer use VirtualBox as much as in the past, I still try to avoid (Docker) for as long as I can.

vv181

legendary

Activity: 1932

Merit: 1273

Quote from: ABCbits on April 07, 2022, 06:37:44 AM

Quote from: NeuroticFish on April 07, 2022, 05:00:59 AM

Quote from: ABCbits on April 07, 2022, 04:49:21 AM

But currently ElectrumX is most popular version.

Imho ElectrumX is the most popular for "public" use. For local use it may be a bit of overkill. But actually that's only a feeling, I don't know that for sure.

I can understand why you think it's overkill for personal usage. But i'm not fan of lighter alternative Electrum Personal Server[1] since you need to enter each master public key (xpub, ypub or zpub) or address manually. electrs[2] might be best option for personal usage, but i never tried it.

Got interested to seek how the Electrum server performs and found out this article: sparrowwallet: server performance and J. Lopp: Electrum server performance report. The former is a recently dated test compared to the latter.

And yea as far as I understand, choosing which implementation suits the best is tied to how's the hardware capability and what the intended use of the server. Personally, so far I've been only tried using Electrs on my node, and to this date, there haven't been any major issues and fits my usage.

In rough summary, the test conducted by Sparrowwallet is as follows:

Tested on Raspberry Pi 4 8GB, Ubuntu 21.10 x64, and 1TB external USB SSD.

Quote from: https://sparrowwallet.com/docs/server-performance.html#results

Results

Test 1: Initial load
Tl;dr: Fulcrum is 22x faster than ElectrumX, ~300x faster than Electrs

Test 2: Wallet refresh
Tl;dr: Fulcrum is 8x faster than ElectrumX, 1.5x faster than Electrs

Quote from: https://sparrowwallet.com/docs/server-performance.html#conclusion

Conclusion

We can be relatively sure of two trends continuing into the next few years - wallet depths will increase, and storage costs will decrease.

For this reason, Fulcrum emerges as a clear winner in this benchmark. Although it takes slightly longer to index than Electrs, once that indexing is complete the benefits of a scalable and performant server will greatly outweigh this once-off cost, not only in query speed and reliability but in prolonging hardware lifespan.

Although Electrs may have some application when disk space is very limited and wallet depths are small, Fulcrum is recommended as the ideal server to pair with Sparrow.

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: NeuroticFish on April 07, 2022, 05:00:59 AM

Quote from: ABCbits on April 07, 2022, 04:49:21 AM

But currently ElectrumX is most popular version.

Imho ElectrumX is the most popular for "public" use. For local use it may be a bit of overkill. But actually that's only a feeling, I don't know that for sure.

I can understand why you think it's overkill for personal usage. But i'm not fan of lighter alternative Electrum Personal Server[1] since you need to enter each master public key (xpub, ypub or zpub) or address manually. electrs[2] might be best option for personal usage, but i never tried it.

Quote from: NeuroticFish on April 07, 2022, 05:00:59 AM

Quote from: ABCbits on April 07, 2022, 04:49:21 AM

Yes, if you want to install from source code, install some dependency and configure things manually[1]. If you want easy time, consider using it's Docker image[2]. Some time ago i made the short guide for using ElectrumX Docker image for Linux-based OS[3].

[1] https://electrumx-spesmilo.readthedocs.io/en/latest/HOWTO.html#howto
[2] https://github.com/lukechilds/docker-electrumx
[3] https://bitcointalksearch.org/topic/m.56915327

I may also give it a go under WSL when I get a bit more free time. Not necessarily because my current setup would be bad, it's just that I like a bit more the interface of mempool.space than of the block explorer I use now. So I'll bookmark myself these links Wink

If you plan to try Docker, AFAIK you could just install Docker for Windows so you don't need to setup WSL.

[1] https://github.com/chris-belcher/electrum-personal-server
[2] https://github.com/romanz/electrs

nemomaximus

newbie

Activity: 10

Merit: 18

Many thanks! I really appreciate all information! Time to dig into it and see if I can get it up and running

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: ABCbits on April 07, 2022, 04:49:21 AM

But currently ElectrumX is most popular version.

Imho ElectrumX is the most popular for "public" use. For local use it may be a bit of overkill. But actually that's only a feeling, I don't know that for sure.

Quote from: ABCbits on April 07, 2022, 04:49:21 AM

Yes, if you want to install from source code, install some dependency and configure things manually[1]. If you want easy time, consider using it's Docker image[2]. Some time ago i made the short guide for using ElectrumX Docker image for Linux-based OS[3].

[1] https://electrumx-spesmilo.readthedocs.io/en/latest/HOWTO.html#howto
[2] https://github.com/lukechilds/docker-electrumx
[3] https://bitcointalksearch.org/topic/m.56915327

I may also give it a go under WSL when I get a bit more free time. Not necessarily because my current setup would be bad, it's just that I like a bit more the interface of mempool.space than of the block explorer I use now. So I'll bookmark myself these links Wink

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: NeuroticFish on April 07, 2022, 04:03:13 AM

Quote from: nemomaximus on April 07, 2022, 03:40:20 AM

Where do I download the "best" version from?

For Electrum server it's debatable what's a "best version". Imho any version that works for you it's just fine.

But currently ElectrumX is most popular version.

Quote from: nemomaximus on April 07, 2022, 04:14:54 AM

Is this the correct place to download the ElectrumX server: https://github.com/spesmilo/electrumx

Yes, if you want to install from source code, install some dependency and configure things manually[1]. If you want easy time, consider using it's Docker image[2]. Some time ago i made the short guide for using ElectrumX Docker image for Linux-based OS[3].

[1] https://electrumx-spesmilo.readthedocs.io/en/latest/HOWTO.html#howto
[2] https://github.com/lukechilds/docker-electrumx
[3] https://bitcointalksearch.org/topic/m.56915327

nemomaximus

newbie

Activity: 10

Merit: 18

Thanks!

It is the server I am looking for at this time.

Is this the correct place to download the ElectrumX server: https://github.com/spesmilo/electrumx

Please share the download places to the other flavors of ElectrumX if possible. Don't like to guess based on searches

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: nemomaximus on April 07, 2022, 03:40:20 AM

I have been able to install the Bitcoin Core node (version 22.0) on my own server. But now I want to continue with installing an Electrum server that connects to my node.

Good. Keep in mind that Bitcoin Core needs to download the whole blockchain (425+GB) and you'll probably also store it, so make sure you have Bitcoin Core set correctly, you have disk space and patience to have it synchronized.

Quote from: nemomaximus on April 07, 2022, 03:40:20 AM

What is the difference between Electrum and ElectrumX?

Electrum is the client software. ElectrumX is one of the popular servers for Electrum.
An Electrum Server (ElectrumX, Electrs, EPS and so on) works with a Bitcoin node (Bitcoin Core). Electrum (i.e the client) connects to an Electrum server in order to work. If you want it to connect to your own server, you have to specify that as command line parameter for Electrum.

Quote from: nemomaximus on April 07, 2022, 03:40:20 AM

Where do I download the "best" version from?

For Electrum server it's debatable what's a "best version". Imho any version that works for you it's just fine.
For Electrum itself (Electrum client) you download the last version from electrum.org and verify it.

Quote from: nemomaximus on April 07, 2022, 03:40:20 AM

How do I do that?

Any good tutorial that I can read?

If you are under Windows, I've made this tutorial.
Since then the versions evolved a little, but it should be pretty much still good. It installs Electrs and not ElectrumX simply because that one I could make it work.
If you want only Electrum server and no block explorer, just skip the block explorer part.

nemomaximus

newbie

Activity: 10

Merit: 18

Hi,

I have been able to install the Bitcoin Core node (version 22.0) on my own server. But now I want to continue with installing an Electrum server that connects to my node.

How do I do that?

Where do I download the "best" version from?

Any good tutorial that I can read?

What is the difference between Electrum and ElectrumX?

Thanks!

Topic: Installing my own Electrum server (Read 319 times)