Topic: Electrum server discussion thread - page 9. (Read 61507 times)
What's the disk size required at the moment to run an Electrum server?
Has anybody profiled the Python code for electrum server? It seems slow to me, about four times slower than bitcoind. Not a problem right now keeping up with blocks, but if the block size gets expanded and transaction rate continues to increase then there is likely to be a problem. I am using the latest code from github. I am running a private server on a 1.4 GHz Haswell based atom server with 8 GB of RAM and an SSD. A recent 1 MB block took about 200 seconds to process according to the log. This had 99% CPU utilization while processing the large block.
Perhaps there are some tuning suggestions around, but I wasn't able to find any.
Perhaps there are some tuning suggestions around, but I wasn't able to find any.
Is it possible to patch the memory allocation bug manually, in the previous version of electrum-server? I attempted to update to the new version but ran into problems, including permission errors.
I really think what this project needs is a release cycle. The version on github is a development version. That's fine but we are going to get frustrated if we try and use it as a production version. I gave up running the new server, if there was a stable tag I would have gone back to that.
I did think of taking a known good version, scripting all setup, and putting it in a docker container. That way setting up a new server would be as easy as 'docker run blahblah/electrum-server' and waiting until it set itself up.
I've been running an Electrum server for almost 2 years now and to be honest it's been running very stable. Yes, there are the occasional bugs, but they are solved quickly by Thomas. You should hang out in the IRC channel though to follow up on the latest developments..
Is it possible to patch the memory allocation bug manually, in the previous version of electrum-server? I attempted to update to the new version but ran into problems, including permission errors.
I really think what this project needs is a release cycle. The version on github is a development version. That's fine but we are going to get frustrated if we try and use it as a production version. I gave up running the new server, if there was a stable tag I would have gone back to that.
I did think of taking a known good version, scripting all setup, and putting it in a docker container. That way setting up a new server would be as easy as 'docker run blahblah/electrum-server' and waiting until it set itself up.
Is it possible to patch the memory allocation bug manually, in the previous version of electrum-server? I attempted to update to the new version but ran into problems, including permission errors.
I fixed setup.py yesterday.. let me know
I get different problems now.
Code:
zipimport.ZipImportError: bad local file header in /usr/local/lib/python2.7/dist-packages/electrum_server-0.9-py2.7.egg
This apparently means there is a syntax error somewhere, possibly python code that only works with a higher version. The error message makes it look like a ZIP error, which it isn't. If I run 'python setup.py install' again it seems to work and it seems to run. I can't connect but that may be because it's updating.
I really need to test this on a clean machine.
Give Thomas few days, it is completely new version of the server 
I will install it soon and I might be able to improve the docs.
I will install it soon and I might be able to improve the docs.I got it running with some minor hacks. I didn't see any tags in the repo. Release tags might be a good idea so if something doesn't work I can just go back to the last tag.
EDIT: Actually, no I didn't get it running. It's choking on a futex.
I fixed setup.py yesterday.. let me know
Give Thomas few days, it is completely new version of the server I will install it soon and I might be able to improve the docs.
I will install it soon and I might be able to improve the docs.I got it running with some minor hacks. I didn't see any tags in the repo. Release tags might be a good idea so if something doesn't work I can just go back to the last tag.
EDIT: Actually, no I didn't get it running. It's choking on a futex.
Can you update the install instructions? Right now HOWTO.md references start and stop scripts that don't exist. The TLDR section at the top of INSTALL doesn't result in a running server, nor do the full instruction in that file.
Give Thomas few days, it is completely new version of the server
I will install it soon and I might be able to improve the docs.
Thomas,
You put a lot of great work into Electrum but it's all a bit lost if people can't easily get the server running. Can you update the install instructions? Right now HOWTO.md references start and stop scripts that don't exist. The TLDR section at the top of INSTALL doesn't result in a running server, nor do the full instruction in that file.
I tried to do it myself and send a pull request but I've no idea what the instructions should be. Whatever I try 'python setup.py install' won't install the src module on ubuntu 14.04 so electrum_server.py won't run. It was all working so nicely until I upgraded.
If you want people to run servers it should be as easy as possible for those guys.
You put a lot of great work into Electrum but it's all a bit lost if people can't easily get the server running. Can you update the install instructions? Right now HOWTO.md references start and stop scripts that don't exist. The TLDR section at the top of INSTALL doesn't result in a running server, nor do the full instruction in that file.
I tried to do it myself and send a pull request but I've no idea what the instructions should be. Whatever I try 'python setup.py install' won't install the src module on ubuntu 14.04 so electrum_server.py won't run. It was all working so nicely until I upgraded.
If you want people to run servers it should be as easy as possible for those guys.
I'm running the most recent electrum server from GIT. The logfile seems to be missing blocks, is the following normal?
Why do I not get a log message for every block?
Code:
[25/07/2014-20:48:22] blockchain: 312457 (82.964s)
[25/07/2014-20:58:35] blockchain: 312458 (194.778s)
[25/07/2014-21:02:21] blockchain: 312460 (128.478s)
[25/07/2014-21:10:27] blockchain: 312462 (250.398s)
[25/07/2014-21:16:38] blockchain: 312463 (166.109s)
[25/07/2014-21:22:49] blockchain: 312464 (212.260s)
[25/07/2014-21:51:48] blockchain: 312466 (670.598s)
[25/07/2014-21:56:06] blockchain: 312468 (168.989s)
[25/07/2014-22:04:08] blockchain: 312469 (166.773s)
Why do I not get a log message for every block?
if several blocks are available, it processes them together, before it prints a log line.
I agree that it can be confusing, it should probably print a message on each block
I'm running the most recent electrum server from GIT. The logfile seems to be missing blocks, is the following normal?
Why do I not get a log message for every block?
Code:
[25/07/2014-20:48:22] blockchain: 312457 (82.964s)
[25/07/2014-20:58:35] blockchain: 312458 (194.778s)
[25/07/2014-21:02:21] blockchain: 312460 (128.478s)
[25/07/2014-21:10:27] blockchain: 312462 (250.398s)
[25/07/2014-21:16:38] blockchain: 312463 (166.109s)
[25/07/2014-21:22:49] blockchain: 312464 (212.260s)
[25/07/2014-21:51:48] blockchain: 312466 (670.598s)
[25/07/2014-21:56:06] blockchain: 312468 (168.989s)
[25/07/2014-22:04:08] blockchain: 312469 (166.773s)
Why do I not get a log message for every block?
I don't know how to edit/create/activate any server on my Electrum. I use a Window and any expert could just put a link here where I can learn about it. I know what you're thinking, but it's effortless to help me out here. Thanks
At the risk of being rude, https://github.com/spesmilo/electrum-server/blob/master/HOWTO.md
If this is not sufficient to get you going, you probably shouldn't be running an Electrum server.
grnbrg.
I don't know how to edit/create/activate any server on my Electrum. I use a Window and any expert could just put a link here where I can learn about it. I know what you're thinking, but it's effortless to help me out here. Thanks
What does the ulimit command show for the userid running the server, though? On most Linuxes, max open files is set to 1024, and this limit can not be raised by non-root users.
As said the 'ulimit' is not the problem:
Code:
$ ps -efa|grep server.py |grep -v grep
electrum 24942 1 0 06:59 ? 00:00:00 /usr/bin/python -u server.py
$ cat /proc/24942/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 15862 15862 processes
Max open files 32768 32768 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 15862 15862 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Do you have the open files limit set sufficiently high?
should show 16k or higher.
Code:
ulimit -n
That shouldn't be the problem - nofile is set to 32k in the upstart script.
My setup:
Ubuntu 12.04 LTS (kernel 3.2.0) w/Python 2.7.3; latest Electrum server from git; leveldb 1.9.0; plyvel 0.8
What does the ulimit command show for the userid running the server, though? On most Linuxes, max open files is set to 1024, and this limit can not be raised by non-root users.
You probably need to add a line to /etc/security/limits.conf to give your server user a higher soft/hard limit. ie:
Code:
electrum soft nofile 16384
electrum hard nofile 16384
grnbrg.
Do you have the open files limit set sufficiently high?
should show 16k or higher.
Code:
ulimit -n
That shouldn't be the problem - nofile is set to 32k in the upstart script.
My setup:
Ubuntu 12.04 LTS (kernel 3.2.0) w/Python 2.7.3; latest Electrum server from git; leveldb 1.9.0; plyvel 0.8
Third time in 2 or 3 months that the database of my server e2.pdmc.net broke ("wrong_hash"). Is this just me or are other operators experiencing this as well?
Anyway, I'll take my server offline for now and bring it back online once I find enough time to move it faster hardware (probably in August).
Anyway, I'll take my server offline for now and bring it back online once I find enough time to move it faster hardware (probably in August).
Do you have the open files limit set sufficiently high?
Code:
ulimit -n
should show 16k or higher.
grnbrg.
I have nothing extraordinary in the config to show
Third time in 2 or 3 months that the database of my server e2.pdmc.net broke ("wrong_hash"). Is this just me or are other operators experiencing this as well?
Anyway, I'll take my server offline for now and bring it back online once I find enough time to move it faster hardware (probably in August).
Anyway, I'll take my server offline for now and bring it back online once I find enough time to move it faster hardware (probably in August).
I have the same, I moved to Samsung 840 SSD but database is still breaking sometimes, without a clear reason.
I never had this problem, while running the server on a very old and busy box.
Do both of you want to share particulars of your setup to see if there are any similarities?
Jump to: