Thank you for the comprehensive answer!
Just wanted to report back and say that the setup seems to be running fine now.
FWIW, a bare Debian VM running just bitcoind in listen=0-mode and ArmoryDB uses (w/o buffers and cache) a tad over 1 GB RAM plus ~200 MB swap in normal operation, so 2 GB should be plenty. That's without touching any of the RAM tunables. For the initial build of Armory's DB I had it at 3 GB, no issues. Then again, I haven't actually done much with it yet, just let it run for a few days, to see if it was stable.
One thing I still don't get is FCGI. Setting up a web server/proxy, even a very lightweight one, seems to me a very heavy-handed solution to the problem of getting two pieces of software on different machines to talk to one another. That on its own doesn't even handle security, you'd need to set up SSL with self-signed certificates or something. For the time being I just tunnel the hardcoded port over ssh, works like a charm.
Topic: Questions about configuration utilising separate ArmoryDB (Read 335 times)
I'm running a setup very much like what you describe - bitcoind and armoryDB on Ubuntu VM, with ArmoryQT running on Windows.
I've got the bitcoind blocks directory living on a hard-drive array, while the bitcoind chainstate and armorydb live on an SSD. I give each VM (I have separate VMs for Core, BIP148, soon-to-be-ChinaDumpCoin) 5.5GB of RAM.
I've got the bitcoind blocks directory living on a hard-drive array, while the bitcoind chainstate and armorydb live on an SSD. I give each VM (I have separate VMs for Core, BIP148, soon-to-be-ChinaDumpCoin) 5.5GB of RAM.
1) Yes, that does it.
2) The less RAM/threads you got, the longer it takes for the DB to build. As for how little RAM you can get away with, you're looking at:
a) Core eats around 3GB RAM. Most of its memory use to maintain the UTXO set, the bigger that gets (and it ought to with SW), the higher that requirement will go.
b) ArmoryDB needs ~1GB RAM tops to just run maintain itself and run the client, but it will be looking at ~2.5-3GB by default to bootstrap. There are cli args to reduce the resource cost of the bootstrapping part if you are running on a tight budget. You can also split the setup where you let Core bootstrap first, shut it down, let ArmoryDB bootstrap then run the whole stack to avoid concurrent costs.
c) Core needs space for blockchain data and its DB. 200GB should get you an extra year from August 1st maybe?
d) Armory's DB is fairly lean. Currently it's <1GB. 5GB should last you at least till 2020.
e) Assuming you have lots of swapping space, you could get away wtih 2GB RAM on a very lean setup. 4GB is better, 8 is what I would recommend.
3) Right now you want client and server in lock step. Once this stuff congeals (it came out in 0.95, we're at 0.96, so maybe another version or 2?), the requirement should loosen.
4) It would hold the address script hashes you register with it from your clients. That's WO data only, no chaincode.
5) This is the network stack:
a) Node talks to other nodes over the WAN.
b) DB tries to talk a node on localhost only (hardcoded), through 2 different ports, one for P2P layer, one for RPC. RPC is not mandatory but preferable.
c) DB listens on localhost (hardcoded again) as a FCGI service. Local clients connect directly through that interface, remote obviously clients can't, therefor you need you put a web daemon in front of that FCGI service (since it's works over HTTP) to allow for clients to connect to a remote DB.
d) Client connects to the DB/port you give it. By default it looks on localhost. There is currently no encryption of the client to DB layer, that stuff is for a future release. If you want to use TOR on top of the HTTP daemon, you would need to turn your daemon into a hidden service, run Tor on the client machine and somehow tunnel the daemon over localhost:port for the client to figure that out. Or you can VPN into the server VM. If the VM is a guest on your machine, you can just connect to the DB over whatever IP:port the VM makes itself visible as.
Neither the client nor the DB use nor need the WAN. All phone home code has been removed since 0.94. All possible operations that would open a browser to an external link (that's only to show you your tx on a blockchain explorer) come with a warning message box (in case you miss click it).
2) The less RAM/threads you got, the longer it takes for the DB to build. As for how little RAM you can get away with, you're looking at:
a) Core eats around 3GB RAM. Most of its memory use to maintain the UTXO set, the bigger that gets (and it ought to with SW), the higher that requirement will go.
b) ArmoryDB needs ~1GB RAM tops to just run maintain itself and run the client, but it will be looking at ~2.5-3GB by default to bootstrap. There are cli args to reduce the resource cost of the bootstrapping part if you are running on a tight budget. You can also split the setup where you let Core bootstrap first, shut it down, let ArmoryDB bootstrap then run the whole stack to avoid concurrent costs.
c) Core needs space for blockchain data and its DB. 200GB should get you an extra year from August 1st maybe?
d) Armory's DB is fairly lean. Currently it's <1GB. 5GB should last you at least till 2020.
e) Assuming you have lots of swapping space, you could get away wtih 2GB RAM on a very lean setup. 4GB is better, 8 is what I would recommend.
3) Right now you want client and server in lock step. Once this stuff congeals (it came out in 0.95, we're at 0.96, so maybe another version or 2?), the requirement should loosen.
4) It would hold the address script hashes you register with it from your clients. That's WO data only, no chaincode.
5) This is the network stack:
a) Node talks to other nodes over the WAN.
b) DB tries to talk a node on localhost only (hardcoded), through 2 different ports, one for P2P layer, one for RPC. RPC is not mandatory but preferable.
c) DB listens on localhost (hardcoded again) as a FCGI service. Local clients connect directly through that interface, remote obviously clients can't, therefor you need you put a web daemon in front of that FCGI service (since it's works over HTTP) to allow for clients to connect to a remote DB.
d) Client connects to the DB/port you give it. By default it looks on localhost. There is currently no encryption of the client to DB layer, that stuff is for a future release. If you want to use TOR on top of the HTTP daemon, you would need to turn your daemon into a hidden service, run Tor on the client machine and somehow tunnel the daemon over localhost:port for the client to figure that out. Or you can VPN into the server VM. If the VM is a guest on your machine, you can just connect to the DB over whatever IP:port the VM makes itself visible as.
Neither the client nor the DB use nor need the WAN. All phone home code has been removed since 0.94. All possible operations that would open a browser to an external link (that's only to show you your tx on a blockchain explorer) come with a warning message box (in case you miss click it).
In a nutshell, I'd like to run as much as possible of the "Armory stack" in a Linux VM on a server, while keeping the GUI on a Windows desktop.
1) It looks like running bitcoind, ArmoryDB, a minimal web server and Tor in the VM, armory-qt on Windows should do the trick, correct?
2) What would the system requirements for such a VM be? I have lots of reliable storage, but RAM is at a premium.
3) How picky is this about versions -- do I have to keep the DB and GUI in lockstep or ...?
4) In this configuration, the VM would not hold any sensitive data, correct?
5) How'd I set up bitcoind, ArmoryDB and armory-qt if I didn't want incoming connections at all and outgoing ones via Tor only?
1) It looks like running bitcoind, ArmoryDB, a minimal web server and Tor in the VM, armory-qt on Windows should do the trick, correct?
2) What would the system requirements for such a VM be? I have lots of reliable storage, but RAM is at a premium.
3) How picky is this about versions -- do I have to keep the DB and GUI in lockstep or ...?
4) In this configuration, the VM would not hold any sensitive data, correct?
5) How'd I set up bitcoind, ArmoryDB and armory-qt if I didn't want incoming connections at all and outgoing ones via Tor only?
Jump to: