Topic: [ANN] BitcoinArmory-Daemon - armory on web servers - page 2. (Read 20542 times)

Well the thing is, our server making the calls to the daemon is in .net(C#) It is much easier and cleaner to simply make json rpc calls to the coin daemon. What are the advantages of programming the hooks directly?


Also looking at armoryd.py i notice many new jsronrpc methods not found in the original bitcoin API calls list (https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list)  these methods have a prefix of 'jsonrpc' so im thinking they are newly added to the api call list but wiki does not show them.

My current envoirement with ArmoryD

If its an help for someone

I have "X" Online servers (at the moment I am trying to know how much Users I can support on each)
And "X" "Warm-Cold" Wallets server instances. (Each online server instance has an offline Server instance)

Process to broadcast:

It starts with: Web requests to Middle-Api-Server to make a broadcast from AddressX to AddressZ

The Middle API Server communicates to Online Watching Server and checks the balance.
If balance is OK, I create a encrypted Queue of another Middle-Backend with an Unsigned Transaction.

Then another Process Server Checks the Encrypted Queue for unsigned Tx's and sends it to the Warm-Cold Server to sign.
This middle Server stores the SignedTx back to the Encrypted Queue and then the Online Server Queue get a HexTx from ArmoryD (Online server)
and broadcasts this rawtransaction over BitcoinD RPC.

Its a little bit confusing maybe as 5 Servers are needed for this process.
(1 web server, 1 Api server, 1 Online Watching Server, 1 Offline Server, 1 Queue process Server For Signing)
and then increasing on Servers depending on Users.


The testings right now are 100.000 Users per Wallet
- You need to tweak the ArmoryD a little bit for this porpose on some places
   1. The most functions are on Wallet scope - I added functions to support the RPC on Address Scope.
   2. The Offline server Unlock function is very slow on 100.000 Addresses, so I modified the Unlock function to just unlock the PrivKey which I need to sign.

Its just a little info for somebody to get started with armory and Services on a website.

I would preffer Wallet per User, but until armoryd does not support multi Wallet its just impossible - at least I didnt found a suitable solution.


In a few days I can tell if the wallet supports 100.000 Addresses / Users with a avg. of 2-3 Transactions per User. (would be 300.000 Transactions per Wallet / Server)


Best regards

I'm not sure what you're asking with the last question.  If you are signing with bitcoind (can it do offline signing?) then you can just broadcast it with any network-connected app once you get it back online.  Armory's default is to pass in this decorated-unsigned-tx, and return the same decorated-signed-tx to the online computer.  That Armory instance will "prepareFinalTx().serialize()" which converts the decorated-tx to a byte string appropriate to be sent over the wire.

Btw, we do have many users leveraging armoryd as their backend, it just requires a bit of work to fill in the gaps.  One of the nice things about armoryd.py is that it's pretty easy to adapt it to whatever you need to do:  there's a clear place to put code everytime a new tx comes in, everytime a new wallet-relevant tx comes in, and every new block.  You don't have to operate through RPC, you can just program your hooks directly in.

Ok I am actually trying to build an exchange and the idea is to have cold warm and hot wallets. The cold would be using offline armory and the hot would be using online armory. The warm wallet will be built using either armory daemon or bitcoind.  I didnt know armory daemon was not complete. So like you explained, I can use bitcoind and then before sending the unsigned transaction to cold storage, I would need add extra data to the transaction.

What about signing a transaction, can that be done using bitcoind or do i need to implement a command specific to  armory's requirements?

Armory unsigned transactions are different.  It's because they store all the extra data needed for the offline computer to verify the transaction without the blockchain.  It can be a lot of data, but required to maximize security. 

On the other hand, you might be able to go the other way, but you'd have to build some helper functions to convert a raw unsigned transaction from bitcoind, to collect all the blockchain data that is needed for Armory to verify the tx before signing.  Armory assumes all transactions will always be signed by a dumb device with no blockchain -- there is no alternative form.

This is not standardized in any way.  Maybe one day it will be.

ok so at the moment, can i sign/unsign transactions using bitcoin daemon since they already have a json api ? or will that not work with armory wallets?

At the moment, "getledger" and "getledgersimple" are the biggest divergences of them, though we attempted to implement "listtransactions" the same way Bitcoin Core does.  I really don't like what Core does with "listtransactions":  it provides multiple outputs with overlapping entries, which require a bit of extra logic to adapt to your application.

On the other hand "getledger" provides you exactly what you would see on the Armory GUI ledger -- it's a mutually-exclusive list of entries, each one corresponding to a single transaction, and theoretically all the values produced from it can be summed to give you your balance. 

We will also soon have an interface for:

- Multiple wallets (context switching)
- Create unsigned transaction (to take to offline computer)
- Sign unsigned transaction (to run from offline computer)

If you'd like to recommend more stuff you'd like to see, post in the following thread:
https://bitcointalksearch.org/topic/what-would-you-like-to-see-in-armoryd-588227

What commands are specific to the armory daemon that are not in the bitcoindaemon?
 

In one line: easier to implement.

Up until a few months ago, etotheipi was working on Armory alone. Cost of implementation is a very high priority in this case. Now we're 5, so things are changing, notably Armory's backend model.

Wallet operations are not threadsafe. Not at all!  
can someone please explain why armory wallets are not thread safe and why armory runs as a singleton? please explain in detail.

You and me both! I think I'll be looking into getting the Raspberry Pi to act as a USB client. It would then be necessary to use the USB protocol, which is a lot more complicated than just serial, but would not - as far as I know - allow login to the offline machine.

EDIT: I just discovered that the Raspberry Pi Model A (as opposed to Model B that is out now) will support acting as a USB device/client, because it uses a different communication-chip than Model B. Model A doesn't have an Ethernet port, and only has a single USB port, but I figure this might even be an advantage, since after the software has been installed on the device (using, for example, some USB-to-Ethernet type device to update packages), the Ethernet port will have to be removed because the USB connection needs to be used to communicate with the online computer. Perhaps one will be less tempted - and feel safer - if the device doesn't even have a port to plug in an Ethernet cable.
Additionally, it costs only $25. That's really not much for the ultimate in safe Bitcoin wallets. That being said, Model A isn't out now. And USB client mode won't necessarily work out-of-the-box. This is a software issue, as far as I can gather. Not OS level, I think, but firmware level - so it might not be that easy to implement. But still, looks like a device is coming to the market that fulfils the requirements of an offline hardware wallet. Off the top of my head, I can't see what else one would need other than a Raspberry Pi Model A, and perhaps a USB cable. But most people have USB cables lying around. Sure beats that $40 2xUSB-to-serial-plus-null-modem solution.
[/quote]

Wonder where things are at with serial ports. I own a rasp pi model A and I like the idea of using it for running armory in offfline mode. The rasp pi model A can work with serial ports no problem using a shield. Has there been any development on armory for making serial ports work? I gather we have to send the 'unsigned transaction' over from online armory to an offline pc over a serial connection?

Here is a link to the latest code:
https://github.com/etotheipi/BitcoinArmory/blob/0.91-dev/armoryd.py

Hi guys

Thanks for all your support in trying to get Armory working on server, but it's still a bit difficult for me.

I need it to work easily in different environments... No hard setup or installs.

I have an idea though. If you guys tell me what classes and methods I should be looking at in the source, I might be able to write my own version in Java. So I'll look at your code, figure out what it does to produce the unsigned transaction, and then I'll write some code that replicates the output from the same input.

If I get it working, I'll send you the source files so you can let others use them too, if you want that.
I'm writing the tests as well of course, so that anyone can see that they get expected result.

I'm imagining something like creating a class and method in Java:

public String getArmoryUnsignedTransaction(String[] unspentInput, String[] addressesTo, long amountSatoshi){
           //Do some magic
           return theWholeTransactionAsASingleString;
}

Would you be able to point me to some folders, classes and methods in the source where I can see what's going on when creating the unsigned transaction?

I just now noticed this thread.  This is an amazing feature, given all the hacks / thefts that I've seen, this kind of webserver daemon seems like it'd be the holy grail of bitcoin daemons since it'd be effectively unhackable.

Getting armoryd working on windows will be much more challenging than linux (which is five commands plus getting txjsonrpc).  I'm not sure we've even tried armoryd.py in Windows without having the whole build environment setup.   As CircusPeanut said, you might only need to get the _CppBlockUtils.pyd and .py from a preinstalled version of Armory, but there might be other .dlls you need as well.  Maybe just copy those two files plus all the .dlls from the installed Armory dir, into your checkout directory.  Or use Linux... which is dramatically easier to setup.


We should have a function in there that gets you an unsigned transaction for a list of recipients.  To make the signing process easier, you can look at extras/cli_sign_txdp.py which is a full-featured, command-line interface for signing offline transactions.  The two arguments for it are wallet file and a unsigned.tx files from Armory. 

Be aware that the master branch may not have all the armoryd.py features implemented.  The testing branch will have all the updated features, but of course is the unstable version.  Hopefully we'll have that ironed out in a couple weeks.

You are missing _CppBlockUtils.pyd and CppBlockUtils.py. You can either get them from your armory installation or build cppForSwig to produce these files.

More detailed instructions are here:
https://bitcoinarmory.com/download/building-armory-from-source/

Re: Question 2...I don't have any experience building on linux so I can't help trouble shoot that.


Yes that can definitely be done. Send your command from the Java program and the result of any JSON method you call will return a text representation of the returned value or values (as a name/value mapping).

OK, I see I wasn't expressing myself clear.

The question wasn't about JSON per se, it was about how to interact with armory. I have written software in Java, and I need it to interact with Armory, to create an offline transaction.
I want my program to be able to connect for example to localhost:8999, send command jsonrpc_sendmany([array of bitcoin addresses : amount to send])
Then I'd like to get in return the offline transaction, unsigned, into my program, and my program stores file to disk/USB or whatever. Then I want to take that offline transaction to an offline computer and sign it before broadcasting it from an online computer.

Think it can be done?

Hi

Thanks for the help thus far, but I still have problems. Sorry for this massive post, but I think it makes it easier to debug.

I'm trying to get this thing working on both windows and linux, but encounter difficulties on both.

1) Windows:
I put the  txjsonrpc into the suggested folder, I used this one: https://github.com/oubiwann/txjsonrpc
I installed twisted, and that required zope, so I installed that too.
Then I installed psutils-1.2.1 and confirmed that it's working.
I'm running this thing in Python 2.7.6, opening the file from the shell, then hitting run (F5)

So now I get this error:
----------------------
(CRITICAL) armoryengine.py:1043 - C++ block utilities not available.
(CRITICAL) armoryengine.py:1044 -    Make sure that you have the SWIG-compiled modules
(CRITICAL) armoryengine.py:1045 -    in the current directory (or added to the PATH)
(CRITICAL) armoryengine.py:1046 -    Specifically, you need:
(CRITICAL) armoryengine.py:1047 -        CppBlockUtils.py     and
(CRITICAL) armoryengine.py:1051 -        _CppBlockUtils.pyd

Traceback (most recent call last):
  File "X:\BitcoinArmory-master\armoryd.py", line 49, in
    from armoryengine import *
  File "X:\BitcoinArmory-master\armoryengine.py", line 1039, in
    import CppBlockUtils as Cpp
ImportError: No module named CppBlockUtils
----------------------

Any suggestions?
I'm using the latest source from https://github.com/etotheipi/BitcoinArmory, downloaded as zip.

2) Ubuntu 12.04
I put the txjsonrpc into the suggested folder, I used this one: https://github.com/oubiwann/txjsonrpc
I tried installing psutils-1.2.1:
----------------------
$ python setup.py  install
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib.linux-i686-2.7
creating build/lib.linux-i686-2.7/psutil
copying psutil/_pslinux.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psbsd.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_compat.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psosx.py -> build/lib.linux-i686-2.7/psutil
copying psutil/__init__.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_pssunos.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_error.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psmswindows.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psposix.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_common.py -> build/lib.linux-i686-2.7/psutil
copying psutil/error.py -> build/lib.linux-i686-2.7/psutil
running build_ext
building '_psutil_linux' extension
creating build/temp.linux-i686-2.7
creating build/temp.linux-i686-2.7/psutil
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-i686-2.7/psutil/_psutil_linux.o
psutil/_psutil_linux.c:13:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------
Since install failed, I put psutils into the same folder that I'm trying to run the armoryd.py from, then try to run it:
----------------------
$ python armoryd.py
Traceback (most recent call last)
  File "armoryd.py", line 49, in
    from armoryengine import *
  File "/home/linuxuser/Documents/arm/armoryengine.py", line 47, in
    import psutil
  File "/home/linuxuser/Documents/arm/psutil/__init__.py", line 88, in
    import psutil._pslinux as _psplatform
  File "/home/linuxuser/Documents/arm/psutil/_pslinux.py", line 20, in
    import _psutil_linux
ImportError: No module named _psutil_linux
$
----------------------

More suggestions?

I haven't had a need to use JSON across languages yet, but I understand that is one of it's features.

From http://en.wikipedia.org/wiki/JSON

You are trying to run it as a bash script.  You need to run it as a python script:

"python armoryd.py".

Also, as someone else noted, you'll need to download the txjsonrpc project and put it in your base directory (there should be a txjsonrpc dir next to armoryd.py).  Or actually get txjsonrpc installed -- it's an extra dependency not needed by regular Armory but needed for armoryd.py.