Topic: [ANN] BitcoinArmory-Daemon - armory on web servers (Read 20594 times)

Awesome, thanks to everyone involved!

Thanks to Doug we actually have a new version of that method with an optional minconf argument  on the "dev" branch:

Hi everyone!

I made a thread over here (click me) about getreceivedbyaddress.

In short, does getreceivedbyaddress only include confirmed transactions or also unconfirmed ones?

Looking through the source gives me the impression that it counts both confirmed and unconfirmed transactions:

Also looking through Transaction.py and PyBtcWallet.py did not give me any clues on how one would add an extra check to the lambda ( something like x.getConfirmations() > minconfs ) to only include confirmed transactions.

Any advice / inputs?

As stated before, that library will be included with the next version of Armory.


It is. To be honest, armoryd basically won't work out of the box if you download the latest version. I had to fix at least two or three bugs before armoryd even ran at all, and I think other team members fixed a bug or two. I don't know what happened. It just fell out of sync with the rest of Armory, I guess.

That being said, the latest version works great. There are a few rough edges and a couple of bugs that need to fixed before it gets released, but trust me, it'll be a much smoother experience.

Ok I fixed the txjsonrpc problem.

1) sudo apt-get install python-setuptools
2) sudo easy_install txJSON-RPC

The second command must be run IN the armory directory. Now the error I get is:

root@cloud-server-07:/BitcoinArmory# python armoryd.py
(ERROR) Traceback (most recent call last):
  File "armoryd.py", line 1158, in
    rpc_server = Armory_Daemon()
  File "armoryd.py", line 852, in __init__
    self.lastChecked
AttributeError: 'Armory_Daemon' object has no attribute 'lastChecked'

Traceback (most recent call last):
  File "armoryd.py", line 1158, in
    rpc_server = Armory_Daemon()
  File "armoryd.py", line 852, in __init__
    self.lastChecked
AttributeError: 'Armory_Daemon' object has no attribute 'lastChecked'


Which seems to be more Armory specific.

All I need is to be able to get the balance. So I think the version on the master branch will be fine?

The only problem I have right now is to get armoryd.py to find txjsonrpc.auth module.

Traceback (most recent call last):
  File "armoryd.py", line 55, in
    from txjsonrpc.auth import wrapResource
ImportError: No module named txjsonrpc.auth


I'm probably going to run into some other issues after I solve it but still...

You're probably cloning the master branch, which doesn't have any of the new armoryd code. You need to clone the devel branch to get all the new armoryd code. Just be aware that it is a work-in-progress. Various things may be incomplete or completely broken.

I cloned the repo to BitcoinArmory directory. But I still get


I'm guessing I need to install it but I could not figure out how to do that.
 

Hi. As part of my ongoing cleanup of armoryd, I grabbed a copy of txjsonrpc and am now distributing it along with Armory. If anybody can't wait for the next version of Armory and wants to do what I did, https://github.com/etotheipi/BitcoinArmory/commit/5a75164e90e9898cddf66d13199744e8d94b873b has the details.

Speaking of armoryd, I've added quite a bit of functionality over the past couple of weeks. armoryd is still rough around the edges but is quickly becoming a pretty useful tool. Feel free to check it out, with the caveat that there's still plenty to do and that any version you download now definitely won't be the version that goes into Armory 0.92. (In fact, I'll be checking in some more lockbox functionality in the next hour or two.)

How can I install txjsonrpc on the BitcoinArmory directory?

Hi!

I noticed that some transactions are not getting to mainbranch, and for this reason the Tx never gets confirmed.
At this moment I dont know why it happens. Over 25.000 transactions on Testnet and maybe 50 cases on which happens the mainbranch issue.

Is there a way to detect the "scrap" items on armory core?
At this moment the one and only working solution I got is to restart the watching only wallet service.
After restarting armory service the TxId just dissapears and I mark them as "Scrap" in database and reissue the Tx creating a new one.


Is there a way to ask Armory core if the transaction is really a "scrap" item?
And how can it happen that the Tx does not get into the Mainbranch? (all tests are done over testnet so far)

Best regards

If you don't know the answer to that question, you should probably spend some time with the Armory GUI to understand the vast functionality provided by Armory.   Especially the offline wallets.  Most people are interested in armoryd because of the ease with which you can create watching-only wallets to put on your webserver, and keep the private keys on an offline computer.

Armory uses Bitcoin Core solely for communicating with the Bitcoin network, peer finding, and blockchain validation.   Armory provides 100% new wallet infrastructure to provide deterministic wallets (only needeing to be backed up one time), plethora of backup features, the ability to manage multiple wallets at once, and the ability to manage wallets created offline but still monitor the wallet activity and distribute payment addresses from an online computer.
 
Armory uses the filtered datastream provided from Bitcoin Core, since no other software can reliably and securely replicate the validation/verification behavior of Bitcoin Core (if you use something else, you risk getting forked).   Armory doesn't even try to do blockchain validation -- that's Core's job.  Therefore you need to run both, and Armory watches the blk*.dat files from Core and the tx messages that are sent over P2P.

I am a bit confused, Why does armory run bitcoind in the background and what is the use of armory daemon?

Wondering if etotheipi can chime in, why does the hashing function need changing for ltc if I am just using armory wallets for storing coins. or are wallets and hashing function connected together?

Can armory not use the coin daemon (litecoin daemon via rpc calls ) and manage offline wallets that way. I can modify armory to convert the unsigned tx to work with armory's offline signing and then all should be good?

I'm basing that assessment on this thread in github https://github.com/etotheipi/BitcoinArmory/issues/32

Its the work I'm afraid  I will have to do if I'm to use armory for our exchange. Etotheipi mentioned some where it requiring some work. The transaction format is the same across coins only the hashing algorithm changes from sha256 to scrypt ? 
Indulge me what makes you say its a lot of work ?

No need to recompile since armoryd is written in Python.

Modifying Armory to run with different crypto currencies requires a lot of work.

OK armory would need to be modified and recompiled. Can Armorybe modified to create wallets for a coin type ? So I want multiple wallets maybe one wallet per coin type. 

Assuming you mean that  you want to convert an "unsigned transaction" for armory to sign. Yes, you can do this with a bit of coding in armoryd.

Take whatever raw transaction hex from bitcoind and use the unserialize method on the appropriate object to get an Armory object. You probably want PyTx or PyTxDistProposal. Note that in the devel branch (and others) PyTxDistProposal is renamed UnsignedTransaction. So whatever the code base you're using look for the appropriate object and call unserialize on it.

I need to figure out how to use bitcoind and armory offline wallets together via json rpc or some other method.
I think this is the post I am looking for  https://bitcointalksearch.org/topic/solved-made-script-signing-raw-transactions-from-bitcoind-with-armory-448284
this is able to convert a signedtransaction from bitcoind into something armory can sign in offline mode. correct me if i am wrong?

Well the thing is, our server making the calls to the daemon is in .net(C#) It is much easier and cleaner to simply make json rpc calls to the coin daemon. What are the advantages of programming the hooks directly?


Also looking at armoryd.py i notice many new jsronrpc methods not found in the original bitcoin API calls list (https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list)  these methods have a prefix of 'jsonrpc' so im thinking they are newly added to the api call list but wiki does not show them.

My current envoirement with ArmoryD

If its an help for someone

I have "X" Online servers (at the moment I am trying to know how much Users I can support on each)
And "X" "Warm-Cold" Wallets server instances. (Each online server instance has an offline Server instance)

Process to broadcast:

It starts with: Web requests to Middle-Api-Server to make a broadcast from AddressX to AddressZ

The Middle API Server communicates to Online Watching Server and checks the balance.
If balance is OK, I create a encrypted Queue of another Middle-Backend with an Unsigned Transaction.

Then another Process Server Checks the Encrypted Queue for unsigned Tx's and sends it to the Warm-Cold Server to sign.
This middle Server stores the SignedTx back to the Encrypted Queue and then the Online Server Queue get a HexTx from ArmoryD (Online server)
and broadcasts this rawtransaction over BitcoinD RPC.

Its a little bit confusing maybe as 5 Servers are needed for this process.
(1 web server, 1 Api server, 1 Online Watching Server, 1 Offline Server, 1 Queue process Server For Signing)
and then increasing on Servers depending on Users.


The testings right now are 100.000 Users per Wallet
- You need to tweak the ArmoryD a little bit for this porpose on some places
   1. The most functions are on Wallet scope - I added functions to support the RPC on Address Scope.
   2. The Offline server Unlock function is very slow on 100.000 Addresses, so I modified the Unlock function to just unlock the PrivKey which I need to sign.

Its just a little info for somebody to get started with armory and Services on a website.

I would preffer Wallet per User, but until armoryd does not support multi Wallet its just impossible - at least I didnt found a suitable solution.


In a few days I can tell if the wallet supports 100.000 Addresses / Users with a avg. of 2-3 Transactions per User. (would be 300.000 Transactions per Wallet / Server)


Best regards

I'm not sure what you're asking with the last question.  If you are signing with bitcoind (can it do offline signing?) then you can just broadcast it with any network-connected app once you get it back online.  Armory's default is to pass in this decorated-unsigned-tx, and return the same decorated-signed-tx to the online computer.  That Armory instance will "prepareFinalTx().serialize()" which converts the decorated-tx to a byte string appropriate to be sent over the wire.

Btw, we do have many users leveraging armoryd as their backend, it just requires a bit of work to fill in the gaps.  One of the nice things about armoryd.py is that it's pretty easy to adapt it to whatever you need to do:  there's a clear place to put code everytime a new tx comes in, everytime a new wallet-relevant tx comes in, and every new block.  You don't have to operate through RPC, you can just program your hooks directly in.

Ok I am actually trying to build an exchange and the idea is to have cold warm and hot wallets. The cold would be using offline armory and the hot would be using online armory. The warm wallet will be built using either armory daemon or bitcoind.  I didnt know armory daemon was not complete. So like you explained, I can use bitcoind and then before sending the unsigned transaction to cold storage, I would need add extra data to the transaction.

What about signing a transaction, can that be done using bitcoind or do i need to implement a command specific to  armory's requirements?

Armory unsigned transactions are different.  It's because they store all the extra data needed for the offline computer to verify the transaction without the blockchain.  It can be a lot of data, but required to maximize security. 

On the other hand, you might be able to go the other way, but you'd have to build some helper functions to convert a raw unsigned transaction from bitcoind, to collect all the blockchain data that is needed for Armory to verify the tx before signing.  Armory assumes all transactions will always be signed by a dumb device with no blockchain -- there is no alternative form.

This is not standardized in any way.  Maybe one day it will be.

ok so at the moment, can i sign/unsign transactions using bitcoin daemon since they already have a json api ? or will that not work with armory wallets?

At the moment, "getledger" and "getledgersimple" are the biggest divergences of them, though we attempted to implement "listtransactions" the same way Bitcoin Core does.  I really don't like what Core does with "listtransactions":  it provides multiple outputs with overlapping entries, which require a bit of extra logic to adapt to your application.

On the other hand "getledger" provides you exactly what you would see on the Armory GUI ledger -- it's a mutually-exclusive list of entries, each one corresponding to a single transaction, and theoretically all the values produced from it can be summed to give you your balance. 

We will also soon have an interface for:

- Multiple wallets (context switching)
- Create unsigned transaction (to take to offline computer)
- Sign unsigned transaction (to run from offline computer)

If you'd like to recommend more stuff you'd like to see, post in the following thread:
https://bitcointalksearch.org/topic/what-would-you-like-to-see-in-armoryd-588227

What commands are specific to the armory daemon that are not in the bitcoindaemon?
 

In one line: easier to implement.

Up until a few months ago, etotheipi was working on Armory alone. Cost of implementation is a very high priority in this case. Now we're 5, so things are changing, notably Armory's backend model.

Wallet operations are not threadsafe. Not at all!  
can someone please explain why armory wallets are not thread safe and why armory runs as a singleton? please explain in detail.

You and me both! I think I'll be looking into getting the Raspberry Pi to act as a USB client. It would then be necessary to use the USB protocol, which is a lot more complicated than just serial, but would not - as far as I know - allow login to the offline machine.

EDIT: I just discovered that the Raspberry Pi Model A (as opposed to Model B that is out now) will support acting as a USB device/client, because it uses a different communication-chip than Model B. Model A doesn't have an Ethernet port, and only has a single USB port, but I figure this might even be an advantage, since after the software has been installed on the device (using, for example, some USB-to-Ethernet type device to update packages), the Ethernet port will have to be removed because the USB connection needs to be used to communicate with the online computer. Perhaps one will be less tempted - and feel safer - if the device doesn't even have a port to plug in an Ethernet cable.
Additionally, it costs only $25. That's really not much for the ultimate in safe Bitcoin wallets. That being said, Model A isn't out now. And USB client mode won't necessarily work out-of-the-box. This is a software issue, as far as I can gather. Not OS level, I think, but firmware level - so it might not be that easy to implement. But still, looks like a device is coming to the market that fulfils the requirements of an offline hardware wallet. Off the top of my head, I can't see what else one would need other than a Raspberry Pi Model A, and perhaps a USB cable. But most people have USB cables lying around. Sure beats that $40 2xUSB-to-serial-plus-null-modem solution.
[/quote]

Wonder where things are at with serial ports. I own a rasp pi model A and I like the idea of using it for running armory in offfline mode. The rasp pi model A can work with serial ports no problem using a shield. Has there been any development on armory for making serial ports work? I gather we have to send the 'unsigned transaction' over from online armory to an offline pc over a serial connection?

Here is a link to the latest code:
https://github.com/etotheipi/BitcoinArmory/blob/0.91-dev/armoryd.py

Hi guys

Thanks for all your support in trying to get Armory working on server, but it's still a bit difficult for me.

I need it to work easily in different environments... No hard setup or installs.

I have an idea though. If you guys tell me what classes and methods I should be looking at in the source, I might be able to write my own version in Java. So I'll look at your code, figure out what it does to produce the unsigned transaction, and then I'll write some code that replicates the output from the same input.

If I get it working, I'll send you the source files so you can let others use them too, if you want that.
I'm writing the tests as well of course, so that anyone can see that they get expected result.

I'm imagining something like creating a class and method in Java:

public String getArmoryUnsignedTransaction(String[] unspentInput, String[] addressesTo, long amountSatoshi){
           //Do some magic
           return theWholeTransactionAsASingleString;
}

Would you be able to point me to some folders, classes and methods in the source where I can see what's going on when creating the unsigned transaction?

I just now noticed this thread.  This is an amazing feature, given all the hacks / thefts that I've seen, this kind of webserver daemon seems like it'd be the holy grail of bitcoin daemons since it'd be effectively unhackable.

Getting armoryd working on windows will be much more challenging than linux (which is five commands plus getting txjsonrpc).  I'm not sure we've even tried armoryd.py in Windows without having the whole build environment setup.   As CircusPeanut said, you might only need to get the _CppBlockUtils.pyd and .py from a preinstalled version of Armory, but there might be other .dlls you need as well.  Maybe just copy those two files plus all the .dlls from the installed Armory dir, into your checkout directory.  Or use Linux... which is dramatically easier to setup.


We should have a function in there that gets you an unsigned transaction for a list of recipients.  To make the signing process easier, you can look at extras/cli_sign_txdp.py which is a full-featured, command-line interface for signing offline transactions.  The two arguments for it are wallet file and a unsigned.tx files from Armory. 

Be aware that the master branch may not have all the armoryd.py features implemented.  The testing branch will have all the updated features, but of course is the unstable version.  Hopefully we'll have that ironed out in a couple weeks.

You are missing _CppBlockUtils.pyd and CppBlockUtils.py. You can either get them from your armory installation or build cppForSwig to produce these files.

More detailed instructions are here:
https://bitcoinarmory.com/download/building-armory-from-source/

Re: Question 2...I don't have any experience building on linux so I can't help trouble shoot that.


Yes that can definitely be done. Send your command from the Java program and the result of any JSON method you call will return a text representation of the returned value or values (as a name/value mapping).

OK, I see I wasn't expressing myself clear.

The question wasn't about JSON per se, it was about how to interact with armory. I have written software in Java, and I need it to interact with Armory, to create an offline transaction.
I want my program to be able to connect for example to localhost:8999, send command jsonrpc_sendmany([array of bitcoin addresses : amount to send])
Then I'd like to get in return the offline transaction, unsigned, into my program, and my program stores file to disk/USB or whatever. Then I want to take that offline transaction to an offline computer and sign it before broadcasting it from an online computer.

Think it can be done?

Hi

Thanks for the help thus far, but I still have problems. Sorry for this massive post, but I think it makes it easier to debug.

I'm trying to get this thing working on both windows and linux, but encounter difficulties on both.

1) Windows:
I put the  txjsonrpc into the suggested folder, I used this one: https://github.com/oubiwann/txjsonrpc
I installed twisted, and that required zope, so I installed that too.
Then I installed psutils-1.2.1 and confirmed that it's working.
I'm running this thing in Python 2.7.6, opening the file from the shell, then hitting run (F5)

So now I get this error:
----------------------
(CRITICAL) armoryengine.py:1043 - C++ block utilities not available.
(CRITICAL) armoryengine.py:1044 -    Make sure that you have the SWIG-compiled modules
(CRITICAL) armoryengine.py:1045 -    in the current directory (or added to the PATH)
(CRITICAL) armoryengine.py:1046 -    Specifically, you need:
(CRITICAL) armoryengine.py:1047 -        CppBlockUtils.py     and
(CRITICAL) armoryengine.py:1051 -        _CppBlockUtils.pyd

Traceback (most recent call last):
  File "X:\BitcoinArmory-master\armoryd.py", line 49, in
    from armoryengine import *
  File "X:\BitcoinArmory-master\armoryengine.py", line 1039, in
    import CppBlockUtils as Cpp
ImportError: No module named CppBlockUtils
----------------------

Any suggestions?
I'm using the latest source from https://github.com/etotheipi/BitcoinArmory, downloaded as zip.

2) Ubuntu 12.04
I put the txjsonrpc into the suggested folder, I used this one: https://github.com/oubiwann/txjsonrpc
I tried installing psutils-1.2.1:
----------------------
$ python setup.py  install
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib.linux-i686-2.7
creating build/lib.linux-i686-2.7/psutil
copying psutil/_pslinux.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psbsd.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_compat.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psosx.py -> build/lib.linux-i686-2.7/psutil
copying psutil/__init__.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_pssunos.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_error.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psmswindows.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psposix.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_common.py -> build/lib.linux-i686-2.7/psutil
copying psutil/error.py -> build/lib.linux-i686-2.7/psutil
running build_ext
building '_psutil_linux' extension
creating build/temp.linux-i686-2.7
creating build/temp.linux-i686-2.7/psutil
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-i686-2.7/psutil/_psutil_linux.o
psutil/_psutil_linux.c:13:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------
Since install failed, I put psutils into the same folder that I'm trying to run the armoryd.py from, then try to run it:
----------------------
$ python armoryd.py
Traceback (most recent call last)
  File "armoryd.py", line 49, in
    from armoryengine import *
  File "/home/linuxuser/Documents/arm/armoryengine.py", line 47, in
    import psutil
  File "/home/linuxuser/Documents/arm/psutil/__init__.py", line 88, in
    import psutil._pslinux as _psplatform
  File "/home/linuxuser/Documents/arm/psutil/_pslinux.py", line 20, in
    import _psutil_linux
ImportError: No module named _psutil_linux
$
----------------------

More suggestions?

I haven't had a need to use JSON across languages yet, but I understand that is one of it's features.

From http://en.wikipedia.org/wiki/JSON

You are trying to run it as a bash script.  You need to run it as a python script:

"python armoryd.py".

Also, as someone else noted, you'll need to download the txjsonrpc project and put it in your base directory (there should be a txjsonrpc dir next to armoryd.py).  Or actually get txjsonrpc installed -- it's an extra dependency not needed by regular Armory but needed for armoryd.py.

Hi

I guess this is really basic, but I still need some help getting this script running.
So I'm running this on a linux machine, I've installed python, and I'm trying to run the script.

$ ./armoryd.py
from: can't read /var/mail/twisted.internet
from: can't read /var/mail/twisted.web
from: can't read /var/mail/txjsonrpc.web
from: can't read /var/mail/txjsonrpc.auth
from: can't read /var/mail/twisted.cred.checkers
from: can't read /var/mail/armoryengine
./armoryd.py: line 51: import: command not found
./armoryd.py: line 52: import: command not found
./armoryd.py: line 53: import: command not found
./armoryd.py: line 54: import: command not found
./armoryd.py: line 55: import: command not found
./armoryd.py: line 56: import: command not found
./armoryd.py: line 59: import: command not found
from: can't read /var/mail/jsonrpc
./armoryd.py: line 61: syntax error near unexpected token `('
./armoryd.py: line 61: `class UniversalEncoder(json.JSONEncoder):'
$

So then I installed twisted:
$ sudo apt-get install python-twisted

But I still get the same message.
Trying to run it on Windows I get the same import problem.

Any suggestions?

Question2
When I get the script running and can use it to create offline transactions etc, can I have it interact with another program in another language I've written?
If so, can I do it over localhost, and send RPC or something, getting replies in json?

Great news, web-armory people! The daemon is being included with armory itself so will be up to date and more fully featured than I ever could have achieved.

see https://bitcointalksearch.org/topic/m.1443054 for details.

I will keep this thread open to discuss the particulars of using armory on webservers, as the daemon is not strictly only for use on webservers (may be used for, say, running your uniquely common yet complex task easily, or remotely controlling your armory application).

You are right, this was the problem.
Thanks!

Status on my Debian 6 server:
- operational bitcoind daemon
- download BitcoinArmory-Daemon (https://github.com/thedawnrider/BitcoinArmory-Daemon.git)
  with latest armoryengine.py by mav
  (added Armory and Armory-Daemon dependencies)
- Replaced some code in armory-daemon.py (line 217 https://bitcointalksearch.org/topic/m.1437378 by etotheipi)
- download and make BitcoinArmory (https://github.com/etotheipi/BitcoinArmory.git)
- replace CppBlockUtils.py and _CppBlockUtils.so on Armory-Daemon from BitcoinArmory
-> start Armory-Daemon ($ python armory-daemon.py)

Now starting to explore how the armory-daemon works...

Then how are you running with the new code?  The .so most definitely needs to be recompiled to use the new stuff.  I would expect seg faults, not hanging, but computers are weird....

I recommend checking out the current Armory branch to a different directory, type "make", and then copy the CppBlockUtils.py and _CppBlockUtils.so over to the BitcoinArmory-Daemon directory.  Then it should work.

With this change I get the following error:

If I set:

There are no errors, also armory-daemon does not get started.
Well...I think it does not get started because I do not receive this message:

The process seems like to hang. Most of the time a ^C helps me out, now I have to shut down the terminal.

I expect the online mode needs a TheBDM.setOnlineMode(true) but I can't get to the code triggering the error.

By the way, I am not able to compile the CppBlockUtils.py or _CppBlockUtils.so code.

Actually, this may not be so bad.  I was busy preaching about all the changes and how it was all worth the effort given the usability improvement for Armory GUI ... but actually you may be relatively unaffected as long as you set blocking=True.  In general, the GUI is advancing, not the library, so you don't have to upgrade like this often.

I just browsed your code and didn't see a whole lot that has to be changed, except you need to replace "TheBDM.LoadBlockchain()" with "TheBDM.setBlocking(True);  TheBDM.setOnlineMode(True)".  There might be a couple minor things that you have to update, and please post here with any more issues and I bet we can clear them up quickly.  But overall you don't appear to be using many things that changed.

Aaaaack been a long time since I looked at this and forgot to copy the new _CppBlockUtils.so to the repo. I have not changed any c++ code, this is just a wrapper. I am going to put this on hold until the changes are made. Then I will properly re-assess the situation and put this in a fork of BitcoinArmory rather than as a standalone daemon with the .so binary included.

That's definitely going to have to be recompiled. 

On that note: mav, did you change any of the C++ code?  Or were you just using the stock armory code, and just putting a wrapper around it?  If you just wrapped it, it may be better to just fork the BitcoinArmory repo and add your .py files to it.  Then updates are just a "make" away, and goes pretty smoothly on all linux distros (few dependencies, and no version requirements on them).

MAV thanks!

I have a fully operational bitcoind daemon (Debian 6) and also get the same errors.

Does the CppBlockUtils.py and/or _CppBlockUtils.so can stay on the same version or do they need a more accurate version also?

Oh boy.  I forgot about this thread, and should've warned you that I converted Armory to a multi-threaded application.  As such, some of the interfaces changed a bit.  There is an example of blockchain loading in the extras/sample_armory_code.py file, but I'm not sure all the samples are up-to-date for the new multi-threaded code.  On the upside, just about any function that doesn't return something (like blockchain scanning) you can use wait=False and it will run in the background allowing you to do other operations while you wait. 

You'll see what I mean if you look at the blocking vs. non-blocking methods, shown in sample_armory_code.py:


As above, if you set blocking to True, it should behave much like the original... the main thread always waits for a response from the BDM before continuing.  On the other hand, if blocking=False (default):


In this case, the blockchain scan happens in the background and the main thread continues running immediately, giving you an opportunity to do other calculations or operations in the main thread.  If you use the setBlocking=False, you should ALWAYS check theBDMState() before making a call like getTopBlockHeader() which you expect to return data -- anything that returns data is blocking by default.  Use a conditional like this:


"Uninitialized" and "Offline" are pretty much the same, so I usually use "if TheBDM.getBDMState() in ('Uninitialized','Offline'): ..."

Anyways, I'm sure there's other things that are new, but obviously ArmoryQt.py is functioning correctly, so there's lots of sample code in there.  Really, 95% of it is the same, just some new/renamed calls and slightly different arguments.  For this application, you probably want setBlocking(True), since you might just be adding complication to an app that doesn't actually need the multi-threading.  If you run the latest Armory GUI, you'll see why I made the change.  MUCH better user experience...

Sorry about that!  Please pester me about more errors, and I'll help you recover from this under-the-hood overhaul...
(and sorry in advance, there's probably another one coming... upgrading to completely new wallets and unicode support...)

I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py

Unfortunately I am unable to load the blockchain, and am getting these errors.


Any ideas why these errors are being thrown? This is happening in TheBDM.run()

The daemon still runs and correctly responds to getnewaddress however I haven't got a suitable wallet to test the other calls, namely:
getbalance
getreceivedbyaddress
listtransactions
sendtoaddress

Please see https://github.com/thedawnrider/BitcoinArmory-Daemon/blob/master/armory-daemon.py for the source

Thanks! Great software!

You are right, there is no alternate chain for change addresses.  It is possible to end up with accidentally re-using an address once, especially if your server is very active in giving out addresses.

However, in version 0.82 I added a new feature to the Expert interface (in the GUI) that lets you specify the change address to use.  It allows you to send change back to the first input address (may be better than the alternative), or you can supply your own address.  You can't specify another wallet yet, but I want to allow that, eventually.

However, the new wallet format that will come after beta, is based on endless discussion with Pieter Wiulle (bitcoin-qt dev), and that will result in wallets, by default, having two different subchains for each wallet -- the second chain will be used solely for change.  The new wallet format is a ways off (hopefully Beta isn't!), but it is coming.  For now, it sounds like switching your desktop to Expert usermode is your solution.

I made a simple experiment and noticed that change addresses come from the same sequence. Is there a way around this feature?

I am sorry, I am misunderstanding something. I like the idea of deterministic address generation without private keys. But does Armory deterministically separate change addresses from normal ones?

Let's say I don't want to use offline transactions. I just want to generate addresses with watching-only wallet (being run on the web server machine) and then spend the received bitcoins from the full wallet (being run on my desktop computer).

I like this idea since it doesn't require any synchronization between wallets.

But would it be safe, or would Armory generate change addresses that may conflict with addresses generated on server? Is there any chance to make this simple scheme work?

I tested several send transactions in a row (with same parameters), and for each transactions the inputs and outputs are the same. Which is logical since armory look at the existing adresses balances to construct a transaction, if a transaction is not broadcasted it can't be taken into account.

I don't know if etotheipi is gonna change this behavior in the future, but for now you have to broadcast an unsigned transaction before creating the other one, no other way around it.

What I'm thinking to get around this limitation is to have a computer, completely behind a firewall, which pull from production servers all unsigned txs when they are created, signed and broadcast them. Then it send a signal to the productions servers (delete the unsigned tx file?) to go ahead and create the next unsigned tx if needed.

That's not as secure as offline signing, especially if signing is automated, but I think it's still relatively secure compared to existing solution with bitcoind.

This is an interesting point you make. On one hand it seems fairly certain you are right about this, but maybe etotheipi can confirm whether the wallet can handle the creation of multiple unsigned transactions at a time? It seems like a fairly likely situation to happen under most normal use cases, even with the gui client. If the daemon software can avoid having to implement a strict 'sequential' series of transaction where the server must create-a-tx-then-announce-before-creating-the-next-tx, it would make life much much easier.

TBH I always assumed transactions could be grouped and processed offline as a batch. If armory does require one-at-a-time, that would become a very limiting factor for even a moderately popular web-based service.

Well thank you, you did the most part, and I had an interest in this as I'm launching a new bitcoin service using this script.

I hope you tested it before pushing to git, I did but the more testing the better. Thanks for pushing it to github anyway

Anyway I'll keep updating the code if I find so way to improve it with. But for now I have to focus on a way to generate an offline TX and push it to an offsite computer to be signed and broadcast immediately. Because if you don't broadcast your unsigned tx immediately, the next generated one should be incorrect :/

Massive kudos for making this happen unclescrooge. Armory takes a fair bit of poking around, so it seems you've done well with this.

I added the code and have written some basic tests, unfortunately time is unexpectedly short for me this weekend so later in the week I will try adding the missing fields from listtransactions (and properly confirm that the existing fields are correct).

See the latest on github for the changes. I appreciate the efforts you have put in unclescrooge. It's nice to know people are looking at this, gives me motivation to continue on it. I was planning to use this in a project I was working on, but that has been put on the backburner and as such armory-daemon has languished.

Ok so I updated the previous code which didn't work at all.

Now I manage to get working  "getbalance()", "listtransactions()" (only a few items that I think important are returned though), "sendtoaddress()" (create a file containing the unsigned transaction in the current directory and return a random string).

Do not use in production server without testing though:

Thanks for the response mav. Although I never touched python in my life nor am I a programmer, I added some codes from ArmoryQt and manage to get getbalance and listtransactions working (for this last only a few items are returned and some are wrong so I need to work more on that).

Here is yur modified code so far, ugly as hell lol:

Sorry for the slow response, I am subscribed to this thread but very rarely come on this forum any more so have not been updated on the new posts. I recently started at a new job which has sadly diverted my attention away from bitcoin dev.

I will be looking at this code on the weekend. I made some changes a while ago but never pushed them to git. By the end of the weekend check back on this thread, there should be updates. It's been a while since I've looked at this daemon, and certainly have not been using it in production on any of my systems. But from memory I also could only get getNextUnusedAddress to work properly, although the other calls did something slightly related to the required task, just not something completely useful yet.

The end of the weekend - I am sure some updates will be ready.

Hi mav,

You daemon is very very useful. However I haven't been able to make any function beside getNextUnusedAddress work. getbalance() return 0 even though there's one adress with 1 btc on it, listransactions doesn't return anything (but seems like the function isn't developped yet in your code) and getreceivedbyaddress raises an unknown error.

Did you manage to make it work?

Any way I'm not a python programmer but I'll try to debug that and post what I found, if any

unclescrooge

Just thought to check this thread again, and that daemon looks like it could be extremely useful.  Thanks for working on this and putting it out for the public to use!

I was pointed to this thread by etothepi when I sent an email asking about generating addresses for my server to use, and ended up deciding that the best implementation for my needs was to generate chunks of addresses at a time with the method described in the first post, and putting them into a redis list (acting as a queue) which would get refilled when it dropped below a certain amount of addresses.  I'll probably stick with my current implementation as it is more than sufficient for what I need right now, but I will certainly keep an eye on this daemon and might decide to switch to it later on. 

The api for everything except getnewaddress depends on loading the blockchain. If you stripped it right down to just that one call, then yes it would be great when combined with bitcoinmonitor.net, and as you say, it's not even required to have bitcoin running. But for any functionality beyond getting new addresses from the wallet, you will still need to have bitcoin running so there's access to the blockchain.

This looks very interesting. Combining this with bitcoinmonitor.net url callbacks (using the API) will provide you a complete solution to accept payments without the need to put your hot wallet online anywhere and without the hassle to set up/maintain a long list of pre-generated addresses with your bitcoinmonitor.net agent. Awesome! 
As soon as I find some time I will set this scenario up for one of my sites.

-  There is full re-org/double-spend handling, which has been tested at the library level.   But it hasn't been tested at the GUI/interface level, because I never set up a way to test block/tx injection over the networking interface.  I see re-orgs happen all the time -- you'll see output to the console that says "Invalidating old chain tx -- Marking new chain tx valid".  For web-servers, this should be all you need. 

- Armory is pretty dumb when it comes to... lots of network-stuff.  It was written with the assumption that Bitcoin-Qt is going to feed it trustworthy, reasonable data.  I have noticed, under strange testing conditions, if a conflicting tx happens to make it in, it will show up in the ledger but it actually won't affect your balance at all.  Again, this is not all that relevant for web-servers, but it's worth noting:  Armory receives the conflicting tx, and immediately detects it's relevant, and adds it to the ledger on the GUI.  Then Armory tries to add it to it's pool of tx for each address, etc, but finds those outputs have already been spent.  So it gets dropped, and the underlying UTXO list is untouched.  It rarely ever matters, because Bitcoin-Qt/bitcoind can't pass it conflicting tx...

So I need to fix the auto-display-in-ledger bug.  But balances and spendability should maintain correctness in this situation.  Please let me know if they appear not to.

As for storing data between loads:  some people complain about the load times, etc, but it comes with a tremendous benefit of robustness, because there's no way for Armory to ever be "out of sync" with the blockchain.  One issue with Bitcoin-Qt is that it can mark outputs as spent, or believe that some transaction succeeded that wasn't accepted and get stuck in a state that isn't consistent with the rest of the network.  Armory avoids all this by having no memory.  At the expense of load times.

This functionality will eventually have to be added to do this (especially with the blockchain size getting so big), but right now it's not there, and it will probably require a lot of testing.  If the server is going to process lots and lots of transactions separately, you might consider taking my examples above for creating transactions, and add that memory/tracking around it (maintain the tracking yourself).  Maybe that's too much work...  I don't know.

---

Btw, I haven't looked at your stuff yet, but this sounds fantastic!  Thanks so much for doing this, and the JSON-RPC interface is something I've wanted to do for a while.  Perhaps we'll find a way to merge functionality together...

The simple Armory Daemon is now on github. It aims to be as close to a direct replacement for the Satoshi interface as possible.

https://github.com/thedawnrider/BitcoinArmory-Daemon

Available json-rpc calls are based on the Satoshi rpc interface and are as follows:

getbalance
getnewaddress
getreceivedbyaddress
sendtoaddress

getbalance:
Returns a decimal value in BTC for the total remaining balance in the wallet.

getnewaddress:
Returns the next address in the wallet as a string.

getreceivedbyaddress:
Returns a decimal value in BTC for the amount received by the address.

sendtoaddress:
Returns an unsigned transaction as a string. Implementation of signing and broadcasting is left to the client.



Features that may be included in the future:
User authentication
SSL
More API methods as required (the current set fills my needs). listtransactions is one method I am looking to include in the future.
Please suggest anything else you personally may find useful, as I have designed this only to fill my specific needs.



Clarifications required:

- Are incoming transactions which eventually do not become part of the blockchain dealt with correctly? I am unsure how / unable to test this.

- What happens if multiple transactions are made which would result in the balance being less than zero? Presumably when it comes to broadcasting the last of the signed transactions there will be an error from the Armory client which is performing the signing/broadcasting. Should tracking of the likely future balance in the daemon be enforced even though the transactions have not yet been broadcast and maybe never will be? How should this be managed, if at all?

I see you point, but a slim install and a firewall would be all you need.  A lockdown of 'only allow connection to port XYZ from host ABC' would be sufficnt.

That's true. The thing is though, that I imagine IP connectivity opens up a lot more potential attack surfaces than a USB connection. I mean, any program on the key box listening for incoming connections will suddenly be an attack surface. And it's really hard to control which programs do this; a lot will. With the device acting as a USB client this won't be an issue. You can't - not as far as I know at least - "listen" for connections via USB, and so, there will only be one central point - one attack surface - that we need to secure, which is the USB driver and the program that communicates over this connection. If this is secure, the key box should be secure.

I would like to point out that you do not have to use USB. Ethernet would actually work fine for this. Just have to cards in the webserver; one for Internet and the other for the key-box.

Then you can just set static addresses (or even run a dhcp server on the keybof, but that increases your attack surface).

Anyway you achieve the same goal. There is only one connection to the keybox and it is not on the interwebz.

The private key (let's call it the master private key) would basically belong to the person who needs this setup (web server plus key-storage box (the latter having the private keys for the wallet (lets call these wallet private keys))) to produce and send out valid transactions, spending coins from the wallet(s) on the key-storing box. The key-storing box would then have the corresponding "master" public key, and it will only sign a transaction sent to it over the USB connection if it is signed using the master private key.

My initial use case was, as mentioned, me selling bitcoins for fiat currency. I have a site that informs the user to send an amount of fiat currency to my bank account, and include the address - to which the bitcoins are to be sent - in the "comments" field, when sending the wire transfer via his or her e-banking site. The bank I would use to provide my account would have the master private key. Whenever a new bank transfer to my account takes place, the bank sends me an email containing information on the transfer - including the destination bitcoin-address that the buyer included in the "comments" field - signed with the master private key. Then it's simply a matter of having the online web server read new incoming mails from the bank, and pass on the content of the mails to the key-storage box. The key-storage box checks that the wire transfer message is signed with the master private key, and if it is, constructs a transaction that sends an amount of bitcoins to the address specified in the wire transfer message. It passes this to the web server which publishes it. All automated, and - as far as I can see - unhackable, provided the software on the key storage box is trusted.
The only thing that struck me while writing this, is that we need some way of telling the storage box the amount to send. This can't really be included in the signed wire transfer message, as the customer shouldn't be in charge of this. It seems this would be an unsafe operation. Ie. a customer could place an order, hack into the web server, and change the amount being sent to his address, since the storage box can't know the bitcoin exchange rate, since it isn't online. I guess we'd need Mt. Gox to sign a piece of data containing the current time and exchange rate, so the box can know how many coins to send.

Are you imagining three devices? I only imagine two: the web server and the device. The device stores the keys, and does the signing - provided it receives a properly signed message that tells it where to send the coins.

You and me both! I think I'll be looking into getting the Raspberry Pi to act as a USB client. It would then be necessary to use the USB protocol, which is a lot more complicated than just serial, but would not - as far as I know - allow login to the offline machine.

EDIT: I just discovered that the Raspberry Pi Model A (as opposed to Model B that is out now) will support acting as a USB device/client, because it uses a different communication-chip than Model B. Model A doesn't have an Ethernet port, and only has a single USB port, but I figure this might even be an advantage, since after the software has been installed on the device (using, for example, some USB-to-Ethernet type device to update packages), the Ethernet port will have to be removed because the USB connection needs to be used to communicate with the online computer. Perhaps one will be less tempted - and feel safer - if the device doesn't even have a port to plug in an Ethernet cable.
Additionally, it costs only $25. That's really not much for the ultimate in safe Bitcoin wallets. That being said, Model A isn't out now. And USB client mode won't necessarily work out-of-the-box. This is a software issue, as far as I can gather. Not OS level, I think, but firmware level - so it might not be that easy to implement. But still, looks like a device is coming to the market that fulfils the requirements of an offline hardware wallet. Off the top of my head, I can't see what else one would need other than a Raspberry Pi Model A, and perhaps a USB cable. But most people have USB cables lying around. Sure beats that $40 2xUSB-to-serial-plus-null-modem solution.

This seems like a good idea to post on the Improving Offline Wallets Thread.

I like the hardware idea, except I don't understand where the private key & signature comes from?  It's not stored on the online system, is it? 

Why do you need any signing?  Why not just use that device to facilitate passing ASCII-only between the two systems?  Any non-ASCII buffers are rejected, and the offline computer will do nothing except collect the data and wait for the user to act on the received data.  Really, it would just behave like a serial port.

Damnit!  I wish I could make the serial port idea work...

Another possible web server setup that is possible with Armory (that I've actually thought of setting up myself), is having the web server connect via USB to a small Linux ARM device (just something cheap) that has a USB OTG port (allowing the device to act as a USB client). This USB client device stores the private keys and only responds to request over USB. With this setup, it's possible to simply send transaction requests to the web server that are signed with a certain private key, let the web server forward the request to the device, and then let the device - connected only via USB and not connected to the Internet - verify that request with the corresponding public key, and only return a signed transaction over USB if the signature on the request can be verified with the public key on the device.

This should - if implemented correctly - make the wallet unhackable. The only thing an attacker can do - even if he gains full access to the web server - is send request over the USB connection to the device that hold the keys, but without the proper private key the device will ignore the requests.

This could be useful for use cases such as an exchange needing to transfer coins from its cold storage wallet to it's online wallet. The exchange owner simply has to make sure that the private key used to sign requests is inaccessible to attackers.
Another use case could be a company exchanging fiat currency for bitcoins. Here the bank is in possession of the private key. It promises to sign all incoming wire transfers with this private key. If the wire transfer - in a comments-field of some sort - includes a receiving address, it's possible to let someone wire a certain amount of fiat currency to the business owners account, and let the business send out coins to that person - completely securely - given that the customer includes the send-to address in the wire transfer. If the bank can keep the private key secure, this - again - prevents an attacker from being able to do anything were he to gain full control of the web server.

I mentioned in my original post about this that this part is not trivial.  It's because selecting coins for a transaction is not a one-shot process -- you have to declare what your desired output amount is and your desired fee, then Armory will create the best transaction it can based on those parameters.  If the transaction has properties that require a bigger fee, then you need to re-select coins with the new fee.  Theoretically, this should be part of a while-loop or something (in case the new selection requires an even-bigger fee), but I designed the SelectCoins algorithm to use a selection that leaves some margin (if possible) so that a higher fee can be included without changing the coin selection.

The functions for reference are:
validateInputsGetTxDP() at qtdialogs.py:4867,
createTxAndBroadcast() at qtdialogs.py:4811
(Maybe)broadcastTransaction() in ArmoryQt.py:1889).  
PySelctCoins() in armoryengine.py:5049.  

There's a lot of code here, because it is also validating the data was entered properly on the Send-Tx dialog.  But what you need is there.  I'll try to summarize it here.

PySelectCoins arguments:

• unspentTxOutInfo: a list of unspent TxOuts, retrieved via wallet.getTxOutList("Spendable") (qtdialogs.py:4968)
• targetOutVal: total amount to send to (all) recipients(s), in satoshis  (i.e. 10.0*ONE_BTC for 10BTC)
• minFee: you can always set this to 0 the first time, and re-call it with higher if the output suggests more
• numRand (optional): number of randomized SelectCoins solutions to try (more about this in the appendix, below)
• margin (optional): how much margin you'd like to build into the selection, for the reasons stated above

Actually executing the transaction -- you must get all the spendable coins, make a coin selection, then calculate the change and generate the recipient list (you will need to wallet.getNextUnusedAddress() generate a change address, or you can do something else).  Shuffle the list.   Then create the "PyTxDistProposal" -- this is an intermediate step where the transaction is constructed as a BIP 0010 packet, in case it needs to be signed by an offline computer.  In the case that you have the full wallet -- this TxDP will be signed immediately, then converted to a standard transaction and broadcast.

And finally, here's the code sample you've been waiting for:  Create one transaction from a wallet, to send 10.0 BTC to one recipient


I suppose I could encapsulate much of this code in a more friendly way.  On the other hand, you can pretty much copy this code and make the appropriate adjustments, and you'll get a signed transaction.  You can modify it for multiple recipients, handling fees differently (but you should trust the calcMinSuggestedFee[1] value), and saving off copies of the transaction before sending.

I will include more about actually broadcasting the transaction, in a bit.  The problem is that you need to start the networking engine in order to actually broadcast it.  On the other hand, you have a PyTx object (finalTx), so you can call binary_to_hex(finalTx.serialize()) to get the raw hex version of it, which you might be able to broadcast in other ways.  (like "sendrawtransaction" RPC call with bitcoind)

I know you'll have questions.  Fire away!

---

Appendix: The way PySelectCoins works:  a bunch of deterministic coin-selections (solutions to the pseudo-knapsack problem) are generated based on a variety of standard coin pools (the types and sizes of unspent TxOuts for your wallet).  A few random solutions are thrown on top to generate some variety.  Usually, the deterministic algorithms find the best answer, but for strange coin-pool make-ups, some semi-randomized selections will produce decent answers.  In all, we get about a couple dozen possible solutions, and then each one is evaluated for "fitness" ... how big is the fee?  how many addresses are combine?  how obvious is it which output is the recipient and which one is change?  The relative weightings of each of these is defined in armoryengine.py:5000 -- the WEIGHTS variable defines how much relative weighting to use for each factor.  There's six of them, though it really could've been reduced to 3 important ones:  tx-fee/allow-free, input-anonymity, and output-anonymity.  


You can manually modify these weightings, or find a way to inject a new weights matrix into the PyEvalCoinSelect (I apologize, I didn't make that easy, you might be better to just manually change armoryengine.py, for now).  What do these mean?  The default behavior is to avoid using zero-conf coins at all costs.  Any solution with no zero-conf coins will be scored higher than any solution with zero-conf.  The next two biggest weights are "ALLOWFREE" and "NUMADDR".  This means that equal weighting is given to "please give me a transaction that can be free" and "please use as few addresses as possible on the input side".   "TXSIZE" and "PRIORITY" are next, but probably shouldn't even be there, because they are already tied into the ALLOWFREE and NOZEROCONF scores.  

Finally, OUTANONYM is the last critieria.  Given a set of coin-selects that are basically all the same for the other 5 criteria, it will select the solution that has better output-anonymity.  i.e. -- if the outputs are 10.0 and 1.3482023, it's fairly obvious to someone observing the blockchain which one is the recipient, which is the change-back-to-self (1.3482023).  However, if a solution is found that gives 11.34 and 5.11 -- it's not so obvious anymore, and that solution will get a higher score.  It gets extra credit if it's deceptive:  if the recipient is getting 1.384 and it finds a solution that has a change of 8.1:  then most would observers would make the wrong conclusion.

The point of this description was not only to describe the weightings, but suggest you change them for your own usage.  I had planned to make this customizable in Armory, I just hadn't gotten around to it.  If all you care about is input-anonymity and output-anonymity, I suggest changing NUMADDR and OUTANONYM to high numbers, and change the other 4 to something tiny.  You might pay a fee when you otherwise wouldn't have to, but it will be more anonymous.

DISCLAIMER:  I make no guarantees about transaction anonymity at all.  Use of this feature is completely at your own risk.  This is implemented solely for the purposes of declaring a rough ordering for an under-defined set of solutions.  Setting any particular weight to 10¹⁰ and others to 0 does not mean you are guaranteed a "good" solution -- there may only be one solution.

This functionality is very simple to achieve and is outlined in the first post thanks to etotheipi providing the example. This speaks volumes about how nicely armory is designed, it's very easy to work with. I have started on an api interface which currently is able to get the next address in the watch-only wallet.

I have had difficulty in achieving my next two goals with armory which is to get the balance of an address, and to make a transaction for signing by the offline wallet. Unlike generating the next address, both of these require the BDM to be loaded (ie the blockchain to be buffered), which I have been able to achieve. However, implementing the functionality I desire has not yet happened, I have had too many other things which have taken priority, and I have not looked closely enough at the source to realise these goals.

If anyone is able to make this extra functionality work I would love to hear how you did it. I have written a very basic foundation which allows interface to armory wallets, however it's so basic that I have not yet uploaded it for public consumption. The project deserves a more complete implementation than what I have started. I don't seem to recall anything being in the example file about generating transactions.

Delay in upload is also due to considering whether to implement a satoshi-clone api or to create an api specific to armory. Any suggestions on this are also welcome.

Exactly.

I recommend using an offline computer to make the full wallet, then make a watching-only copy.  Put it on your webserver, and also import it into Armory on your primary system.  Then your webserver will distribute addresses, and you can see the full transaction history from your desktop.  If someone gets the watching-only wallet... well all they can do is watch, but they can't take any funds!

I don't recommend generating addresses from the watching-only wallet, while the webserver is doing the same.  There won't be any fireworks, but you might end up re-using an address by accident.  If you want to also be able to distribute addresses from your primary system, I recommend making a second wallet on the offline computer just for that purpose.  You can watch them both from Armory, but only distribute addresses from the one the server isn't using.  Then you can also execute offline transactions from either wallet using the online computer.

So i can basically create as many receiving addresses on the webserver as I want, but without any risk in case the server gets compromised? Awesome!

That kind of documentation has not made it into my priority list, yet.  However, threads like this are specifically for users to share ideas about how they successfully leveraged armoryengine for their purposes.  And while I don't have time to make such documentation (yet), I will respond to such threads and help folks accomplish their goals. 

Also, if you checkout the git project, there is a file called "extras/sample_armory_code.py" which has a lot of example code.  It's not as good as actual documentation, but working example code is quite useful.

I have been looking for this information on the Armory website (like a developer API section), but I'm sure it will be up there at some point.

Thanks for this! It will be highly useful.

(1) Ultimate/Full includes every, confirmed and not confirmed
(2) Spendable includes everything (SentToSelf || >1 confirmation)
(3) Unconfirmed is (NotSentToSelf && <6 confirmations)

EDIT:  I realize now it would've been clearer to say avoid "Spendable" and just say "Confirmed".  Though maybe not: you send coins to yourself in Armory and they show with 0 confirmations, yet it still shows up as confirmed.  That doesn't fit well with folks conditioned to believe that nothing is confirmed until 1+.  Ehhh... whatever.  I'm leaving it.

Indeed, twisted seems to be the way to go. I've written a twisted web server which allows for threaded interaction with the armory wallet and blockchain.

I'm looking for feedback about what functionality to incorporate (and in what format) from people who may want to use this.

Currently I've got a basic web server with the following url structure (only basic calls for my needs so far but I plan to extend it further)

URL	DESCRIPTION	ARGS	OPTIONAL default	CONSIDERATIONS
/wallet/getAddress	gets the next unused address from the watch-only wallet
/blockchain/getAddressBalance	gets the balance of an address (not necessarily in your wallet)	address=X	balType=Spendable | unconfirmed | ultimate	Unsure to use balType from armory client or minconf from satoshi client

My doubt with the url design is whether to emulate as closely as possible the internal class structure of armoryengine or whether to design an entirely new abstraction with wallet and blockchain etc (like above table) which is independent of the wallet type (satoshi / armory etc). My thoughts right now are to replicate the internal structure of armory, since this is a wrapper for armory and not a generic application. But then I think perhaps a generic web interface would be good to have...

Any particular feature requests will be given priority over general coverage (once I've satisfied my own needs), so make them in this thread.

Suggestions about how the implementation are most welcome.




Also, I am unsure about the difference between spendable, unconfirmed and ultimate in address balances. Can someone clear this up for me? It didn't behave how I expected. eg:

Into one address I had 1.2 btc transferred with 8 confirmations and 2.1 btc with 1 confirmation, and I got this result:
spendable - 3.3
unconfirmed - 0
ultimate - 3.3

This is why I prevent Armory from opening twice.  Wallet operations are not threadsafe. Not at all!  

However, one benefit of python-twisted is that it is guaranteed to run as a single-thread, so as long as there is only one instance of Armory, it's perfectly safe.

For your purposes, not using python-twisted, need to do something else.  Actually, I would recommend using python-twisted to create a server/daemon process to handle (serially) all requests for wallet interaction.  Make sure everything goes through that interface.  The wallet design has been remarkably robust (I even test it with mid-write interrupts and it recovers fine).  But as you can see, two simultaneous operations can break it pretty easily.  

Anticipating that servers often create multiple threads to handle concurrent requests, I wanted to get a feel for how armoryengine would respond to attempts at fetching many addresses at the same time. I ran the following test code and it borked my watch-only wallet file, to the point where I couldn't even get an address out of it without threading.


Whenever I call readWalletFile on a borked wallet file, with or without threading, I get the following error:


The purpose of this post is just to gather the info, I will look more into how to handle this in the near future. Any suggestions from wise heads are welcome.

Hah!  I didn't even notice this thread.  I'll offer lots of advice for you...

Most importantly, checkout extras/sample_armory_code.py.   That has a lot of details about how to load the blockchain and access address balances.  Constructing transactions isn't so obvious, but it's do-able.

I'm kinda swamped this month, but I'm subscribed now and will provide tips on using the armoryengine.py where I can.

Btw, I did intend for armoryengine.py + _CppBlockUtils.so/.dll/.pyd to be a nice standalone, full-featured library.  I'm pretty sure that all the ArmoryQt.py and qtdialogs.py code only uses armoryengine.py, but does not expand it (thus, anything that Armory does, you can do with just those files).  There's no JSON interface yet, but I think I can add it...

 
P.S. -- The dependencies for just armoryengine+cppBlockUtils is:


All of it is in the Ubuntu repos.  Doing this in Windows is far from easy, but I'm pretty sure the original instructions posted 6 months ago still work.

I found an answer for these questions and have posted some code about how to get the balance of an address.

Calling BDM_LoadBlockchainFile is very slow (tens of seconds on my machine) since I'm guessing it loads and parses the entire blockchain. I'm going to look into how to incorporate that command (probably with some others to make it useful) and create a kind of 'bitcoind' for armory, which will be able to access the watch-only armory wallet, and also be able to get balances for that wallet. It seems from the comments in the armory source code that any address balance can be easily fetched, not just addresses in the wallet (can't find the comment sorry).


As for a Ruby port, sorry I'm not going to be much help cause I don't know any ruby. Perhaps someone else will step up and investigate. Possibly if a bitcoind style implementation of armory can be developed, with a json api, perhaps your ruby could access armory through that interface rather than as a direct language port. Time will tell.

I would totally donate a few coins for an Armory port to Ruby

Here is some more info which I found from testing today.

I wanted to cut the armory source down to remove dependency on qt, and thus run just a simple script to access the watchonly wallet on my web server without installing the whole of armory.

My goal in this post is to get new addresses from a watch-only wallet using python, on a server without qt installed.

Steps:

From the source at https://github.com/etotheipi/BitcoinArmory I took only the following file:

• armoryengine.py

There were two more files I needed, which are not in the github repo (presumably compiled on install), so I got them from my armory install, located in /usr/share/armory and added them to the folder containing the above three files.

• CppBlockUtils.py
• _CppBlockUtils.so

Note there is no dependency on qt for any of these files.

I copied a watchonly wallet from armory into the folder containing the specially selected files above, and created a test.py file as below. It worked! The test script read the wallet file and created / printed the details of the next address.


This allows the web server to run a light-weight non-gui version of the watch-only capabilities of armory, which utilises the watch-only wallet.

Note that this code does not seem to allow interaction with the bitcoin network, only with the wallet. Thus, getting the balance of an armory address from the watchonly wallet is not possible with this code (which is my next task to figure out). Any suggestions how to go about it are most welcome. I imagine this functionality should be possible from the satoshi bitcoin client, but I have yet to figure out how to get the balance of an address not in the current (satoshi) wallet. I'm also looking into how armory connects to bitcoind and how it gets balances for addresses.

Any suggestions or comments are most welcome, this is all new to me.

Armory Daemon is now available on github:

https://github.com/thedawnrider/BitcoinArmory-Daemon

More info on page 2 of this thread - https://bitcointalksearch.org/topic/m.1197291

-------------------------

I thought I'd start this thread as a place to collate information about using Armory client on a webserver, since the only info I found so far was hidden on page 16 of the Armory Discussion topic.

The info below is about getting an Armory address on a webserver and it comes from https://bitcointalksearch.org/topic/m.765880

Anyone who has information or questions about using Armory on a webserver, please post it here so webdevs can make the most of Armory, which currently seems to be focusing on GUI interaction rather than API (understandably).




I put some thought into how Armory could be used for a webserver application, and I realized that with just a couple lines of python, you can generate addresses no problem on your webserver.  I added a couple convenience functions and committed them to the master branch of Armory. Here's what you do:

(1) Load Armory on your computer.  
(2) Create a regular wallet.  
(3) In Wallet Properties, "Create Watching-Only Copy"
(4) Upload watchonly wallet to webserver
(5) Setup webserver to load wallet and call wlt.getNextUnusedAddress() for each request
(6) Sit at home with the full wallet and watch/confirm/send transactions while the webserver does the work!

Demonstration code to see how to access/manipulate the wallet from a python script:


Here's the output on a watching-only wallet (this is actually the output of the second time I ran it):


Your webserver can run blissfully ignorant about what is actually going on with the wallet, it just gives out a new address on every request.  You can sit at home on your computer with Armory running and the full wallet there, with full control.  But someone who compromises the webserver will not get anything!

If you want the webserver to track incoming transactions, it will require loading the blockchain and checking for new blocks periodically.  It's not a lot of code, but I'll save it for later if you are interested.

One other thing:  Armory uses the crypto++ library, which is not particularly fast at ECDSA operations (and my key generation algorithm is kinda slow, which is why I'll be upgrading to a new one, soon).   My [decent] system can generate about 200 addr/sec.  So this script takes a few sec to run.  But all keys are stored in the wallet, so they don't have to be recomputed when you restart the script.