Topic: dev branch currently broken? (Read 218 times)

This issue should be fixed and the code in a better place atm. It's still wonky in places since I'm in the midst of refactoring the giant hardcoded qt gui files, but at least most of what's left over is limited to Qt5 migration and/or refactoring those files. Give it a spin and let me know.

Can you post the terminal output for this?

Ran the python module to generate the CFFI output, and then modified the path as you suggested. That seemed to work ok.

But whatever I've tried, I always get the AuthorizedPeersException, unexpected public key size of 0 when try to run cppBridge. I'm using --public for both cppBridge and armory-db.

Also tried letting the armory-qt start the cppBridge, and looking at the parameters it passed to make sure I'm not making a mistake there, but it's getting the same result. Arguments passed are:

 --db-type=DB_FULL --armorydb-ip=127.0.0.1 --armorydb-port=9001 --datadir=(my datadir) --satoshi-datadir=(my satoshi datadir) --public

Sorry, I'm still in the middle of a few major changes.

Connectivity between ArmoryDB and CppBridge should work. You should see a "Registered bdv xxxxxxxxx" in the db logs/command line prompt on successful connection. You should run both cppbridge and armorydb with --public at first, easier to setup this way.

The testnet flag is for the db to parse the Bitcoin testnet. If you start the db with --testnet, you need both ArmoryQt and CppBridge to run with --testnet.

For now CppBridge automation is a bit off. I suggest you comment out this line to prevent automation and spawn CppBridge yourself with the relevant command directly.

https://github.com/goatpig/BitcoinArmory/blob/dev/armoryengine/CppBridge.py#L88

As for BIP151.py, you have to build it manually atm, after building the rest of the code. You need to run the command from within the c20p1305_cffi folder to generate the python lib:


The import for this library a custom path atm. I haven't really dealt with the whole build/deploy/automation part yet, as you can tell. I suggest you change the import path to something more generic (seen here https://github.com/goatpig/BitcoinArmory/blob/dev/armoryengine/BIP15x.py#L19), such as:


to:


It should work at this point, although I ran into a circular import issue while converting some more GUI to qt5 and decided it was time to refactor the GUI files (qtdialogs.py has some 13k lines of code implementing most of the GUI, and it gets imported everywhere). Then I had to deal with an issue with the mempool code (it starts choking around 100k tx) so I had to drop progress on the GUI for a few weeks unfortunately.

Noticed your other comment about --testnet flag, I also tried this with cppBridge and armory-db but got same result.

Hi, I tried out the latest dev branch commit I could find (5ba0f0). Still failing to get things to run.

Had to make a few changes to PySide2 stuff in UI to get the QT UI to not crash on startup. It looks like you've written some BIP151 code but it there seems to be some missing stuff on the Python side. I see in BIP15x.py a "sys.path.insert" to what seems to be a local path on a developer machine. I tried commenting out use of this module.

cppBridge no longer starts up on --public mode, as it did when I was looking at things back in November. It throws an AuthorizedPeersException of "unexpected public key size", and it's getting sent a key of size 0.

That's about as much poking around as I felt comfortable doing at this point.

Sorry, lost track of this thread this week.

I'll try out your changes when I have a chance.

Maybe none of this actually affects you. I just remembered the arg list passed to CppBridge has --testnet hardcoded in there. You need to run ArmoryDB with --testnet too.

satoshi-datadir in ArmoryQt/CppBrdige does not currently have an effect. This an argument that only ArmoryDB uses. You are spawning ArmoryDB manually.


Sorry about the delay. I was gonna tell you to give me the log files but then I realized CppBridge overwrites ArmoryDB's log file, so I went about fixing that and... got into a giant refactoring. Pulled a string, unraveled the whole sweater kinda thing. I'll let you know when I push the new code, couple days from now I expect. Then you'll be able to run CppBridge again and feed me the logs for troubleshooting.

Did "ArmoryDB --datadir=(...) --satoshi-datadir=(...) --public". Verified that CppBridge was run with "db-type=DB_FULL --armorydb-ip=127.0.0.1 --armorydb-port=9001 --datadir=(...) --satoshi-datadir=(...) --public".

Here I had to add:
   bridgeArgs.append(["satoshi-datadir", BTC_HOME_DIR])

to armoryengine/ArmoryUtils.py line 3661.

At this point it launches the UI but still doesn't show any progress just as before, and state remains disconnected.

Moving away from both py2 and qt4. The dev branch as you've found out runs on py3. I have not migrated to qt5 yet, that will come soon.


A quick history on the architectural changes:

- Prior to 0.94, all of Armory C++ code was made available as a shared library to Python, using SWIG.

- The next step was to isolate the blockchain service (ArmoryDB) from the rest of the wallet. The client (ArmoryQt/Python) talked to the blockchain server (ArmoryDB/C++) over a socket, using the FCGI protocol (some older protocol from the CGI days, meant to sit behind a HTTP daemon). This connection was not encrypted. ArmoryQt would spawn the DB on its own, connect to it locally and do its thing. Connecting the client to a remote ArmoryDB was not recommended, due to the lack of encryption. Savvy users would typically go around that limitation by SSH'ing into their remote system.

- The current iteration (in the dev branch) uses WebSockets (via LWS). LWS is a robust lib that can handle exposure to the WAN. It can also be paired with a load balancing front (nginx is easy to setup for one) and/or operated behind a cloud host (cloudflare/linode and the likes). The connection has its own AEAD layer added (chacha20poly1305). You can enable TLS in LWS at your own discretion but it isn't on by default and will require setting a few flags in the code and rebuilding ArmoryDB. Since the connection between client and server is now encrypted and authenticated, you need to share keys between the 2 before a connection can be successfully established. This is where your setup is failing atm. More on this at the end.

- SWIG has also been dropped. It didn't mesh well with py3 and imposed annoying restrictions on the build system (particularly with OSX). I didn't find a satisfactory replacement for SWIG so I ended up writing what amounts to a RPC API (CppBridge). This new process talks to ArmoryDB as well as handles wallets and all the crypto. The Python client has been relegated to a dumb interface: all it does is render GUI, the heavy lifting is done in CppBridge. CppBridge will also automate ArmoryDB (this isn't implemented yet).

- CppBridge talks to ArmoryQt through a socket as well. This socket is not yet encrypted, I am currently working on this. There is no fitting chacha20poly1305 Python package out there so I am in the process of creating one myself using CFFI. This connection will be authenticated as well, so it will too require a key share. ArmoryQt automates CppBridge.

-----------------------------------------

Regarding key shares, the default mode for the AEAD channels is 2-way (client authenticates server, server authenticates client). There is a 1-way mode as well (client authenticates server, server does not authenticate client). This mode is only available for ArmoryDB. It needs to be started with --public, and so does the client (1-way clients can't speak to 2-way servers, 2-way clients can't speak to 1-way servers).

In your specific case, to get the dev branch running you would have to share the ArmoryDB's key with CppBridge and vice versa. Assuming you use a 1-way setup, only CppBridge (the client) needs to know of ArmoryDB's key (the server). The keys are stored in dedicated key stores. CppBridge looks for its keys in clients.peers. ArmoryDB looks for them in server.peers. Both these files (when used) are found in your datadir (~/.armory). Ephemeral setups ignore these files. These are used for on the fly key shares when one process automates the other. Since you are running ArmoryDB manually (no other way to do this atm), you are using static keys, hence the key stores.

You can view the content of keys stores and manipulate them with the help of a plain command line tool found in /cppForSwig/KeyManager. I have yet to write any documentation for this tool, but the code file is small and fairly easy to parse: https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/KeyManager.cpp

Currently, CppBridge is hardcoded to accept any server key in a 1-way setup. This means you do not need to actually setup a manual key share, but instead you would need to run both CppBridge and ArmoryDB with --public. ArmoryQt should already pass it this argument to CppBridge, so chances are you are missing --public when running ArmoryDB. In the future I'll remove this shortcut. By then CppBridge will have the code to automate ArmoryDB and will setup an ephemeral key share to secure the socket. It will do the same with the socket to ArmoryQt. The key share process will be for people looking to access a DB remotely and/or allow access to select known peers (or open it up to the public).

Hi,

In Fedora 32, PyQt4 is no longer shipped with python2, so I failed to get most recent tags of Armory to build.

I was able to get the dev branch to build and ArmoryDB starts successfully, and ArmoryQt.py with python3 starts up but I discovered it was having trouble launching CppBridge because of wrong working directory. With that fixed, the UI came up but state showed disconnected and had progress bars that wouldn't move.

Running the same configuration on Fedora 30 build of v0.96.5 tag, seems to run fine against my full node using satoshi port.

Happy to do some further debugging and make pseudonymous contributions, any advice on where to start looking on the UI issue?