Pages:
Author

Topic: dev branch work (Read 304 times)

legendary
Activity: 3794
Merit: 1375
Armory Developer
April 07, 2022, 04:26:41 PM
#23
1. Haven't imported a wallet with private keys in a while, time to retest it I guess.

2. How old is your DB? Current code received a set of fixes for txfilters vs what 0.96.5 puts out. Did you setup from scratch or used an existing db state?

3. I forgot this button even existed o.o', will fix

Thanks for the report.

Quote
It seems like porting to a different Qt version/lib was pretty clean in terms of outcome, so that's something. The dashboard wallet scan animation was always a little "janky" under Xen, so some gui bugs are already fixed!

There are some minor syntax changes from Qt5 to Qt4, but its essentially the same lib. I was surprised how little impact the migration had on looks. I was expecting a struggle just to port it and then end up with botched dialogs. Py2 to Py3 was a lot more disruptive in the end.
legendary
Activity: 3430
Merit: 3080
April 06, 2022, 06:35:48 PM
#22
CppBridge loads all wallets in the datadir and converts them to the new format on the fly. Old wallet files are ignored past that point. Those wallets that carry encrypted private keys will lead to a prompt in GUI to unlock them. The same password used to unlock them will be used to encrypt the new file.

some issues I found:

1. one (encrypted) wallet throws this exception:
Code:
(INFO) ArmoryQt.py:4486 - Dashboard switched to "Scanning" mode
terminate called after throwing an instance of 'Armory::Wallets::IdException'
  what():  [EncryptionKeyId] invalid key size
(INFO) ArmoryQt.py:5277 - BDM is safe for clean shutdown


2. Another converts, then scans, then logs these errors when resolving tx hints:
Code:
-INFO  - 2022-04-06 - 23:08:29.411: (BlockchainScanner.cpp:1671) xxx blocks hit by tx filters
-ERROR - 2022-04-06 - 23:08:34.169: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters:
-ERROR - 2022-04-06 - 23:08:34.184: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.184: (BlockchainScanner.cpp:1470) Skipping this block
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters:
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1470) Skipping this block
-ERROR - 2022-04-06 - 23:08:34.186: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters:
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1468) Block deser error while processing tx filters:
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1469)   raw data does not match expected block hash
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1470) Skipping this block
-ERROR - 2022-04-06 - 23:08:34.187: (BlockchainScanner.cpp:1470) Skipping this block

3. Upon file->quit, we have this:
Code:
(INFO) ArmoryQt.py:5277 - BDM is safe for clean shutdown
[2022/04/06 23:18:39:3578] N: __lws_lc_untag:  -- [wsi|0|pipe] (0) 22.654s
[2022/04/06 23:18:39:3826] N: __lws_lc_untag:  -- [vh|1|default||-1] (1) 22.677s
[2022/04/06 23:18:39:3826] N: __lws_lc_untag:  -- [wsicli|0|WS/h1/default/127.0.0.1] (0) 22.677s
[2022/04/06 23:18:39:3826] N: __lws_lc_untag:  -- [vh|0|netlink] (0) 22.677s
(ERROR) CppBridge.py:220 - Socket error: [Errno 9] Bad file descriptor
(ERROR) ArmoryQt.py:5288 - Strange error during shutdown
Traceback (most recent call last):
  File "/home/user/bitcoinarmory/./ArmoryQt.py", line 5280, in closeForReal
    TheBridge.shutdown()
  File "/home/user/bitcoinarmory/armoryengine/CppBridge.py", line 390, in shutdown
    self.clientFut.result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 440, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/user/bitcoinarmory/armoryengine/CppBridge.py", line 259, in readBridgeSocket
    response = self.bip15xConnection.decrypt(\
  File "/home/user/bitcoinarmory/armoryengine/BIP15x.py", line 312, in decrypt
    raise AEAD_Error("failed to decrypt payload: " + str(decryptionResult))
armoryengine.BIP15x.AEAD_Error: failed to decrypt payload: -1
(INFO) ArmoryQt.py:5290 - Attempting to close the main window!

the ArmoryQt.py process does terminate ok.


It seems like porting to a different Qt version/lib was pretty clean in terms of outcome, so that's something. The dashboard wallet scan animation was always a little "janky" under Xen, so some gui bugs are already fixed! Tongue
legendary
Activity: 3794
Merit: 1375
Armory Developer
April 06, 2022, 02:25:25 AM
#21
I can't see how it makes sense to scan the blocks for transactions in that case, but that happened, and all was well. Sadly, there was no surprise BTC Undecided Grin

Failure on my part, "isNew" flag not being propagated correctly I expect.

Quote
It seems there are methods in cppforswig/BridgeAPI/WalletManager to convert the 0.96.x format to the new one? Can I call them somehow, or is there no userspace interface yet?

CppBridge loads all wallets in the datadir and converts them to the new format on the fly. Old wallet files are ignored past that point. Those wallets that carry encrypted private keys will lead to a prompt in GUI to unlock them. The same password used to unlock them will be used to encrypt the new file. There is no client facing API to call the conversion code directly. It is all baked into loadWallets (https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/BridgeAPI/WalletManager.cpp#L199).

It will be part of the last major new piece of GUI I need to implement. I want to add a settings dialog that precedes ArmoryQt itself to do a few things:

---------
- Wallets: need a tab that lists all wallets found in the data dir and offer users the choice to convert. I don't like the tacit conversion but it was a cheap solution at the time. I'm very bad at GUI development =D. Also need to offer the option to unlock wallets. The new wallet container encrypts public data with a different passphrase. This is set to a cleartext key by default, but with some minor GUI at wallet creation time, users can have the option to set a passphrase of their own choosing.

With fully encrypted wallets, users need unlock the public data prior to loading Armory itself. I could have gone with a simple message box prompt but that would get very obnoxious with more than 2-3 wallets loading.

---------
- AEAD key management: CppBridge couldn't talk to ArmoryDB because ArmoryDB wasn't set to --public. All sockets are encrypted now (this means from ArmoryDB to CppBridge and from CppBridge to ArmoryQt). The AEAD allows both 1-way auth (client authenticates the server) and 2-way auth (server authenticates the client back).

It defaults to 2-way, meaning ArmoryDB's key vault needs to know of CppBridge's key prior to the connection. As for ArmoryDB's key, if it's missing from CppBridge's vault, the user should be prompted to accept the key, but since I'm lazy, I defaulted the accept to true instead of writing a prompt (https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/BridgeAPI/CppBridge.cpp#L324).

Since ArmoryQt spawns CppBridge, adhoc keys are setup every run, and discarded after use.I haven't looked at automating ArmoryDB yet, so you're left running it yourself. As it is missing the keys in its vault, it will reject all clients, unless you run it with --public (accept all clients). The 1-way and 2-way handshakes are incompatible on purpose, there is no downgrading of a 2-way handshake to 1-way for convenience purposes.

There's a command prompt tool to manage keys, aptly dubbed "KeyManager" (I'm good with names 'n stuff) -> https://github.com/goatpig/BitcoinArmory/blob/dev/cppForSwig/KeyManager.cpp

This needs some GUI around it, considering it isn't just for your local instances, but remote ones as well (considering running a supernode for those people who can't get fullnode Armory up locally, just so they can move their coins out). With 2-way auth, you can actually manage who gets in a WAN exposed DB and/or CppBridge.

This tab will also be a good place to manage IPs for remote DB/Bridge. The bridge handles all things wallets, ArmoryQt is more or less a dumb interface now, so you could run CppBridge in say a Docker container and plug ArmoryQt to it from somewhere else.

---------
- DB/Core management: Need some GUI for people to discover block/db paths to ease the pain around initial setup. Also a place to reintroduce the settings for Core and DB automation. It's a lot harder to make the bootstrap consistent if ArmoryQt starts before the DB or Core. It's simpler to assume these pieces of software have to run before ArmoryQt is up and handle the complexity in this new pre-start dialog.
legendary
Activity: 3430
Merit: 3080
April 05, 2022, 01:34:58 PM
#20
yep, that worked Cheesy

I created a new wallet. I can't see how it makes sense to scan the blocks for transactions in that case, but that happened, and all was well. Sadly, there was no surprise BTC Undecided Grin

It seems there are methods in cppforswig/BridgeAPI/WalletManager to convert the 0.96.x format to the new one? Can I call them somehow, or is there no userspace interface yet?
legendary
Activity: 3794
Merit: 1375
Armory Developer
April 05, 2022, 07:55:23 AM
#19
ok, so I finally have the dev branch in some kind of working state (real life + building problems intervened before)

one problem persists, namely that I cannot get the CppBridge to launch ArmoryDB in latest dev branch. No clue whether this is because it's broken or just that I did something wrong. ArmoryDB works fine on it's own, but launching ArmoryQt.py once ArmoryDB is started/synced does not create the bridge connection. Yes, I re-ran the c20p1305_cffi.py script. And also make clean. Not in that order Tongue

Try running ArmoryDB with --public
legendary
Activity: 3430
Merit: 3080
April 04, 2022, 04:58:25 AM
#18
ok, so I finally have the dev branch in some kind of working state (real life + building problems intervened before)

one problem persists, namely that I cannot get the CppBridge to launch ArmoryDB in latest dev branch. No clue whether this is because it's broken or just that I did something wrong. ArmoryDB works fine on it's own, but launching ArmoryQt.py once ArmoryDB is started/synced does not create the bridge connection. Yes, I re-ran the c20p1305_cffi.py script. And also make clean. Not in that order Tongue
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 18, 2022, 06:54:33 AM
#17
Works for me. Thanks for the help.
newbie
Activity: 8
Merit: 0
January 17, 2022, 05:45:43 PM
#16
I will try to clean this up a little more based on your comments, then after this can try to keep my commits more granular. This changeset became too unwieldy as it got bigger.
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 09, 2022, 10:33:09 AM
#15
Reviewed the full changeset, here are my comments:

Quote
######################
## review comments

ui/MultiSigDialogs.py:
    HighPrioTODO and some commented lines are in infraction of python indent
    rules. This code will not run as a result (intentional?)

qtdialogs/qtdefines.py:
    STRETCH as seen in qtdialogs.py has been redefined in qtdefines.py.
    Files importing STRETCH still look for it in qtdialogs.py (mostly). It
    should be imported from qtdefines.py only and STRETCH should be removed
    from qtdialogs.py (it makes more sense in qtdefines)

qtdialogs/ArmoryDialog.py:
qtdialogs/DlgHelpAbout.py:
qtdialogs/MsgBoxWithDNAA.py:
    Hardcoded path to images in the root img folder is invalid
    (i.e ./img/MsgBox_info48.pgn). These paths are probably invalid now that
    the files live one folder deeper (should be ../img instead, I suspect)

qtdialogs/MsgBoxCustom.py:
    Same image pathing issue as with MsgBoxWithDNAA.
    Also, the file uses the old connect/SIGNAL syntax which is invalid in
    pyside2, e.g:

        self.connect(btnOk, SIGNAL('clicked()'), self.accept)

    it should be

        btnOk.clicked.connect(self.accept)

qtdialogs/DlgWltRecoverWallet.py:
qtdialogs/DlgCorruptWallet.py:
    These dialogs are deprecated, new wallets do not make use of it. Shouldn't
    spend time fixing these. There is no cause to remove them however, they're
    not imported anywhere. May revive this stuff later but the whole functionality
    is obsolete due to how the new wallet files work: they're fully encrypted,
    with MACs, so file corruptions will always be detected natively instead of
    potentially altering wallet data silently.

qtdialogs/DlgSettings.py:
    This dialog is super broken and will take a lot of work to fix besides
    the pyside2 migration stuff. I'm thinking of removing those features in
    there that are broken and reimplementing them in the "manager" dialog
    instead of fixing them in place. Shouldn't spend time on this code as
    a result.

qtdialogs/RestoreWOData.py:
    This dialog requires a decent amount of time to work with both Armory
    legacy wallets and BIP32 wallets. The new wallet format also allows for
    exporting the wallet structure and meta data (all data that isn't private
    keys), as well as encrypt that data for transit, so all things related to
    WO and meta data import/export will need a UX rework.

qtdialogs/DlgRestoreSingle.py:
    The stuff that applies to RestoreWOData applies to wallet restore in
    general. Put another way, time shouldn't be spent trying to simply fix
    these dialogs, that effort will be lost when reworking the restore
    dialogs to fit the new wallet functionalities.

qtdialogs/DlgRestoreFragged.py:
    Same as above. The SSS code is dead atm anyways so this feature is nowhere
    near funcitonal.

qtdialogs/DlgPasswd3.py:
    Invalid img path.
    Invalid connect/SIGNAL syntax.

qtdialogs/DlgInflatedQR.py:
    Missing a few imports.

dynamicImport.py:
    Depreacted file used to load plugins, a feature that was never really
    fleshed out. Ignore.

jasvet.py:
    Python implementation of the Bitcoin message signing protocol. Unsafe and
    deprecated for signing, only used for verifying signatures. Should be fully
    replaced with calls to CppBridge.

pytest/*.py:
    The python tests are deprecated. A few of them can be revived but over
    95% of what they cover has been moved to CppBridge. Coverage only
    really make sense for CppBridge now. The python client is now a dumb UI
    that does not implement any core logic, only GUI elements. CppBridge is
    an API process that communicates via protobuf, so it's pretty easy to
    flesh out a headless test suite for it instead.

qtdialogs/.pylintrc
    This looks like some sort of linter dependency and/or a syntax helper.
    These kind of files shouldn't be committed.

ui/toolsDialogs.py:
    Minor indent infractions in the import section.

######################
## files with nothing to comment on:
ui/QrCodeMatrix.py
ui/Wizards.py
ui/CoinControlUI.py
ui/FeeSelectUI.py
ui/TxFramesOffline.py
qtdialogs/DlgWalletSelect.py
qtdialogs/DlgWalletDetails.py
qtdialogs/DlgUnlockWallet.py
qtdialogs/DlgShowKeyList.py
qtdialogs/DlgSetComment.py
qtdialogs/DlgSendBitcoins.py
qtdialogs/DlgRequestPayment.py
qtdialogs/DlgReplaceWallet.py
qtdialogs/DlgQRCodeDisplay.py
qtdialogs/DlgProgress.py
qtdialogs/DlgOfflineTx.py
qtdialogs/DlgNewAddress.py
qtdialogs/DlgMigrateWallet.py
qtdialogs/DlgKeypoolSettings.py
qtdialogs/DlgIntroMessage.py
qtdialogs/DlgExportTxHistory.py
qtdialogs/DlgEULA.py
qtdialogs/DlgDispTxInfo.py
qtdialogs/DlgConfirmSend.py
qtdialogs/DlgChangePassphrase.py
qtdialogs/DlgBrowserWarn.py
qtdialogs/DlgAddressBook.py
ArmoryQt.py
armorymodels.py
armoryengine/ArmoryUtils.py

I'm considering merging this stuff into the dev branch and moving to a file by file or feature by feature basis past this point. What do you think? Do you want to cleanup this PR some more or move on?
newbie
Activity: 8
Merit: 0
January 07, 2022, 03:56:16 PM
#14
The reindent tool also deleted some continuation lines at the end of files, so I restored them.

Would like to help where I can. I'll let you know what I'm thinking after I've spent some more time using the dev branch build.
newbie
Activity: 8
Merit: 0
January 06, 2022, 06:21:27 PM
#13
Looks like the reindent tool has some problems. Will send whitespaces fix patch. Will respond to your other comments after.
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 06, 2022, 09:53:29 AM
#12
I've gone over the change set, it's fairly straight forward. A few comments:

- The indent issue still exists in ui/MultiSigDialogs.py and ui/Wizards.py. Could you please fix those and submit only these changes as a new commit on top of the attic_qtdialogs branch? The rest of the work is good, we can start fixing the minor issues on a step by step basis.

- The patch introduced a lot of new and unnecessary white spaces, as well as deleting the new line at the end of several files, both of which are style infractions and tend to pollute changesets. Fortunately, git can fix a lot of this on its own (git am --whitespace=fix *path/to/patch*). It can't do anything about the missing new line at file end however, but can be addressed in a small commit. In the future, please try not to commit whitespace changes, possibly by having git ignore them when creating a patch.

- Apparently, I've fucked up with the license header in the segregated dialog files. It looks like I've elected to set the header to "Armory Technologies Inc 2011-2021" when I'm not allowed to do this (I'm not ATI), and it's wrong (I've done the work, whereas ATI is defunct). You've copied that license header along in your work and that made me realize it needs fixed. The license itself is also wrong, ATI uses AGPLv3, whereas I use MIT (more permissive). I do not have the right to loosen ATI's license, therefor the new code has to be copyrighted to me so that it can benefit from the looser license, hence the double license header which is the correct formulation. I'll fix those as I go, feel free to do the same.

- This syntax doesn't not carry over from PyQt4 to PySide2:
Code:
self.connect(self.btnAccept, SIGNAL("clicked()"), self.verifyUserInput)

The correct syntax is:
Code:
self.btnAccept.clicked.connect(self.verifyUserInput)

This stuff amounts for a lot of the syntax errors left.

------------------------

With regard to development, currently the following features are FUBAR and need a lot of attention. I don't intent to fix these in the near term:

- fragmented backups
- multisig/lockbox dialogs
- message signing

The following GUI elements are still broken but fairly easy to fix. I will likely give them attention:

- Wallet creation/restore wizard
- Changing the password on wallets
- Some minor display issues such as wallet labels, security mode, ownership

On top of that, there are obsolete features that are/will be replaced:

- The settings menu
- Wallet corruption checks and recovery

There is a set of new features that I want to add in the scope of the next release:

- A new "manager" dialog that will appear before the main lobby. It would have a few tabs with the purpose of walking the user through setting up Armory prior to start up. The tabs would be:
Quote
. Wallets: load all wallet files in the datadir, help with migration from the old to the new format, assist with unlocking public data (new wallet format encrypts all wallet data, public data can be locked with a separate password that requires a one time unlock at load)

. ArmoryDB: prepare all things related to setting up ArmoryDB. The new DB enforces AEAD, and supports both 1-way and 2-way auth. While local ArmoryDB requires path settings, connecting to a remote DB requires key exchange. This tab would assist with that.

. bitcoind: This tab will be succinct for now, just help with detecting bitcoind and custom pathing, possibly automate the process. Something to flesh out in the future.

- Transaction export format: There is a new format for passing transactions around now. This format is not backwards compatible with previous Armory versions. The new signer code also handles PSBT, so the idea is to allow (when eligible) for users to choose between formats when exporting signed/unsigned transaction blobs. The formats would be legacy, with a cutoff at 0.92, one at 0.96.x, PSBT and the new format.

- Support for paying to taproot addresses.

If you want to help with any of these, name the task and we can coordinate.
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 03, 2022, 11:39:48 AM
#11
Thanks a lot, will get to it sometimes tomorrow, busy with another PR atm.
newbie
Activity: 8
Merit: 0
January 03, 2022, 11:10:41 AM
#10
Ok. I am not using an IDE but ran the reindent tool from pypi, which only allows indent with 4 spaces for some reason.

Fixed them and am sending you a new patch.
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 03, 2022, 07:01:47 AM
#9
First remark, this may seem silly but it is what it is:

The Armory code uses 3 spaces per tabs instead of the usual 4. Your editor is set to use 4 so it auto rearranged a bunch of files (all the new files, as well as completely re-indenting MultisigDialogs.py and Wizard.py). Could you please stick to 3 spaces?
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 02, 2022, 01:54:08 PM
#8
Thanks for the patch! I made a branch with it, named attic_qtdialogs. Ill review the files and ask for changes as I go (if they're too big for a quick fix here and there). You can then contribute updates as stand alone patches. Once I'm happy with it, I'll merge the branch into dev. It may take me the whole week to review the change set, it's pretty big and I'm busy with a few other tasks atm.
newbie
Activity: 8
Merit: 0
January 02, 2022, 11:03:36 AM
#7
I just sent the patch to the email address you use in git.
legendary
Activity: 3794
Merit: 1375
Armory Developer
January 01, 2022, 04:49:00 AM
#6
Go ahead. It's gonna be pretty hard to back and forth on it though. Please wait till the end of the day however, so that I can push some changes I have locally, so that you can rebase on top of what I'm working against.

EDIT: new code is up.
newbie
Activity: 8
Merit: 0
December 31, 2021, 05:00:03 PM
#5
I'd like to to create a PR of the work I've done, but I don't have an adequately anonymous github account and they've made it very difficult to create one.

Could I email you a patchset?
legendary
Activity: 3794
Merit: 1375
Armory Developer
December 23, 2021, 04:11:48 AM
#4
Ok. So what about CppBlockUtils.py, which no longer exists? There are a few calls into that module in dialog code (e.g. MultiSigDialogs.py calling CryptoECDSA()).

CppBlockUtils.py was auto generated by SWIG to expose the C++ code. I don't use this anymore so all calls to CppBlockUtils in Python are now invalid and need to be replaced equivalents in CppBridge. They won't look quite the same since with the previous model, the py code would call cryptographic routines directly, whereas I've elected to isolate all of that in cpp binary now.

Most of this you're running into needs to be reworked to fit the new design, and in some cases the entire routine needs migrated to the cpp side still. I suggest you focus on those parts of the codebase that can work without that kind of heavy intervention and list those that require that. I'll get to them eventually.
Pages:
Jump to: