This will probably never work in regtest.
Armory has 2 paths to broadcast a transactions:
1) P2P
Armory is connected to your Bitcoin node as a node itself. It uses that to grab zero confirmation tx from the network, new block notifications as well as broadcasting transactions. The P2P layer is not meant to operate according to a client server paradigm however.
When a node broadcasts a transaction on the network, it first announces it has transactions by hash in an inv_tx packet. The recipient nodes then decide at their own discretion if they want any of the transactions in that inventory batch. They will request these tx by hash from the origin node. At this point, the exchange is over.
These 2 steps are the "send" mechanic, where Armory sends to the tx to the node.
Next is the "get" step. In a client/server setup (i.e. Armory operations), you want an ACK on whether the tx was accepted. What we are trying to achieve when broadcasting a transaction from Armory is to get that tx in the Bitcoin network's "mempool". Simply pushing that tx to our node is not enough, we need to know whether that tx was accepted by the node as well, which is a strong enough guarantee the tx will be passed around the entire network in consequence.
To do so, after the node has grabbed our tx, we then try to grab that tx back from the node's own mempool, which demonstrates the node has in fact accepted the new tx. The issue with this method is 2 fold.
a) First, the tx will not be available in the node's mempool until the node pushes the inv packet itself to its connected peers. This process can take a while, as Core like to batch their inv_tx together before pushing that stuff out. This is why P2P broadcast is unreliable on testnet/regtest, as there's not enough traffic to prompt quick inv_tx forwarding.
Keep in mind that your node thinks this tx is just one of many floating around the network, that it may or may not need to bounce it to other nodes for propagation. It has no sense of priority nor duty to push that stuff out. Your node just doesn't know this is its own original tx and it needs to propagate it ASAP, and there is no facility to convey that over the P2P.
b) Next, since Armory pushed that tx to your node, it will never see the inv_tx packet in return. From the perspective of your node, it is wasted bandwidth to return the inv of this tx to its point origin. Therefor, Armory has to poll your node's mempool blindly, hoping to find that tx in there. If it fails to, the result will be a "timeout (get)".
2) RPC
The tx is passed to your node's RPC interface. This is guaranteed behavior with ACK/nACK and error verbose.
-------------------------
- Why not always use the RPC?
There's no guarantee the user has gone through the steps to enable the RPC, nor that it is desirable. Only the node is required to operate Armory, not the RPC. Also, in terms of performance, the RPC is at least an order of magnitude slower.
- Why use the RPC interface at all then?
If the RPC is available, ArmoryQt will use it as fallback for tx broadcasting in case the P2P method fails.
- RPC broadcast in armoryd
There's method for this in armoryd atm. You can add it easily by copying the code in jsonrpc_sendasciitransactionraw and replacing the one line with RPC broadcast (https://github.com/goatpig/armoryd/blob/master/armoryd.py#L323):
Armory has 2 paths to broadcast a transactions:
1) P2P
Armory is connected to your Bitcoin node as a node itself. It uses that to grab zero confirmation tx from the network, new block notifications as well as broadcasting transactions. The P2P layer is not meant to operate according to a client server paradigm however.
When a node broadcasts a transaction on the network, it first announces it has transactions by hash in an inv_tx packet. The recipient nodes then decide at their own discretion if they want any of the transactions in that inventory batch. They will request these tx by hash from the origin node. At this point, the exchange is over.
These 2 steps are the "send" mechanic, where Armory sends to the tx to the node.
Next is the "get" step. In a client/server setup (i.e. Armory operations), you want an ACK on whether the tx was accepted. What we are trying to achieve when broadcasting a transaction from Armory is to get that tx in the Bitcoin network's "mempool". Simply pushing that tx to our node is not enough, we need to know whether that tx was accepted by the node as well, which is a strong enough guarantee the tx will be passed around the entire network in consequence.
To do so, after the node has grabbed our tx, we then try to grab that tx back from the node's own mempool, which demonstrates the node has in fact accepted the new tx. The issue with this method is 2 fold.
a) First, the tx will not be available in the node's mempool until the node pushes the inv packet itself to its connected peers. This process can take a while, as Core like to batch their inv_tx together before pushing that stuff out. This is why P2P broadcast is unreliable on testnet/regtest, as there's not enough traffic to prompt quick inv_tx forwarding.
Keep in mind that your node thinks this tx is just one of many floating around the network, that it may or may not need to bounce it to other nodes for propagation. It has no sense of priority nor duty to push that stuff out. Your node just doesn't know this is its own original tx and it needs to propagate it ASAP, and there is no facility to convey that over the P2P.
b) Next, since Armory pushed that tx to your node, it will never see the inv_tx packet in return. From the perspective of your node, it is wasted bandwidth to return the inv of this tx to its point origin. Therefor, Armory has to poll your node's mempool blindly, hoping to find that tx in there. If it fails to, the result will be a "timeout (get)".
2) RPC
The tx is passed to your node's RPC interface. This is guaranteed behavior with ACK/nACK and error verbose.
-------------------------
- Why not always use the RPC?
There's no guarantee the user has gone through the steps to enable the RPC, nor that it is desirable. Only the node is required to operate Armory, not the RPC. Also, in terms of performance, the RPC is at least an order of magnitude slower.
- Why use the RPC interface at all then?
If the RPC is available, ArmoryQt will use it as fallback for tx broadcasting in case the P2P method fails.
- RPC broadcast in armoryd
There's method for this in armoryd atm. You can add it easily by copying the code in jsonrpc_sendasciitransactionraw and replacing the one line with RPC broadcast (https://github.com/goatpig/armoryd/blob/master/armoryd.py#L323):
Code:
#TheBDM.bdv().broadcastZC(pytx.serialize()) #comment this out
TheBDM.bdv().broadcastThroughRPC(pytx.serialize()) #use this instead
TheBDM.bdv().broadcastThroughRPC(pytx.serialize()) #use this instead
Thanks a lot for the explanation goatpig! To me it wasn't clear from the documentation/source of sendasciitransaction() or even the raw variant that it was never going to work. I ended up hacking around my stuff by getting the raw hex for the tx and then broadcast via bitcoin-cli. I understand your point about blindly relying on the RPC though.
The other thing I've noticed is that it desyncs ArmoryDB and seems to prevent armory (even after a fresh build and new .armory folder being created to ever catch up to the regtest blockchain. So basically if you, inadvertdly try it once, you are stuck redownloading/building/scanning.
I didn't care too much about the performances here as I only wanted to see that the tx that I was pushing was good enough.
Thanks for the clarifications once again.
