Topic: Constructing raw P2WSH by hand: witnessScript does not match scriptPubKey (Read 229 times)

Alright, makes sense. Thanks again for your help! I sent a tip to express my gratitude. I was stuck for some time on these issues and you really helped me figure out what was going on and some details I was missing, so I'm very grateful.

With Bitcoin Core's tooling, there is no way. You will have to do it manually or find other tools to make the signature for you.

Ah, I see. Thank you for the detailed reply!
I suppose "concatenate any signatures to the witness script" is just so common that I assumed it would be somewhat standard, but you're absolutely right, it definitely doesn't have to be.

If I wanted to press on, is there a way to generate a signature (and leave me to create the final witness manually), or at that point am I left with basically reimplementing it myself or hooking directly into the Bitcoin Core code? Since the "concatenate the signature with my script" is the only step that I'm missing for signing the thing automatically.


Thanks, I'll give that a try! Looks more promising than doing it by hand.

Yes, that is the error.

It is because the script is non-standard.

Generating the signature is easy, it's constructing the witness that is hard. While for this particular script constructing the witness is easy, the only way you know that is because you have analyzed the script and determined how to make the witness. For other scripts it's not so easy. As a script gets more complicated, you need to reason about what it does in order to figure out how to create the final witness. Since it is hard to do, Bitcoin Core just does template matching - it will only sign and make a witness for scripts that match some predetermined templates. The script that you have chosen does not match any of those templates.

That's a bug and has been fixed for 0.21.

Yes, there is no way to use Bitcoin Core to sign for arbitrary P2WSH and P2SH outputs.

There is a project which does allow for a general witness creator. It's called Miniscript and it is able to work on arbitrary scripts because it limits the scripts to certain opcodes and requires certain opcode ordering in order for it to be able to understand what a script does. Once this is implemented in Bitcoin Core, it will be possible to create and sign for arbitrary scripts so long as they conform to Miniscript. However we are still a ways out from this getting into Core.

I will also admit that my original attempt used a much more complex witnessScript, which would succeed either on a 2-of-2 multisig, or on a single sig after a checksequenceverify timelock expired. In my attempts to simplify it yesterday night, I must have botched something in the process. Now that I've gotten the witness script to at least match and the error is different, I'll give it another try with my original script, and if I get anything interesting I'll add it here.

You can try using btcdeb

So I tried to do it again and somehow got a different error this time:

Is this what you meant? I've included a full log below just in case if not (if that's the expected error, then feel free to skip the second half of my post).

Could you clarify why it wouldn't be able to sign my transaction? Would it be because it's non-standard? I'm not sure how that would affect things. From reading the spec, in particular BIP141 and the Script definitions, my impression is that signing a transaction is a simple matter of serialising part of it and calculating the ECDSA signature, which shouldn't depend on the actual script being used (taken from https://en.bitcoin.it/wiki/BIP_0143). Then, as https://en.bitcoin.it/wiki/BIP_0141 defines it, constructing the witness script would simply involve concatenating the resulting signature with whatever is passed as the input to the command. Am I misunderstanding something here?

I did give it a try using OP_CHECKSIG for a canonical pay-to-pubkey script, but bitcoin-tx recognised it as a pubkey script and created a P2WPK output, which is not what I want. Does this mean that there's no way to use Bitcoin Core to sign P2WSH outputs, or am I misunderstanding what you said about OP_CHECKSIGVERIFY? If so, how would one even sign and use such a transaction?

I will also admit that my original attempt used a much more complex witnessScript, which would succeed either on a 2-of-2 multisig, or on a single sig after a checksequenceverify timelock expired. In my attempts to simplify it yesterday night, I must have botched something in the process. Now that I've gotten the witness script to at least match and the error is different, I'll give it another try with my original script, and if I get anything interesting I'll add it here.

EDIT: After re-running with my full script, I am still getting a script mismatch. This makes me believe I am making some mistake in the serialisation of the full script. Is there a canonical tool that I can use to "compile" Script? I have found some javascript library which does the job, and was using a combination of that and just reviewing it by hand, but I must be missing some subtlety, possibly in the encoding of the sequence number or similar. I am slightly surprised that there is a decodescript RPC but no encodescript or similar; what would be a good way to make sure I canonically serialise it? I suppose I could dig inside bitcoin core and find a way to call the code it's using directly, but surely there's a more end-user facing interface somewhere?

Anyway, as promised, here's the full log of what I'm doing - in case that error wasn't what you expected, or if you by any chance spot something I'm doing wrong. I'll be fully verbose to make sure every step of what I'm doing is clear, so apologies I include some things which seem trivial or unnecessary (and also for the large size of the upcoming post).


At this point I have a wallet with an address, corresponding pub/priv keys, a balance and a funding transaction to spend from. So I proceed to create the P2WSH transaction:


Now we have out P2WSH transaction confirmed:

So I try to spend from it:


At this point I also serialise my script:


And now trying to sign the spending transaction:

Hey, at least the witnesscript matches, unlike yesterday.

Yes. Private keys aren't necessary, only if you want to share them.

I tried doing exactly the same thing you did, but did not run into your error (although I ran into another one which I'll explain below). The likely culprit is that you have specified the scriptPubKey incorrectly.

The error I ran into is one that you will run into too. Specifically, it's that Bitcoin Core cannot sign this transaction because you are using OP_CHECKSIGVERIFY. It looks like you are going for the typical Pay to Pubkey script, but the Pay to Pubkey script uses OP_CHECKSIG, not OP_CHECKSIGVERIFY. Unfortunately Bitcoin Core does not have a general signer (note that this is a hard problem) and thus cannot sign your transaction because it does not understand a script that uses OP_CHECKSIGVERIFY.

The witnessScript I'm using is just a push of a pubkey followed by op_checksigverify. Would you like me to post the verbose log of what I did, including the actual values for the public keys, private keys and addresses?

I'm not sure what you mean by the funding transaction here - the address was initially funded by coinbase transactions, from running the generatetoaddress command (this is all on regtest). Then I created a transaction spending from one of the coinbases, with a P2WSH output - this was successful, and my process is also described in the post here (that's TX when being built, and the txid is saved as NEWTX once it's broadcast - granted my naming isn't the best for this). Then I'm trying to spend the P2WSH output, which is where I'm encountering the issue.

Can you post the actual witnessScript and the funding transaction (the one that sent money to the address)?

I've been trying to create and spend from a P2WSH transaction, for learning purposes, and have been unable to construct a correct spending transaction.

Here are all my steps, starting from a fresh wallet :
1. Get wallet address, public key, and private key, mine some blocks to it:

2. Grab an unspent output from "bitcoin-cli listunspent" and create a P2WSH output transaction:

3. Sign and broadcast:

4. Create a transaction spending from NEWTX (still sending to myself):

5. Attempt to sign it:

At this point I'm getting the error
.

I cannot figure out why it wouldn't match. My serialisation would be the obvious culprit, but decodescript confirms it is correct. I have found https://bitcointalksearch.org/topic/generate-p2wsh-redeemscript-does-not-correspond-to-witnessscript-5236818, but that bug was apparently fixed in Bitcoin Core 0.20, and I am on the latest release (0.20.1).

Any pointers would be greatly appreciated.

Here's the standard support template, if necessary:
Bitcoin Client Software and Version Number: 0.20.1
Operating System: Linux 64-bit
System Hardware Specs: N/A
Description of Problem: See above
Any Related Addresses: N/A (regtest network)
Any Related Transaction IDs: N/A
Screenshot of the problem: N/A
Log Files from the Bitcoin Client: N/A