Author

Topic: Segwit raw transaction decoder (Read 238 times)

full member
Activity: 161
Merit: 168
January 17, 2024, 12:26:43 AM
#16
In that case, please show me a witness transaction only Legancy inputs in the blockchain.

You were right about only legacy inputs transaction since that's what the last sentence of the quote also says. It's at the beginning of the document btw. My question now is how to construct the witness in the following case:

but what if only some of the inputs are segwit?
= segwit

Good question.
Funnily enough, I am currently busy with the same task.
The legacy inputs are signed in the Sigscript field and the witness block remains empty.
The witness inputs are signed in the witness block. And the Sigscript field is put down different things.
In any case, it's a terrible mess. And I hope to never have to do this again.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 18, 2024, 11:16:22 PM
#15
Quote
It's kinda hard to write a decoder that can tell you what went wrong.
That's true. However, it should be possible to tell the user, "where" it went wrong. What about using byte position in a decoded transaction to do that? I guess telling the user that "we parsed your transaction, and since this byte, it failed our assumptions" should improve some things.

And yes, I know, "PRs welcome", and I know that such improvement should be submitted into Bitcoin Core. But for now, I will just share this idea, as it may be helpful, if OP wants to improve his own parser.

OpenTimeStamps website has a legacy transaction decoder that colorizes the fields: https://timestampserver.com/decode_bitcoin_transaction/ so it is possible.
copper member
Activity: 900
Merit: 2243
January 18, 2024, 02:10:48 PM
#14
Quote
It's kinda hard to write a decoder that can tell you what went wrong.
That's true. However, it should be possible to tell the user, "where" it went wrong. What about using byte position in a decoded transaction to do that? I guess telling the user that "we parsed your transaction, and since this byte, it failed our assumptions" should improve some things.

And yes, I know, "PRs welcome", and I know that such improvement should be submitted into Bitcoin Core. But for now, I will just share this idea, as it may be helpful, if OP wants to improve his own parser.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 17, 2024, 03:34:50 AM
#13
Thanks. I tried doing it this way, and was able to make segwit transactions.

Just need to be careful to not include extraneous witness data lengths, for example:
Code:
    for w in witness_stack: # for each input
        signed_transaction += create_varint(len(w)) # stack element count
        witness_bytes = b""
        for w_elem in w:
            witness_bytes += create_varint(len(w_elem)) + w_elem # element varint length + count
        signed_transaction += witness_bytes # Push only the stack itself, we must not include a varint length for the stack here like we would in legacy P2PKH/P2SH (!)

In this case, hypothetically if there were 8 witness items, they would not be specified as 00 x8 times, but 08 as the stack length and then each stack (containing the relevant scriptsig)?
It's each input's number of stack items followed by the stack items.
To be simpler, here's an example for 3 witness items:
Code:
...
...
...

Quote from: NotATether
And if only half of them were witnesses, then it's 04 as stack length and then the 4 stack items?
If there are non-witness amongst the inputs, witness data should be (let's say input1 above is p2pkh):
Code:
...
00
...

Refer to BIP143 for examples of mixed witness and non-witness inputs: https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#example
The number of witness items doesn't have to be specified and the arrangement is based from the inputs' index as mentioned in BIP144:
Quote from: bip-0144
The number of script witnesses is not explicitly encoded as it is implied by txin_count. Empty script witnesses are encoded as a zero byte. The order of the script witnesses follows the same order as the associated txins.
legendary
Activity: 2534
Merit: 6080
Self-proclaimed Genius
January 17, 2024, 02:53:45 AM
#12
In this case, hypothetically if there were 8 witness items, they would not be specified as 00 x8 times, but 08 as the stack length and then each stack (containing the relevant scriptsig)?
It's each input's number of stack items followed by the stack items.
To be simpler, here's an example for 3 witness items:
Code:
...
...
...

Quote from: NotATether
And if only half of them were witnesses, then it's 04 as stack length and then the 4 stack items?
If there are non-witness amongst the inputs, witness data should be (let's say input1 above is p2pkh):
Code:
...
00
...

Refer to BIP143 for examples of mixed witness and non-witness inputs: https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#example
The number of witness items doesn't have to be specified and the arrangement is based from the inputs' index as mentioned in BIP144:
Quote from: bip-0144
The number of script witnesses is not explicitly encoded as it is implied by txin_count. Empty script witnesses are encoded as a zero byte. The order of the script witnesses follows the same order as the associated txins.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 16, 2024, 11:43:58 PM
#11
In that case, please show me a witness transaction only Legancy inputs in the blockchain.

You were right about only legacy inputs transaction since that's what the last sentence of the quote also says. It's at the beginning of the document btw. My question now is how to construct the witness in the following case:

but what if only some of the inputs are segwit?
= segwit
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 16, 2024, 10:38:23 PM
#10

They're all UTXOs from a legacy address so I figured that the witness array still needs to be the total length of the inputs.


I am relatively certain that in this case you are not allowed to create a Segwit transaction. It must be a legancy TX!
Witness Field also not permitted in this TX!
Witness field may only be inserted if at least one input comes from a Segwit address.

BIP141 says this:

Quote
The witness is a serialization of all witness fields of the transaction. Each txin is associated with a witness field. A witness field starts with a var_int to indicate the number of stack items for the txin. It is followed by stack items, with each item starts with a var_int to indicate the length. Witness data is NOT script.

A non-witness program (defined hereinafter) txin MUST be associated with an empty witness field, represented by a 0x00. If all txins are not witness program, a transaction's wtxid is equal to its txid.

So if there is no segwit input, it basically means there is no marker, flag, witness field, etc. but what if only some of the inputs are segwit?

I am not sure what the witness is supposed to be in this case.

In this case, hypothetically if there were 8 witness items, they would not be specified as 00 x8 times, but 08 as the stack length and then each stack (containing the relevant scriptsig)?

And if only half of them were witnesses, then it's 04 as stack length and then the 4 stack items?

How does Core even keep track of which input a witness belongs to if there are legacy inputs scattered in the middle?
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 16, 2024, 09:26:27 PM
#9
The witness data + locktime at the end (hex):

0000000000000000

There are 8 inputs in the transaction.

Are you sure that at least one of the 8 inputs comes from a Segwit address?
At least one input must contain witness data!
I'm guessing this transaction needs to be signed in Legancy format.

They're all UTXOs from a legacy address so I figured that the witness array still needs to be the total length of the inputs.

I have not tested with a bech32 address yet, because I was busy verifying that legacy transaction creation works (it does).
hero member
Activity: 1722
Merit: 801
sr. member
Activity: 860
Merit: 423
January 16, 2024, 03:25:54 PM
#7
I could've sworn I saw a link to a raw transaction parser somewhere on this board recently, but now I can't find it. I remember it was in the OP of the topic.

I'm trying to diagnose why my code is generating wrong segwit transactions and a decoder would really help me. Most of the others either only support legacy transactions or show unhelpful errors instead of color-coding the raw hex.

https://coinb.in/#verify

https://btc.com/tools/tx/decode

Are you looking for any of the above?
staff
Activity: 3458
Merit: 6793
Just writing some code
January 16, 2024, 03:01:21 PM
#6
It's kinda hard to write a decoder that can tell you what went wrong. That generally requires being able to do analyses and identifying patterns - things that are a lot easier for humans to do. There's also the aspect that you know what you intended on doing and so can spot what is wrong, whereas a decoder probably doesn't know what you were trying to do. Usually I just decode by hand for these kinds of problems.
copper member
Activity: 900
Merit: 2243
January 16, 2024, 02:01:40 PM
#5
Quote
Does its error message show exactly which part of the raw transaction is wrong and the bytes?
It shows you the error code. And then, you can find it in Bitcoin Core source code. And then, you can jump directly to the code, to know exactly, what was parsed, and what was not.

Code:
$ grep -r "superfluous" .
./test/functional/p2p_segwit.py:        self.test_superfluous_witness()
./test/functional/p2p_segwit.py:    def test_superfluous_witness(self):
./doc/release-notes/release-notes-0.18.1.md:- #15893 Add test for superfluous witness record in deserialization (instagibbs)
./doc/release-notes/release-notes-0.20.0.md:- #17537 Cleanup and move opportunistic and superfluous TopUp()s (achow101)
Binary file ./src/test/test_bitcoin-script_tests.o matches
Binary file ./src/test/test_bitcoin matches
./src/test/script_tests.cpp:                                "P2SH(P2WPKH) with superfluous push in scriptSig", SCRIPT_VERIFY_WITNESS | SCRIPT_VERIFY_P2SH, true, WitnessMode::PKH
./src/test/data/script_tests.json:    "P2SH(P2WPKH) with superfluous push in scriptSig"
./src/txrequest.cpp:                // requested and/or completed for other reasons and this is just a superfluous RequestedTx call.
./src/script/signingprovider.cpp:    // Also note that having superfluous scripts in the keystore never hurts.
./src/secp256k1/examples/examples_util.h:     *    might try to eliminate "superfluous" memsets.
./src/support/cleanse.cpp:     *    might try to eliminate "superfluous" memsets.
./src/wallet/test/wallet_tests.cpp://! until after it is loaded. The notifications are superfluous in this case, so
./src/wallet/scriptpubkeyman.cpp:        // Related CScripts are not removed; having superfluous scripts around is
./src/wallet/rpc/backup.cpp:        auto subscript = std::move(import_data.redeemscript); // Remove redeemscript from import_data to check for superfluous script later.
./src/wallet/rpc/backup.cpp:        auto subscript = std::move(import_data.witnessscript); // Remove redeemscript from import_data to check for superfluous script later.
./src/wallet/rpc/backup.cpp:            // RecurseImportData() removes any relevant redeemscript/witnessscript from import_data, so we can use that to discover if a superfluous one was provided.
I wonder, which version of Bitcoin Core you have, because I cannot find the exact error message on master. But there are Python tests, including "test_superfluous_witness", so maybe it could help in reproducing the problem.

You can also try to construct a transaction with "createrawtransaction", and even sign it with "signrawtransactionwithkey" or "signrawtransactionwithwallet". Which means, that you can use some dummy example, and show it there, if you don't want to show real transaction ID and other real data. You can even use regtest, to make some dummy example from scratch.
member
Activity: 66
Merit: 5
Eloncoin.org - Mars, here we come!
January 16, 2024, 11:09:38 AM
#4
I'm not really into tech so I might be wrong though just try my best to learn.

So I did a quick search and I found this https://live.blockcypher.com/btc/decodetx/ I don't know if you have tried it already.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 16, 2024, 11:05:12 AM
#3
What about using Bitcoin Core, and typing "decoderawtransaction" in console?

By the way, each time I see a question like that, I wonder, how many people are not familiar with all the tools, which are built into Bitcoin Core. And many of them can be even used offline, even if you won't have a full node, with the whole chain.

Does its error message show exactly which part of the raw transaction is wrong and the bytes?

Because if not, then it's not much better than blockchain.info's decoder (which I think uses Core under the hood)

To be more specific... it's saying:

Code:
{"error":"Transaction has superfluous witness data"}

The witness data + locktime at the end (hex):

0000000000000000

There are 8 inputs in the transaction.
copper member
Activity: 900
Merit: 2243
January 16, 2024, 10:38:11 AM
#2
What about using Bitcoin Core, and typing "decoderawtransaction" in console?

By the way, each time I see a question like that, I wonder, how many people are not familiar with all the tools, which are built into Bitcoin Core. And many of them can be even used offline, even if you won't have a full node, with the whole chain.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
January 16, 2024, 10:23:47 AM
#1
I could've sworn I saw a link to a raw transaction parser somewhere on this board recently, but now I can't find it. I remember it was in the OP of the topic.

I'm trying to diagnose why my code is generating wrong segwit transactions and a decoder would really help me. Most of the others either only support legacy transactions or show unhelpful errors instead of color-coding the raw hex.
Jump to: