A bit could be added for an extended SIGHASH.  Maybe SIGHASH could be a varint or something.

No, I'm dismissing offline transactions with hardware where getting n*100K of data to the device is infeasible. Microcontrollers these days are very fast - the Trezor for instance has a USB interface which is perfectly capable of getting a MB of data (10 txins given the 100K tx limit) over the wire in a second or two. QR codes are an issue, but by then you're talking about a single use-case.


Any protocol upgrade is controversial and takes a long time to implement - might as well knock off some other use cases.

For instance an alternate approach, as Mike hinted at, is a OP_CHECKSIG2. It'd be extremely valuable to make the SIGHASH flags more general, and in addition it would also be very valuable if you could sign a signature to spend a specific set of CTxOut's - scriptPubKey:value - rather than a specific COutPoint - txid:n - which would make transaction mutability far less of an issue. (also forces people to stop re-using addresses) We can add pubkey recovery at that point too, which makes transactions a fair bit smaller.

Again, you've got support from myself and gmaxwell, and it seems Mike, going down that approach. And again, remember that a set-based CHECKSIG with a merkle tree for the txouts allows you participate in big CoinJoin tx's without a lot of device overhead.

It was this transaction in the Coinjoin thread.

I don't know if it would have helped, but if the transaction generation engine had to include tx fees or something maybe it would have failed more gracefully (i.e. produced an invalid tx).

I would not dismiss offline transactions a "meh, just a single use case."   That is the future of any serious Bitcoin users and HW wallets.  Accommodating them is in everyone's best interest.

Also, you are talking about a major protocol upgrade that is far from uncontroversial and could take months/years of debate, testing, etc.  I'm talking about a very simple, modification of a couple lines of code that completely solves a problem that a lot of Bitcoin innovators having today.

The big problem is it's specific to OP_CHECKSIG, and in the future it's quite possible that OP_CHECKSIG will be changed/made obsolete/who knows? Doing this via a SIGHASH bit is inelegant because it's so specific to a single use-case.

With a v2 transaction format, with a merkle tree extending into the tx and including a hash of the CTxOut structure being spent your hardware wallets would just take that data and operate only on that. For instance this is helpful if you are signing just a small part of a much bigger combined transaction with funds from multiple parties - there might be 1000 people chipping in to pay a large bounty for instance, and with a v2 tx format and set-based SIGHASH's your hardware wallet only needs to be given the small part of the txouts you are signing for.

Interestingly in that use-case your signature can still cover all the other txouts safely. Your wallet only needs to know that you've contributed xBTC of funds, and the txouts it knows about, via the merkle paths its been provided, are for (x-fee)BTC of funds. Perfectly safe no-matter what malware is on your computer, yet the outside world only knows that you added funds to some huge txout set. This is perfect for CoinJoin tx's.

I'm all for UTXO commitment proposals.  After all, I have a pretty elaborate one I'd like to see pushed forward.  But that still serves a different purpose than this:  sending the offline computer UTXO proofs/branches is going to be a lot more than 8 bytes per input.  Part of the goal is to minimize the data sent across the online-offline channel, and the complexity of what needs to be done by the offline signing device.   

I'm not sure where the inelegance comes from -- we add 8 bytes to the data to be hashed and we close up a major oversight/inefficiency in the protocol.  If this was some kind of epic, controversial change, I'd agree that it may not be worth it.  I've already said it's not even worth its own hard-fork.  But given how simple-yet-useful the change is, I see little reason not to do it if the timing is right.  Unless it will be will feasibly be solved by some other major upgrade in the near future.

The all I see is gmaxwell: "I think the sighash with input value seems pretty inelegant." and Mike saying he likes the idea of doing something, lets talk more at the conference.

The seriously-considered proposals are the UTXO commitment proposals. As for other uses for SIGHASH bits, see IRC discussions about the need to be able to specify sets of txins and txouts being committed to. I also posted a quick idea myself for txout summation.

FWIW one of my main objections to this is that we really need general ways to prove fees paid by transactions for inflation fraud proofing, and short proofs of txout existence are very helpful to SPV nodes who are able to use them to check that transactions are valid - you're focusing on something very specific.


Additional merkle roots can be committed to in the coinbase txout; adding them is a soft-fork just like adding additional SIGHASH bits is. The best way to do this is to put the digest of the secondary merkle root in the last txout of the coinbase (with a P2Pool style 0-value OP_RETURN) so the proof can use a SHA256 midstate of the coinbase tx to keep the size independent of the size of the coinbase tx. It's at this point that a "v2" transaction format can be defined to fix all this stuff for once.

I'd be happy to pitch in on a BIP defining that transaction format, and we can do it in a way that leaves room to add the UTXO commitment digest when that is fleshed out as well.

You seem to suggest there is consensus about your opinion it should not be done, and that there are seriously-considered proposals to change the meaning of the merkle root.  But, if you read the rest of this thread, there is a quite a bit of high-level support for this modification, as an "add-on" to any future hard forks.  Of course, we're all happy to discuss alternatives, and what other uses there are for the other SIGHASH types, but you haven't mentioned what they are or where I can go read about it.

On the other hand, if we were to implement such a drastic/breaking change in the protocol as you suggest, to actually change what the merkle root means in the header, we'd have no problem simply "fixing" the existing hash types to include the output values, as that would equally break all existing parsing/verification code as changing the meaning of the merkle root.  If your suggestion was implemented, we could remove the "wasted" SIGHASH_WITHINPUTVALUE bit and recover it for these other use cases you mention.

On a related note:  TierNolan, can you link to the thread where this was reported?  I'd like to see how this massive fee occurred.  Was it malicious?  Or just likely a bug in the way the online computer was reporting input values to the offline computer?  If I had to guess, it was someone who tried to manually execute the offline signing procedure, but did not include all the supporting transactions, and ended up mis-reporting input values to the offline computer (which would've been caught if they supplied and checked the supporting tx, or SIGHASH_WITHINPUTVALUE was implemented).

There are much better ways to do this than using up what limited SIGHASH bits we have, and we already have other uses for SIGHASH bits that should be implemented.

In particular it has been proposed to extend the merkle tree of transactions down into the transactions themselves, which would allow you to easily provide compact proofs of the value of the transaction outputs that a transaction is spending.

tl;dr: Strong NACK on SIGHASH_WITHINPUTVALUE

This thread was pinged from another thread where someone (probably) lost 200 BTC to fees.

It is probably worth creating a BIP and reference implementation so that it is tested and included in any subsequent hard fork.

From here, it looks like only the bottom 5 bits make any difference.

This suggests that you can add extra sighash bits and it won't cause anything to explode.

Sighash decides how the transaction is hashed, so you can't do it in a soft fork kind of way.  Either the bit has no effect (so is inherently backward compatible) or it changes something and so will fail checksig for older clients.

Yeah, go for it.

I am making a new proposal for colored coin soft-fork: https://bitcointalksearch.org/topic/opcheckcolorverify-soft-fork-for-native-color-coin-support-253385

To make it works with lightweight HW wallet, OP_CHECKFEEVERIFY is not enough because colored coin is not fungible with normal coin. In this case only OP_CHECKVALUE or OP_TRUE solution would work.

As I mentioned, there could be multiple OP_CHECKVALUE in the same dummy output, e.g.

OP_CHECKVALUE

OP_CHECKVALUE ........

Quote from: etotheipi on July 09, 2013, 11:43:58 AM

Couldn't you just have a single output specifying the fee for the whole tx?  It would essentially do the same thing, and the OP_NOP/OP_CHECKFEEVERIFY would still work with zero output value and be pruned. 

Interesting. This could be more efficient than my previous OP_TRUE solution since the miner won't need to transfer the fee once again. Just redefine an unassigned OP code to OP_CHECKFEEVERIFY (We don't need to consume an OP_NOP because the output is not supposed to be spent)

Quote from: etotheipi on July 09, 2013, 11:43:58 AM

That would make it somewhat complicated to do more-advanced contracts though... where not all inputs and/or outputs are known at the time of signing (like the lighthouse contract).  Just more food for thought.

Yes, that would require the OP_CHECKVALUE or the OP_TRUE solution.

I thought something similar (see my crossed out reply above), and found it won't work for the same reason

Does it have to be one extra output per input?  Couldn't you just have a single output specifying the fee for the whole tx?  It would essentially do the same thing, and the OP_NOP/OP_CHECKFEEVERIFY would still work with zero output value and be pruned. 

That would make it somewhat complicated to do more-advanced contracts though... where not all inputs and/or outputs are known at the time of signing (like the lighthouse contract).  Just more food for thought.

Does it have to be an extra output?  Couldn't we just put this in the TxIn script being signed directly?  Right now, a standard TxIn script is:


Couldn't we do:


EDIT: Never mind, I just realized that that those op-codes don't get signed, so they can be altered without breaking the signature.

I have a ugly but backward compatible way to implement the OP idea:

1. Redefine one of the unassigned OP code (e.g. 0xfc) as OP_CHECKVALUE
2. When signing of input value is required, add an extra output like " OP_CHECKVALUE" is added, with 0 bitcoin assigned to this output
3. If an output of this format is found when validating a transaction, the engine will check whether the value of the specified output is same as TxPrevOutputValue. If not, the whole transaction is invalid. If yes, the code remove the top 3 stack items.

More rules:
1. There could be multiple OP_CHECKVALUE in the same dummy output.
2. The whole transaction is invalid if the specified output is not found in the input list.

Advantage:
This is transparent to existing clients, making it a perfect soft-fork. For both old and new clients, the dummy output is provably unspendable and will be pruned from the UTXO set.

Disadvantage:
Bloats blockchain, and looks hacky

Any comments? At least I want to know if it works in theory

After thinking carefully I believe your original proposal is better. It allows the HW wallet to sign a partial transaction (e.g., a user wants to sell smartcoins for ordinary bitcoin. The HW wallet has to know the exact amount of smartcoins being sent.)

With scriptSig Malleability (https://en.bitcoin.it/wiki/Transaction_Malleability), a variation of the original proposal could be done with a soft fork:

First, redefine OP_NOP1 to OP_CHECKVALUE. If the top stake item does not equal to the value of the input, the script fails. If they are equal, OP_CHECKVALUE does nothing.

The scriptSig for a standard send-to-key-hash transaction will look like this: OP_CHECKVALUE OP_DROP
and the combined script will be OP_CHECKVALUE OP_DROP OP_DUP OP_HASH160 OP_EQUALVERIFY OP_CHECKSIG

For old version clients, OP_CHECKVALUE is simply OP_NOP1, which will do nothing. OP_DROP will then remove . So the design is completely backward compatible.

(This is my first time designing a script. Please comment)

Edit: sorry it doesn't work, as the does not sign

On the other hand we're probably going make miners create proofs of transaction fees sometime in the future, either as part of the UTXO proof stuff or something separate, so I'm not sure a hacked up mechanism like this is really worth it.

Users aren't going to want to have to check some checkbox due to some strange technical reason that they have no understanding of.

Also, it is only needed for transfers to cold storage addresses. 

A client could have a checkbox to create the appropriate transaction.