Topic: Segwit details? N + 2*numtxids + numvins > N, segwit uses more space than 2MB HF - page 13. (Read 21415 times)

So according to you it is not a soft fork, but it is not a hard fork either... so how many other forks are there? I think you need to explain this. Pitch forks maybe?

Funny

Is this thing just a hack to fool miners into storing more data?  As you point out it's not just more TX but also more bytes per TX.  I don't see it happening.  Who is going to send that first transaction that might be spendable by anyone?  Probably I am naive here and missing some political or alchemical posturing nuance.  If so that's fine, in the end nobody is going to break the installed base as you put it.  It's not worth the risk.     

There is also the claimed advantage that it removes malleability.  But again, is malleability really a problem?  If you are concerned about a TXID changing, for some scheme that relies on using unpublished TXs, then submit your transactions directly to a mining pool.  This is the way we need to go eventually anyway, as TX relaying by random nodes is purely altruistic and in a long term perspective looks like a temporary condition.   

Anyway thanks for your analysis. 





 

maybe you didnt say it, but the official claims are here: https://bitcoincore.org/en/2016/01/26/segwit-benefits/#block-capacitysize-increase

maybe its just me misinterpreting the english as my second language and the above isnt claiming that it will increase the block capacity. I avoid political stuff so maybe I am just not understanding the nuances of the english. recently I found out that "sick" meant "cool", but cool wasnt about the temperature, but something else. So I guess it just matters what the meaning of the words "size increase" means.

you know that segwit is being marketed as the magic solution to allow blocksize increase without hardfork. that is the issue here, disingenuous marketing

it might be that the "only" thing that is lost is the ability to verify the tx, but hey, what's wrong with needing to trust things, right? and the if the receiving node for the p2sh spend also doesnt run segwit, he is able to validate the tx because it is an anyone can spend signature? If anyone can spend it, what prevents somebody else from spending it? I am really confused as to how older nodes without the signature witness data can spend it to another node that doesnt have the witness data, without also allowing any random person from spending. What magic am I missing? Is that what the zeroknowlege commitment stuff is?

If you can only spend a wtxid to a segwit node, that is considered good enough? It seems segwit demotes all older nodes, even if they are fully validating and relaying nodes, into SPV nodes. This is acceptable?

I am not convinced at all that there is any space savings and pretty sure that it will take more space per tx if you count the witness data and we really need to count that data too.

This needs to be stopped being marketed as the softfork that increases tx capacity as that is not proven at all and likely the exact opposite when you make the reasonable assumptions that people will have to update to segwit and thus get all the witness data

So, I'm on mobile right now so I can't give as detailed of a response. I will edit this with my full response later in half a day when I get back to my computer to give you a full response. This is just the gist of what I want to day.

I'm not sure about the space reducing part. Was that actually mentioned anywhere? I don't think I said that it would reduce the space used. It is essentially a 2Mb block size limit increase but with the added benefit of making transaction malleability impossible.

About the txid thing, you are incorrect. Reread all the BIPs carefully and multiple times. It can take a couple reads to fully comprehend and understand what is happening. Also keep in mind that there will probably be changes when segwit is actually released. The changes will only be omissions of what was specified e.g.I don't think they will include the new address type.

With the addresses, the witness program is nested in a p2sh address so it will be 3xxxxx. These should all be able to be spent to under the current system. You as the sender don't need to know the whether the address is segwit or not, but the receiver will need to in order to properly spend from it.

ok, it seems the wtxid is not included in the witness data, however I cannot imagine how it can be encoded such that the space taken in the original blockspace + space in witness blockspace is smaller than just using 2MB of ordinary blockspace.

and if it doesnt REDUCE the total space, then it has no net gain and is a failure for increasing tx capacity. So where is the proof that it will reduce the total space used? We still trust in math around here, dont we?

and if the details about the total space used is not known by you, then the question arises about who has peer reviewed this. Using this for scalability has negative effect unless the combined space is reduced and in almost all cases when you have a single reference to something else, you cant save space as the something else needs to exist and also the reference to it. The best that would be possible is to have the position in the witness space be the implicit reference and that is probably how it is done.

however, there is still the issue of:

Transaction ID

A new data structure, witness, is defined. Each transaction will have 2 IDs.

Definition of txid remains unchanged: the double SHA256 of the traditional serialization format:

  [nVersion][txins][txouts][nLockTime]
  
A new wtxid is defined: the double SHA256 of the new serialization with witness data:

  [nVersion][marker][flag][txins][txouts][witness][nLockTime]

from the BIP...

the wtxid is based on all of the original, plus marker (1 byte?) flag (1 byte) and witness, which appears to be:

 1-byte - OP_RETURN (0x6a)
   1-byte - Push the following 36 bytes (0x24)
   4-byte - Commitment header (0xaa21a9ed)
  32-byte - Commitment hash: Double-SHA256(witness root hash|witness nonce)

all this seems to be above and beyond what would be needed for the normal, plus the nVersion (4 bytes) and nLockTime (4 bytes) are duplicated. To a simple C programmer like me it sure looks like instead of reducing the net amount as required by anything claiming to save space, it is increasing the size by approx 50 bytes.

So even if we say the cost for all the work in all the project across the bitcoin world is ZERO, it still reduces the overall tx capacity of bitcoin permanently. The fact that such a anti-space saving mechanism is marketed at all, let alone as a space saving "softfork", well you see my concerns about the technical reputations of anybody that supports the segwit for increased tx capacity.

I dont want to take away from the brilliance of the tech to solve the malleability and increase the potential usecases. The problem is that it is being backdoored through the softfork mechanism and being marketed without objective peer review.

James

P.S. So my understanding is that you need a special segwit address (that is somehow determined to be a segwit address using what mechanism?) so both sender and receiver need to already have the segwit version. I guess just ignoring all the existing nodes is at least some level of backward compatibility. But are you sure all users will quickly get used to having to deal with two types of addresses for every transaction and they will make sure they know what version the other party is running. Doesnt this bifurcate the bitcoin universe? maybe the name should be "bifurcating softfork"

"should not have been marketed as a scalability solution but rather that it is for transaction malleability with a side effect of some scalability a 75% fee discount for signature heavy settlement transactions."

I don't know, so I can't agree or disagree. I couldn't find anything about how that would be serialized. I do agree that upgraded nodes would have to download all of the witness data though, whether they include the wtxids, I cannot say.

I still say that it is a soft fork, albeit not entirely a soft fork but not a hard fork either. Let's agree to disagree.

No, that is not possible. An old node would not know about the new witness program and how that works in an p2sh output. It would only know of p2pkh outputs that are meant for it, even if the witness program is just a p2pkh but segwit spendable only. The old node wouldn't know that it received such a payment so this attack wouldn't work.

Now that I think about it, I don't think the wtxids are included and that the witness data is included in a separate structure. If the witness data is requested, then I think (don't quote me on this) that it is just appended to its respective transaction in the block when it is sent. The wtxids are probably generated on the fly just like regular txids are generated on the fly as well. So in fact, it would be a 2 Mb increase but the "official" size of the block cuts out all of that witness data. If the block were to be requested normally as it is now, the witness data would not be included and the size of the block would be at most 1 Mb.

I do agree that it should not have been marketed as a scalability solution but rather that it is for transaction malleability with a side effect of some scalability.

OK, so you agree that if segwit achieves the activation level, all nodes will have to update and download the 2MB of data, which contains 300kb+ of wtxids that otherwise wouldnt be needed in a straight 2MB hardfork.

so it is a total fail from a "increasing tx capacity without requiring a hardfork" point of view. Let us not quibble if it technically is a softfork or hardfork, the reality is users will have to update or not be  able to spend. it looks like a hardfork, walks like a hardfork, quacks like a hardfork.

It sounds like it is possible to make it less of a problem, but it will be possible for segwit to be used to make unspendable payments to old nodes, so this creates an attack vector where the attacker simply sends to thousands of users some small amount of unspendable segwit wtxids. Once users get the bitcoins, they wont care about whether it is softfork or whatever, they will want to spend the bitcoins.

So, in the case where segwit is adopted, then all the nodes must update and get the full 2MB blocks that are bloated with needless wtxids. Now I just briefly looked at segwit details for the first time today, so maybe there is some super magic negative knowlege antimatter spacetime warping data compression that allows the segwit to actually save blockchain space. but calling the witness data not the blockchain since it is separate, again it becomes the type of stuff politicians do and not what technical guys should be doing. So if you are a politico, then fine, but I had always seen your posts as from an objective technical guy and was totally shocked at what you wrote. I am assuming the witness data is treated the same as the normal blockchain data, so it is in the same category and thus the statement that segwit is a total fail for increasing tx capacity without hardfork is fully justifed

Since segwit was started to fix malleability, maybe it should stick to that and not try to solve a problem it cannot solve. Officially claiming that it solves this is damaging to bitcoin's technical credibility and the other coins will take FULL advantage of this.

You cannot claim to be intelligent while advocating idiotic things, right?

James

Segwit defines a new address type, but I don't think that will actually be implemented. Rather it will be using the witness program in p2sh addresses instead. A non-upgraded node will not have such addresses, they will still be using p2pkh addresses like we do now. If a segwit transaction were to be made which spent from a segwit output to an old p2pkh output, an old node would still be able to spend from it. The transaction would validate because the node sees it as an anyonecanspend input and the output is just like any p2pkh output in use right now. There is no need for the witness data to spend, it just cannot know whether the transaction it spends from was legitimate or not. Then it can still spend the p2pkh output normally as it does now.

Well they aren't actually in the blockchain as we know it. It would essentially be like a secondary blockchain of all of the witness data. Either way, yes upgraded nodes would have to download 2 Mb of data.

Sorry, I actually spoke incorrectly there. I forgot about the whole address thing.

it is a hardfork pretending to be a softfork that increases tx capacity without doing a hardfork, but it actually wastes space permanently.

Of course it only wastes space on nodes running the segwit "softfork"

but if you received any wtxid and want to spend it, you need to run an updated wallet, which I am sure will be available within 3 days from when you get such a wtxid. After all the changes required are quite simple: https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki

Anybody that has written a bitcoin core from scratch (like me) should be able to implement the dozen or so changes in a month or two, and we can ignore any issues about the customer support as they can just do manual rawtx manipulations with all the great tools available for that if they really want to and dont want to update to the only wallet that supports segwit

So it breaks the installed base
Creates customer support and field update issues
And permanently wastes 30%+ of the new space as opposed to a simple 2MB hardfork (or 4MB)
But it will single source wallets for the months it takes for everybody to "fix" their software

Dont get me wrong, I think the segwit tech is pretty cool, but instead of pushing it into BTC mainnet under the innocent sounding "softfork" and exposing the entire installed base to pain and suffering, it seems this magnitude change should start in a side chain, get field tested and then if it makes sense to do a HARDFORK for it. DO NOT BREAK THE INSTALLED BASE

James

I would advise that you actually read all of the BIPs.

The one you are looking for is BIP 143: https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki.

How does the wtxid waste space? The only place it ends up in the blockchain is in the coinbase transaction as the witness root hash where the wtxids are hashed together.

It is a soft fork because after the upgrade old nodes and wallets still function perfectly fine with the old system. They can still receive segwit transactions, they just can't spend from them. They can still receive and spend from traditional transactions which will still be valid under the new rules

No. It simply means that the new rules can go into effect and are now considered valid. No one is being forced to upgrade and you can still function fine without upgrading for a while.

No. It will not fork the blockchain. All of the blocks produced after the fork are still valid under the old rules. This is part of what makes it a soft fork. It doesn't fork the blockchain like a hard fork does.

Segwit is needed for its solution to the transaction malleability problem. It makes transaction malleability impossible to occur now since the txids now contain only data that is already signed. If everyone were to upgrade to segwit, it would indeed be a very good thing for Bitcoin. It also solves the O(n^2) hashing problem.

Additionally, you can still use Bitcoin as it is now after the fork. A lot of people seem to forget that.

Lastly, I will say that marketing segwit as a scalability solution was probably a bad idea. Its original intent was to solve the transaction malleability problem and a side effect was that the block space was effectively doubled. People use the 60% - 80% figure because the assumption is that people won't upgrade to segwit and make use of its advantages. Otherwise it would be the same as a block size doubling.

ACK

I dun get it either. 

considering the obstacles it faces, I will not implement segwit support into initial iguana versions. If a bunch of users end up with unspendable funds, then I guess I will  be forced to massively complicate the blockchain handling.

does anybody know the BIP that defines the new network message(s)? I assume each block would need a getsegwitdata equivalent so existing nodes just do the existing getdata and the segwit softforks (ha ha) do the additional call, process the new data format, update the internal dataset, validate the signatures and enable spending. Seems like a LOT of work to get a 60% increased capacity.

Since this is basically a hardfork (let us not kid ourselves that creating INCOMPATIBLE bitcoins is in any way backward compatible!!), and it requires the additional data, then just hardfork 2MB.

That would not require wtxid wasting precious space in the blockchain.

The logic used to justify wtxid permanently using up space that otherwise wouldnt be needed is that it allows a softfork. But this is a fake softfork, as existing nodes wont be able to validate or spend any segwit payments it gets. HOW ON EARTH IS THAT BACKWARD COMPATIBLE?? which is the industry's understanding of softfork behavior (I know technically that is not what softfork is, I speak of what users are thinking)

So, if segwit gets 75%, then it forces all the nodes to update. How is that not a hardfork?

The logic is segwit allows 60% increase in capacity without a hardfork, but this premise is wrong. It is effectively a hardfork, actually I think it is worse. If it was a hardfork, then users who know about such things will understand that they have to update. Saying it is a softfork will make users think they dont have to upgrade. Then they find out that they cant spend the bitcoins they got. So we end up with two incompatible bitcoins. this is not a good plan at all

Let us call a hardfork a hardfork. segwit is a hardfork from the most important aspect that is it NOT BACKWARD COMPATIBLE with existing wallets and independent cores.

OK, so that then means that segwit is WASTING precious blockchain space and not avoid a hardfork. It makes no sense to me that a defacto hardfork that breaks ALL EXISTING wallets and also requires all independent cores to make significant changes, testing, field updates, customer support, and it is all to permanently waste space with the redundant wtxids that would not be needed if we just hardforked 2MB

Please tell me there is some sanity here. There is no logical justification for segwit and plenty of risk factors of creating the impression that bitcoin is broken, and if you dont consider the existing installed base not being able to validate or spend bitcoins as not broken, then something about how you evaluate brokenness is broken.

James

P.S. Technically segwit is very clever and I can see plenty of use cases it enables. However, positioning it as a softfork is a disaster. But I guess bitcoin has nothing to worry about, after all there are no altcoins out there that have any chance at all against bitcoin. So we can just do whatever we want and it wont matter since there are no alternatives out there for users. Who is afraid of LTC anyway, right?

It's not just you, it's me too.
Wow, that is a lot of changes.

Sipas implementation is available here: https://github.com/sipa/bitcoin/pull/8

In essence, yes.

Yeah, pretty much. Although it still allows for the old type transactions so old wallets will still work.

IIRC segwit was originally proposed as a hard fork.

when you say "receive" but not spend, it is received and unverifiable and unspendable, right?

is it just me, or does it seem like calling this a softfork might be technically accurate, the market confusion and incompatibility it will cause is pretty much like a hardfork

The txid of a segwit transaction will be the same segwit or not since the signatures are not part of the transaction. Unupgraded users will be able to receive but not spend from validate segwit transactions fully. They can spend from segwit transactions because the output will still have to be a p2pkh output to non-upgraded wallets.

Edit: Fix error where I said that segwit txs could not be spent from.