Topic: Commitment-based node for Initial Blockchain Download (Read 89 times)

Why do you think it is nearly impossible? It is effectively the same system, the only thing that was changed, was the way of looking at the system. It is like using functional programming, instead of object-oriented programming. Everything can be translated from one viewpoint to another, and it can be proven, that a lossless translation is always possible.

No, because you don't need the whole commitment, but only the part, which is covered by signature. Of course, in practice, everything is covered, because for example the previous block hash guarantees, that you need the full data to accept the new header. But still, it is the same things as with pointers: you don't need "unwrapped" commitments, that would be an equivalent of a system without any pointers. If you have transaction hash, and if that part is signed, then you can keep it as it is, and not expand it by putting transaction data in that field directly.

In practice, there will be more tables, for example sorted by hash functions. Which means, you could have a table of HASH160 entries, and map them to the indices in the table of public keys. And in a similar way, you could have a table of SHA256 entries, to cover transaction hashes, merkle root, block hashes, and all things like that. But I didn't explain full details behind those tables, because they are not yet ready.

I think it will be smaller, after processing the whole chain. Why? Because some data are repeated. Every time you have address reuse, or R-value reuse, or when anything is reused, it can be easily compressed, just by using simple commands, like "repeat N bytes M times". And just by applying that simple compression, it will probably shrink at least a few gigabytes, if not more.

Not only hardest path, personally i'd say it's near impossible.


For now i have one thought. Wouldn't size of the commitment is extremely huge which makes it impractical? I say extremely huge based on naive calculation which multiply 32 bytes with total of all signature and public key.

I thought about it for some time, and judging by some posts, I think it is a good moment to release that idea publicly. The implementation is not yet ready, but again: this idea is worth sharing, because I want to inform some more experienced people about it, and I want to tell other users, what is possible, and what kind of new clients could be published in the future.

This sentence is more important than you can probably imagine. If you look at each address type we have, you will notice, that public keys are important, and are harder to implement than other opcodes.

This is a good start: https://en.bitcoin.it/Script

You can read this page, and see, how easy or hard it is, to implement each opcode.

1. Constants? Easy, just some stack push.
2. Flow control? Also quite easy, each programming language has it, even if it is functional.
3. Stack? Classical examples can be found for any language.
4. Splice? Only OP_SIZE is allowed, but string-based operations are usually available in any standard library.
5. Bitwise-logic? It comes from assembly, but is also quite simple, and if you don't have it, then by having addition, subtraction, multiplication and division, you can represent it on any architecture (again, only OP_EQUAL and OP_EQUALVERIFY is available, but the rest is also easy to implement, if needed).
6. Arithmetic? As I said, if you have any basic math in your language, then you can do that as well.
7. Crypto? This part is the hardest one, but only OP_CHECKSIG-based opcodes are hard. All hash functions are quite easy, they have a lot of rounds, but all operations are quite simple.
8. Locktime? It only prevents transactions from being pushed, but after some time, it is just some OP_NOP (and any language contains some library for time handling).
9. Pseudo-words and reserved words are easy, they are just "OP_FALSE OP_VERIFY", and you are done.

So, what is hard? OP_CHECKSIG, and similar opcodes. The rest is sometimes easier, sometimes harder, but it has one nice property: it is transaction-independent. Which means, if you have OP_SHA1, then it works in the same way with every single transaction. However, OP_CHECKSIG is transaction-specific. It works differently on different transactions, and in one transaction it may evaluate into OP_TRUE, but in another into OP_FALSE. But " OP_SHA1 OP_EQUAL" is always only true, or only false, in all transactions you can imagine. Which means, it is easier to handle.

It is hard to handle public keys and signatures. We have sigops limit, but there is no "hashops limit". But it is essential for any real-life scenario. Read that quote above again: "You can't use a coin without exposing its pubkey". This is very important to understand the whole system. And if you read the whitepaper, then you can notice, that only P2PK is described there, no Script is needed to start the system. It can be added later, and wrapped into P2PK. And in the current BTC implementation, Taproot did exactly that.

So, what is the idea from the topic? It is very simple, but as always, the devil is in the details. All you need, to bootstrap a node, is to collect all public keys you see, and put them into some kind of table. And when I say "all public keys", I mean all of them, so also R-values in all signatures.

In the initial code, I decided to always include "02" or "03" prefix. I did it even in R-values, because then things are easier to handle. Even in Taproot, public keys can be converted, just by adding missing "02" prefix, and it will be fine. If some public key has "04" prefix, it can be converted properly into "02" or "03" prefix, and stored in compressed form. Invalid keys are provably unspendable, and can be ignored, the same with invalid scripts.

If someone wants to strip "02" or "03" prefix, it is possible, but it is probably not worth it. And after collecting more keys, and sorting them, there is more room for more optimizations, for example public keys can be sorted by their prefixes, and you can have the whole file, with all public keys, starting with "02badc0ded...". But it will come later, if needed, the first version just uses the fixed 33-byte field for a single key. And it is possible to have two files: "even" with "02" keys, and "odd" with "03" keys, if someone needs to have a nice, 32-byte-based padding.

So, there is a table with public keys. What is more? A table with signatures, of course. A signature is just a connection between two public keys. It means, you can select two public keys from the first table, and use "multiplication and addition" to connect those keys. So, all you need, is just two indices from the pubkey table, and two 256-bit numbers, to connect them correctly from ECDSA point of view.

In the title of this topic, you can read "commitment-based". So, obviously, the third table is a table of commitments. It is needed to make sure that signatures are not generated out of thin air, and that z-value is actually covered by some data. Fortunately, in many cases, things are double-hashed, which means, in many cases, you only need some 256-bit number as your commitment.

So, this is a base. Public keys, signatures, and commitments to those signatures. This is just a base to start the system. And on top of that, there are consensus rules, to accept only valid commitments, and to decide, who can create a new commitment. Dealing with commitments, and standardizing them, is the hardest part of this idea, but the basic structure ends there, so I decided to share the part, that is unlikely to change (so can be marked as "discussion-ready", and can be battle-tested on forum). Thoughts?