Fast parallel block validation without UTXO-index - page 2.

misterbigg

legendary

Activity: 1064

Merit: 1001

Quote from: cryptoanarchist on April 07, 2017, 03:50:25 PM

Does this make SegWit obsolete?

No, it just improves the performance of common operations when validating transactions and blocks. Which is very helpful since it lowers the costs of operating a full node. In other words, it increases decentralization (we like that).

I read the entire document, clearly these folks have some skills!

cryptoanarchist

legendary

Activity: 1120

Merit: 1003

Does this make SegWit obsolete?

tomtomtom7

jr. member

Activity: 38

Merit: 18

Quote from: piotr_n on April 06, 2017, 08:27:26 PM

Ok. But what about outputs that never existed in the first place. How do you find that they don't exist?

The spend tree is scanned; if the output is found it is a double spent, if the corresponding transaction is spent it is ok, and if nothing is found it is spending a non-existing output.

Similarly, if the spend-index "catches" the search, two lookups are done: presence of the transaction and absence of the output.

The 'cheat' of spending an output of an existing transaction with an index higher then the output count is caught by script validation.

piotr_n

legendary

Activity: 2053

Merit: 1356

aka tonikt

Quote from: tomtomtom7 on April 06, 2017, 08:12:20 PM

The problem of spending early outputs is solved by the spend-index, which lags behind the tips a few blocks to catch these. This is also explained under the "spend-index" section on the site.

Ok. But what about outputs that never existed in the first place. How do you find that they don't exist?

tomtomtom7

jr. member

Activity: 38

Merit: 18

Current usage:

1.3G   ../data-cmp/block-index
5.6G   ../data-cmp/spend-tree
41M   ../data-cmp/headers
25G   ../data-cmp/transactions2
196M   ../data-cmp/spend-index
3.6G   ../data-cmp/tx-index
85G   ../data-cmp/transactions1
119G   ../data-cmp/

But this can be pruned, just like with Core.

As for memory, this uses memory mapped files which means lots of virtual memory, but leave it to the OS to swap. By using proper locality of reference, the data that isn't used is kept out of memory.

The problem of spending early outputs is solved by the spend-index, which lags behind the tips a few blocks to catch these. This is also explained under the "spend-index" section on the site.

piotr_n

legendary

Activity: 2053

Merit: 1356

aka tonikt

How much memory and disk space does it need?

Do you keep it all in files on disk?

If you get to verify a transaction (block) that tries to spend an unexisting output, do you have to go back through all the files just to find out that it doesn't exist?

tomtomtom7

jr. member

Activity: 38

Merit: 18

I have build a storage engine as the core of a new Bitcoin Node in development. The idea is to abandon the UTXO index and use a fast concurrent spend tree instead.

The results are very promising. Further information is here: https://bitcrust.org

Performance results: https://bitcrust.org/results

Source code in Rust is available at https://github.com/tomasvdw/bitcrust

Topic: Fast parallel block validation without UTXO-index - page 2. (Read 4668 times)