Are Blocks in blockchain are indexed ?

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: r1s2g3 on September 18, 2018, 08:33:29 PM

Quote from: ABCbits on September 17, 2018, 11:04:30 PM

There's no way any developer would think scan entire database just to get a data.

Quote from: nc50lc on September 18, 2018, 01:30:03 AM

Then, by comparing the speed between (ex. not indexed Window Explorer) and Bitcoin's Blockchain, this question pops in to your mind?

I am a Performance DBA, I know even if the table size in a couple of GB and you have to do Full table scan to fetch the data ,required data cannot be fetched within the seconds . (Debarring parallel query operations).

Sometimes telling obvious things is safer/better choice Roll Eyes

Quote from: r1s2g3 on September 18, 2018, 08:33:29 PM

Quote from: Thirdspace on September 18, 2018, 05:36:25 PM

you can read more in details on bitcoin wiki Bitcoin Core 0.11 (ch 2): Data Storage

That link perfectly satisfy my query.

Thanks to you all. Slightly off topic question, I already gone through all the chapters of "Mastering Bitcoin" book and I do not remember if I have read anything about it. Did I missed in reading or there are some other source/book from which I should start.

This should be obvious, but the link is outdated and UTXO database/chainstate and LevelDB used by Bitcoin Core has been changed few times.
AFAIK 0.15.1 change chainstate format and there are multiple levelDB optimization.

"Mastering Bitcoin" mostly discuss about things necessary in Bitcoin protocol and database isn't part of it (except block header/TX format) since each client have different way to store/index blockchain/UTXO. A bit off-topic, you might find source/book you want in list that i mentioned in past

Quote from: ?? on ??

Since you have intermediate blockchain knowledge, surely these lists shouldn't too difficult for you :
https://github.com/decrypto-org/blockchain-papers
https://github.com/coderplex/awesome-blockchain

You also could check these cryptocurrency specific lists, but might be difficult for you (and me) :
https://github.com/igorbarinov/awesome-bitcoin
https://github.com/Scanate/EthList
https://github.com/jashmenn/bitcoin-reading-list

The lists i mentioned aren't courses, but i'm sure you would get more knowledge/info with shorter time.

r1s2g3

sr. member

Activity: 742

Merit: 395

I am alive but in hibernation.

Quote from: ABCbits on September 17, 2018, 11:04:30 PM

There's no way any developer would think scan entire database just to get a data.

Quote from: nc50lc on September 18, 2018, 01:30:03 AM

Then, by comparing the speed between (ex. not indexed Window Explorer) and Bitcoin's Blockchain, this question pops in to your mind?

I am a Performance DBA, I know even if the table size in a couple of GB and you have to do Full table scan to fetch the data ,required data cannot be fetched within the seconds . (Debarring parallel query operations).

Quote from: Thirdspace on September 18, 2018, 05:36:25 PM

you can read more in details on bitcoin wiki Bitcoin Core 0.11 (ch 2): Data Storage

That link perfectly satisfy my query.

Thanks to you all. Slightly off topic question, I already gone through all the chapters of "Mastering Bitcoin" book and I do not remember if I have read anything about it. Did I missed in reading or there are some other source/book from which I should start.

Thirdspace

hero member

Activity: 1232

Merit: 738

Mixing reinvented for your privacy | chipmixer.com

Quote from: r1s2g3 on September 17, 2018, 09:54:45 PM

It means it need to be scanned in entire blockchain that it is not spent.

Quote from: r1s2g3 on September 17, 2018, 09:54:45 PM

I understand each block has address of the previous block so do it have kind of index to reach that last block and we traverse all block one by one?

all databases always have indexes to make searching specific data faster
for bitcoin, metadata for blocks are stored under blocks/index/* directory
you can read more in details on bitcoin wiki Bitcoin Core 0.11 (ch 2): Data Storage

nc50lc

legendary

Activity: 2534

Merit: 6080

Self-proclaimed Genius

Quote from: r1s2g3 on September 17, 2018, 09:54:45 PM

-snip-

By saying "indexed" was it meant like index in books and file storage systems?
Then, by comparing the speed between (ex. not indexed Window Explorer) and Bitcoin's Blockchain, this question pops in to your mind?

Clients have a blockchain_headers which contains something similar (for indexing) that enables it to easily find the transaction hashes which have the accurate "links" to where the UTXOs are stored:

Code:

"inputs": [
        {
            "addresses": [
                "39EK***********************"
            ], 
            "age": 513486, 
            "output_index": 82, 
            "output_value": 10000, 
            "prev_hash": "24dc16f626085e7c75e6fbdbd2533f27d98d884fe73f58fed46a3e319157****", 
            "script": "160014d0ef607e4dc338d4dd67df92e2090b770a781***", 
            "script_type": "pay-to-script-hash", 
            "sequence": 4294967293, 
            "witness":

prev_hash: the previous transaction where it belongs.
output_index: where it sits between all of the other outputs of the particular transaction.

File explorers and Blockchain works differently, this is just "in-relation".

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Bitcoin core (other client might do things differently) build and use UTXO database which stored on storage (not RAM) which makes transaction verification is faster. The UTXO database updated every time node receive transaction/block or create and broadcast transaction.
Besides, AFAIK most transaction can be verified in hundred millisecond.

There's no way any developer would think scan entire database just to get a data. Indexing, database balancing, sorting or/and separate database for linking obviously used in here.

If you're curious with UTXO size/count, you can check https://statoshi.info/dashboard/db/unspent-transaction-output-set

r1s2g3

sr. member

Activity: 742

Merit: 395

I am alive but in hibernation.

When we do a transaction then it is made sure that whatever we are going to send to receiver are unspent outputs (UTXO). It means it need to be scanned in entire blockchain that it is not spent.
But I see that transaction are validated within a seconds (or few seconds). Since blockchain size is now is 170 GB plus so it look to me quite a fast speed to traverse all blocks in blockchain.

I understand each block has address of the previous block so do it have kind of index to reach that last block and we traverse all block one by one?

Topic: Are Blocks in blockchain are indexed ? (Read 382 times)