Author

Topic: How was the average size of blk*.data chosen? (Read 115 times)

newbie
Activity: 12
Merit: 0
blk*.dat files have a maximum size of 128 MiB, and new files are simply started if the block that is about to be written to disk would cause the file to exceed this size limit.

The PR that introduced this limit, https://github.com/bitcoin/bitcoin/pull/1677, suggests that this limit was chosen to reduce disk fragmentation of the files. However, I don't think there is anything that determines or indicates that this size is "most efficient", whatever that means. This is often the case with many of the maximums that exist throughout Bitcoin and other protocols - maximums are often "it just feels like that's a big enough number".

It seems that the limit was set to prevent excessive fragmentation, since the blocks range from 1 to 4 megabytes and we don't want to handle large numbers of files. I apologize, when I talk about efficiency I'm referring to how this size can provide better read/write performance within the node. For example, in the Google File System paper, a chunk size of 64 MB was determined by user behavior because a larger chunk size significantly reduces the amount of interaction required between clients and the master node for read/ write operations.
staff
Activity: 3458
Merit: 6793
Just writing some code
blk*.dat files have a maximum size of 128 MiB, and new files are simply started if the block that is about to be written to disk would cause the file to exceed this size limit.

The PR that introduced this limit, https://github.com/bitcoin/bitcoin/pull/1677, suggests that this limit was chosen to reduce disk fragmentation of the files. However, I don't think there is anything that determines or indicates that this size is "most efficient", whatever that means. This is often the case with many of the maximums that exist throughout Bitcoin and other protocols - maximums are often "it just feels like that's a big enough number".
newbie
Activity: 12
Merit: 0
Hello,

I notice that the Bitcoin database is stored sequentially in blk*.dat files, with an average size between 128 to 134MB. I would like to know how this average size is determined and if there are any tests that demonstrate this size is the most efficient.

Regards.
Jump to: