Detailed information about datadir | Bitcointalksearch.org

jgarzik

legendary

Activity: 1596

Merit: 1100

Quote from: DeathAndTaxes on July 04, 2013, 03:25:00 PM

The chainstate sub folder contains leveldb representation of the UXTO right?

Yes.

Quote

What is the significance of multiple xxxxxx.sst files?
There is also LOCK, LOG, and MANIFEST-xxxxxx files?

These are leveldb internal storage setup. Inherent in leveldb design. Analogy: Berkeley DB's low level storage format.

Quote

The "blocks" subfolder contains leveldb representation of raw blocks right?

No. This is the raw bitcoin-specific wire format of blocks.

Quote

Why are the files of difference size?

Bitcoin blocks differ in size.

Also, some blk*.dat are larger because they were originally that way when imported the first time.

If you use bootstrap.dat to initialize a new database then all your files should be nearly the same size.

DeathAndTaxes

donator

Activity: 1218

Merit: 1080

Gerald Davis

Found a little more info here ....

Quote

Used in 0.8.0:
* wallet.dat: personal wallet (BDB) with keys and transactions
* peers.dat: peer IP address database (custom format); since 0.7.0
* blocks/blk000??.dat: block data (custom, 128 MiB per file); since 0.8.0
* blocks/rev000??.dat; block undo data (custom); since 0.8.0 (format changed since pre-0. Cool

* blocks/index/*; block index (LevelDB); since 0.8.0
* chainstate/*; block chain state database (LevelDB); since 0.8.0

https://github.com/bitcoin/bitcoin/blob/cb2e1bdaa902197ba0f04aebd00420a4680548e0/doc/files.txt

ok "custom format". Well that helps. Wink

DeathAndTaxes

donator

Activity: 1218

Merit: 1080

Gerald Davis

Thanks. That makes sense and yes this is from a very old install so it fits.

Any idea on the format of peers.dat file?

etotheipi

legendary

Activity: 1428

Merit: 1093

Core Armory Developer

Quote from: DeathAndTaxes on July 04, 2013, 03:25:00 PM

For example there are 30 blkxxxxxx files the first few are very large then most are 134MB. The last one is smaller but that would make sense as it is the block file in progress.
Why 134MB? Why roughly 2GB for the first two block files and then the third one breaks that pattern?

If you redownload the blockchain, they will be 134 MB. But if your blockfiles were mostly downloaded prior to version 0.8, then it used a 2 GB blkfile size, and just kept them when they switched it in 0.8.

If you were to wipe the directory and redownload with 0.8+, it would be about 70 files of 134 MB each. If you redownloaded with version 0.7 or prior, it would be 5 files of 2 GB each.

DeathAndTaxes

donator

Activity: 1218

Merit: 1080

Gerald Davis

Looking for detailed information on the data directory and file formats for QT client (0.8.3).

There is this wiki page but it is pretty sparse:
https://en.bitcoin.it/wiki/Data_directory

For example what format is the peers.dat file in?

The chainstate sub folder contains leveldb representation of the UXTO right?
What is the significance of multiple xxxxxx.sst files?
There is also LOCK, LOG, and MANIFEST-xxxxxx files?

The "blocks" subfolder contains leveldb representation of raw blocks right?
Why are the files of difference size?

Code:

08/05/2012 02:16 AM 2,097,361,271 blk00000.dat
12/16/2012 02:37 PM 2,097,361,271 blk00001.dat
12/16/2012 07:04 PM 2,097,299,522 blk00002.dat
02/22/2013 01:23 PM 1,879,601,111 blk00003.dat
02/27/2013 07:07 AM 134,217,728 blk00004.dat
03/04/2013 02:05 AM 134,217,728 blk00005.dat
03/08/2013 01:52 PM 134,217,728 blk00006.dat
03/13/2013 05:51 AM 134,217,728 blk00007.dat
03/18/2013 11:53 PM 134,217,728 blk00008.dat
03/24/2013 03:29 AM 134,217,728 blk00009.dat
03/29/2013 04:37 AM 134,217,728 blk00010.dat
04/02/2013 06:30 PM 134,217,728 blk00011.dat
04/07/2013 01:42 AM 134,217,728 blk00012.dat
04/11/2013 06:15 AM 134,217,728 blk00013.dat
04/16/2013 12:28 AM 134,217,728 blk00014.dat
04/21/2013 04:08 AM 134,217,728 blk00015.dat
04/25/2013 09:57 PM 134,217,728 blk00016.dat
05/01/2013 09:47 AM 134,217,728 blk00017.dat
05/06/2013 02:32 PM 134,217,728 blk00018.dat
05/11/2013 12:56 AM 134,217,728 blk00019.dat
05/15/2013 07:33 AM 134,217,728 blk00020.dat
05/20/2013 09:52 AM 134,217,728 blk00021.dat
05/25/2013 11:00 AM 134,217,728 blk00022.dat
05/30/2013 03:57 PM 134,217,728 blk00023.dat
06/04/2013 05:22 PM 134,217,728 blk00024.dat
06/12/2013 11:01 AM 134,217,728 blk00025.dat
06/16/2013 06:07 PM 134,035,313 blk00026.dat
07/02/2013 06:03 PM 134,216,638 blk00027.dat
07/02/2013 06:16 PM 134,108,901 blk00028.dat
07/03/2013 06:25 PM 83,886,080 blk00029.dat
02/22/2013 01:42 PM 264,241,152 rev00000.dat
02/22/2013 01:54 PM 242,221,056 rev00002.dat
02/22/2013 02:25 PM 236,978,176 rev00003.dat
02/27/2013 07:07 AM 17,825,792 rev00004.dat
03/04/2013 02:05 AM 16,777,216 rev00005.dat
03/08/2013 01:52 PM 17,825,792 rev00006.dat
03/13/2013 05:51 AM 16,777,216 rev00007.dat
03/18/2013 11:53 PM 16,777,216 rev00008.dat
03/24/2013 03:29 AM 16,777,216 rev00009.dat
03/29/2013 04:37 AM 16,777,216 rev00010.dat
04/02/2013 06:30 PM 17,825,792 rev00011.dat
04/07/2013 01:42 AM 16,777,216 rev00012.dat
04/11/2013 06:15 AM 16,777,216 rev00013.dat
04/16/2013 12:28 AM 16,777,216 rev00014.dat
04/21/2013 04:08 AM 16,777,216 rev00015.dat
04/25/2013 09:57 PM 16,777,216 rev00016.dat
05/01/2013 09:47 AM 16,777,216 rev00017.dat
05/06/2013 02:32 PM 16,777,216 rev00018.dat
05/11/2013 12:56 AM 16,777,216 rev00019.dat
05/15/2013 07:33 AM 16,777,216 rev00020.dat
05/20/2013 09:52 AM 16,777,216 rev00021.dat
05/25/2013 11:00 AM 16,777,216 rev00022.dat
05/30/2013 03:57 PM 17,825,792 rev00023.dat
06/04/2013 05:22 PM 17,825,792 rev00024.dat
06/12/2013 11:01 AM 17,825,792 rev00025.dat
06/16/2013 06:07 PM 17,127,799 rev00026.dat
07/02/2013 06:03 PM 17,186,353 rev00027.dat
07/02/2013 06:16 PM 17,151,788 rev00028.dat
07/03/2013 06:25 PM 9,437,184 rev00029.dat

For example there are 30 blkxxxxxx files the first few are very large then most are 134MB. The last one is smaller but that would make sense as it is the block file in progress.
Why 134MB? Why roughly 2GB for the first two block files and then the third one breaks that pattern?

Topic: Detailed information about datadir (Read 1167 times)