Will it ever be possible to separate chainstate and blocks folder?

theymos_away

member

Activity: 82

Merit: 26

Quote from: Thirdspace on March 02, 2018, 06:10:22 PM

I was wondering about the actual connection between blocks and chainstate folders
can we recover/rebuild one by using the other? which one is more important?
recently my wallet (btx based on btc) ask to rebuild the blocks because corrupted issue
but it keeps failing on rebuilding the blocks and I ended syncing from my last backup
every now and then I make backup by copying blocks and chainstate folders, is this the correct way to backup blockchains?

See:
https://en.bitcoin.it/wiki/Data_directory#Transferability
https://en.bitcoin.it/wiki/Splitting_the_data_directory

I've never heard of any problems symlinking the blocks or chainstate directories, as long as they remain accessible, of course. Symlinking files related to the wallet is more dangerous due to BDB's weirdness.

Thirdspace

hero member

Activity: 1232

Merit: 738

Mixing reinvented for your privacy | chipmixer.com

Quote from: LoyceV on March 02, 2018, 11:20:49 AM

Quote from: cellard on March 02, 2018, 08:09:18 AM

But I remember I asked about symlinking method to separate chainstate folder from blocks folder and also other members also asked about it and basically the consensus was that it was very unsafe and would corrupt the block or chainstate files...

As far as I know, the unsafe part happens if your symlinked location becomes unavailable. That's not something I worry about on my own system though (worst case I'll need to restore a backup).

I was wondering about the actual connection between blocks and chainstate folders
can we recover/rebuild one by using the other? which one is more important?
recently my wallet (btx based on btc) ask to rebuild the blocks because corrupted issue
but it keeps failing on rebuilding the blocks and I ended syncing from my last backup
every now and then I make backup by copying blocks and chainstate folders, is this the correct way to backup blockchains?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: achow101 on March 01, 2018, 03:11:31 PM

It should be possible to do now with symlinking.

I can confirm this, it's exactly what I did with my blocks-directory (my .bitcoin-directory is on my SSD now).

As an alternative (although not something I prefer) you could mount a small dedicated partition on your chainstate location.

Quote from: cellard on March 02, 2018, 08:09:18 AM

But I remember I asked about symlinking method to separate chainstate folder from blocks folder and also other members also asked about it and basically the consensus was that it was very unsafe and would corrupt the block or chainstate files...

As far as I know, the unsafe part happens if your symlinked location becomes unavailable. That's not something I worry about on my own system though (worst case I'll need to restore a backup).

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: cellard on March 02, 2018, 08:09:18 AM

But I remember I asked about symlinking method to separate chainstate folder from blocks folder and also other members also asked about it and basically the consensus was that it was very unsafe and would corrupt the block or chainstate files... what has changed since then? Or when you say possible, you mean it can be done but it is still unsafe even with the latest version? Im just not sure what you mean.

I think you mean that nothing has changed and it's still unsafe but I want to be sure... but I predict that im going to be stuck with an HDD.

Nothing has changed and it's probably still unsafe to symlink the blocks folder.

cellard

legendary

Activity: 1372

Merit: 1252

Quote from: achow101 on March 01, 2018, 03:11:31 PM

Quote from: cellard on March 01, 2018, 01:52:33 PM

Will this ever be possible

It should be possible to do now with symlinking. But to actually have configuration options, I'm not sure.

Quote from: cellard on March 01, 2018, 01:52:33 PM

and how long are we from seeing that?

No such time estimates can be given since no one is working on it and it is not a priority.

Quote from: cellard on March 01, 2018, 01:52:33 PM

I remember asking a long time ago about wallet separated from root folder and got told it was not possible and unsafe, yet it is now possible, so I would like to ask about this.

What people were asking for in the past is not possible and is still unsafe. What happens now is only possible because of datadir structure changes that move wallet things into their own separate directory.

But I remember I asked about symlinking method to separate chainstate folder from blocks folder and also other members also asked about it and basically the consensus was that it was very unsafe and would corrupt the block or chainstate files... what has changed since then? Or when you say possible, you mean it can be done but it is still unsafe even with the latest version? Im just not sure what you mean.

I think you mean that nothing has changed and it's still unsafe but I want to be sure... but I predict that im going to be stuck with an HDD.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: cellard on March 01, 2018, 01:52:33 PM

Will this ever be possible

It should be possible to do now with symlinking. But to actually have configuration options, I'm not sure.

Quote from: cellard on March 01, 2018, 01:52:33 PM

and how long are we from seeing that?

No such time estimates can be given since no one is working on it and it is not a priority.

Quote from: cellard on March 01, 2018, 01:52:33 PM

I remember asking a long time ago about wallet separated from root folder and got told it was not possible and unsafe, yet it is now possible, so I would like to ask about this.

What people were asking for in the past is not possible and is still unsafe. What happens now is only possible because of datadir structure changes that move wallet things into their own separate directory.

cellard

legendary

Activity: 1372

Merit: 1252

I have been looking at the documentation on the latest 0.16 update and I see that we can finally allocate the wallet.dat file anywhere we want including any other hard drive which is cool, but still this doesn't solve the problem of having to have chainstate and blocks in the same place.

The ideal setup would be to have your blocks folder in an HDD and then the rest on an SSD for maximum speed while not cluttering your expensive SSD space with blocks. Even if you use prune mode eventually it will be a pain in the ass to sync on your SSD since SSD space is very expensive and usually you will want to use it for other things too. If we could use HDD for blocks and SSD for chainstate it would lower the costs of running a node at decent speeds a lot.

Will this ever be possible and how long are we from seeing that? I remember asking a long time ago about wallet separated from root folder and got told it was not possible and unsafe, yet it is now possible, so I would like to ask about this.

Topic: Will it ever be possible to separate chainstate and blocks folder? (Read 89 times)