Author

Topic: Back up my blockchain data (Read 189 times)

member
Activity: 162
Merit: 65
January 03, 2023, 06:39:12 PM
#13
google:
"The rev*. dat files (the "undo files"), contain all UTXOs spent by the inputs of a block. It was introduced in Bitcoin Core 0.8, and contains a concatenation of records, one for each block. This mimicks the structure of the blk*"

it basically holds a copy of txs that are spent tx....(opposite of a utxo set) incase there was a re-org/rollback the node doesnt have to go around requesting old blocks that got previously spent to be relisted as unspent, to then update the utxo database

its more of a internal time saver in cases of rollback

and yes many advise its helpful to copy the rev files too

(my personal node had full blockchain but i edited it to only have a certain amount of other files like rev files as its not often needed to re-org that far back, but still helpful to have recent rev files to speed up any re-org)

i have not personally messed to much with my version. so best to try to copy as much as possible to save as much time as possible
rev files should be ~60gb total

if you are only copying the first 50% of blk's then just copy the reciprocal first 50% of revs so they can correlate to the blk data

copying over just blk may save time of having to re download the initial block download process. but without the other files like rev, your system then need to read each block again the create the utxoset and rev sets. which is a bit extra of "rescan" time as they call it


Thank you for your detailed explanation. It's fucking helpful.
legendary
Activity: 4410
Merit: 4766
January 03, 2023, 10:45:19 AM
#12
google:
"The rev*. dat files (the "undo files"), contain all UTXOs spent by the inputs of a block. It was introduced in Bitcoin Core 0.8, and contains a concatenation of records, one for each block. This mimicks the structure of the blk*"

it basically holds a copy of txs that are spent tx....(opposite of a utxo set) incase there was a re-org/rollback the node doesnt have to go around requesting old blocks that got previously spent to be relisted as unspent, to then update the utxo database

its more of a internal time saver in cases of rollback

and yes many advise its helpful to copy the rev files too

(my personal node had full blockchain but i edited it to only have a certain amount of other files like rev files as its not often needed to re-org that far back, but still helpful to have recent rev files to speed up any re-org)

i have not personally messed to much with my version. so best to try to copy as much as possible to save as much time as possible
rev files should be ~60gb total

if you are only copying the first 50% of blk's then just copy the reciprocal first 50% of revs so they can correlate to the blk data

copying over just blk may save time of having to re download the initial block download process. but without the other files like rev, your system then need to read each block again the create the utxoset and rev sets. which is a bit extra of "rescan" time as they call it
member
Activity: 162
Merit: 65
January 03, 2023, 09:00:44 AM
#11
yes you can just copy over half of the files and then copy over the other half (in two copy/paste rounds)

the blk files are sequential so make sure you (eventually) copy all the files or your sync(re-scan) will stop at a certain point if a middle blk file is missing
hey man, i see besides those blk.dat files, there are also rev.dat files. What are these? Can i remove rev.dat files to save some space?
newbie
Activity: 62
Merit: 0
January 03, 2023, 07:10:42 AM
#10
For me is better to look for a hard drive that can accommodate all the data to avoid mistakes you might regret later. Speaking out of experience
legendary
Activity: 1512
Merit: 7340
Farewell, Leo
January 03, 2023, 04:53:04 AM
#9
Yes, that's what i was asking. Can i just copy a part of the blockchain data that my external drive can hold maximum? And later i will just need to redownload some of the recent data? Is it dorable?
I haven't ever tried it, but it's reasonable to copy a few .blk files, and have the rest downloaded, yeah.
member
Activity: 162
Merit: 65
January 03, 2023, 12:12:08 AM
#8
So here i am asking if possible to reduce the blockchain data (perhaps to delete couple recent blocks) to make it small so it can fit in my external drive.
You can copy part of the blockchain, until your external drive is full. The rest of it has to be downloaded again I'm afraid.

Yes, that's what i was asking. Can i just copy a part of the blockchain data that my external drive can hold maximum? And later i will just need to redownload some of the recent data? Is it dorable?
legendary
Activity: 4410
Merit: 4766
January 02, 2023, 11:04:23 AM
#7
So here i am asking if possible to reduce the blockchain data (perhaps to delete couple recent blocks) to make it small so it can fit in my external drive.
You can copy part of the blockchain, until your external drive is full. The rest of it has to be downloaded again I'm afraid.

yes you can just copy over half of the files and then copy over the other half (in two copy/paste rounds)
As far as I understand, he's reinstalling an OS? Thus, he needs to move the blockchain data externally, wipe out everything internally, re-install the system, and move the blockchain back into it. If he can't move part of the data outside the internal drive, it needs to be re-downloaded.

he has an external drive just not one big enough to do the job in one go..
i was reading it as if he was getting out/away from an old bloated system and wanted to install on a new system. without having to redownload the blockchain

but either way. copying as much as he can saves some time
legendary
Activity: 1512
Merit: 7340
Farewell, Leo
January 02, 2023, 10:51:32 AM
#6
So here i am asking if possible to reduce the blockchain data (perhaps to delete couple recent blocks) to make it small so it can fit in my external drive.
You can copy part of the blockchain, until your external drive is full. The rest of it has to be downloaded again I'm afraid.

yes you can just copy over half of the files and then copy over the other half (in two copy/paste rounds)
As far as I understand, he's reinstalling an OS? Thus, he needs to move the blockchain data externally, wipe out everything internally, re-install the system, and move the blockchain back into it. If he can't move part of the data outside the internal drive, it needs to be re-downloaded.
legendary
Activity: 4410
Merit: 4766
January 02, 2023, 10:41:16 AM
#5
yes you can just copy over half of the files and then copy over the other half (in two copy/paste rounds)

the blk files are sequential so make sure you (eventually) copy all the files or your sync(re-scan) will stop at a certain point if a middle blk file is missing
copper member
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
January 02, 2023, 10:27:57 AM
#4
Could your internal drive store the extra 450GB?

If it can then you could make a drive partition for it and reinstall your operating system just on main partitions it already used (make sure you actually concentrate when setting things up though as you might overwrite it) - also I don't know if windows or mac like to reinstall over the whole drive so you might have to look up how to make a host protected area (HPA) that they then can't touch.

There's the alternative chance that if you have a few GB free you could make the new partition and slowly move things across and then resize the partition.
legendary
Activity: 2030
Merit: 1569
CLEAN non GPL infringing code made in Rust lang
January 02, 2023, 10:01:21 AM
#3
Yeah, unfortunately you need the space to keep the whole blockchain and if you prune it it does the opposite: Only the latest blocks remain. Ideally you should have the space to hold that blockchain like a 1tb hard drive. If your internet is good (measured in tens or hundreds of mbps) you can probably just let it go and sync it again.
legendary
Activity: 2044
Merit: 1018
Not your keys, not your coins!
January 02, 2023, 08:38:15 AM
#2
I wanted to get out of my bloated system and reinstall the system. So i want to back up my blockchain data so i don't get re-synced.
You can run Bitcoin Core prune node, not Bitcoin Core full node.

Quote

The problem: i don't have a spare external drive that is large enough to store that. Mine is like 450gb or something. but the blockchain data at this point in time is larger than that.
With prune node, you can choose how many latest GB of the blockchain you want to store on your disk. 15GB to 30GB is good for a prune node.
member
Activity: 162
Merit: 65
January 02, 2023, 08:22:13 AM
#1
I wanted to get out of my bloated system and reinstall the system. So i want to back up my blockchain data so i don't get re-synced.
The problem: i don't have a spare external drive that is large enough to store that. Mine is like 450gb or something. but the blockchain data at this point in time is larger than that.
So here i am asking if possible to reduce the blockchain data (perhaps to delete couple recent blocks) to make it small so it can fit in my external drive.

Is it feasible?
How can i make it?

Again, thank you all! 
Jump to: