Bitcoin livenet size in gigabytes | Bitcointalksearch.org

Rath_

legendary

Activity: 1876

Merit: 3132

Quote from: msaikat on September 25, 2019, 02:15:13 PM

Thanks for the answer.

Whoops, I looked at 478 GB in the previous quote instead of subtracting 261 GB from it. Right now, you have downloaded 217 GB out of about 227 GB of block data. You should need about 35 GB of space more to finish the sync. Keep in mind that about 6 blocks are mined every hour so you will need an extra space but definitely less than 1 TB to keep your node operational for now.

msaikat

newbie

Activity: 7

Merit: 4

Quote from: Rath_ on September 25, 2019, 02:05:24 PM

It looks like it's because of the txindex set to 1. According to this reddit post, 2 years ago, one had to download 108 GB without txindex turned on and 226 GB once enabled. It looks like you will need about 1 TB of space. I haven't run a node with txindex turned on so someone else should confirm if this is true.

Thanks for the answer.

Rath_

legendary

Activity: 1876

Merit: 3132

Quote from: msaikat on September 25, 2019, 01:57:29 PM

261G ./blocks/index
478G ./blocks
481G

It looks like it's because of the txindex set to 1. According to this reddit post, 2 years ago, one had to download 108 GB without txindex turned on and 226 GB once enabled. It looks like you will need about 1 TB of space. I haven't run a node with txindex turned on so someone else should confirm if this is true.

msaikat

newbie

Activity: 7

Merit: 4

Quote from: bob123 on September 25, 2019, 01:25:20 PM

Those ~500GB do not come from core, right ?

I am not 100% clear what you meant by core here? Here is the folder structure contains everything...

$ ls -l
total 116
Sep 18 14:22 bitcore-node.json
Sep 25 18:50 data
Sep 18 14:22 node_modules
Sep 18 14:22 package-lock.json
Sep 18 14:22 package.json

$ sudo du -sh
481G

$ cd data
$ ls -l
Sep 18 13:00 bitcoin.conf
Sep 24 07:43 blocks
Sep 24 07:54 chainstate
Sep 24 07:54 database
Sep 18 13:00 db.log
Sep 25 18:53 debug.log
Sep 24 07:54 fee_estimates.dat
Sep 24 06:56 peers.dat
Sep 24 07:54 peers.dat.077e
Sep 24 07:37 peers.dat.7e3a
Sep 24 07:52 wallet.dat

$ sudo du -h
16K ./database
3.6G ./chainstate
261G ./blocks/index
478G ./blocks
481G

Quote from: bob123 on September 25, 2019, 01:25:20 PM

It seems like your only option is to get more storage. At least if you want to develop something which requires all transactions being indexed.

Can you give an idea how much disk-space I need to add to the AWS EC2 instance?

bob123

legendary

Activity: 1624

Merit: 2481

Well, if you need all transactions accessible then you can't use pruning mode since you need to have txindex enabled.

Those ~500GB do not come from core, right ?
For core fully synced you'll need roughly 250GB.

It seems like your only option is to get more storage. At least if you want to develop something which requires all transactions being indexed.

msaikat

newbie

Activity: 7

Merit: 4

Quote from: bob123 on September 25, 2019, 12:44:15 PM

Does your appllication only need to handle transactions affecting your wallet ? Or all transactions ?
If it is the latter one, you need txindex enabled (which implies no pruning possible), otherwise not.

All transaction will be required to be considered

Quote from: bob123 on September 25, 2019, 12:44:15 PM

How much storage do you have accessible ?
If your node is pruned and you remove it, then you'll need to resync.
Same applies to if you had txindex disabled, and enable it.

Otherwise, it should not be necessary.

I have 495G of space which is already occupied! I didn't run with -prune mode earlier but there is a "txindex=1" in "bitcoin.conf"

bob123

legendary

Activity: 1624

Merit: 2481

Quote from: msaikat on September 25, 2019, 12:20:04 PM

I need bitcore to build my own application to perform transactions with bitcoin address creation, view tx status etc.

Does your appllication only need to handle transactions affecting your wallet ? Or all transactions ?
If it is the latter one, you need txindex enabled (which implies no pruning possible), otherwise not.

Quote from: msaikat on September 25, 2019, 12:20:04 PM

I am now facing only the storage issue and wondering if I need to start re-sync the full node!

How much storage do you have accessible ?
If your node is pruned and you remove it, then you'll need to resync.
Same applies to if you had txindex disabled, and enable it.

Otherwise, it should not be necessary.

msaikat

newbie

Activity: 7

Merit: 4

Quote from: Rath_ on September 25, 2019, 11:36:33 AM

Sorry, I don't have any experience with Bitcore so I will have to rely on the information available on their Github and the Internet since I don't have any spare VPS at the moment to test it. What guide did you follow? Did you create bitcoind config file? Take a look at this guide and see if it helps you. Skip 'testnet=1' in the bitcoind config and change 'testnet' to 'mainnet' in the bitcore-node.json. Do you need bitcore for some kind of service or you just want to run a full node?

Hi,

I actually followed the exact same steps described in the guide you mentioned and yes, bitcoin.conf file was created. I changed to "network": "livenet". I need bitcore to build my own application to perform transactions with bitcoin address creation, view tx status etc.

I am now facing only the storage issue and wondering if I need to start re-sync the full node!

msaikat

newbie

Activity: 7

Merit: 4

Quote from: OmegaStarScream on September 25, 2019, 10:11:16 AM

Are you hosting it on VMWare? This looks like the answer you're looking for: https://bitcoin.stackexchange.com/questions/65005/bitcored-vmware-disk-space-increases-incredibly-fast

Thnaks for asking. I am using AWS EC2 instance

Rath_

legendary

Activity: 1876

Merit: 3132

Quote from: msaikat on September 25, 2019, 10:37:47 AM

(2) What options should I use while running bitcoind. Now I am using only "exec": "/usr/lib/node_modules/bitcore/node_modules/bitcore-node/bin/bitcoind" in bitcore-node.json

Sorry, I don't have any experience with Bitcore so I will have to rely on the information available on their Github and the Internet since I don't have any spare VPS at the moment to test it. What guide did you follow? Did you create bitcoind config file? Take a look at this guide and see if it helps you. Skip 'testnet=1' in the bitcoind config and change 'testnet' to 'mainnet' in the bitcore-node.json. Do you need bitcore for some kind of service or you just want to run a full node?

bob123

legendary

Activity: 1624

Merit: 2481

Quote from: msaikat on September 25, 2019, 10:37:47 AM

(1) The -prune mode is incompatible with -txindex and -rescan. Is that -txindex helps to reduce the disk space?

-txindex is for indexing all transactions, not just the ones which affect your wallet.
For regular use this is not needed. If you however are developing something like a blockchain explorer or chain anyalysis software, you definitely need txindex enabled.

If you are simply just running a full node to either support the network or to not rely on any 3rd party, you can freely disable txindex. And if you wish, enable pruning. This will only save X MB of the last blocks (default = 500).

msaikat

newbie

Activity: 7

Merit: 4

Quote from: Rath_ on September 25, 2019, 09:59:49 AM

You can use pruning if you don't mind getting rid of old blocks after they have been verified. Only about 288 recent blocks will be kept in case of chain reorganizations by default.

#Edit: I have no experience with bitcored which might be the root of the problem.

Hi @BitCryptex Thank you for the reply. I have two things to know here...

(1) The -prune mode is incompatible with -txindex and -rescan. Is that -txindex helps to reduce the disk space?

(2) What options should I use while running bitcoind. Now I am using only "exec": "/usr/lib/node_modules/bitcore/node_modules/bitcore-node/bin/bitcoind" in bitcore-node.json

FYI I installed using "npm install -g --unsafe-perm=true bitcore" and "apt-get install bitcoind"

I really appreciate your help.

OmegaStarScream

staff

Activity: 3500

Merit: 6152

Are you hosting it on VMWare? This looks like the answer you're looking for: https://bitcoin.stackexchange.com/questions/65005/bitcored-vmware-disk-space-increases-incredibly-fast

Rath_

legendary

Activity: 1876

Merit: 3132

Quote from: msaikat on September 25, 2019, 09:50:37 AM

Mine is having 495G in total and the node syncing took all of it and now shows "No space left on device" in debug.log. Can anybody mention what is the recommended disk size for a full livenet node. Earlier I heard about 200G which didn't work for me. Or any suggestion that can improve this scenario?

You can use pruning if you don't mind getting rid of old blocks after they have been verified. Only about 288 recent blocks will be kept in case of chain reorganizations by default.

#Edit: I have no experience with bitcored which might be the root of the problem.

msaikat

newbie

Activity: 7

Merit: 4

Hello,

I have a full node running with "bitcored" and it's getting a huge disk space and running out of it several times! Mine is having 495G in total and the node syncing took all of it and now shows "No space left on device" in debug.log. Can anybody mention what is the recommended disk size for a full livenet node. Earlier I heard about 200G which didn't work for me. Or any suggestion that can improve this scenario?

/msaikat

Topic: Bitcoin livenet size in gigabytes (Read 186 times)