Bitcoin-core: only keep last x blocks on disk

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: Hyena on December 18, 2015, 11:23:22 AM

Quote from: joschua011 on December 18, 2015, 10:42:23 AM

Quote from: Hyena on December 18, 2015, 07:58:21 AM

Things get even stranger in a configuration where you need several wallets running in parallel on different VMs on the same host. It's obviously a big overhead to store a duplicate block chain on both of those VMs.

You could just share the data volume that contains the blockchain among different vm's.

Won't it cause any conflicts, deadlocks and corruption? But if you say it's possible then I guess it would be a good idea. To my understanding block chain has been notorious for being overly sensitive and getting corrupt at the sight of slightest problems so that it has to be downloaded from scratch. But that's just like my subjective opinion.

It isn't possible to use the same blockchain data with more than one instance of bitcoin core. There are locks on the files which prevent that from being possible.

Quote from: Hyena on December 18, 2015, 11:23:22 AM

Quote from: DannyHamilton on December 18, 2015, 10:38:12 AM

I'm curious why you don't just use an SPV interface, bitcoinj for example?

I haven't looked into those alternatives yet. I always assumed that a different node software comes with a different API/RPCs which would require my personal software to be rewritten/changed.

They usually are different API/RPCs but you might be able to find some that are similar.

Hyena

legendary

Activity: 2114

Merit: 1015

Quote from: joschua011 on December 18, 2015, 10:42:23 AM

Quote from: Hyena on December 18, 2015, 07:58:21 AM

Things get even stranger in a configuration where you need several wallets running in parallel on different VMs on the same host. It's obviously a big overhead to store a duplicate block chain on both of those VMs.

You could just share the data volume that contains the blockchain among different vm's.

Won't it cause any conflicts, deadlocks and corruption? But if you say it's possible then I guess it would be a good idea. To my understanding block chain has been notorious for being overly sensitive and getting corrupt at the sight of slightest problems so that it has to be downloaded from scratch. But that's just like my subjective opinion.

Quote from: DannyHamilton on December 18, 2015, 10:38:12 AM

I'm curious why you don't just use an SPV interface, bitcoinj for example?

I haven't looked into those alternatives yet. I always assumed that a different node software comes with a different API/RPCs which would require my personal software to be rewritten/changed.

joschua011

member

Activity: 86

Merit: 10

Quote from: Hyena on December 18, 2015, 07:58:21 AM

Things get even stranger in a configuration where you need several wallets running in parallel on different VMs on the same host. It's obviously a big overhead to store a duplicate block chain on both of those VMs.

You could just share the data volume that contains the blockchain among different vm's.

DannyHamilton

legendary

Activity: 3472

Merit: 4801

I'm curious why you don't just use an SPV interface, bitcoinj for example?

Hyena

legendary

Activity: 2114

Merit: 1015

Quote from: achow101 on December 18, 2015, 09:03:54 AM

Wallet functionality in a pruned node was disabled because it was still untested when 0.11.0 was released. It will be enabled in 0.12.

You should still be able to broadcast transactions using a pruned node, but I don't know about getting outputs since there is no wallet and txindex is disabled with pruning.

However you can just build the master branch from github for now and get the wallet functionality and then upgrade to 0.12 whenever they release it. It should be coming out in a few weeks.

ok this is really great news. If it's about to be released soon then I can wait. I have long been promoting circular block chains so that blocks older than a year for example would be deleted. With this pruning option it is pretty much the same effect. Those who want to keep the full history can do it if they have the space but most people probably only would keep last year or something on their disk. If you have coins older than 1 year sitting in your wallet you would just send those coins to yourself once in a year and everything would work perfectly. block chain size is no longer an issue

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: Hyena on December 18, 2015, 08:49:47 AM

Quote from: achow101 on December 18, 2015, 08:46:55 AM

Bitcoin core supports pruning. You can set it using the -prune= where is the size of the blockchain you want to store. The minimum you can do is 550 Mb. Unfortunately you can't use the wallet yet but that will be fixed in 0.12

Ok that's good news. What do you mean by not being able to use the wallet? I would need to sign transactions and listen for new incoming transactions, that's all what I need. The latter requires a wallet where to keep the private keys for signing.

edit:
But I could do my thing without wallet functions, all I need is my own (possibly C++) function for signing a raw transaction with the private key I'd provide. I need the pruned mode to allow me to construct raw transactions based on specific outputs, sign them and broadcast them. Is it possible?

Wallet functionality in a pruned node was disabled because it was still untested when 0.11.0 was released. It will be enabled in 0.12.

You should still be able to broadcast transactions using a pruned node, but I don't know about getting outputs since there is no wallet and txindex is disabled with pruning.

However you can just build the master branch from github for now and get the wallet functionality and then upgrade to 0.12 whenever they release it. It should be coming out in a few weeks.

Hyena

legendary

Activity: 2114

Merit: 1015

Quote from: achow101 on December 18, 2015, 08:46:55 AM

Bitcoin core supports pruning. You can set it using the -prune= where is the size of the blockchain you want to store. The minimum you can do is 550 Mb. Unfortunately you can't use the wallet yet but that will be fixed in 0.12

Ok that's good news. What do you mean by not being able to use the wallet? I would need to sign transactions and listen for new incoming transactions, that's all what I need. The latter requires a wallet where to keep the private keys for signing.

edit:
But I could do my thing without wallet functions, all I need is my own (possibly C++) function for signing a raw transaction with the private key I'd provide. I need the pruned mode to allow me to construct raw transactions based on specific outputs, sign them and broadcast them. Is it possible?

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Bitcoin core supports pruning. You can set it using the -prune= where is the size of the blockchain you want to store. The minimum you can do is 550 Mb. Unfortunately you can't use the wallet yet but that will be fixed in 0.12

Hyena

legendary

Activity: 2114

Merit: 1015

It's getting embarrassing to demand 60+ GiB from server hosting providers just to have bitcoin-core running in a situation where I even don't care about the full block chain. In my situation I would only want to store last 1000 blocks on disk because I know that I don't care about transactions that are older than that. The number could be even smaller, I would probably be happy to only have last 144 blocks (24 hours) stored on disk. Perhaps such a configuration is already possible? If not then how doable it is? Does anyone else feel they would need such functionality?

Things get even stranger in a configuration where you need several wallets running in parallel on different VMs on the same host. It's obviously a big overhead to store a duplicate block chain on both of those VMs. What I would propose is to host only one VM with the full block chain and configure other VMs to communicate with the full node whenever they need info about historic blocks. I'm pretty sure these sort of things have been discussed before but I feel like this is sort of an important improvement for the bitcoin-core and it deserves up to date discussion.

Topic: Bitcoin-core: only keep last x blocks on disk (Read 803 times)