Can I just run a pruned node without downloading historical data? - page 2.

DannyHamilton

legendary

Activity: 3472

Merit: 4801

Quote from: pooya87 on October 07, 2021, 10:12:39 PM

Quote from: Beetkoin on October 07, 2021, 05:12:52 AM

So I want to know if there is an implementation of Bitcoin where I can just start with a pruned node. I will sacrifice the security of verifying the entire chain from genesis for the convenience of just downloading what is necessary to construct a pruned node.

Yes there is, the category is called SPV clients and the example of it is Electrum.
In this category, you download a very small amount of data known as block headers and perform minimal verification on a lot of things. Your communication with nodes will also be minimal each time you want to synchronize.

Quote from: Beetkoin on October 07, 2021, 02:29:27 AM

I just want to run a full node where I can just type in a certain public key, and it tells me which UTXOs are associated with this key

Maybe you should first explain what your real goal is.

This is definitely starting to sound like a classic XY Problem.

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: Beetkoin on October 07, 2021, 05:12:52 AM

So I want to know if there is an implementation of Bitcoin where I can just start with a pruned node. I will sacrifice the security of verifying the entire chain from genesis for the convenience of just downloading what is necessary to construct a pruned node.

Yes there is, the category is called SPV clients and the example of it is Electrum.
In this category, you download a very small amount of data known as block headers and perform minimal verification on a lot of things. Your communication with nodes will also be minimal each time you want to synchronize.

Quote from: Beetkoin on October 07, 2021, 02:29:27 AM

I just want to run a full node where I can just type in a certain public key, and it tells me which UTXOs are associated with this key

Maybe you should first explain what your real goal is.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: witcher_sense on October 07, 2021, 05:41:31 AM

The only implementation I am aware of is this: Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!) It is no longer being maintained, though.

This doesn't include a UTXO list. I'm not even sure if a pruned node does that nowadays. I'd like the feature though: that would allow to load an old wallet.dat into a pruned node.

Quote from: Beetkoin on October 07, 2021, 05:12:52 AM

That triggers me a lot. Feels like the developers forcing me to do a lot of extra work.

It's part of the "verify, don't trust" mantra. As long as you're not limited on data, any half decent computer should be able to download and prune Bitcoin Core in about a day. If you have enough RAM you could even fit it on a RAM drive to beat SSD speed (I've done this as a test on /dev/shm).

BlackHatCoiner

legendary

Activity: 1512

Merit: 7340

Farewell, Leo

Quote from: n0nce on October 07, 2021, 04:23:45 AM

If you don't mind verifying everything, just run an SPV node. It's as simple as that

You mean, connect with an SPV node? If you run your own SPV node, you're verifying everything.

Quote from: n0nce on October 07, 2021, 05:39:13 AM

Quote from: Beetkoin on October 07, 2021, 05:12:52 AM

However, when I first run Bitcoin Core / Bitcoin Knots, it only gives me two options. Both options require me to download the entire 350GB blockchain, regardless of whether I want to run a full node or a pruned node.

This is the only thing giving you more security over an SPV node though.

You mean privacy? Connecting to an SPV node has no difference from running your own in terms of security.

Quote from: Beetkoin on October 07, 2021, 05:02:13 AM

Unpruned costs 350GB. Pruned requires trusting third parties for historical data.

Both pruned and unpruned requires you to trust none. With pruned you just then get rid of the blocks you had downloaded to save up storage. Note that once you start syncing as a pruned node, you can only choose specific wallet(s). Once you finish the synchronization, no new wallets can be loaded unless you're fine by re-syncing it.

vjudeu

copper member

Activity: 903

Merit: 2248

Quote

The list of all non-zero UTXO can be constructed from an Unpruned full node. Or it can be downloaded from somewhere.

UTXO's with zero satoshis are valid. Creating and spending them is valid, so if you get rid of them, you will have an incompatible node.

Quote

And if I reinstall my computer or my harddisk fails, then I have to do everything all over again.

No, because you can make a backup of your pruned node and store it somewhere. All you need is keeping some hash of that backup, then you can later download something like 7 GB of data and check if the hash is correct, in that way you can do initial block downloading once and trust your own backup since then.

PrimeNumber7

copper member

Activity: 1666

Merit: 1901

Amazon Prime Member #7

Quote from: Beetkoin on October 07, 2021, 02:07:43 AM

Forgive me, Toxic Maxis, but is there an implementation of Bitcoin Core where I can only download the most recent 2GB of blocks?

Furthermore, I have another idea.

Is it possible to build a full node that stores only the latest UTXO set and the latest 2016 blocks for difficulty adjustment?

If I store all current UTXOs and the latest 2016 blocks, is it to all intents and purposes, the same as storing the entire 350GB block chain?

No, this is not possible.

Each new block that is found is a derivative of the previous block, and every block before it. In order to validate if a block is valid or not, you need to know if all inputs spent in that block are valid, and the only way to know if an input is valid is by at one point looking at all previous blocks. If your node has never looked at all previous blocks, it will have no way of knowing the UTXO set.

One solution might be to change the bitcoin protocol so that blocks must contain a hash of the UTXO set so that someone has the ability to run a full node without processing the entire blockchain.

Quote from: Beetkoin on October 07, 2021, 02:29:27 AM

Quote

the pruned blockchain contains data relevant to your wallet
hence other nodes cannot really give you "filtered" data

Do I have to start a new wallet in order to run Bitcoin core?
What if I just want to use Bitcoin core as a block explorer?

I just want to run a full node where I can just type in a certain public key, and it tells me which UTXOs are associated with this key. Basically a local version of blockchain.com/explorer

If you want to use a full node as anything resembling a block explorer, you must store the entire blockchain. When you run a pruned node, you are disregarding the history of all transactions, and are keeping a list of all unspent outputs that can be part of a block, plus a portion of the recent blockchain in case of a reorg.

If all you want are unspent outputs, you can run a pruned node, but this will omit a lot of information that pretty much all other block explorers provide.

witcher_sense

legendary

Activity: 2450

Merit: 4415

🔐BitcoinMessage.Tools🔑

Quote from: Beetkoin on October 07, 2021, 05:12:52 AM

Quote from: n0nce on October 07, 2021, 05:05:20 AM

That's exactly what a pruned node does actually. You construct the UTXO list yourself while downloading all blocks from genesis onwards, so you can be 100% sure that it's correct.
But looking up how much funds were on an address at an earlier point in time, whose block has been deleted from your drive, that has to be fetched from non-pruned full nodes.

Wow that is good news. I will want to run a pruned node.

However, when I first run Bitcoin Core / Bitcoin Knots, it only gives me two options. Both options require me to download the entire 350GB blockchain, regardless of whether I want to run a full node or a pruned node.

That triggers me a lot. Feels like the developers forcing me to do a lot of extra work.

And if I reinstall my computer or my harddisk fails, then I have to do everything all over again.

So I want to know if there is an implementation of Bitcoin where I can just start with a pruned node. I will sacrifice the security of verifying the entire chain from genesis for the convenience of just downloading what is necessary to construct a pruned node.

The only implementation I am aware of is this: Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!) It is no longer being maintained, though.

You may have noticed a big "DON'T DO THIS" sign, so you probably shouldn't try to run a pruned node without validating of all previous history.

It is not developers who are forcing you to download and verify everything that occurred in bitcoin blockchain, it is trust issues with third-party providers of blockchain data that require developers to not implement questionable things that may destroy the whole network.

n0nce

hero member

Activity: 882

Merit: 5834

not your keys, not your coins!

Quote from: Beetkoin on October 07, 2021, 05:12:52 AM

However, when I first run Bitcoin Core / Bitcoin Knots, it only gives me two options. Both options require me to download the entire 350GB blockchain, regardless of whether I want to run a full node or a pruned node.

This is the only thing giving you more security over an SPV node though. Anything else would be pointless really.
Just let it sit for a few days, it's not as bad as you might think

Beetkoin

newbie

Activity: 15

Merit: 18

Quote from: n0nce on October 07, 2021, 05:05:20 AM

That's exactly what a pruned node does actually. You construct the UTXO list yourself while downloading all blocks from genesis onwards, so you can be 100% sure that it's correct.
But looking up how much funds were on an address at an earlier point in time, whose block has been deleted from your drive, that has to be fetched from non-pruned full nodes.

Wow that is good news. I will want to run a pruned node.

However, when I first run Bitcoin Core / Bitcoin Knots, it only gives me two options. Both options require me to download the entire 350GB blockchain, regardless of whether I want to run a full node or a pruned node.

That triggers me a lot. Feels like the developers forcing me to do a lot of extra work.

And if I reinstall my computer or my harddisk fails, then I have to do everything all over again.

So I want to know if there is an implementation of Bitcoin where I can just start with a pruned node. I will sacrifice the security of verifying the entire chain from genesis for the convenience of just downloading what is necessary to construct a pruned node.

n0nce

hero member

Activity: 882

Merit: 5834

not your keys, not your coins!

Quote from: Beetkoin on October 07, 2021, 05:02:13 AM

Ok, so currently the only two options for running a full node are Unpruned and Pruned.

Unpruned costs 350GB. Pruned requires trusting third parties for historical data.

I am proposing a "partially pruned" full node.

Basically, a Pruned full node plus all currently non-zero UTXO.

That's exactly what a pruned node does actually. You construct the UTXO list yourself while downloading all blocks from genesis onwards, so you can be 100% sure that it's correct.
But looking up how much funds were on an address at an earlier point in time, whose block has been deleted from your drive, that has to be fetched from non-pruned full nodes.

Quote from: Beetkoin on October 07, 2021, 05:02:13 AM

The list of all non-zero UTXO can be constructed from an Unpruned full node. Or it can be downloaded from somewhere.

This makes absolutely no sense. If you 'just download something somewhere', you're trusting and might as well just use an SPV node software.

Beetkoin

newbie

Activity: 15

Merit: 18

Ok, so currently the only two options for running a full node are Unpruned and Pruned.

Unpruned costs 350GB. Pruned requires trusting third parties for historical data.

I am proposing a "partially pruned" full node.

Basically, a Pruned full node plus all currently non-zero UTXO.

This can save disk space, plus not having to trust third parties for historical data.

The list of all non-zero UTXO can be constructed from an Unpruned full node. Or it can be downloaded from somewhere.

n0nce

hero member

Activity: 882

Merit: 5834

not your keys, not your coins!

Quote from: Beetkoin on October 07, 2021, 02:29:27 AM

So I am thinking this:
If I can just store the complete set of all current UTXOs, plus the most recent 2016 blocks, then it is to all intents and purposes the same as running a 350GB full node.

How would you plan to get all current UTXOs without downloading the blockchain from the beginning?
It's simple: if I bought BTC in 2010, that UTXO was mined and recorded in the blockchain back then. If you only download the latest blocks, you will never know about it and think that that address is empty. This is why you need to download everything to build the UTXO cache yourself and verify everything.

If you don't mind verifying everything, just run an SPV node. It's as simple as that

If you don't have the storage for 350GB, I think pruned node is a good idea. It gives you many benefits over SPV, while only requiring a few GB of storage. Sure, the initial block download won't be faster than when doing a regular full node, but if you don't have bandwidth issues, it's really a no-brainer.

I'm not sure, what's the state on this, but it seems you can just run Bitcoin Core in SPV mode actually. Thus having access to all of the commands etc.:
https://github.com/bitcoin/bitcoin/pull/9076 It seems it was never merged (?)

Charles-Tim

legendary

Activity: 1512

Merit: 4795

Leading Crypto Sports Betting & Casino Platform

Quote from: Beetkoin on October 07, 2021, 02:29:27 AM

Is it possible to build a full node that stores only the latest UTXO set and the latest 2016 blocks for difficulty adjustment?

You can check this out to know more about pruned node, but ways to privacy can be costly and best to download the full blockchain.

Pros and Cons of Bitcoin Node types (Full node and Prune node)

Quote from: Beetkoin on October 07, 2021, 02:29:27 AM

As far as I can tell, if I run a pruned node, when I want to look up a certain public key I will have to reference other people's nodes. Basically a pruned node is no different from blockchain.com/explorer.

Completely different. If you have pruned node, you are running full node, not necessary to download the whole blockchain, but it is advisable to download the whole blockchain if you want to enjoying running full node. Blockchain explerers are their parties, many people will use their IP address to look up bitcoin addresses, just a way many addresses can be connected together and resulting to no privacy, you can check this including the mempool on Bitcoin Core if you are running full node.

Also is the issue of SPV wallets connecting to central servers that can link your bitcoin addresses together, linking it to your IP addresses, making privacy not possible, this is the reason privacy oriented bitcoin users will not depend on central servers but run their own node.

Beetkoin

newbie

Activity: 15

Merit: 18

Quote

the pruned blockchain contains data relevant to your wallet
hence other nodes cannot really give you "filtered" data

Do I have to start a new wallet in order to run Bitcoin core?
What if I just want to use Bitcoin core as a block explorer?

I just want to run a full node where I can just type in a certain public key, and it tells me which UTXOs are associated with this key. Basically a local version of blockchain.com/explorer

Furthermore I don't have the space to store 350GB of data.
As far as I can tell, if I run a pruned node, when I want to look up a certain public key I will have to reference other people's nodes. Basically a pruned node is no different from blockchain.com/explorer.

So I am thinking this:
If I can just store the complete set of all current UTXOs, plus the most recent 2016 blocks, then it is to all intents and purposes the same as running a 350GB full node.

NeuroticFish

legendary

Activity: 3668

Merit: 6382

Looking for campaign manager? Contact icopress!

Quote from: Beetkoin on October 07, 2021, 02:07:43 AM

is there an implementation of Bitcoin Core where I can only download the most recent 2GB of blocks?

Nope. The idea is to trust nobody, hence you (your local wallet software) will verify/validate the whole blockchain to ensure the 2 GB you keep is correct / not tampered.
Furthermore, the pruned blockchain contains data relevant to your wallet, hence other nodes cannot really give you "filtered" data, since they don't know your addresses.

Quote from: Beetkoin on October 07, 2021, 02:07:43 AM

If no such software exists I will build it from scratch. I am currently reading Programming Bitcoin by Jimmy Song.

I suggest you read more first. Also maybe what you want is a SPV wallet.

Beetkoin

newbie

Activity: 15

Merit: 18

I am trying to run Bitcoin core, but when I started the software, it says that I can either:

1) Download the entire blockchain and keep it on my disk, 350GB
2) Download the entire blockchain, verify historical data, and only keep the most recent 2 GB of blocks.

Either way, I have to download 350GB of historical data.

Forgive me, Toxic Maxis, but is there an implementation of Bitcoin Core where I can only download the most recent 2GB of blocks?

Furthermore, I have another idea.

Is it possible to build a full node that stores only the latest UTXO set and the latest 2016 blocks for difficulty adjustment?

If I store all current UTXOs and the latest 2016 blocks, is it to all intents and purposes, the same as storing the entire 350GB block chain?

If no such software exists I will build it from scratch. I am currently reading Programming Bitcoin by Jimmy Song.

Topic: Can I just run a pruned node without downloading historical data? - page 2. (Read 531 times)