Topic: UTXO set size does not increase storage costs, you just suck at computer science (Read 3160 times)

You can always verify the data on DHT with the full blockchain, and you only need to do it once

i.e. you still scan whole blockchain, you just don't store UTXO set locally.

You have the Bitcoin security model wrong.

How did all these exabytes of data get into the DHT?  If you didn't put them there yourself, you're trusting whoever put them there that they constitute a well-formed blockchain.

What you're describing is SPV UTXO security, and it's also been brought up many times before. Perhaps you should save for allegations of unimagniativeness for ideas that area actually original.

SPV UTXO security means that the reward for someone who can produce N blocks isn't just reversing the transactions they can make and replacing them, it's on the order of the value of the whole bitcoin economy, past and into the future.

They could steal anyones coin,

they could freely inflate the supply, etc.

You have the Bitcoin security model wrong:

For all practical intents and purposes it is enough to have N latest blocks + UTXO set.

There are challenges already before that though like spam and who can change which entries in the database and so on so it might be best to do just the UTXO set first and see how that works out before risking one's precious blockchain.

I am not sure offhand why thin client weren't already thinking DHT

(Or maybe phones just aren't good at being DHTs, maybe they aren't "always on" or have limited bandwidth.)[/quotes]

Yes, it is likely impractical to make phones DHT nodes, although phones will be able to query DHT.

It could work for desktop clients, though. A lot of people think that running Bitcoin-Qt is the right thing, I doubt they will mind running a DHT node for themselves and for mobile brethren.

When the user is offline, where then can that data be found, or is it even needed at such times at all?

It's just a protocol which requires users to keep all data required to verify their transactions.

In simple wordings: a distributed and shared UTXO dataset, right?

So we could also have a distributed and shared full blockchain?

In simple wordings: a distributed and shared UTXO dataset, right?

So we could also have a distributed and shared full blockchain?

Haha, sorry, I didn't mean to offend anyone, I'm just somewhat disappointed that people consider only the simplest solutions and do not want to see a bigger picture.

Let's start with an example... Suppose I am a miner who is in kinda exotic situation: I have unlimited computational and networking resources, and enough temporary storage (RAM) to verify blockchain, but local permanent storage is incredibly scarce. But if I want to store something, I can store it externally over network. Say, in a DHT like Freenet, Tahoe-LAFS or something like that.

So I want to download and scan blockchain up to block N just once, then I need to be able to verify validity of any transaction. Is it possible?

Of course. As I go through blockchain I will build an index and will store it in an external DHT. I only need to keep hash of the latest block locally, 32 bytes that is.

Having this hash, I can retrieve things from DHT until I find transaction outputs I need to verify in index I've built. (I do not care whether DHT is secure: if something was tampered with, hashes won't match.)

This is kinda obvious: if secure distributed file systems can exist, then I can simply store data in such file systems instead of a local file system.

But... how much would it cost me to verify a transaction? Well, tree-like datastructures generally have look-up costs on scale of log2 N where N is a number of elements. In the worst case each individual satoshi is an individual UTXO, so we have 21000000*100000000 = 2100000000000000 ~= 2^51 UTXOs. Thus I need something like 51 lookups to find an UTXO in a binary search tree. Or just 9 lookups if I have a 64-ary tree.

But people can argue that 9 lookups per UTXO is a lot... Network latency yada yada. How about zero?

That's possible. Suppose that person which sends transaction knows how I store index in DHT, it isn't secret. To make sure that I'll include his transaction into block, he will fetch all the data I need from DHT himself, and will send me a message with his transaction and all information I need.

I don't need to look up anything in a DHT myself, I only need data which was stored in the message. And this is secure: if data was tampered with, hash won't match.
 
So, basically, number of transactions I can include into a block is limited by my computational and network abilities, but storage capability/cost is irrelephant.

But what's about blocks mined by others? Oh...

Well, it is possible to do 9 DHT lookups per UTXO mentioned in block. Number of outputs is limited, I can do lookups in parallel, so it isn't such a big problem. But still...

You know, miners are friendly guys, how about they all use same DHT, and then include confirmation information together with the block they have just mined?

So I receive a new block and supplementary information which is all what is needed to confirm that block is valid.

In the end, it is possible to do all mining having only 32 bytes of permanent secure storage. It requires somewhat more bandwidth, though. But extra bandwidth costs are approximately proportional to block size. So maybe not a big problem...

E.g. I either need 128 GB of RAM, array of hard drives and 100 MBit/sec pipe. Or I need 1 GB of RAM, no hard drives at all and 1 GBit/sec pipe. Which is cheaper?

So what I'm talking about storage/bandwidth trade-off. Using less storage might increase latency, but possible in such a way that it won't be critical.

Next time I will teach you how to implement a blockchain-based cryptocurrency in such a way that new miners can start mining right away without downloading whole blockchain, stay tuned...

I'm just somewhat disappointed that people consider only the simplest solutions and do not want to see a bigger picture.