Thanks for that analysis DeathAndTaxes! Very informative.
Based on a convo I just had about this topic over on #bitcoin-dev, I'm realizing that I have some knowledge gaps that I need to fill to better understand some of the alternative proposals, but I think others here might find this interesting so I'm sharing it here (with sipa's permission):
1:31:31 PM sipa: sugarpuff: now, about dealing with ways to improve performance, IF you are giving up zero trust
1:31:35 PM sipa: sugarpuff: see it this ways
1:31:47 PM sipa: sugarpuff: internally, every full node maintains a database of all unspent transaction outputs
1:32:13 PM sipa: sugarpuff: this is a high-efficiency, compact database with just outputs (not signatures for example) and not those that have already been spent
1:32:31 PM sipa: sugarpuff: every block you can see as sort of an authenticated patch against that database
1:32:45 PM sipa: (add outputs of trenasactions in the block, remove the outputs spent by transactions in it)
1:33:08 PM sipa: if we can somehow agree on a snapshot of that database at some point in time, and you trust it, we can just copy the database
1:33:13 PM sipa: instead of copying all blocks before it
1:34:01 PM sugarpuff: sipa: right, and that sounds like going into checkpoint territory. the rolling-root proposal is similar but not entirely the same
1:34:27 PM sugarpuff: but old blocks should have almost unanimous agreement in the network
1:34:36 PM sipa: sugarpuff: not done yet
1:35:07 PM sipa: sugarpuff: there are proposals to make such a snapshot of that database essentially for every block, and put a hash of that snapshot in the block itself
1:35:14 PM sipa: sugarpuff: and have this validated by miners too
1:35:43 PM sugarpuff: sipa: ah, yeah, that's a good idea. is that what UXTO is about?
1:35:56 PM sipa: the UTXO set is just what we call that database
1:36:10 PM sugarpuff: ah, lol, i just got the acronym
1:36:11 PM sipa: (Unspent TransaXtion Outputs)
1:36:14 PM sugarpuff:
1:36:29 PM sipa: now, this has a very significant impact on the network
1:36:41 PM sipa: as validating and maintaining these database snapshots is not cheap
1:37:16 PM sipa: (there are ways to make it pretty efficient, by using a merkle tree for that entire database, but the impact is not neglectable)
1:37:48 PM sipa: now, you can have a node that just asks a peer, "give me your current UTXO set", and it can validate that at least the majority of hashing power agrees with it
1:38:02 PM sipa: without needing to downloading any old blocks
1:38:36 PM sugarpuff: sipa: just by asking enough nodes for it and assuming the majority is right?
1:38:56 PM sipa: sugarpuff: no no, not the majority of your peers
1:39:06 PM sipa: sugarpuff: you can actually just validate the entire old blockchain, but only the headers
1:39:14 PM sipa: sugarpuff: which is very cheap (80 bytes per block)
1:39:18 PM sipa: sugarpuff: like SPV nodes do now
1:39:31 PM sipa: sugarpuff: but the headers would be committing to the hash of these UTXO set snapshots
1:39:42 PM sipa: so you can verify that the chain agrees with the snapshot
1:39:50 PM sipa: which implies the majority of miners do
1:40:31 PM sugarpuff: sipa: i'd like to do more reading on this and not use up more of your time, do you have some recommended links about this topic? Think SPV + UTXO thread is a good start?
1:40:54 PM sipa: several mailing list posts, forum posts, ...
1:41:00 PM sugarpuff: too broad...
1:41:05 PM sugarpuff: i could spend a year doing that
1:41:18 PM sugarpuff: i love the bitcoin paper, i've read it like 3.5 times now
1:41:18 PM sipa: there's no nice centralized knowledge repository of all these experimental ideas
1:41:28 PM sugarpuff: ah that's unfortunate
1:42:15 PM sugarpuff: sipa: ok, well thanks, i think what you said here is enough to get me started. i'll go do some more research on it. right now there are too many gaps in my knowledge to give you useful replies on this
1:42:23 PM sipa:
1:42:23 PM sipa: yw