This is special to our use case: bootstrap.dat is essentially an append-only file. Blocks are simply concatenated onto the end.
Today's torrent at height 193000 is 2,491,771,562 bytes in size.
The next torrent, a few months from now, will have the same first 2,491,771,562 bytes.
Thus, to bittorrent, the next torrent will simply appear to be a truncated / not fully downloaded bootstrap.dat. Bittorrent is built to fill in the missing pieces of a file, so that is what it does here
Does anyone know if bittorrent can share streams between multiple versions of the same file?
I mean, lets suppose we publish the torrent for the first x bytes, add y bytes to the file, then publish another torrent for the new version. Will people downloading the new, longer torrent, be able to request blocks from people running clients that have only downloaded the shorter torrent? There does exist a Bittorrent streaming protocol, TS Engine, but as far as I can tell it's purely block based and doesn't efficiently handle the case where every client needs the
whole stream, right from the beginning. I know internally bittorrent can identify blocks that is already has using a merkle tree system, but the tree can only have one tip. (1)
It's not a very important optimization for bitcoin, just publishing up to the latest checkpoint is fine for us even if old seeds aren't useful anymore, but I have an application where torrenting a file that is continuously being extended would be useful.
(1) Ironically the data I want to distribute via bittorrent in this fashion is a forest of merkle trees, exactly the sort of data structure that you could use to implement a continuously-appended-to torrent...