Start from the Ending Index Location. Begin hunting through Pi using all combinations of 1s and 0s in some systematic or mathematical way, looking for efficiency and intelligence in solving how to do it the most quick possible way, to find the one path through Pi that ends at our Ending Point exactly using the same exact number of steps, and calculating intelligently the number of 1s encoded in the formula using the Flip Count information (so we don't waste time searching for paths that have more 1s than are even in the timeline we've created. Since any fork in the road will shrink and compress a given file, the overall chance of that file having the same 64-bit Base Key, number of steps to reach the end goal, and same number of 1s, would be significantly reduced. Add to that, since we are not encoding the file all the way into Pi for one single MetaData Key, but splitting the files up into more and more MetaData Keys, the chances of the file's uniqueness will keep going up with more divisions, while the number of Metadata keys added to our final file output would create a larger and larger file.
It my goal to be able to make this work with 500k files up to 5 TB files using this system if it can be made to work.
Like I said before. This changes nothing. there are several factors that condemn this to fail. Even if you could solve the uniqueness problem, which you can't, there are still several factors. your "chunks" are good if you want to shorten the jumptime; you need a chunk everytime you reach a certain value, say 33 for the heck of it. so pretty much every 33th bit you need to write the corresponding chunk in your file. the higher the value the longer you have to search through pi.
So, now here is the problem: you want to continously iterate through pi, which is fine by itself - BUT you have to store all that in a memory. you could use certain formulas which enable you to jump to a specific place in pi to shorten the amount of time you need and memory used, but we are talking about a supercomputer here to effectively iterate trough pi, get the corresponding values, write them in a file, remember where you where/use a formula to get there again, do that over and over again.
Don't misunderstand me - You had a really nice idea. But you are not the first to try that. The main factor of a Decoder/Encoder is the speed. You don't have any speed.
Well, if you mean the time needed to load 2GB of the Pi Index into memory, I assume that would take 1 minute or so to, upon starting the software, generate the Pi Index (of that size) needed. Who cares if it's in memory? I don't. Besides, who knows if I will end up using 2GB, it could end up being 500MB of Pi, with smaller chunks. First we need to just get a basic 2 meg version running, and try to encode files between 500K and 1 MB and just see what happens.
But if I get this working, imagine the time saved over the internet to send your friends or family the 20GB of videos taken on your vacation. You would be sending a small file 4-64k large, in their email in moments. Then they'd decompress out the videos overnight, while sleeping perhaps. Wake up, the videos are there. The internet did not need to be congested with all of those 0s and 1s. And if a lot of people were using it, the internet would work more and more smoothly all the time. Think of the difference!
Another thing is, you still can't convince me that just because it's possible to have 2^N files TO encode, that there are that many unique files. For all we know, our research into this could reveal a fundamental characteristic of Nature to only allow organized patterns to assemble in random data at a given ratio, like the Golden Means (8/7 is it)? Just trying to solve this problem itself could lead to a breakthrough. What if there are only (2^N)/10 file in existence of each size, and they already happen to be written in Nature? It would mean all of our ideas already exist in the Global Intelligence of Nature and that our brains are receiving the information via DNA alterations that come from eating plants. Because science just recently confirmed that by eating some plants, our DNA is altered, and they hypothesize that new ideas come from this phenomenon. If Nature is the one providing original ideas, it stands to reason the answer is already in Nature for every created thing.
I don't wish to go too deepy into philosophy here, though. Just saying, you don't know for sure that even though there are 2^N files possible, that all of those combinations will ever be used to organize intelligent data into a file that ends up on someone's computer that could be put through my system. In that case, there might be all the uniqueness I'll ever need. Won't know until we try it and watch it either (Get busted) or work as conceived.