Topic: Crypto Compression Concept Worth Big Money - I Did It! - page 5. (Read 13895 times)

Hey wait, I've got an idea. Let's compress those 0.02 bits again! Genius!

Eventually we might be able to compress it to zero. 

^^  pretty darn clear

If your idea would work, then how about the following approach:

We compress 1280 different files of 5MB each, into 4KB crypto keys. We concatenate all 1280 4KB crypto keys together into one file, which then becomes 5MB in total. We compress that file as well, in 4KB.

So effectively, we have now compressed each 5MB file into 4KB/1280 = about 3.2 bytes. Really?

And we could actually do the same thing again, and compress 1280 sets of 1280 5MB files, compress each set into 1280 4KB crypto keys, concatenate the set's crypto keys and compress that to a second crypto key (like above), then concatenate all the 2nd crypto keys and compress the resulting 5MB into a third, final 4KB crypto key. We have now compressed each 5MB file into 0.02 bits.

Really??

Dude, you remain ignorant of simple logic, and keep on wandering in meaningless details and experiments. I like your creativity, but there is a much more fundamental problem here, that has nothing to do with Pi, hunt values, flipping bits, decimal encoding paths, or ANY other approach you may think of.

The proof HAS been shown here (numerous times) that it is simply NOT possible, but you choose to ignore that.


Your idea is essentially the same as this: (just replace "2-bit" with "5MB" and "1-bit" with "4KB")


See, you're completely ignoring the actual argument, and keep fiddling with details of an interesting yet inherently impossible algorithm.

I think it's interesting conversation over a bowl, but not feasible. You're almost guaranteed to end up with a larger file size than you started with by using the methods you suggest.

Oh, I love DemoScene!  I was an an Amiga fan in late 80's and early 90's until long after Windows DOS came on the scene, which was totally bankrupt of efficiency compared to the Amiga.  Long live Amiga, forever.  (Okay, fanboy rant over).

I have all the Demoscene demos and have also made a collection of all their music, too, it's some of my favorite music in the world.  


ANSWER:
Yes, to get on with your input, I was trying to think of a way to store a mathemetical version of my compressed key as a jpeg, where each pixel would be black (on white background) to show the movement thru Pi in a stock-exchange sort of drawn squiggly line representing the path taken from all possible paths.  It would draw from left to right when encoding, going from bottom and rising toward the top right, with every 0 going down and every 1 going up and groups of 0s (but not 1s) going level (left toward right).  Then the image would be analyzed and decoded back out using the black bits to help the program to accurately calculate all the ups and down of the timeline, which would be unique.  Because even if the overall file ended up at the same Index, the internal bits would rise and fall differently, creating a key effect from the actual shape of the path taken.  Then you could do optical image recognition on the image, counting pixels, overlaying the computer's approximation until it finally matches 100%, and then you'd know the actual path taken bit for bit, not just because of the end place thru Pi.  It would become a signature based on unique shape (which WOULD BE UNIQUE).  

But I wonder how much data it would take to create such a drawing even for the smallest of files?  The image would become enormous and would have to be saved in a lossless format otherwise if even one bit got changed, it would be a disaster.  For a 50 MB file, it might take a 50 MB image, defeating the whole purpose.  (Although the image would be only in black and white, not color, so that might help some.)  

Just a thought though.  What do the rest of you think of that?

This is indeed impossible, and is simply proven. It bothered me for months about a decade ago when I was working on a brainstorm team to do something very similar, where a small amount of mathematical data could be processed into a much larger file. Herein lies the problem:

In any given amount of bytespace, represented by x as the number of total bytes, you have 256^x different possible combinations.

So, for instance, in a 20 KB compressed file, you have 256^20,000 total possibilities. Assuming that is 98% compression, the original file would be 1 MB. Problem is that a 1 MB file would have 256^1,000,000 different possible outcomes.

This kind of compression does not exist because you cannot use mathematics to create an all-inclusive compression method. That is why most compressions are very specific to the type of file you are working with.

Generative Art
"Generative Art" is often used to refer to computer generated artwork that is algorithmically determined.

Some examples of generative art (5 examples in 451 KB)
You can see how much data is algorithimically compressed in these demos.

More resources
Scene 
Assembly
Demojs

Maybe you're right, but I want to wait until Murraypaul redoes his long list of Indexes again showing that they still end at the same place every time.  Maybe the list will shrink considerably to almost nothing, in which case, I only then need to find a few more criteria to increase uniqueness again.  It may still be possible, but I won't know until the proofs are shown here.  Thanks BurtW.

That does not fix the problem.

Okay, thanks for your assessment of my situation, all of you.

I just want to know a few more things from you all.  

Let's say I made a modification to my ruleset (actually, it's not a modification, its the ruleset I started with, but felt it would add too much complexity, but perhaps that's what is needed here anyway) so I am going to give the ruleset here again from my original template.  Please someone, the ones who did the proofs before, do a few proofs again to show me my error, or if it's viable now:

Here goes everything:


1) Open file to processed.  Analyze the data.  Then we open our file and begin to record the following:
   A) Original Filename.  Size of the Original File.  Size of Pi Index Used (how big each chunk is to be split into).  Size of the last Mega Chunk in bits (if applicable).
   B) Basekey (the first 64 bytes of the original file, giving the program enough room to establish a unique path given a number of hops.
  
2) Begin reading the data one character at a time (converting hex to Ascii Binary) all in memory using the loaded Pi Index to the Pi Index Size shown in example A above.  Convert everything (all file contents) to Ascii Binary (so every incoming piece of data is exactly 8 bits).  Begin moving forward through Pi by hopping on a single solitary digit called a "Hunt Value" meaning the number we are to be hopping on in Pi.  Starting from the decimal point, begin encoding by hopping.  Start at the 1st 4 in Pi if our first bit is a 0.  If it's a 1, start at the first 5 in Pi.  Hop along Pi, encoding 0s and 1s using these rules.

>>Here comes the rule change!

0 = no change in Hunt Value. 1s = +1 to Hunt Value.   However, how we search for our Hunt Value changes here:   We have to double confirm every Hunt Value against the Index at that location being even (0) or odd (+1).  We can only accept a Hunt Value which double confirms the bit we are encoding.  For example:   Here is  our data to be encoded:

0010 0001  (space added for readability only)

Our first Hunt Value is 4, but we can't stop on just any 4 now, it must be an EVEN 4!  Here goes:   The first 4 in Pi is at index 2 (if you are counting indexes 1-50 and not 0-49 like coders do, so let's please just use 1-50 to keep this easy to read (no offsetting for now))  That means the first 4 IS a 4 AND it's EVEN.  

Let's continue:  the 2nd 0 in our encoding byte example ....  the next four we find in Pi is at Index 19 (odd) but we are looking for an EVEN 4 because our encoding bit is still 0.   Our third 4 is at Index 23 (no good still) so we keep going ....  ah, at Index 36, we have our 2nd EVEN 4!   So now our 2nd bit is encoded ....   Next we are encoding (our third encoding bit) a 1, so now we need to find the next ODD 5 (it increments+1 and because it's a 1, its ODD too) ...  so our first 5 from that location is at Index 48 (an even, so its no good) ... but at Index 51, we have our first ODD 5, we just encoded that bit.  

This process continues accordingly.  This spreads out the hops a great distance, ensuring the path taken is more unique, but at a cost of having to use way more of the Index for every byte "stored" ...

>SNIP


DECODING CHANGES:
We now have an additional checksum for helping to narrow down the possible paths that could have been taken.  

Please someone (murraypaul that means YOU-wink) run the proofs you ran before using this new method and see if they are not all unique. I beg you.  I am very curious to see what comes from this.

My statement above was more general: "this can happen somewhere therefore you will not get unique results".

However, here is exactly the same issue happening right there in the first few digits of pi.

Maybe by carefully studying this specific example you will discover this is the general problem with the entire idea:


This is a very specific example of exactly what I described as a general problem.

This. It seems like you're simply ignoring logical facts. Claiming that people don't understand your idea. Well, we do. Really.

B(asic)Miner, do you even consider at all that it might be possible that your idea is just flawed? As in, plain wrong? Cause if not, no reasoning whatsoever will cure this delusion and we might as well end this discussion, and leave you to your attempts without further hindrance (good luck in that case).

Thanks for your hard work, and this information is invaluable to me.

However, you are still trying to assume I'm going to try and do the entire file in one attempt through Pi, when I already said I was going to limit the size to 500 MB.  But now I see that 100 MB requiring 8.4 billion digits of Pi is far too much, so I must lower my expectations for each chunk.  I will now say each chunk should be 25 MB, which is 1.1 Billion digits of Pi.

We would end up with something like this:

[OPENFILE]
[FileSplitSizeInKB=25600_&_LastChunkSizeInKB=9500]
[Filesize=409600_&_Name="400MBofStuffForWhomever.zip"]
[MChunks=8_&_REM:  Begin Chunk(s) on Next Line! ]
[1,w, xxxxxx, y, zzzz]
[2,w, xxxxxx, y, zzzz]
[3,w, xxxxxx, y, zzzz]
[4,w, xxxxxx, y, zzzz]
[5,w, xxxxxx, y, zzzz]
[6,w, xxxxxx, y, zzzz]
[7,w, xxxxxx, y, zzzz]
[8,w, xxxxxx, y, zzzz]
[CLOSEFILE]

This chunking method would have an additional obvious benefit of making the MetaData more unique.  But I still have to find a solution as to the problem of having 6 bits being 1s and 10 bits being 0s inside of 16 bit pairs and what that means toward my theory first.  

I do also need to work out a way of creating more uniqueness too, I agree.

We have proven it can't work, repeatedly.
You just don't believe us.

I downloaded 1 billion digits of Pi from here: http://stuff.mit.edu/afs/sipb/contrib/pi/
I downloaded the complete works of Gilbert and Sullivan (text format) from here: http://www.gutenberg.org/files/808/

The ~1.3MB file needs just over 100 million digits of Pi to encode.

After byte 1295987, pi position is 103683939
After byte 1295988, pi position is 103684011
After byte 1295989, pi position is 103684099
After byte 1295990, pi position is 103684138

That is almost exactly 80 digits of Pi required for each 1 byte of input files.
(Which is exactly what you would expect.)

So a 100MB file would require 8.4 billion digits of Pi.
A 50GB file would require 4.2 trillion digits of Pi.

The current record for calculating digits of Pi is 10 trillion, and that took 371 days to do, so I think we agree real-time calculation is out of the question?
4.2 trillion digits of Pi require 4.2 trillion bytes to store uncompressed. That is a 4 Terabyte drive just to store Pi to be able to perform your 'compression'.
(Which doesn't work anyway, because it is not reversible.)

Please read about Information Theory and Coding Theory.

The ability to compress any given set of data is based on the entropy of the data. Random noise is essentially uncompressible, regardless of what type of compression you try to apply.

I think it's good for a person to have a problem like this in the background of your brain; if for no other reason than it can be energizing and help you to become a better problem solver.

I myself am stuck on a stabilization system that is probably impossible, but it's a great brain exercise, and not letting go of it helps me to solve problems in the physical world often better than before I got stuck on the idea.

I had previously not heard of Jan Sloot and being that I'm a sucker for a good conspiracy theory and this makes the conversation even more fun for me:)

BTW thanks for supplying those links Moto:

A single 5 byte file: "Hello", results is many many more than two million collisions.
There will only ever be more collisions the longer the file is.
A 100MB movie file would result in more collisions than could be saved to a hard drive.

You are going to ignore this, by saying that your theory magically only works on large files not small ones, which conviently means that we can't run a test program to demonstrate you are wrong.
But you are wrong.
If two 5 byte combinations give the same hash, then any pair of files which are exactly the same, except that they each start with a different one of those 5 byte combinations, will also give the same hash.
Making the file bigger just stops us demonstrating that you are wrong, it doesn't make you right.