Topic: Crypto Compression Concept Worth Big Money - I Did It! - page 9. (Read 13895 times)

I am not trying to get the system, as described, to "work" but out of mathematical curiosity I wonder if there is a starting search digit that would create unique ending positions for all 16 of the 4 bit combinations.  We would have to try starting at 0, then 1, then 2, ... up to 9.

If there is one of more starting locations that do in fact lead to unique ending locations for all 16 of the four bit combinations then we could see if any of those could be extended to 8 bits.

If there is even one starting digit that creates a unique map of the 4 bit values or the 8 bit values then it could be used as an esoteric code mapping.

This could also be tried on all 16 starting locations of a hexidecimal representation of pi.

Pretty simple program.  If I get time I might try it for fun.

The biggest question is not whether a computer can do the "compression" but whether it is possible to reconstruct the file.

Here are the chains of digit positions for just 4 bits:

Pi:    314159265358979323846264338327950288419716939937510
0000     4                4   4            4
0001     4                4   4       5
0010     4                4           5                5
0011     4                4           5         6
0100     4 5   5 5
0101     4 5   5           6
0110     4 5  6            6
0111     4 5  6     7
1000       5   5 5                    5
1001       5   5 5         6
1010       5   5           6 6
1011       5   5           6        7
1100       5  6            6 6
1101       5  6            6        7
1110       5  6     7               7
1111       5  6     7    8

As you can see, the bit combinations 0101, 0110 and 1001 all lead to the same ending digit position. The same is true for 0001/1000, 1010/1100 and 1011/1101/1110. This means that the algorithm's output of a file starting with one of the bit patterns in such a group is indistinguishable from the output if the file had started with one of the other bit patterns in the group.

It follows that it is impossible to uniquely decompress the output.

Don't take it personally, but your scheme is not usable.

Onkel Paul

Now the next question is if this is possible, can a consumer computer do a compression within few seconds if not minutes for average file sizes?

 I am still waiting for 1mb file mail Send it to me , I will try to compress it , what can I lose but time

Not true at all.  If the idea worked, and it does not, but if it did the idea would produce just three small numbers, something like this:


This is far less than the input of 50 megabytes.

B(asic)Miner questions and specific answers for you:

Where did you get this 100-150 number?  

We would need to see all the rules in order to help you figure out if this idea is theoretically sound.

As pointed out previously a 50 megabyte file contains 50*1024*1024 = 52,428,800 bytes.  So using your number of 100 digits /byte you would need at least 5,242,880,000 digits of pi.

Please stop using this term.  It has a meaning and this is not it.  Use "here is my metadata"

Is this the full description?

I don't think this is true.

Even if the path was unique, which I don't think it is, this process sounds very very hard, possibly impossibly hard.

A byte is always 8 bits by definition.  How that byte and those 8 bits are used to encode data varies.  (Simple) ASCII maps all the various printed symbols into these 8 bits.  8 bits per character.

Don't worry too much about these details.  Just describe the idea itself as well as you can and let the geeks and engineers worry about the details - like whether it is possible or not.

Makes no difference. There is no way to 'cheat' the problem at hand. On average, the index will be at least as large as the actual data itself, and very likely (in 99.999999% cases) even larger.

Besides some exceptional (very rare) 'lucky' numbers, the vast majority will require more data to store the index.

Consider this: even if it would be possible to compress (or 'encode' or 'represent' or 'index') just a tiny bit off any file, you could repeat the same process over and over, and eventually reducing every file to a few or even one bit. I think you'll intuitively feel that's not gonna work.

Pi is even a transcendental number (as opposed to e.g. √2 which is also irrational, but algebraic).

I changed my program to use a precalculated file of Pi to 1000000 digits, rather than compute the digits on the fly.

When I apply it to the same 1000000 digits file, it runs out of Pi digits at only 12500 bytes of the input file (out of 1000000 bytes total).

'Compression' stats at that point:
After byte 12505, pi position is 999428
After byte 12506, pi position is 999525
After byte 12507, pi position is 999561
After byte 12508, pi position is 999628
After byte 12509, pi position is 999729
After byte 12510, pi position is 999795
After byte 12511, pi position is 999856
After byte 12512, pi position is 999950

OOPS, reading back through the previous posts I see this has already been shown here:

https://bitcointalksearch.org/topic/m.3120822

Oh well, might as well post my work here anyway....

-----------------------------------------------------------------

Thanks for writing the program!  Cool, I was just about to do the same.

Here is a quick example showing that the paths are not unique for the idea as described.  We may not have the full description and there may be more to it than we have been told but using the start at 4, jump to next 4 as long as the input is zero and increment when the input is one idea:

The following two sequences must have different paths:

00001000
00010000

Assume what if we are in an area of pi with the following sequence of digits:

   4 ... 4 ... 4 ... 5 ... 4 ... 5 ... 5 ... 5 ... 5  (where ... means digits that do not matter in this example)

If my understanding of the idea is correct both inputs land at the same end location.

00001000 would map to 4 4 4 4 5 5 5 5
00010000 would map to 4 4 4 5 5 5 5 5

but with the totally possible distribution given we end up at the same ending location.

Is our understanding of your idea complete and correct or is there more to it than described so far?

Ok, to go the extra mile, I have written a toy program which does what I think you are describing:
- Set the first Pi search digit to 4
- Start at the beginning of the digits of Pi, and the beginning of an input file
- Repeatedly:
  - Take the next byte of the input file and convert it into binary
  - For each binary bit:
    - if it is is 1 increment the Pi search digit, wrapping round to 0 as necessary
    - find the next digit of Pi which matches the search digit
- Print the resulting index digit of Pi where the last binary bit of the last byte of the input file was found.

The program is extremely slow, so I'm not going to run it on a huge file.
For the first just over 1.5KB of its own source file, the program produces:

After byte 1610, pi position is 130339
After byte 1611, pi position is 130407
After byte 1612, pi position is 130468
After byte 1613, pi position is 130531
After byte 1614, pi position is 130666
After byte 1615, pi position is 130747
After byte 1616, pi position is 130857
After byte 1617, pi position is 130915
After byte 1618, pi position is 130995

I hope you can see that your 50MB file is never going to have a final index position of 500,000, if I have already reached 131,000 with only 1.5KB.

[Note that you haven't demonstrated any decompression routine, and this 'compression' routine is not proved to be reversible, so that two input files may give the same index value, which means that decompression is not possible.]

That is why I called it a "very large out of band reproducible random data set (like pi)"

That is why it is an encoding/decoding system, not a compression system.

Very large amounts of information can be encoded in very small metadata:  ISBN numbers, URLs, etc.

He is not talking about data compression at all.  He just does/did not know the proper terminology is all.

Although I highly appreciate creativity and thinking out of the box, this whole idea is fundamentally flawed, as could have been proven in advance, regardless of the exact implementation details.

Not because of 'stubborn thinking' or refusing to look beyond known methods, but simply because it is mathematically impossible to create a compression (or 'encoding' or whatever you wanna call it) algorithm that reduces any file to less than its own size. No matter if you use Pi, or black magic, or voodoo, or whatever.

Let's repeat the same argument one more time: there are N possible files of 1MB. There are M possible 4K crypto keys. N is much, much, MUCH larger than M. So, by definition, the vast majority of 1MB files (that is, way over 99.9999999999999999999999%) can NOT be represented by a 4K crypto key. Period.

Even if you would create a HUGE (de)compression program that contains a vast library of ALL possible 1MB files, and when compressing a file it just stores some index to specify the exact 1MB file you need, it's still impossible. There's much more possible input files, than the number of unique index keys (or 'compressed files') you can output.

What riddles me, is why people who are obviously not stupid, keep believing in fairy tales and ignoring plain simple logic.

Everyone:

I also know how much fun it can be to bait, flame and troll some of the losers and scammers who post here.  I think this guy is simply trying to find out if his idea will work.  I know I could be wrong and he may just be having fun or trying to pull a scam - it is so hard to tell sometimes but so far he looks genuine to me.

Carry on...

Oh God, please just give OP a bitch-slap!