Topic: Crypto Compression Concept Worth Big Money - I Did It! - page 10. (Read 13895 times)

I find your idea described here:

https://bitcointalksearch.org/topic/m.3120662

very interesting from a purely mathematical, theoretical point of view.

Could a very large out of band reproducible random data set (like pi) be used in this way?

Can we uniquely encode a binary input stream as a very small set of metadata in this way?

Correct me if I am wrong but your theory can be summarized as follows:

The metadata (your very small file description) consists of an end point in the number pi and number of hops it took you to reach that end point into the number pi.

Your encoding is just finding the end point and counting the number of hops.

Your decoding is finding the one unique path that ends up at that specified endpoint using that specific number of hops.

First, I must say that I do not think the general purpose decoder for anything other than trivial examples is possible by current computational means.  As an exercise it might be fun to code up a simple encoder and decoder that could be used to try this out on trivial (up to a hundred bits or so) examples.  Who knows, something interesting might come out of the exercise of trying to find an efficient way to find the path (or paths see the next point below).

Second, you would need to prove that given an endpoint and a number of hops there is only one solution (path).  I have not put too much thought into this but this looks to be a very hard thing to prove.  However, the good/bad news is that the opposite assertion:  given an endpoint and a number of hops there is not always one unique path looks to be very easy to prove.  We just need one single example where we find more than one path to an endpoint that takes the same number of hops.  The simple program described above could be used to prove this.  All you have to do is pick random input data sets, calculate the encoding and then run the decoder to find all possible solutions.  If there is more than one solution in a few cases (my expectation) then we have proved the negative assertion.

I for one am very glad you thought this up and have spent so much time and effort on it and want to personally thank you for bringing it to my attention.  It is fun to dream up new things and share them.  You could have saved yourself a lot of grief and flames by just publishing your idea in your original post but I know from personal experience that is very hard to do when you think you have really got something new and valuable.

Can you compress compressed file? Can you RAR .rar file? :-]
Even 99% compression (1% reduction of file size) working for ALL files would be impossible as you could run the file through this compression multiple times.

 Let me put this out there , I have no technological or math skills , I was merely joking and would probably put whatever you send me int o a .rar file or search google for some methods and if i fail i fail

 So basically , we could already see where this is going , I wont win , but I will try just for the heck of it

 I do not understand your propositions , both of them . Am I allow to try to .rar whatever you send me? How much time do I have? Its like asking for a 12 year old to score a touchdown at an NFL game , there might be a sheer luck where a 12 year old may run 1 yard and score , very low chance but it worths the try for 1 bitcoin since there is nothing I could lose

 Btw that means : sure mail me some stuff and I will try to compress them , you can find my mail on my profile.

Sure, I'll be happy to do that.

Let's be clear about the definition of 'compress' though. Either of these conditions is fine with me:

1) I provide the 1MB file in advance. Then you provide a smaller ('compressed') file in return, plus some decompression program or script or anything, which can recreate the original 1MB file. The size of the compressed file and your program or script together, may not exceed 950K.

or

2) I provide the 1MB file in advance, in encrypted form. You provide two programs or scripts or whatever. The first is able to take an input file and reduce it to 950K or less. The second is able to take the compressed file and restore the original 1MB file. These two programs or scripts can be as large as you want. May also be just one program that does both things. Then I provide the password to get access to the 1MB file I supplied earlier.

These two approaches are to avoid nonsense solutions where the data is actually just moved into the decompression program. E.g. simply cutting off the last 50K of my file and storing that in the 'decompression' program.

In both cases, things must still work if I rename the program and/or compressed file. This is to avoid nonsense solutions where data is stored in the filename.

Let me know if you want some 1MB files to test

1 BTC for every success. No action (other than the effort you put into compressing them) required in return if you fail.

 Well , if I had 1 bitcoin to bet , I would try to get you on that bet but I dont. I would tought take you on , if I can compress one 1mb file you send me into 950kb you will pay me 1 bitcoin , if I fail I will do something of your choice that doesnt involve any sorts of money (since I cope with poverty after some bad investments :/)

Oh, he'll probably be able to compress some 1MB files to 950KB. But not the ones I'm supplying

See, compression is all about information density. Quite some everyday files of 1MB contain much less than 1MB of actual information. A text file, for example, or an image with lots of 'empty' spaces, will not have a very high information density. Meaning they use 1MB of disk space to store a lower amount of actual data.

But if an 1MB file actually holds (close to) 1MB of information, there's no way to represent that same information in significantly less disk space. Brilliant compression algorithm or crypto key magic or not. And to get the proper persective: the vast majority (like 99.9999%) of all 1MB files actually do hold pretty close to 1MB of information.

Either way, I have enough faith in these principles to set a 100 BTC bounty to be proven otherwise. Hope to hear from B(asic)Miner soon.

Your right, but finding the position where it is is the problem, and the position index will be likely even bigger (in Bytes) than the data you get (and dont start the loop - we will encode this way the positin as well  )

Entropy is the answer! :-]

True. I forgot about unproven normality. So lets rephrase this and say that it is instead in sequence of truly random numbers. (Well, actually you can only prove, that every finite-lenght sequence is encountered [i random number sequence] with probability which converges to 1.)

Yes, the B(asic)Miner's algorithm is there. That does not mean it does work :-]. All algorithms exists there even non-functiong ones.


But there is interesting question... How many different computer files are there? I mean... the number of possible files is easily calculated given length limit. But how many different files truly exist up today?
So imagine central database with one indexed copy of every file. (Every time any new file is created or existing file is somehow changed in the slightest, not yet encountered way, it is stored into central database under new index.) Such set of indexes would be manageable. However, the process of upgrading the database and downloading files based on index would be not.

Another interesting question: Is it possible to create "supercompression" as non-deterministic probabilistic algorithm? That means one compressed file can produce multiple outputs but the probability of correct one is >99.999997% for long enough processing.

 I thought this was possible? Aren't we able to rip games and all? I dont get it why cant he compress 1mb into 950kb ?

 This is not 'of course it can be done' message , it is more like 'I really dont know if we are able to do this or not , I tought we were able to do it but I might be wrong' type of message

B(asic)Miner, how about this: wanna bet? Walk your talk? Put your money where your mouth is? After all, money talks and bullshit walks

I give you a few 1 MB files (1048576 bytes). If you can compress them to 950 KB (972800 bytes) or less, I pay you 100 BTC. If you are unable to pull this off within a reasonable time limit, let's say one or two months from now, you pay 1 BTC to any charity of your choice (must post tx here )

Deal?

I happen to have done so recently:

Ah, see, conjectured. This is NOT AT ALL proven to be true whatsoever. To this very day, no mathematical proof exists that states whether or not Pi is normal.

(and even if it is, the idea is still flawed for obvious reasons already described above)

I'm not sure I got it right...

You store in your 4k file final position and path length... how can you say the path is unique?

Take this part of pi:
3.141592653589793238462643383279502884197169399375

These two sequences have same lenght and end on final 5:
00010
00001

This isn't even certain, it is not known whether Pi is a normal number or not.

So, even the algorithm that B(asic)Miner claims to have written, is located somewhere in Pi already. Therefore it must exist. Or, ehh... wait

Yes. Every known file in the universe is somewhere inside of Pi. (As a matter of fact, even every finite-length unknown file is located inside Pi. That means every wikipedia article that will be published, description of every invention that will be invented on Earth or elsewhere, every novel, short-story or book that was, is or will be written. Incuding all variants of ending, all misspellings, etc.)

But no matter how you traverse the Pi you always need a starting point, starting index, first decimal place to begin with. (Or maybe you need last index, last step of your path for recreating the path from finish to the start.) Obviously simplest way is to just proceed sequentially, start eg. with index 12345678 and proceed to 12345679, 12345680, 12345681, ... But even if you traverse or backtrack Pi some different algorithmically predefined way, you need that starting point.

Number of starting points can not be smaller than the number of possible compressible files.

This. You're compressing multiple different input files to the same 4K crypto key. When decompressing a 4K crypto key, how do you determine the result, since there are multiple (or actually: infinite!) possible outcomes.

You seem to be missing this critical point, B(asic)Miner. There are infinitely more files larger than 4K, than there are 4K crypto keys.

You don't seem to understand when I try to explain very clearly why your process will not work.
You've even seen a joke page created by someone else with broadly the same process, and another page explaining why it will not work.


a) 50,000 is 50KB, not 50 bytes. 50MB is 50*1024*1024, not 50*1000.

b) And what if you don't find the last bit of data until index location 501,500,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000...(repeat up until 50MB)?

You entire process only works if you assume you can find all possible index locations in less space that the original file took.
You cannot do this.


And what people keep telling you, but you refuse to accept, is that on average, the index required to store the final position will be as large as the file you are trying to store.

You do realize that this whole "Pi FileSystem" was a practical joke, right?

Yes, it works, in theory, for 0.000000001% (prolly much less) of all possible input files. But it's way, way, WAAAAY too slow for practical usage AND it doesn't work for the remaining 99.999999999% (prolly much more) input files.

And "too slow" in this context is not just a matter of requiring faster processors, it's "too slow" as in "takes longer than the age of the universe to process".

There are little more than 1.415461031044954789 x 1E9864 possible 4k files. These are ALL possible results of your compression. Although this number is extremely high, it is much lower than number of possible 1MB or 100MB or 1000MB files.
So how can you make a correspondence (1 to 1 connection) between a set of files and much larger set of files without any code repeating (correspondig to several different decompressed files) ??

Listen, if I sat down with a programmer who was able to listen to what I'm saying and not throw up objections that have no relevance to my method, and we could sit and work on this until it fits my theory exactly, then I know it would work.  You are right, I cannot do it myself, but I have been very clear about that from the very first post.  I have asked for help.  But it could be done soon.  Very very soon.  A few days for the encoding portion, it's just a file lines really.  The decoding portion would take a lot of brain work, research, testing, modification, etc ...  and we'd have to make our choices about how to resolve the solution of this software by seeing the results of those tests.  If I had the money to hire a coder myself, I would not have even come here at all.  Again, not asking for money now, but I am asking for someone to just TRY to do this with me.  How much time could it take to prove me wrong?  Maybe less than the time it's taking you to actually type out these responses to belittle me so you can go on believing everything is impossible.

Stop trying to push this off to ... someday, when now would be just as good.  Another person besides myself, his name is Philip Langdale, a programmer, has already created something like my idea here:  https://github.com/philipl/pifs

It is a method (like I've been telling all of you) to push data into Pi using a very hard mathematical solution.  It barely works because the encoding is just too slow.  But he has already done this!   You can download the app yourself and give it a try.  He is encrypting data into Pi.  But he's doing it the wrong way in my opinion.  But if you read his page, you will see that he has rational support behind that idea that every known file in the universe can fit inside of Pi.  Go read his arguments yourself, so you can stop beating me up for things you don't understand, despite being geniuses no doubt.  I want to work with you, I don't want to fight with you.  Thanks for your time.