Stupid question - Why don't we just compress the blocks? - page 2.

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: AlexGR on January 16, 2016, 10:33:05 AM

This is not about video per se. It's about an algorithm that a neural network discovered, which could compress a lot of data with very high percentage ratio.

The fact that they publish nothing about this supposed algorithm suggests that it is in fact a hoax rather than some revolutionary new thing.

It's strange that people will just accept "we can't publish stuff because of X" when in fact they could publish the specific algorithm used (for the supposed video mentioned) without giving away how that algorithm was created (as supposedly the algorithm was simply one of an infinite number that this amazing AI could create).

AlexGR

legendary

Activity: 1708

Merit: 1049

Quote from: AliceWonderMiscreations on January 16, 2016, 10:27:56 AM

Quote from: AlexGR on January 16, 2016, 10:19:59 AM

Quote from: Lauda on January 16, 2016, 10:13:05 AM

This is a nice example of why people with IT degree need to decide on the technicalities (not trying to be offensive). Compressing random data usually results in 0% saved space or the compressed file ends up actually being bigger than the original.

We just need a revolutionary new compression method. Something like that:

http://www.theserverside.com/feature/Has-a-New-York-startup-achieved-a-99-compression-rate

From presentations I've seen, this is not only for video. It was thought that it would be marketed best for video because video takes up most internet bandwidth nowadays.

Perhaps people with neural networks will start competing on finding increasingly more efficient compression, compared to what we have now.

Video has a lot of predictable redundant data. Random data is, well, random.

And video compression is typically lossy. Lossy would never work for the blockchain.

This is not about video per se. It's about an algorithm that a neural network discovered, which could compress a lot of data with very high percentage ratio. Video is just one of the deployment markets because it takes up >50% of internet bandwidth, so, naturally, they went after it. But, from what I saw in one of the presentations, it's more like data agnostic.

Plus, in that example, the original file is an MP4, which has already reduced redundant data (and has loss of quality).

AliceWonderMiscreations

full member

Activity: 182

Merit: 107

Quote from: AlexGR on January 16, 2016, 10:19:59 AM

Quote from: Lauda on January 16, 2016, 10:13:05 AM

This is a nice example of why people with IT degree need to decide on the technicalities (not trying to be offensive). Compressing random data usually results in 0% saved space or the compressed file ends up actually being bigger than the original.

We just need a revolutionary new compression method. Something like that:

http://www.theserverside.com/feature/Has-a-New-York-startup-achieved-a-99-compression-rate

From presentations I've seen, this is not only for video. It was thought that it would be marketed best for video because video takes up most internet bandwidth nowadays.

Perhaps people with neural networks will start competing on finding increasingly more efficient compression, compared to what we have now.

Video has a lot of predictable redundant data. Random data is, well, random.

And video compression is typically lossy. Lossy would never work for the blockchain.

AlexGR

legendary

Activity: 1708

Merit: 1049

Quote from: Lauda on January 16, 2016, 10:13:05 AM

This is a nice example of why people with IT degree need to decide on the technicalities (not trying to be offensive). Compressing random data usually results in 0% saved space or the compressed file ends up actually being bigger than the original.

We just need a revolutionary new compression method. Something like that:

http://www.theserverside.com/feature/Has-a-New-York-startup-achieved-a-99-compression-rate

From presentations I've seen, this is not only for video. It was thought that it would be marketed best for video because video takes up most internet bandwidth nowadays.

Perhaps people with neural networks will start competing on finding increasingly more efficient compression, compared to what we have now.

AliceWonderMiscreations

full member

Activity: 182

Merit: 107

Quote from: smith coins on January 16, 2016, 10:04:29 AM

Quote from: SamusNi on January 16, 2016, 09:59:15 AM

Quote from: CIYAM on January 16, 2016, 09:49:58 AM

Quote from: SamusNi on January 16, 2016, 09:47:23 AM

instead of all of this discussions to increase the block size, why don't we just compress the blocks, leaving the size as it is?

Blocks consist of transactions that for the most part are effectively random numbers (such as hashes, public keys and signatures) so they simply won't compress much at all (as you can't in any sensibly usable way compress random information).

The efforts that are going on behind the scenes will make a much bigger difference than any tiny percent you could compress the content of a block.

Are you sure about that? Wouldn't something like gzip applied to the blocks reduce their size by like 99%?

Which efforts are going on behind the scenes exactly?

I don't know how the network will work after compressing but 7zip is more efficient than gzip

7zip is rarely found on *nix systems. Available, sure, but rarely found. bzip2 or xz is more common better alternatives.

moot anyway though since compression isn't the issue.

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: Lauda on January 16, 2016, 10:13:05 AM

Compressing random data usually results in 0% saved space or the compressed file ends up actually being bigger than the original.

It's actually a very simple (and I think probably standard) way to check whether an encryption algorithm is badly flawed (i.e. if the encrypted data can be shrunk by any standard algo like gzip then it obviously hasn't been encrypted properly).

Lauda

legendary

Activity: 2674

Merit: 2965

Terminated.

This is a nice example of why people with IT degree need to decide on the technicalities (not trying to be offensive). Compressing random data usually results in 0% saved space or the compressed file ends up actually being bigger than the original.

SamusNi

newbie

Activity: 32

Merit: 0

Quote from: CIYAM on January 16, 2016, 10:00:56 AM

Quote from: SamusNi on January 16, 2016, 09:59:15 AM

Are you sure about that? Wouldn't something like gzip applied to the blocks reduce their size by like 99%?

Yes - gzip won't do much at all - why don't you try it and see for yourself?

Quote from: SamusNi on January 16, 2016, 09:59:15 AM

Which efforts are going on behind the scenes exactly?

Things like SegWit (you could search to find out if you're interested although I suspect you're just trying to age your account in order to qualify for an ad sig).

Yes I was doing some checks, only about 25% gain with compressions like gzip, 7zip... so doesn't help much.

Thanks for the info.

Looking forward for that SegWit.

Any plans for a release date on that?

smith coins

hero member

Activity: 504

Merit: 500

Quote from: SamusNi on January 16, 2016, 09:59:15 AM

Quote from: CIYAM on January 16, 2016, 09:49:58 AM

Quote from: SamusNi on January 16, 2016, 09:47:23 AM

instead of all of this discussions to increase the block size, why don't we just compress the blocks, leaving the size as it is?

Blocks consist of transactions that for the most part are effectively random numbers (such as hashes, public keys and signatures) so they simply won't compress much at all (as you can't in any sensibly usable way compress random information).

The efforts that are going on behind the scenes will make a much bigger difference than any tiny percent you could compress the content of a block.

Are you sure about that? Wouldn't something like gzip applied to the blocks reduce their size by like 99%?

Which efforts are going on behind the scenes exactly?

I don't know how the network will work after compressing but 7zip is more efficient than gzip

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: SamusNi on January 16, 2016, 09:59:15 AM

Are you sure about that? Wouldn't something like gzip applied to the blocks reduce their size by like 99%?

Yes - gzip won't do much at all - why don't you try it and see for yourself?

Quote from: SamusNi on January 16, 2016, 09:59:15 AM

Which efforts are going on behind the scenes exactly?

Things like SegWit (you could search to find out if you're interested although I suspect you're just trying to age your account in order to qualify for an ad sig).

SamusNi

newbie

Activity: 32

Merit: 0

Quote from: CIYAM on January 16, 2016, 09:49:58 AM

Quote from: SamusNi on January 16, 2016, 09:47:23 AM

instead of all of this discussions to increase the block size, why don't we just compress the blocks, leaving the size as it is?

Blocks consist of transactions that for the most part are effectively random numbers (such as hashes, public keys and signatures) so they simply won't compress much at all (as you can't in any sensibly usable way compress random information).

The efforts that are going on behind the scenes will make a much bigger difference than any tiny percent you could compress the content of a block.

Are you sure about that? Wouldn't something like gzip applied to the blocks reduce their size by like 99%?

Which efforts are going on behind the scenes exactly?

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: SamusNi on January 16, 2016, 09:47:23 AM

instead of all of this discussions to increase the block size, why don't we just compress the blocks, leaving the size as it is?

Blocks consist of transactions that for the most part are effectively random numbers (such as hashes, public keys and signatures) so they simply won't compress much at all (as you can't in any sensibly usable way compress random information).

The efforts that are going on behind the scenes will make a much bigger difference than any tiny percent you could compress the content of a block.

SamusNi

newbie

Activity: 32

Merit: 0

instead of all of this discussions to increase the block size, why don't we just compress the blocks, leaving the size as it is?

Topic: Stupid question - Why don't we just compress the blocks? - page 2. (Read 1187 times)