Topic: Why is it hard to track backwards from public address to private key? (Read 4321 times)

Great post, you really get right to the heart of it.

In real numbers, you can feed an error term back into the algorithm to improve a guess.  In discrete math, and even worse, modular discrete math, you don't get any error term, so you can't get any feedback.  Well, you can get a little, here and there.  Better algorithms attempt to extract the tiny bits of feedback, which in some cases makes them much better than simple brute force, but that is just a reduction in badness, it doesn't get you down into polynomial territory.

The catch is that we really don't know why we can't get feedback. *  And we can't prove that feedback is impossible either.  For all we know, there is some trick just waiting to be discovered that will reduce a big lump of currently impossible problems to triviality.

* The Wikipedia example is (3^k)%17=13.  If you try solving it, you can see that classical techniques for using the error term to improve successive guesses don't help you much, but why we haven't been able to find any techniques that do work is, again, one of the biggest open questions in mathematics today.

To amplify, in the type of computation we are considering everything is discrete; cf the Church-Turing Thesis (or just consider what types of programs the computer you are using this very moment will actually run). Real numbers per se are ineffable!   Not just in cryptography.

Another issue, even without real numbers, is that manipulating very very big integers takes longer than manipulating small integers. Seems obvious, but if you only count the number of arithmetic steps and ignore the size of the integers (or precision of "real" numbers) then factoring becomes easy (polynomial number of steps).

The key distinction between high-school math and the math used in cryptography is that cryptography is discrete. That is, in high school math your answer could be 8.56, 8.57, 8.565, 8.564, etc, with infinite granularity between any two values. You also have continuity, which means that you can find always find some distance such that within that distance for the inputs the outputs are always within a certain (arbitrarily small) range. Thus, even if you algebraic efforts fail, you can always apply methods like Newtonian approximation and get an answer in the end.

Discrete math is different. Consider the following problem:

Find integers x and y such that 120x - 23y = 1.

How would you solve that? With "normal" methods, you can't. It looks like an underdetermined system (and it is), and the requirement to have integers means that you can't just do the old trick of fixing a random x and finding y to match it. You do not get the benefits of continuity because there is no granularity. The answer is (-9,47), but in order to do that you need to do a series of reduction steps in the form of the Extended Euclidean Algorithm

Now, consider a second problem:

Find integer x such that the last ten digits of 3^x are 3147595467

This one is harder. You have to use tricks like the Chinese Remainder Theorem, and even then a lot of computational effort is required.

Now for the really hard problem:

Find integers x,y such that x * y = 8340368351292114898806993735182512787836108384416112375142633921308502660867470 5717333990587055137541907379907772161559852062233059312453755072535705574833391 32157327857301030605921850352045823680907

We have some shortcuts to help you here, but even then the problem is currently just on the fringes of computability. Add a few more digits and it becomes intractable. The reason is that the hardness of the problem does not come from simplifying a complex equation or finding roots to a polynomial; rather, it comes from the fact that the answers have to be integers, and so you're dealing with a world that is discrete. Thus, you can easily find many instances where the expression x*y - n, where n is the above number, come very very close to zero, but those cases are all useless in helping you find the (unique up to swapping x and y and negating both) solution to the above expression.

It is worth noting that even if ECC were to fall today, cold storage addresses would still be secure.

Of course if that were to happen the price would plummet making your cold storage (at least temporarily) worthless.  

More info on secp256k1

https://en.bitcoin.it/wiki/Secp256k1
http://www.secg.org/collateral/sec2_final.pdf (page 15)

Thanks for the great write up and explaination.

BTW for those that don't know or may be interested in looking it up Bitcoin uses the standard recommended curve secp256k1.

This has nothing to do directly with complexity or with cryptography (public key or otherwise) now, just with the nature of elliptic curves.

They are 1-dimensional as algebraic varieties, the same way a line or a circle is 1-dimensional. A circle is a good example: the set of points in the plane satisfying

  x^2 + y^2 = 1

is a circle, even though we have embedded it in a (2-dimensional) plane and are using two coordinates  (x, y). The set of all points  (x, y)  is the entire 2-dimensional plane, while the set of only those points satisfying  x^2 + y^2 = 1  is 1-dimensional. If you feel like describing a point on a circle using a single coordinate, you can do it using stereographic projection, which gives a correspondence between points on a line and points on the circle.

An elliptic curve is also 1-dimensional in this sense. Instead of  x^2 + y^2 = 1  you have a different equation, for example  y^2 = x^3 + a x + b. It is not "fractal" but smooth.

That is all pretty abstract, but what is important in this context is that since we are doing algebra we can plug in elements of any field for  x  and  y.  For the purposes of cryptography we want to use a finite field. Which finite field, and which elliptic curve for that matter? There is a list of standard curves published by NIST and others. The coordinates of a point on the curve then lie in a certain finite field, while the set of all points on the curve form an interesting finite group.

If you are implementing elliptic-curve cryptography, as in Bitcoin, unless you have a good reason you are better off just using one of the standard recommended curves. You do not need to numb your brain with too much detail to get started on the implementation side; it is more important to have a general understanding of how everything works. You can worry about specific mathematical details if and when they come up.

Nah. Cryptography is a complicated mathematical study.  There are various levels of knowledge, and as people attempt to explain in the simplest terms possible some of these complex concepts, others who understand the mathematics better see a need to either correct an error (so that everyone can learn and avoid the error in the future) or clarify an explanation that might otherwise have been misleading.

I do agree however that this conversation has drifted far enough from the original question that there probably isn't much need to continue the discussion any farther.

This is a thread started by someone trying to learn the basics of what a public and private key is.  The fact that it's diverged this far off topic and is now "my math knowledge is better than your math knowledge" is embarrassing for us as a community.

No, sorry, your math is outdated (by like a century, or at the very least 45 years if you start counting from 1967).

The crisis in Mathematics that resulted from Peano's non-differentiable monster curves was successfully resolved.  We even have the word fractal now specifically to describe curves where the Hausdorff Dimension is not equal to the topological or Euclidean dimension.

That is a stretch.

Well made myself laugh anyway.

In that case we are starting to alter the way the terms are typically used in mathematics.  A line is one dimensional, but the only curve that is one dimensional is the curve that happens to also be a line.  Most curves (those that are actually curvy and not straight) exist in two dimensions.

While a plane is two dimentional, I suppose a set of points in a plane could be one dimensional if they all happened to be on the same line in that plane (I doubt this is what was being stated though).

I am guessing that 1-dimensional refers to the line-like nature of the curve, much like your position on a winding street can be described as a distance from an origin (a single value), even though your latitude and longitude are two-dimensional.

Please explain what you mean by 1-dimensional in this context especially when you go on to say "the set of points  (x, y)  in the plane", here the word plane implying two dimensions.  

The second bold statement is incorrect. The points on the eliptical cuve form a finite group, not a field.

Elliptic curves are 1-dimensional but points on them are not "integers", if that is confusing people. You can visualize a typical elliptic curve as the set of points  (x, y)  in the plane satisfying a certain algebraic equation, and you may speak of integral points or rational points on the curve, but for cryptographic purposes the coordinates should lie in a certain finite field.

Back to complexity, it is worth pointing out that just from the statement of a problem it is not always obvious how hard it is to solve it, and that is what makes life interesting. For example, testing whether a huge number is prime by trying to divide it by 2, 3, 5, and so on will take a huge amount of time, but a 100% reliable way to do it "fast" was not discovered until 2002.

You actually need more than just the x-value, because a quadratic is used to find the y-value.  A single bit (parity) will allow you to distinguish between the two possible y-values and pick the correct one.

And in my opinion, it is compression, in the sense of that word that we normally use.  Redundant information is discarded, and enough is kept to reconstruct the original.

-- feel free to stop reading here --

There is a second flag involved in the compression scheme, by the way.  It is for the private key export format.

privkey(int256 value) -> pubkey(int256 x-value,int256 y-value) -> address
privkey(int256 value) -> pubkey(int256 x-value,int256 y-value) -> pubkey(int256 x,bool y-flag) -> address

(x-value,y-value) and (x-value,y-flag) give different hashes, and thus, different addresses.  While doing the ECDSA verification, both pubkeys will give the same result because the y-value is reconstructed if necessary.  But, the bitcoin software watches for transactions based on "secrets", and it only stores one secret per privkey.  So, when exporting a private key, it will attach the flag if the key is compressed, so that the importing wallet knows which secret to watch for, and which address to give to the user.

Well the compressed version of public key is a single value.

With the X value and the curve one can compute the Y value.  The y value is deterministic to the x value and the curve. This means the public key can be reduced to just the X value of the point on the curve.

"Compressed" is a misleading term IMHO as there is no compression going.
"Normal" ECSA public key (x-value, y-value)
"Compressed" ECDSA public key (x-value)  y-value is computed at runtime using the curve.

Sadly Bitcoin didn't use compressed keys from the beginning so most of the potential savings is wasted.

To further clarify, in ECC the private key is in fact a large scalar (number) but the public key is not a number at all - it is a point on the chosen eliptical curve.  The public key is a set of two numbers, an X and a Y coordinate in the finite group that makes up the points on the selected eliptical curve.

I explained this in some detail in post number 4 in this very thread:  https://bitcointalksearch.org/topic/m.1458864

In the specific set of ECC parameters selected to be used by the Bitcoin system the finite field used for the private keys is in fact a prime field, but the private keys themselves do not need to be prime and will only be prime if you are very lucky and randomly generate a prime number for your private key.

I think you still made your point: you've clearly demonstrated an intractable function, even with just the first three examples.  Someone just learning this isn't actually going to try to factor any large number you throw out, nor are they going to try to use it in any way.  So you had just said it's a real key, but as it turns out it's probably not.  This is not an important detail, so no harm no foul.