Topic: FRAGMENTED BACKUPS VULNERABILITY!! IF YOU USE THEM, READ THIS!! - page 2. (Read 44127 times)

You can choose any x as long as the coefficients are random. I don't quite see how randomizing x achieves anything, as you have to provide entire points (x, f(x)) as fragments, therefor x is public information. Maybe you were trying to say that the x values should be chosen at random, instead of say, as part of a sequence? I don't think that's relevant at all for SSS, since all operation are performed on a finite field.



The coefficients are 32bytes, therefor your backup would have 5 lines (1 header line + 4 * (16bytes of data + 2bytes of checksum) lines) per fragment. If the fragment only has 1 header and 2 data lines, it's using implicit [1,...,N] for x.


It's far worse than that. Look at how the coefficients are constructed, they're hashes of the previous coefficient. Whatever fragment the attacker gets access to, he will be able to compute all following coefficients just by hashing x once.

Read this again and look at the code again. In the first scheme, anyone with access to a fragment that carries the first coefficient can compute *ALL* all of the coefficients. The single point he has with the fragment is then enough to reveal the secret.

This is slightly worsened by the fact that since there are more fragments than coefficients, certain coefficients reoccur on several fragments. Now people being people, when they distribute fragments they tend keep the first few for themselves, by happenstance. If the coefficients were distributed in order with fragments, this very human yet entirely irrelevant quirk would actually save these users from total disaster. Again, the order of coefficients matter since they are derived consecutively from one another

This is even worse than I first thought.  In an N-of-M backup, each of the M sheets encodes a linear equation (linear in the unknown coefficients, not in x).  Together, they can be solved to find the N coefficients of a polynomial.  It requires N equations to find the N unknowns, this is where the most of the security comes from.  In addition, you want to avoid leaking information that could potentially reduce the search space for the unknowns, hence the random x values.

But if each sheet reveals a coefficient, then there are TWO equations on each sheet!  (y = c_0 + c_1 * x + c_2 * x^2 + ...  and c_i = x)

Thus you only need N/2 sheets to recover the key !!!.  The seed can be recovered from a single sheet of what was supposed to be a 2-of-M backup, or from two sheets of a 3-of-M or 4-of-M backup, etc etc.  

At least only the oldest Armory backups have this awful flaw.  The newer ones potentially leak some information that makes finding the key a little less impossible if you already have N-1 sheets.

NB: This is of course assuming that I have not misunderstood goatpig's undoubtedly slightly simplified explanation of the flawed algorithm.

There are two things should be random in SSS.  The coefficients of the polynomial.  And the x-values where the polynomial is evaluated.

Non-random x values is probably relatively benign, since all information about the polynomial is in the f(x) values.  Wikipedia does not even mention that the x values should be random.

Pseudo-random coefficients is certainly a bad idea.  And printing the coefficients of the polynomial by using them as x-values is a horrendous mistake, I can hardly believe that it has been done!

Is there any way I can see on my 3-of-5 backup if it was made before or after that update was made?  I am of course going to move my funds, but doing so before a new 3-of-5 backup is in place is also a risk.



EDIT: Could not spell 'pseudo'

TLDR:

Fragmented backups in Armory use a broken implementation of Shamir's Secret Sharing (SSS). All users of fragmented backups need to treat the wallets backed up in this fashion compromised and migrate all funds to a fresh wallet.

1) The story

A couple days ago I was warned by Gregory Maxwell that the implementation for Shamir's Secret Sharing Armory uses is broken. After reviewing the code in question, I've concluded the implementation of SSS introduces a vulnerability in fragmented backups that requires not only an immediate fix but also a community wide warning to alert all fragmented backup users.

These users need to sweep all funds from these wallets. All version starting 0.96.3 will have the vulnerability fix.

2) A high level look at SSS

SSS is a scheme that takes a secret and outputs a set of N fragments. M of these fragments are sufficient to reconstruct the secret.

In order to do so, you first constructs a polynomial f of degree M over a finite field, in which M-1 of the coefficients are random values, and the last one is the secret itself. You then compute N points of f. Each point, which is a pair (x, f(x)), constitutes a fragment.

To reveal the secret, you need to reconstruct the polynomial, which you do by interpolating M points together.

I will not go into the details of the interpolation, but one element here is crucial: The effect of a partial interpolation is to narrow down the possible candidates for valid coefficients. In other words, the more points on the curve you interpolate, the clearer a picture of the curve you get. The curve is constructed around your secret, a more refined interpolation results in an ever increasing leak of information, to the point where you can brute force the secret.

This property of the interpolation is counteracted by the use of a finite field. Since all points generated with the polynomial belong to the underlying finite field, the cyclic nature of finite fields widens the range of possible solutions that would yield your particular polynomial from a partial interpolation (by making them infinite), in such a way that you simply cannot brute force your way out of the interpolating at least M points.

A simple way to demonstrate that property is as follows:

Consider the equation:


It's obvious that x is 3, and that this is the only solution. Now add this variation to the equation:


Suddenly x can be 3, 14, 25, 36... and so on. There is now an infinite amount of solutions for x, due to the introduction of the modulo operation. This is essentially the effect the finite field has on the interpolation equations.

3) The vulnerability

The whole point of the previous section was to demonstrate how SSS is constructed to prevent partial information leaking. One of the requirements to insure that property is that all of the polynomial coefficients but the secret are chosen at random.

This is precisely what Armory's implementation breaks, by rolling deterministic and chained coefficients, where the first coefficient is deterministically derived from the secret, and the subsequent ones are derived successively from one another.

If you look at it in the context of the previous paragraph, where SSS was crafted to deny any alley to gather information about coefficients short of M fragments, the newly introduced deterministic relationship between all coefficents provides a path that potentially dumbs down the security of SSS to that of the hash function used to generate the coefficients.

This can ultimately lead to a subset of fragments leaking information about coefficients where none should be.

4) In the code

Code wise, there were 2 instances of this implementation, both of which are faulty.

a) The first version

It did 2 things wrong, one more aggravating than the other:

The coefficients were not picked at random, instead they were derived as hashes of the secret, in a fashion that boils down to this:


The fragments themselves were constructed as the following points on the curve:


There are 2 important issues with this setup. First of all, the coefficients are not selected at random, therefor it breaks SSS security assumptions.

Next, and most aggravating, the coefficients were provided as is on the fragments, since you need to provide the pair (x, f(x)), and x in this case were the actual coefficients.

Note that since the coefficients were derived from each other, the second mistake is twice as aggravating, as anyone with fragment n would have coefficient n and could derive all coefficients n+1.

The second mistake was caught and fixed eventually. I was not involved with anything regarding cryptography and security in Armory at the time, therefor I have no recollection of the event. I expect there was a write up of the issue and people were told to cycle wallets, but I can't remember any of it.

You can look at the code in its original form here:
https://github.com/etotheipi/BitcoinArmory/commit/80e373a#diff-27fe88d2c6032fecb93912a17d72081bR1615

b) The second version

The second version only made sure points were generated using the [1...N] sequence for x instead of coefficient themselves. At this point fragmented backups where no longer as broken but the code still did not implement SSS correctly.

Notably the assumption that no amount of fragments less than M can leak any data about the secret is not true with that faulty implementation.

One way to look at it is that this implementation introduces a deterministic relationship between the coefficients in a way that it can effectively reduce the security of the system to that of a single pass of HMAC512 provided an amount L of fragments, with L < M.

Here is the commit for the second version:

https://github.com/etotheipi/BitcoinArmory/commit/0824b632600116bd6395cec939fa6fd398efeb19#diff-27fe88d2c6032fecb93912a17d72081bR1915

5) Affected versions

Fragmented backups were introduced in version 0.88 (04/18/2013) and the first fix was deployed in version 0.90 (11/26/2013).

The final fix was introduced in v0.96.3 (9/21/2017)

6) The fix

The coefficients were made deterministic in order to present deterministic fragments to the user when fragmenting a wallet over a given scheme. In other words, fragmenting wallet W over a M-of-N scheme would always yield the save fragment values for the same fragment index.

This introduces scenarios where any amount of fragments can be recomputed from the private wallet root without invalidating fragments still in the wild.

The fix was to randomize the coefficients at the cost of the deterministic characteristic of the fragments.  The choice was fairly simple:

a) The deterministic attribute gained by bastardizing SSS is worthless in the face of how it damages its security properties.

b) Even if a change to SSS is designed so that it does not so obviously erode its security properties, this is still an act of rolling a custom cryptographic function, which commands a level of review and security analysis that will most likely not be performed at the adequate level.

c) There are no scenarios I can think of in which the feature of determinism in fragments is actually necessary and central to this type of backups. Introducing it at the cost of security is therefor doubly unacceptable. Backups are supposed to be forever afterall, lacing solid crypto with any kind of bootlegged algorithm does not stand to reason.

Therefor, it is without hesitation that the faulty feature was undone, and the faulty code removed from the repo so as to prevent unaware users from copying it into their own projects.

The changes can be seen here:

https://github.com/goatpig/BitcoinArmory/commit/94d2a7556d25cf788da639d81a7162694982f6b7
https://github.com/goatpig/BitcoinArmory/commit/7bd9887891ac88e2e49954ef034bedef88f23eaf

7) GUI changes

Since the fragments are not deterministic anymore, they are now generated with a unique set ID which is reflected on the backup strings and printed backups. Fragments are only useful within a their own set. Another way to put it is that you cannot mix and match fragments from different sets. This is the only difference between the pre and post fix implementation.

The fixed version is compatible with fragments generated from the deterministic version. You will still be able to restore from these with version 0.96.3+

8 ) Recommendations

It is hard to say exactly how effectively this custom take on SSS breaks security at the fragment level. How fewer fragments would it take to reproduce a secret than intended? Honestly, I don't know, but while the first implementation was effectively breaking all security assumptions of SSS, the second version is SSS at least theoretically.

I don't expect an attacker can snatch a single fragment and reveal the secret the next minute with minimal code. This vulnerability reduces the overall complexity of the problem that of a hash function, it doesn't outright bypass all complexity.

Since we're talking 32 byte integers, breaking the scheme isn't trivial, but it has certainly been weakened to a state that is difficult to precisely assess. Therefor, to remain on the conservative side, my recommendation is as follows:

If you created a fragment backup of your wallet, consider that wallet compromised. Create a new wallet and sweep the funds from the compromised wallet to the new one. You can redo your fragmented backup scheme on the new wallet provided you use Armory 0.96.3 and newer.

If you do not use fragmented backups, you have nothing to do.

9) Notes

Special thanks to Gregory Maxwell for finding the vulnerability and helping with the review of the fix.