Pages:
Author

Topic: How can you verify the randomness that's coming from a hardware? - page 2. (Read 1651 times)

legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
we'll grab it as 23.548753℃ . what i offer is to use this 0.008753℃ . you can't predict it in any way . and you can't exploit it.
Just because a sensor outputs 6 decimals, doesn't mean it accurately measures them. The last 5 digits could just as well be made up, and thus be predictable.
sr. member
Activity: 1190
Merit: 469

Actually it's not complicated at all. Of course your average Joe won't build his own avalanche noise PCB, but someone with electrical engineering skills should be able to whip a circuit up and order a PCB within an afternoon. It's honestly a simple circuit.
I mean as far as general electronics circuits go, I guess it is simple if you compare it to something like a computer motherboard but that doesn't mean it is simple to understand how it works. And why it needs so many components to work. I didnt count them all but it looks like around 50 discrete components. I don't understand why it needs that many. I thought just one single zener diode is all you need.

Quote
Trust me, it's not a pain. Foundation Devices have such circuits in their hardware wallets and the USB RNG you linked to, may have the exact same thing inside it, as well.
I think it does:

The TrueRNG Hardware Random Number Generator uses the avalanche effect in a semiconductor junction to generate true random numbers. The avalanche effect has long been used for generation of random number / noise and is a time-tested and proven random noise source.


The cost to buy theirs would probably be less than the cost to try and make one yourself. Not only monetary cost but time costs. Thing I don't like about any of these though is how they use "whitening".

The new TrueRNG v3 algorithm uses a more advanced whitening technique to reduce the bias below levels where it can be measured reliably.

If something is truly random then it doesn't need any type of post-processing. Shouldn't need. But you don't get a choice if you use their device...

Quote
Sampling radiation measurements won't be much simpler than sampling the avalanche noise source, and you'll need specialized components.

Here's actually a project of someone building a Geiger based RNG, DIY, not cheap, though. And I wouldn't vouch for its entropy; there are many steps that can go wrong and introduce bias, e.g. in the ADC.
https://www.instructables.com/Arduino-True-Random-Number-Generator/

Low bit rate, kind of pricey setup but the theory of its operation as explained in that instructable is much simpler to understand. you don't even need uranium. just a banana or two might suffice.  Cheesy
member
Activity: 351
Merit: 37
24bit temp chip is on sale for $6
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!

All the information is laid out nicely here: https://betrusted.io/avalanche-noise.html
yes it is but that's a really complicated process and i wouldn't recommend anyone to try it. they might end up with something that doesn't even work right and has low entropy!  Angry
Actually it's not complicated at all. Of course your average Joe won't build his own avalanche noise PCB, but someone with electrical engineering skills should be able to whip a circuit up and order a PCB within an afternoon. It's honestly a simple circuit.

Quote
I just made a quick web search and seriously surprised that there's no ready-made PCB / DIY kit or similar, that you can plug in and get randomness e.g. through cat /dev/tty.usbrandomdevice.
probably because it is a real pain to make them and they would have to charge so much that no one would buy it they would just buy something like this: https://www.amazon.com/TrueRNG-V3-Hardware-Random-Generator/dp/B01KR2JHTA
Trust me, it's not a pain. Foundation Devices have such circuits in their hardware wallets and the USB RNG you linked to, may have the exact same thing inside it, as well.

I appreciate the open-source and verifiable avalanche noise source (actual circuit from few simple components) on the Passport hardware wallet.
And obviously the ability to import your own custom seed phrase. This allows you to generate it with dice or whatever you deem secure.

But again; these avalanche noise circuits are amazing. You can literally see them on the PCB, take an oscilloscope to it and verify that it does what it's supposed to and that there's no deterministic bullshit going on.




i heard someone made one using a geiger counter and detecting radiation. not sure how hard that is to diy. but maybe it's simpler than this zener diode thing.
Sampling radiation measurements won't be much simpler than sampling the avalanche noise source, and you'll need specialized components.

Here's actually a project of someone building a Geiger based RNG, DIY, not cheap, though. And I wouldn't vouch for its entropy; there are many steps that can go wrong and introduce bias, e.g. in the ADC.
https://www.instructables.com/Arduino-True-Random-Number-Generator/
member
Activity: 351
Merit: 37
we'll grab it as 23.548753℃ . what i offer is to use this 0.008753℃ . you can't predict it in any way . and you can't exploit it. At best you'll get random_num + your_num
staff
Activity: 3304
Merit: 4115
you can use high precision temperature indicator. a lot of these pcb's are on the market
Someone can correct me if I'm misremembering, but I believe this has been exploited in the past. I unfortunately don't have a reference, because I can't quite seem to pinpoint what it was exactly. However, it was to do with a computer generating something based on the operating temperature. It might be have been a game, rather than a password or key, but it was easily exploited since most users computers temperatures will be within a range, in fact the vast majority would be.

Only those that are running specialised systems or have poor ventilation to the extreme would be outliers, and even then the temperatures would be easily to emulate. For entropy you need to be as random as possible. A human or the temperature of a room or machine isn't random, in fact it's incredibly easy to predict with a small degree of error.

Ultimately, the conclusion is that using anything which would have a common value among users, and isn't in fact random at all is a terrible idea when it comes to generating sensitive data.
member
Activity: 351
Merit: 37
you can use high precision temperature indicator. a lot of these pcb's are on the market
I'm pretty sure that temperature readings don't have great entropy, right. Or what's the idea there?
Something along the lines of the coastline paradox? That if you measure too accurately, the results are (within some range) going to be 'all over the place'?

Honestly, I'd prefer a circuit actually made for generating high entropy than using something that has good entropy as a side effect.
right away i found temp chip with 24 bit resolution and temp+humidity one with 14 bit. So  last  13 bits from former can be used and 3 from latter. say it updates every 0.2 s and after 4s you'll have 32b seed

https://www.te.com/commerce/DocumentDelivery/DDEController?Action=showdoc&DocId=Data+Sheet%7FTSYS01%7FA%7Fpdf%7FEnglish%7FENG_DS_TSYS01_A.pdf%7FG-NICO-018

https://www.renesas.com/kr/en/document/dst/hs300x-datasheet
sr. member
Activity: 1190
Merit: 469

All the information is laid out nicely here: https://betrusted.io/avalanche-noise.html
yes it is but that's a really complicated process and i wouldn't recommend anyone to try it. they might end up with something that doesn't even work right and has low entropy!  Angry
Quote
I just made a quick web search and seriously surprised that there's no ready-made PCB / DIY kit or similar, that you can plug in and get randomness e.g. through cat /dev/tty.usbrandomdevice.

probably because it is a real pain to make them and they would have to charge so much that no one would buy it they would just buy something like this: https://www.amazon.com/TrueRNG-V3-Hardware-Random-Generator/dp/B01KR2JHTA

i heard someone made one using a geiger counter and detecting radiation. not sure how hard that is to diy. but maybe it's simpler than this zener diode thing.
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!
you can use high precision temperature indicator. a lot of these pcb's are on the market
I'm pretty sure that temperature readings don't have great entropy, right. Or what's the idea there?
Something along the lines of the coastline paradox? That if you measure too accurately, the results are (within some range) going to be 'all over the place'?

Honestly, I'd prefer a circuit actually made for generating high entropy than using something that has good entropy as a side effect.
member
Activity: 351
Merit: 37
you can use high precision temperature indicator. a lot of these pcb's are on the market
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!
Quote
It should give a high entropy result, even if the PRNG was seeded with a known seed which would then be used to reconstruct the randomness.
it gave Entropy = 4.053136 bits per byte.
that was for a file that had 125,000 hex private keys in it.
apparently it treat each character as 8 bits.
Character = byte = 8 bits.

Quote
Well, the 'avalanche noise source' electronic method can actually be observed ('see what is going on'); you do need some lab equipment, though. Keep in mind that just visually inspecting the circuit can already give you some confidence that you received the circuit actually specified in the schematic. Inspecting the schematic tells you what the circuit does, so sneaking in some backdoor is going to be pretty hard on such a device.
i'd love to have one of those devices but i don't think i can get an oscilloscope and things to help build it. that's the problem i think you need that type of thing.  building the thing while i guess its tedious would not be the hardest part the harder part would be figuring out how to interface it to something and do data collection. hence why you don't see people doing this all the time. and the people that do, they just show a short video of the output on their screen nothing to learn there.
All the information is laid out nicely here: https://betrusted.io/avalanche-noise.html
I just made a quick web search and seriously surprised that there's no ready-made PCB / DIY kit or similar, that you can plug in and get randomness e.g. through cat /dev/tty.usbrandomdevice.
sr. member
Activity: 1190
Merit: 469

I don't think this is suited for your application. This program gives you the entropy per byte / character for evaluating data density of a file.
yeah probably it's not but at least I can see character counts and get a good idea of there's any bias in them. not sure of a tool that could take a file of hex private keys and do what you're suggesting.

Quote
It should give a high entropy result, even if the PRNG was seeded with a known seed which would then be used to reconstruct the randomness.
it gave Entropy = 4.053136 bits per byte.
that was for a file that had 125,000 hex private keys in it.
apparently it treat each character as 8 bits.

When i ran it with the -b option i was kind of surprised though that 1s and 0s did not seem to occur at an equal frequency at all. there was a pretty big imbalance there. but i'm not sure if that's an issue. but we're talking not 50/50 not even close.

Quote
I may be wrong, but I think a program that gives you entropy 'per 64-byte seed' (instead of per-byte) across a large set of generated seeds, instead of calculating entropy across a stream of bytes.
If you're using the H(X) formula for entropy you mentioned earlier in the thread then I don't see how that formula could really be useful since if you're taking your universe of possible outcomes to be all 64-byte seeds, then it would be highly unlikely that any of them were duplicated even in a massive file. Thus you would obtain maximum entropy every time on every test run. Doesn't tell you anything. You will never find a duplicate so all your "objects" will be distinct and have the same probability of happening. nothing useful about that.

I think a useful tool would need not only to calculate frequencies of each hex character but of combinations (permutations) of twos, threes and so on. And analyze if there was any bias in any of those character counts. I don't know of a tool that does that though.

Quote
Well, the 'avalanche noise source' electronic method can actually be observed ('see what is going on'); you do need some lab equipment, though. Keep in mind that just visually inspecting the circuit can already give you some confidence that you received the circuit actually specified in the schematic. Inspecting the schematic tells you what the circuit does, so sneaking in some backdoor is going to be pretty hard on such a device.

i'd love to have one of those devices but i don't think i can get an oscilloscope and things to help build it. that's the problem i think you need that type of thing.  building the thing while i guess its tedious would not be the hardest part the harder part would be figuring out how to interface it to something and do data collection. hence why you don't see people doing this all the time. and the people that do, they just show a short video of the output on their screen nothing to learn there.
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!

Do you already have a plan for evaluating the entropy of it?
I used https://www.fourmilab.ch/random/ in the past to measure the "entropy" of linux /dev/random from one of my machines. I assume I would do the same thing with this one. Their ent program the output is kind of confusing except for the option that shows character counts. Which is what I basically go off of. They claim:
I don't think this is suited for your application. This program gives you the entropy per byte / character for evaluating data density of a file. It should give a high entropy result, even if the PRNG was seeded with a known seed which would then be used to reconstruct the randomness.
I may be wrong, but I think a program that gives you entropy 'per 64-byte seed' (instead of per-byte) across a large set of generated seeds, instead of calculating entropy across a stream of bytes.

Quote
I mentioned it earlier in this thread; I think it's just much easier to trust physical entropy (like dice throws) or a relatively straight-forward open-source 'avalanche' circuit on a PCB.
Yeah physical entropy is the way to go for low volume needs which most of us fall into. I trust that the most at the end of the day. Not any of these electronic methods as good as they might seem, you can't really see what is going on. You have to trust what you can't see. Trust past results, trust that it is performing the same as past results. The electronic methods are fun though to investigate. And they might find uses in higher volume applications.
Well, the 'avalanche noise source' electronic method can actually be observed ('see what is going on'); you do need some lab equipment, though. Keep in mind that just visually inspecting the circuit can already give you some confidence that you received the circuit actually specified in the schematic. Inspecting the schematic tells you what the circuit does, so sneaking in some backdoor is going to be pretty hard on such a device.
sr. member
Activity: 1190
Merit: 469

Do you already have a plan for evaluating the entropy of it?

I used https://www.fourmilab.ch/random/ in the past to measure the "entropy" of linux /dev/random from one of my machines. I assume I would do the same thing with this one. Their ent program the output is kind of confusing except for the option that shows character counts. Which is what I basically go off of. They claim:

We interpret the percentage as the degree to which the sequence tested is suspected of being non-random. If the percentage is greater than 99% or less than 1%, the sequence is almost certainly not random.

But they don't explain why or how.

I did my own chi-square test and it concluded do not reject the null hypothesis (I already knew it would though based on the histogram output) so I'm not worried about it. Their program doesn't appear to be open source (CORRECTION:actually it IS open source: they have a github link on the web page) all you get is an exe file. All you need is the exe file and just run it from a command line. in windows!

With all of that said, to have RDSEED the CPU needs to be intel 5th gen cpu or higher. only one of my machines is that Shocked

Quote
I mentioned it earlier in this thread; I think it's just much easier to trust physical entropy (like dice throws) or a relatively straight-forward open-source 'avalanche' circuit on a PCB.

Yeah physical entropy is the way to go for low volume needs which most of us fall into. I trust that the most at the end of the day. Not any of these electronic methods as good as they might seem, you can't really see what is going on. You have to trust what you can't see. Trust past results, trust that it is performing the same as past results. The electronic methods are fun though to investigate. And they might find uses in higher volume applications.
hero member
Activity: 882
Merit: 5834
not your keys, not your coins!
6 years is long enough to know if there was some affect from this supposed backdooring of "billions of devices".
Why is it? Have you seen just how much mass surveillance the US government was undertaking in secret before the Snowden leaks? And there is no telling what other programs they are running, unbeknownst to the general population.
That's the nasty thing about surveillance: without whistleblowers / leaks, there is no way of knowing whether you're affected. Creating a false sense of security. Combine that with powerful 'nothing to hide propaganda'.. The rest is history.

would just like to sometime testout rdseed to see how random its output looks. if it looks random enough maybe it's good enough.
Do you already have a plan for evaluating the entropy of it?

I mentioned it earlier in this thread; I think it's just much easier to trust physical entropy (like dice throws) or a relatively straight-forward open-source 'avalanche' circuit on a PCB.
legendary
Activity: 2268
Merit: 18711
6 years is long enough to know if there was some affect from this supposed backdooring of "billions of devices".
Why is it? Have you seen just how much mass surveillance the US government was undertaking in secret before the Snowden leaks? And there is no telling what other programs they are running, unbeknownst to the general population.

So what exactly was the result of those 32 lawsuits?
As far as I am aware, and someone can correct me if I'm wrong, the lawsuits in question were only to determine whether or not Intel were liable for making statements which were misleading or not fully revealing the details of their products. They had nothing to do with whether or not a backdoor actually existed or who funded it.

that document never mentions Intel though.
And I wouldn't expect it to. They are obviously not going to go naming individual companies, since if the document leaked (as it has done) then adversaries know exactly which companies are complicit. But "Insert vulnerabilities in to commercial encryption systems, IT systems, networks, and endpoint communications devices" is pretty clear.
sr. member
Activity: 1190
Merit: 469
6 years is "long ago"? Hardly. And how long is enough to forgive a company for sneaking a government funded backdoor in to billions of devices?
6 years is long enough to know if there was some affect from this supposed backdooring of "billions of devices". you would think if its happening to "billions of devices", it would be happening to all of us right now too.

Quote
And you expect a company which might be putting a backdoor in to their products to release documentation which says they have put a backdoor in to their products?

well let's take a look at what intel themself says about IME:

At system initialization, the Intel® Management Engine loads its code from system flash memory. This allows the Intel® Management Engine to be up before the main operating system is started. For run-time data storage, the Intel® Management Engine has access to a protected area of system memory (in addition to a small amount of on-chip cache memory for faster and more efficient processing).

So what exactly was the result of those 32 lawsuits? https://www.theverge.com/2018/2/16/17020048/intel-spectre-meltdown-class-action-lawsuits



Quote

that document never mentions Intel though.

at any rate, i dont see intel as a bad guy and would just like to sometime testout rdseed to see how random its output looks. if it looks random enough maybe it's good enough. but at some point i would like one of those usb hardware RNGs because i think they might be above this type of criticism. but they do cost a good bit. like $60 or $70. for the TrueRNG. off amazon. but then i wouldn't have to worry about if my output was truly random or known by some 3rd party too. Grin
legendary
Activity: 2212
Merit: 7064
Well I don't know how you come to that conclusion but the only way I would believe it is if I seen it with my own eyes. I'd like to test RDSEED out and see what it can do. But in the mean time, feel free to share a 10 MB file of output from it so it can be statistically analyzed. Then we have something to talk about. Grin
You already believe in so many things in your life that you never saw with your own eyes, so I don't know why this would be any different.
It's common sense and you can ask any tech expert that understands more how to achieve random results, but if you want to continue playing this game, than go for it.

They haven't. Intel's Management Engine is still being included in every chip they produce.
Their main competition is called AMD Platform Security Processor (PSP), but it is almost the same thing like in Intel microchips.
It's not impossible to minimize both of this processes in some machines, but this is not exactly newbie friendly task.

Now I know that Management Engine has no reason to be used on home systems the way it is designed, but if someone really doesn't like this feature, they can run a version of the Linux kernel that has disabled vPro support (and whatever the counterpart is called in AMD). No need for strange rituals of using ancient hardware with obscure distributions like what Richard Stallman is doing.
It's impossible to really disable this with kernel or any software patch, because this is hardware based problem.
Even with installing special BIOS version that have option to disable Intel Management, you are not really disabling anything.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org

Are you seriously suggesting that Intel chips have a zero chance to include a backdoor?
Because I've got a newsflash for you: https://fossbytes.com/intel-processor-backdoor-management-engine/

ok in no way am I an expert on computer cpu architecture but that article is from 2016. 8 years ago. maybe things have changed since then. with new cpus that intel put out. at this point we are talking about 8+ year old cpus. maybe it's time to upgrade if someone has concerns about the safety of their computing platform.

Now I know that Management Engine has no reason to be used on home systems the way it is designed, but if someone really doesn't like this feature, they can run a version of the Linux kernel that has disabled vPro support (and whatever the counterpart is called in AMD). No need for strange rituals of using ancient hardware with obscure distributions like what Richard Stallman is doing.
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
ok in no way am I an expert on computer cpu architecture but that article is from 2016. 8 years ago. maybe things have changed since then. with new cpus that intel put out. at this point we are talking about 8+ year old cpus. maybe it's time to upgrade if someone has concerns about the safety of their computing platform.

Nothing changed since then. It's still exist on newest Intel CPU (12th gen Alder Lake). It's explicitly mentioned on their product brief[6].



Quote
Keep in mind it's not just about backdoors; RNGs can also simply be implemented badly, which would be hard to test / identify.
Intel has provided documentation about how their RDRAND and RDSEED work. Believe it or not. Trust it or not. But they provided the docs.

Let's see their documentation[1].

RDRAND retrieves a hardware-generated random value from the SP800-90A compliant DRGB and

RDSEED retrieves a hardware-generated random seed value from the SP800-90B and

Both of them are standard from NIST[2-3]. Based on leak by Edward Snowden, NSA influence NIST to make weaker standard[4] and it's been predicted by expert since 2007[5].

[1] https://www.intel.com/content/www/us/en/developer/articles/guide/intel-digital-random-number-generator-drng-software-implementation-guide.html
[2] https://csrc.nist.gov/publications/detail/sp/800-90a/rev-1/final
[3] https://csrc.nist.gov/publications/detail/sp/800-90b/final
[4] https://web.archive.org/web/20130910030443/http://fcw.com/Articles/2013/09/06/NSA-NIST-standards.aspx
[5] https://archive.ph/20120919094854/http://www.wired.com/politics/security/commentary/securitymatters/2007/11/securitymatters_1115
[6] https://www.intel.com/content/www/us/en/products/docs/processors/embedded/12th-gen-iot-desktop-processors-brief.html
Pages:
Jump to: