Topic: How can you verify the randomness that's coming from a hardware? - page 2. (Read 1657 times)

Just because a sensor outputs 6 decimals, doesn't mean it accurately measures them. The last 5 digits could just as well be made up, and thus be predictable.

I mean as far as general electronics circuits go, I guess it is simple if you compare it to something like a computer motherboard but that doesn't mean it is simple to understand how it works. And why it needs so many components to work. I didnt count them all but it looks like around 50 discrete components. I don't understand why it needs that many. I thought just one single zener diode is all you need.

I think it does:

The TrueRNG Hardware Random Number Generator uses the avalanche effect in a semiconductor junction to generate true random numbers. The avalanche effect has long been used for generation of random number / noise and is a time-tested and proven random noise source.


The cost to buy theirs would probably be less than the cost to try and make one yourself. Not only monetary cost but time costs. Thing I don't like about any of these though is how they use "whitening".

The new TrueRNG v3 algorithm uses a more advanced whitening technique to reduce the bias below levels where it can be measured reliably.

If something is truly random then it doesn't need any type of post-processing. Shouldn't need. But you don't get a choice if you use their device...


Low bit rate, kind of pricey setup but the theory of its operation as explained in that instructable is much simpler to understand. you don't even need uranium. just a banana or two might suffice. 

24bit temp chip is on sale for $6

Actually it's not complicated at all. Of course your average Joe won't build his own avalanche noise PCB, but someone with electrical engineering skills should be able to whip a circuit up and order a PCB within an afternoon. It's honestly a simple circuit.

Trust me, it's not a pain. Foundation Devices have such circuits in their hardware wallets and the USB RNG you linked to, may have the exact same thing inside it, as well.


Sampling radiation measurements won't be much simpler than sampling the avalanche noise source, and you'll need specialized components.

Here's actually a project of someone building a Geiger based RNG, DIY, not cheap, though. And I wouldn't vouch for its entropy; there are many steps that can go wrong and introduce bias, e.g. in the ADC.
https://www.instructables.com/Arduino-True-Random-Number-Generator/

we'll grab it as 23.548753℃ . what i offer is to use this 0.008753℃ . you can't predict it in any way . and you can't exploit it. At best you'll get random_num + your_num

Someone can correct me if I'm misremembering, but I believe this has been exploited in the past. I unfortunately don't have a reference, because I can't quite seem to pinpoint what it was exactly. However, it was to do with a computer generating something based on the operating temperature. It might be have been a game, rather than a password or key, but it was easily exploited since most users computers temperatures will be within a range, in fact the vast majority would be.

Only those that are running specialised systems or have poor ventilation to the extreme would be outliers, and even then the temperatures would be easily to emulate. For entropy you need to be as random as possible. A human or the temperature of a room or machine isn't random, in fact it's incredibly easy to predict with a small degree of error.

Ultimately, the conclusion is that using anything which would have a common value among users, and isn't in fact random at all is a terrible idea when it comes to generating sensitive data.

right away i found temp chip with 24 bit resolution and temp+humidity one with 14 bit. So  last  13 bits from former can be used and 3 from latter. say it updates every 0.2 s and after 4s you'll have 32b seed

https://www.te.com/commerce/DocumentDelivery/DDEController?Action=showdoc&DocId=Data+Sheet%7FTSYS01%7FA%7Fpdf%7FEnglish%7FENG_DS_TSYS01_A.pdf%7FG-NICO-018

https://www.renesas.com/kr/en/document/dst/hs300x-datasheet

yes it is but that's a really complicated process and i wouldn't recommend anyone to try it. they might end up with something that doesn't even work right and has low entropy!  

probably because it is a real pain to make them and they would have to charge so much that no one would buy it they would just buy something like this: https://www.amazon.com/TrueRNG-V3-Hardware-Random-Generator/dp/B01KR2JHTA

i heard someone made one using a geiger counter and detecting radiation. not sure how hard that is to diy. but maybe it's simpler than this zener diode thing.

I'm pretty sure that temperature readings don't have great entropy, right. Or what's the idea there?
Something along the lines of the coastline paradox? That if you measure too accurately, the results are (within some range) going to be 'all over the place'?

Honestly, I'd prefer a circuit actually made for generating high entropy than using something that has good entropy as a side effect.

you can use high precision temperature indicator. a lot of these pcb's are on the market

Character = byte = 8 bits.

All the information is laid out nicely here: https://betrusted.io/avalanche-noise.html
I just made a quick web search and seriously surprised that there's no ready-made PCB / DIY kit or similar, that you can plug in and get randomness e.g. through cat /dev/tty.usbrandomdevice.

yeah probably it's not but at least I can see character counts and get a good idea of there's any bias in them. not sure of a tool that could take a file of hex private keys and do what you're suggesting.

it gave Entropy = 4.053136 bits per byte.
that was for a file that had 125,000 hex private keys in it.
apparently it treat each character as 8 bits.

When i ran it with the -b option i was kind of surprised though that 1s and 0s did not seem to occur at an equal frequency at all. there was a pretty big imbalance there. but i'm not sure if that's an issue. but we're talking not 50/50 not even close.

If you're using the H(X) formula for entropy you mentioned earlier in the thread then I don't see how that formula could really be useful since if you're taking your universe of possible outcomes to be all 64-byte seeds, then it would be highly unlikely that any of them were duplicated even in a massive file. Thus you would obtain maximum entropy every time on every test run. Doesn't tell you anything. You will never find a duplicate so all your "objects" will be distinct and have the same probability of happening. nothing useful about that.

I think a useful tool would need not only to calculate frequencies of each hex character but of combinations (permutations) of twos, threes and so on. And analyze if there was any bias in any of those character counts. I don't know of a tool that does that though.


i'd love to have one of those devices but i don't think i can get an oscilloscope and things to help build it. that's the problem i think you need that type of thing.  building the thing while i guess its tedious would not be the hardest part the harder part would be figuring out how to interface it to something and do data collection. hence why you don't see people doing this all the time. and the people that do, they just show a short video of the output on their screen nothing to learn there.

I don't think this is suited for your application. This program gives you the entropy per byte / character for evaluating data density of a file. It should give a high entropy result, even if the PRNG was seeded with a known seed which would then be used to reconstruct the randomness.
I may be wrong, but I think a program that gives you entropy 'per 64-byte seed' (instead of per-byte) across a large set of generated seeds, instead of calculating entropy across a stream of bytes.

Well, the 'avalanche noise source' electronic method can actually be observed ('see what is going on'); you do need some lab equipment, though. Keep in mind that just visually inspecting the circuit can already give you some confidence that you received the circuit actually specified in the schematic. Inspecting the schematic tells you what the circuit does, so sneaking in some backdoor is going to be pretty hard on such a device.

I used https://www.fourmilab.ch/random/ in the past to measure the "entropy" of linux /dev/random from one of my machines. I assume I would do the same thing with this one. Their ent program the output is kind of confusing except for the option that shows character counts. Which is what I basically go off of. They claim:

We interpret the percentage as the degree to which the sequence tested is suspected of being non-random. If the percentage is greater than 99% or less than 1%, the sequence is almost certainly not random.

But they don't explain why or how.

I did my own chi-square test and it concluded do not reject the null hypothesis (I already knew it would though based on the histogram output) so I'm not worried about it. Their program doesn't appear to be open source (CORRECTION:actually it IS open source: they have a github link on the web page) all you get is an exe file. All you need is the exe file and just run it from a command line. in windows!

With all of that said, to have RDSEED the CPU needs to be intel 5th gen cpu or higher. only one of my machines is that


Yeah physical entropy is the way to go for low volume needs which most of us fall into. I trust that the most at the end of the day. Not any of these electronic methods as good as they might seem, you can't really see what is going on. You have to trust what you can't see. Trust past results, trust that it is performing the same as past results. The electronic methods are fun though to investigate. And they might find uses in higher volume applications.

That's the nasty thing about surveillance: without whistleblowers / leaks, there is no way of knowing whether you're affected. Creating a false sense of security. Combine that with powerful 'nothing to hide propaganda'.. The rest is history.

Do you already have a plan for evaluating the entropy of it?

I mentioned it earlier in this thread; I think it's just much easier to trust physical entropy (like dice throws) or a relatively straight-forward open-source 'avalanche' circuit on a PCB.

Why is it? Have you seen just how much mass surveillance the US government was undertaking in secret before the Snowden leaks? And there is no telling what other programs they are running, unbeknownst to the general population.

As far as I am aware, and someone can correct me if I'm wrong, the lawsuits in question were only to determine whether or not Intel were liable for making statements which were misleading or not fully revealing the details of their products. They had nothing to do with whether or not a backdoor actually existed or who funded it.

And I wouldn't expect it to. They are obviously not going to go naming individual companies, since if the document leaked (as it has done) then adversaries know exactly which companies are complicit. But "Insert vulnerabilities in to commercial encryption systems, IT systems, networks, and endpoint communications devices" is pretty clear.

6 years is long enough to know if there was some affect from this supposed backdooring of "billions of devices". you would think if its happening to "billions of devices", it would be happening to all of us right now too.


well let's take a look at what intel themself says about IME:

At system initialization, the Intel® Management Engine loads its code from system flash memory. This allows the Intel® Management Engine to be up before the main operating system is started. For run-time data storage, the Intel® Management Engine has access to a protected area of system memory (in addition to a small amount of on-chip cache memory for faster and more efficient processing).

So what exactly was the result of those 32 lawsuits? https://www.theverge.com/2018/2/16/17020048/intel-spectre-meltdown-class-action-lawsuits




that document never mentions Intel though.

at any rate, i dont see intel as a bad guy and would just like to sometime testout rdseed to see how random its output looks. if it looks random enough maybe it's good enough. but at some point i would like one of those usb hardware RNGs because i think they might be above this type of criticism. but they do cost a good bit. like $60 or $70. for the TrueRNG. off amazon. but then i wouldn't have to worry about if my output was truly random or known by some 3rd party too.

You already believe in so many things in your life that you never saw with your own eyes, so I don't know why this would be any different.
It's common sense and you can ask any tech expert that understands more how to achieve random results, but if you want to continue playing this game, than go for it.

Their main competition is called AMD Platform Security Processor (PSP), but it is almost the same thing like in Intel microchips.
It's not impossible to minimize both of this processes in some machines, but this is not exactly newbie friendly task.

It's impossible to really disable this with kernel or any software patch, because this is hardware based problem.
Even with installing special BIOS version that have option to disable Intel Management, you are not really disabling anything.

Now I know that Management Engine has no reason to be used on home systems the way it is designed, but if someone really doesn't like this feature, they can run a version of the Linux kernel that has disabled vPro support (and whatever the counterpart is called in AMD). No need for strange rituals of using ancient hardware with obscure distributions like what Richard Stallman is doing.

Nothing changed since then. It's still exist on newest Intel CPU (12th gen Alder Lake). It's explicitly mentioned on their product brief[6].

---

Let's see their documentation[1].



Both of them are standard from NIST[2-3]. Based on leak by Edward Snowden, NSA influence NIST to make weaker standard[4] and it's been predicted by expert since 2007[5].

[1] https://www.intel.com/content/www/us/en/developer/articles/guide/intel-digital-random-number-generator-drng-software-implementation-guide.html
[2] https://csrc.nist.gov/publications/detail/sp/800-90a/rev-1/final
[3] https://csrc.nist.gov/publications/detail/sp/800-90b/final
[4] https://web.archive.org/web/20130910030443/http://fcw.com/Articles/2013/09/06/NSA-NIST-standards.aspx
[5] https://archive.ph/20120919094854/http://www.wired.com/politics/security/commentary/securitymatters/2007/11/securitymatters_1115
[6] https://www.intel.com/content/www/us/en/products/docs/processors/embedded/12th-gen-iot-desktop-processors-brief.html