Topic: Thoughts on type safety and crypto RNGs - page 2. (Read 3658 times)

That's an incredibly bold statement given that there's almost no-one writing Bitcoin applications in C / C++ with the exception of Bitcoin Core itself. Equally the demographics of people writing the tiny amount of C / C++ code out there is very different than the demographics writing in more modern languages.

Fact is right now we just can't say anything about what approach is better based solely on where the most bugs have been found; we can say other industries have consistently been moving away from C and to a lesser extent C++ due to the difficulty of writing secure code in those languages.

You're also conflating two separate problems. It may turn out that writing consensus-critical code in other languages is harder, but that's a very different problem than writing secure code in the more general sense. Equally it may turn out that better approaches to writing consensus-critical code are more important than what language you choose to write it in. But right now we just don't know.

That bug in that library was exemplary for the potentially disasterous consequences of a weak memory model present in C and C++. It did not put Bitcoin at risk, but it likely did if the payment protocol had been in core already. The argument that the bug was in a library is weak and applies to the RNG problem we saw with Java on Android too. We have seen a very similar bad RNG problem in Debian Linux too written in C. Errors like those are not language specific, the consequence of the hearbleed bug however was. The bug itself was not such a desaster was it not paired with a weak memory model.

Bitcoin core can not change its technology as it would likely result in a hard fork between its older and newer versions. We can't touch Satoshi's bugs and should one of the used libraries blurp up or even store some junk, chances are good that those "features" have to be preserved.

On a side chain however the technology is not set in stone. Whatever features, even bugs an other tool and library set displays there defines the consensus of that side chain.

I am using Java and more recently Scala not just because they relieve me from some drudgery, but because their do help me to create more robust and correct programs. Ignoring major advances of computer science should be well justified. I see good reasons to stick with the tool set for Bitcoin core, but not around that. Higher level interfaces and new side chains need not to use the same hammer for all nails.

Mike gave good hints for the selection of new hammers, and that's I applauded.

It was an issue in OpenSSL (bitcoind doesn't expose SSL to the public in a default, or even sane, configuration at least).  Every other language also depends on system libraries too. So the language Bitcoin core was written in was irrelevant in this example.

I'm sorry you feel that I'm nitpicking, but I'm not trying to.

So far our experience in this space is that there is more irresponsible and broken software written in higher level languages, there has been virtually no issues in this space from cryptographic weaknesses (or even conventional software security) in Bitcoin applications written in C / C++. I agree that sounds somewhat paradoxical... but it's not that shocking: The security of these systems depends on the finest details of the behaviour of each part of the software and the interactions, when your system obscures the details some extra work is required to review though the indirection. This somewhat offsets the gains. In cryptographic (and especially consensus) systems it's much harder to "fail safe" and a much wider spectrum of unexpected behaviour is actually bad and exploitable. Languages like Java make some kinds of errored software "more safe" when the software is incorret, but making software more correct is still something that is largely not reaching production industrial software development yet (languages with dependant types and facilities for formal analysis seem like they _may_ result in more correct software).  

There is no replacement for hard work and many view higher level languages as an escape from drudgery, so there may be some language selection bias from the attitude of the authors that has nothing to do with the language itself.  In any case, I think your barb was misplaced, at least in this thread: We've seen bad RNG behaviour from Java software several times, and not just in system libraries. (And not just RNG safety, also things like attempts at full node code being shattered by underlying crypto libraries bubbling up null pointer exceptions that cause false block rejections which would have created forks if it were widely used).

(I do agree though that using untyped languages is basically suicide for any, even moderately large, system where correctness matters.)

You are right in that I should not have used "low level" for but type unsafe, since there are high level unsafe languages, like Javascript or Python. Java is type safer and Scala is even better, and that is what Mike said.

Added:
Not using compile time checks that type safety gives is pure arrogance.

BTW what about the heartbleed bug in SSL was it not in Bitcoin core?

Unfortunatelly you only use your intelligence to pinpoint inaccuracy in my sentences.

Except the issues with poor cryptographic security Mike is talking about have only been observed-- so far-- in tools written in Java, Javascript, and Python in our ecosystem. None of these are low level languages.

+1

unfortunatelly most crypto developer still build on their percieved superior programming skills, instead of using modern languages.

Most exploits arise from programming errors in low level weakly typed languages and not from those exotic "timing" and "memory" attacks that they use to justify their ancient tool set.

Java allows very specific off-heap allocation on OpenJDK's VM, that allows for crypto data to live in a specific place in memory without fear of being copied by an eager GC, and to be erased from memory before deallocation. Netty also has some specific buffer types that are zero-copy for performance, that are useful even in non-network applications.

You should not do crypto in JS or Java in the first place. In those languages, you do not have control about memory management. For example in JS, you have no control over how and were the browser stores your secret data (keys etc.). There is no way to enforce the physical deletion of private data.

I wrote an article about some of the failures in wallet randomness we've seen in the past 12 months:

  https://medium.com/@octskyward/type-safety-and-rngs-40e3ec71ab3a

It's a 6 minute read, but the tl;dr summary is:

1) Find ways to make the type systems you are working with stronger, either through better tools or better languages

2) Try and get entropy as directly from the kernel as possible, bypassing userspace RNGs

I should practice what I preach - bitcoinj could be upgraded to use the Checker Framework for stricter type checking, and we currently only bypass the userspace RNG when Android is detected. I'll be looking at ways to make things stricter and more direct next year.