Thread by @ErrataRob

One thing about this bug is "why does a JSON parser need random numbers?"

The answer is that all libraries need random numbers. It's one of the things they don't teach you in computer science class.

news.ycombinator.com/item?id=264636…

While a JSON parser parses the input, it puts things into a hash table, which is very efficient ON AVERAGE. But if a hacker constructs the input in a special way, it becomes very inefficient -- essentially, it'll crash/hang.

Thus, you need to randomize where things get placed in a hash table such that a hacker can't possibly guess the order. This means grabbing a TRUELY random number from the operating system or hardware.

You can't calculate an unpredictable number yourself with pure math, because hackers can execute the same math. You must instead grab external events, such as random motion of molecules in a chip, or random arrival times of packets on the network.

Unfortunately, there is no way to do this in a standard way -- well, there is NOW, but not for older systems, so libraries do ad hoc ways to figure it out for themselves -- and often make mistakes, such as this one.

The way I do it is simply scavenge 'entropy' (true randomness) from as many sources as I can, and mix it together with a hash function. Mixing with a cryptographic hash never makes it less random.

Here is an example. I start by using SHA-2 512 as the function to mix together all the sources of entropy.
github.com/robertdavidgra…

Next I is a function that gets the highest resolution timestamp in the system. This is usually good for about 20 bits of randomization, sometimes 30 bits. If all else fails, this often will suffice for many threats.

Then I get the basic things that people have always used for randomization, the current time(0) and getpid() (process id). It's bad, very bad, but it adds a couple bits worth of randomness.

Next is the 'proper' way -- a system call. All the newer operating system (Windows, macOS, Linux, etc.) provide one that's guaranteed to be security. Unfortunately, libraries need to support older systems that don't. People deploy new CentOS boxes that don't support it

Next reads from /dev/urandom. One thing to note is that I have no platform-specific #ifdefs around it -- I attempt to open /dev/urandom on all platforms, even windows where it doesn't exist.

/dev/urandom is the canonical answer they teach you in school, but it actually fails in so many ways. For example, there's a tiny chance a signal (interrupt) may cause it to fail. The biggest reason is that it usually doesn't work in containers.

Looking over the code, I realize this file is unfinished. I am supposed to also read ASLR information: the pointers to the stack, heap, and text segments, treating them as integers, which by themselves are usually enough to make sure this is random.

All these thing can fail. You should rely upon a cryptographic library if doing something cryptographic. My goal for this code is to solve the fact that every program needs a random seed for hash tables.

https://twitter.com/Lula_Terminator/status/1371666948862443520