Booted a recent 5.x Linux kernel and found /dev/random is no longer blocking. It's only been what, 20+ years? About time.

Hi forest. Welcome back.

hai

@SqueamishOssifrage Rather than making the effort to deeply understand your argument (which seems excellent from a mile high), I removed the problematic assertion from my answer.

@forest There's some talk about that change in Linux there. Seems the intention and plan is reasonable. The meta-argument that if there was a problem in the implementation, it would have been caught by public scrutiny has failed many times, but that's the only one I have.

@fgrieu But you're still recommending a hash size that's twice as large as it needs to be (and if read as SHA3-512, more than four times the cost it needs to be). There is, at present, no credible quantum threat to collision resistance; there's no need double everything willy-nilly when the word quantum shows up: 256 bits is enough for collision resistance.

@forest It will still block sometimes! Specifically, on machines without hardware random number generators, it will block until the consecutive differences of consecutive differences of consecutive differences of interrupt timings are nonzero enough times, more or less.

@kelalaka The premise of your question about /dev/random is not accurate. Linux did not remove blocking in /dev/random. It removed the blocking pool, which is a manifestation of the cryptographically nonsensical concept of entropy depletion whereby after any 256 bits of ciphertext your key is supposed to grow stale and need refreshing, despite decades of cryptanalysis failing to justify this.

@SqueamishOssifrage yes, you are right. I'll update.

This is not to say that any blocking is right—just that your question reads like ‘Have you stopped emotionally abusing your husband?’.

Beyond that, your question is ill-posed. /dev/urandom is the operating system interface for generating key material for cryptography. As such it is, more or less by definition, the right way (or a right way) for an application to read key material for cryptography. Perhaps a more relevant question: Under what circumstances does the operating system correctly implement that interface, and what does a system engineer or system integrator need to do to make sure it works in an OS install?

That is more like my point.

(an application that asks you to wiggle the mouse is an application that should be thrown in the garbage like gpg decades ago)

However, the link says, Entropy collection is not something that the kernel can reliably do on all of the different hardware that it supports, nor can it estimate the amount of entropy provided by the different sources,

(which, by the way, gpg still does, git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=g10/…)

It is like they say, don't rely on the kernel, use user spaced entropy.

no

Veracypt does, too.

They're saying the kernel developers, writing software that has to work on a wide variety of machines, are not in the same position that a system integrator who knows everything about a specific hardware deployment. On a specific hardware deployment, it may be appropriate to draw a seed from some source there isn't currently a kernel driver for, and write it to /dev/random.

Applications should not follow gpg's astonishingly stupid lead that has excited insufferable crypto nerds and held back cryptography for decades.

You mean, the kernel developers made the means to integrate the sources into the /dev/urandom

(fyi, it's /dev/urandom, not /dev/urandom/; it's not a directory, and /dev/urandom/ will not work)

Yes, it is device, I should be more careful about that.

Writing to /dev/random has essentially always been supported as a way to contribute entropy to the pool.

For a while in Linux, device drivers for hardware random number generators (inexplicably) declined to feed entropy directly into the pool, and it was up to userland to read from /dev/hwrng and write to /dev/random. However, that changed a couple years ago.

Now, the circumstances under which a system engineer needs to take extra action are far outside the scope of crypto.se, and really the whole question is not a cryptography question at all.

I saw that when I've updated. Still considering what to do.

…and I would suggest exercising extreme caution before putting up a question that reads as hackernews-style FUD about /dev/urandom. The answer to the question in the title is an unequivocal YES; where there is uncertainty is in specific OS installations, particularly routers that don't have entropy sources—for example, an appropriate place for the discussion would be on an OpenWrt wiki page about a specific hardware device that lacks an RNG.

Effectively, the impact of asking this question is likely to be that it will become an HNQ with insufferable pontification about entropy by people who aren't competent in the field, and depress confidence in the wrong place, leading to bullshit in applications like gpg's advice to please bang on the keyboard like a monkey to make yourself feel better about cryptography.

please bang.. lol

Meanwhile the real problems with predictable keys will mostly be on noninteractive network appliances where the system integrator doesn't actually run gpg by hand so won't notice those messages.

We have seen examples in RSA keys..

So please be careful putting your name and high reputation on questions that serve only to cast doubt like ‘Is (standard common thing for applications to do) really secure?’ when the real security issues underlying it are not in the power of software application authors to address.

I just wondered, since we always say use /dev/urandom

There is a saying something like that repeating all you heard is enough for your sins.

I've deleted it.

Does the simple version is ok. or do you advice to delete it all. I've already got an answer here. Thanks.

@kelalaka When I first read your question, I thought it came from someone less experimented in cryptography than you. So my answer does not go into details and is for non-cryptographers.

That could be seen as off-topic on crypto.SE.

@A.Hersean Cryptography is a wide subject that one cannot grasp all :)

I think however it could have its place on security.SE (where I go far more).

I can ask for migrate.

As you wish. It's your question.

Or you could ask the security.SE mods for their advice.

Did you check the history of the question?

I read the first version before you deleted it. Then the new one, but I did not do a side-by-side comparison.

I read the chat above too.

@kelalaka FYI, H(H(m) || m) doesn't accomplish much: eprint.iacr.org/2019/755 (It definitely does not provide ‘n-bit security’ in an n-bit hash function.)

sigh, here we go

(I don't recommend migration to security.se—even more overconfident infosec nerds to give scatterbrained advice about keyboard monkeys and haveged and other such snake oil.)

@kelalaka Also FYI: the difference between H(H(m)) and H(H(0^b || m)) is essentially explained in the H(k || H(k || m)) post—just as H(k || 0 || H(k || 1 || m)) would thwart the random oracle distinguisher discussed there, so would H(k || H(k || 0^b || m)). (Similarly, one could do H(0 || H(1 || m)) to defeat length extension attacks on, e.g., SHA-256, without exposing yourself to the random oracle distinguisher for H(H(m)).)

@SqueamishOssifrage how we can so confident that the claims in this article are true?

In which article?

The last one, for example,..

Last one where?

eprint.iacr.org/2019/755

We have seen lots of examples of incorrect proof over time. It leaves until one notices.

Proofs can have errors, yes. If you find an error, perhaps you can show that H(H(m) || m) provides better security than the authors and editors of the Journal of Cryptology were convinced it has. But for now, this suggests (far better than any random blatherer on the internet, or on the authors of a book who only have to sound good to their publisher) there is little benefit—and substantial negative security impact—to using H(H(m) || m).

@kelalaka I just sent you a file with some random numbers.

Got it, thanks.

Man, it has 84 pages!

ugh

Please stop spreading FUD about /dev/random and /dev/urandom.

@bdegnan /dev/random and /dev/urandom are identical on macOS; whatever you did is obviously experimental error.

github.com/apple/darwin-xnu/blob/…

@kelalaka Please don't amplify FUD arising out of experimental error like that.

Deleted.

@bdegnan What are you trying to compute/test there? Can you phrase it as a test about probability distributions, in more meaningful language than ‘how random /dev/random is’?

@SqueamishOssifrage They are now. My initial random work was from 2003. I need to update this, I have a whole toolkit relating to it that I need to write up. I have a whole toolkit that I used to determine random sources, but I've never gotten around to releasing it. I need to a grant to just clean up my shelves and release all of my tools :/

I can prove in semiconductors, from 1st principles, that the channel "noise" is not two-way shot noise, and not thermal noise. People lose their shirts when you do the proof because it's inconvenient. You can make random number generators are provable due to the mean-time to collisions. Anyway, that stuff from initially from what work.

They have been the same in macOS since no later than 2001 or so.

I will have to check back in my Darwin code from 10.3. I don't remember when Yarrow was put in to liven things up either.

Also, i still write all of my papers on my Pismo with 10.3, so I probably can check.

@bdegnan opensource.apple.com/source/xnu/xnu-201/bsd/dev/random/…, from 10.1

Same random_cdevsw for both minor numbers. random_read doesn't even look at which device it is.

There is it, so it's the same node. I stand correctd

(your blog post says Darwin 15.5.0, which is much more recent)

@bdegnan So what are you trying to test empirically with these samples?

oh, so I use something called VPR to route graphs for the FPGAs, and sometimes it gets weird. I was trying to sort that out. I write stuff down for me and I do not expect anyone to actually ever read it.

I have no problem just deleting it if gives you heart burn.

I'm not saying that you haven't found an anomaly worth pursuing here. I'm saying I can't even tell what question you were trying to ask of the data, so the glib conclusion is unverifiable. And you appear to have have three incommensurate things—data, correlation coefficient, and p-value—plotted on the same axes.

<shrug> I'll revisit it when I can get a grant to do FPGAs again.

What does it have to do with FPGAs? You appear to have been trying to ask a simple binary hypothesis test of a sample, under the premise of some null hypothesis (and, as is typical in frequentist analyses, no clearly spelled-out alternative hypothesis), but that null hypothesis is implicit in the Matlab code you fed it into, and not clarified by the plots.

That's why I'm asking: What is the question you were asking of the sample you took?

oh, so I get a series of random number to "walk the graph" on the VPR tools because it's heuristic. The quality of the route depends on the numbers, and the "path" was routing to a spiral on MacOS but not on Solaris. I was trying understand the difference between the MacOS random and the Solaris one created from a nuclear source.

So... that's why FPGAs.

I have to dig into the code to remember the details. I tend to forget things that I do not find important. Documentation is my method of remembering things.

@kelalaka OK, that's a fair criticism—you can't actually feed a ~2^64-byte message through MD5, because you have to compute it sequentially. (I haven't looked into the details of the attack; unclear whether it can be, say, parallelized on shorter messages.)

people are still doing attacks on MD5? I figured people would look at what was actually being used.

@bdegnan There is no way that the inputs into the OS entropy pool (whether from nuclear decay or from thermal noise or from not-really-thermal not-really-noise) or even blocking vs. not blocking in /dev/random vs. /dev/urandom could on its own cause that. That sounds much more likely to be a garden-variety bug that happened to manifest on macOS but not Solaris, e.g. perhaps differences in libm.

@bdegnan MD5 was an example for this particular attack on the generic structure of H(H(m) || m); see eprint.iacr.org/2019/755 for details of the generic attack on that.

(I brought it up as a response to crypto.stackexchange.com/a/84483)

ah...

I need to read a few dozen cryptography books. I can keep up with the abstract algebra, but not really the attacks and practical aspects. :/

You know Squeamish, I can only assume that you are a researcher in the field of cryptography due to the depth of interest and knowledge. It's hard to keep up as someone who's just in the peanut gallery

I'd love to just get some time to read papers that aren't the ones that I have to read. :/

@SqueamishOssifrage I'm reading and don't know when I can finish it :)