randombytes() und /dev/urandom

[Frank Hartmann]

Hi,

I have a question how to consume on linux systems the random provided
by /dev/urandom correctly. As background: I tried to integrate tweetnacl
into libzmq, copied randombytes() code from NaCL and still feel
uncertain.

see https://github.com/zeromq/libzmq/commit/e5a294ec023448450b082306b64c204d04543594

So my question is: What are in such a context the rules for using
/dev/urandom correctly:

* Is it ok reading /dev/urandom from multiple threads using the same
file descriptor?

* Is it ok reading /dev/urandom from multiple threads using one file
descriptor per thread?

* Is it ok reading /dev/urandom and launching later a child process?
Somehow child processes inherit file descriptors. Do I need to
prevent this? Is it essential for correct usage?

kind regards
Frank

[Greg Price]

Good question to ask. All of these are fine. The promise /dev/urandom makes is that, once it's properly initialized (*), every byte you read from it is cryptographically indistinguishable from a uniform random byte, uniformly random independent of every other byte ever read from it.

Even in a scenario where the cryptography of /dev/urandom is broken (and planning for that case in application code would be a rabbit hole anyway), in the way the implementation works it doesn't matter at all whether different reads are from the same file descriptor, the same process, or anything like that; they all execute in sequence against the same data structure.

One thing that does need care is any buffering you do in user space of the results of reading /dev/urandom. If you have a buffer and then you fork and use the same buffer contents in parent and child or multiple children, or you use it from multiple threads, that's obviously bad. The most robust solution is to not try to buffer, and just do a read() every time you need more random bytes. At a quick glance that looks to be what you're doing already.

Cheers,

Greg

(*) This caveat is a flaw which, IIUC, Ted intends to fix. At present /dev/urandom should always be properly initialized except for a short period at first boot. See https://factorable.net/weakkeys12.extended.pdf for what can happen in that period; note Ted has already made changes since that paper to shorten the window where it's not yet properly initialized.

  Frank

--
You received this message because you are subscribed to the Google Groups "Randomness generation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to randomness-gener...@googlegroups.com.
To post to this group, send email to randomness...@googlegroups.com.
Visit this group at http://groups.google.com/group/randomness-generation.
For more options, visit https://groups.google.com/d/optout.

[D. J. Bernstein]

You need to call NaCl's randombytes() before starting threads. The
underlying issues are that

* opening /dev/urandom is slow and
* opening /dev/urandom can fail (with, e.g., EMFILE).

NaCl handles these issues by opening /dev/urandom just once, pausing and
retrying indefinitely in case of failures, and putting the resulting
file descriptor into a static variable. Subsequent use is thread-safe.

An os_randombytes(buf,len) syscall, after #include "os_randombytes.h",
would eliminate these issues.

---Dan

[travis+ml-rando...@subspacefield.org]

Sent-To:

It'd add to the kernel ABI, but possibly also allow intelligent
operation in VMs.
--
http://www.subspacefield.org/~travis/
I'm feeling a little uncertain about this random generator of numbers.

[Frank Hartmann]

Greg Price <gnp...@gmail.com> writes:

> Good question to ask. All of these are fine. The promise /dev/urandom makes
> is that, once it's properly initialized (*), every byte you read from it is
> cryptographically indistinguishable from a uniform random byte, uniformly
> random independent of every other byte ever read from it.

Greg thank you for the explanation and links for further reading.

Do you know know why there some special precautions in various
implementations:

NaCL:

http://sources.debian.net/src/nacl/20110221-4.1/randombytes/devurandom.c?hl=23#L22

reads are "chunked" into blocks of 1048576 bytes. Why? An then sleeping sometimes?


Sodium:

libsodium seems to require sometime another random distribution and offers:


uint32_t randombytes_sysrandom_uniform(const uint32_t upper_bound)

https://github.com/jedisct1/libsodium/blob/master/src/libsodium/randombytes/sysrandom/randombytes_sysrandom.c#L227


which does some shaping. Addionally it seems to check for other error
possiblities (S_ISCHR,errno == EINTR) compared to the NaCL implementation while opening /dev/urandom.
And no sleeps?


I would have expected a blocking read of /dev/random to fill the gap until
/dev/urandom is ready, but both implementations do not provide
it. Thread safety does not seem to be a concern either?

Is there an explanation known for these precautions and design decisions?

kind regards
Frank

[Greg Price]

Hi Frank,

On Wed, May 7, 2014 at 12:48 PM, Frank Hartmann <soun...@gmx.net> wrote:

NaCL:

http://sources.debian.net/src/nacl/20110221-4.1/randombytes/devurandom.c?hl=23#L22

reads are "chunked" into blocks of 1048576 bytes. Why? An then sleeping sometimes?

Dan is on the list and can speak for himself, but to my eye this isn't addressing a security issue. It makes sure the program doesn't try to make a single system call that's going to be busy for a very long time, which could cause practical annoyances. I'm not sure of a concrete scenario where it'd be a problem; Dan might know of something or may have just been cautious.

Sodium:

libsodium seems to require sometime another random distribution and offers:

uint32_t randombytes_sysrandom_uniform(const uint32_t upper_bound)

https://github.com/jedisct1/libsodium/blob/master/src/libsodium/randombytes/sysrandom/randombytes_sysrandom.c#L227

which does some shaping.

This is just a convenience function to do some arithmetic with a random word. The random bytes that make up that word come in the same fashion as all the other random bytes, as they should.

Addionally it seems to check for other error
possiblities (S_ISCHR

This checks that /dev/urandom is a character device. That seems like useless paranoia to me -- if your system's configuration is so messed up that /dev/urandom is something else (like a regular file), I don't see why anyone should think that checking that file type is going to help. /dev/urandom could be an alias of /dev/zero just as easily as it could be a regular file; and if some attacker has deliberately messed with it, it will almost certainly be more insidious than that.

 

,errno == EINTR) compared to the NaCL implementation while opening /dev/urandom.

And no sleeps?

 

This is a matter of general Unix systems programming, not specific to /dev/urandom, crypto, or security. This is a good explanation of EINTR which you might find helpful: http://250bpm.com/blog:12

Note that errno == EINTR is actually the case where libsodium does nearly the same thing as NaCl. They behave differently on other errors, but EINTR is the only one that should happen here anyway.

I would have expected a blocking read of /dev/random to fill the gap until
/dev/urandom is ready, but both implementations do not provide
it.

I don't know of a good way to do this, unfortunately.

 

Thread safety does not seem to be a concern either?

See Dan's response to you. NaCl's randombytes() is thread-safe as long as you call it once before you start threads. (Even if you fail to do that, the worst that happens is that you leak some file descriptors.) libsodium appears to have followed the same strategy.

Greg

[Frank Hartmann]

Greg,

thanks again for your explanations.

Your reference to "Dans response" made me wonder as I did receive
uptonow only your two emails.

If I look in a browser in
https://groups.google.com/forum/#!topic/randomness-generation/

I see indeed more answers.

It seems my subscription did not work, despite the friendly "You have
joined the group randomness...@googlegroups.com" reply I
received.

regards
Frank