info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
In need of "good enough" randomness sources, anyone wanna help out? (OpenSSL 1.1.1 beta)
480 views
Skip to first unread message
Richard Levitte
Apr 3, 2018, 12:00:36 PM4/3/18
to
[disclaimer: this discusses pseudo-random numbers, not true randomness, so any time you see "random" written, think "pseudo-random". Bill Gunshannon, please pay attention and don't derail]
Hey,
The randomness machinery in OpenSSL has changed radically for upcoming version 1.1.1. The implementation is now a block cipher DRBG as specified in https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf, based on AES-256-CTR.
To get some kind of seeding, this implementation does like before, and relies on data from an entropy aqcuisition function (see below). It worked, by delivering just enough entropy (64 bytes at assumed 4 bits of entropy per byte => 256 bits, which is exactly enough for AES-256-CTR).
Now, we've run into a bit of trouble because of a very recent change that adds the possibility of a nonce to the pool. When no nonce is given, but there's still a nonce size specified, and I quote:
/*
* NIST SP800-90Ar1 section 9.1 says you can combine getting the entropy
* and nonce in 1 call by increasing the entropy with 50% and increasing
* the minimum length to accomadate the length of the nonce.
* We do this in case a nonce is require and get_nonce is NULL.
*/
So we suddenly find ourselves in a situation where 256 bits of entropy isn't enough, the requirement has been elevated to 384 bits (96 bytes à 4 bits of entropy per byte).
The VMS entropy acquisition function can be seen here:
https://github.com/openssl/openssl/blob/master/crypto/rand/rand_vms.c
It relies almost entirely on the data returned by sys$getjpiw for these item:
JPI$_BUFIO
JPI$_CPUTIM
JPI$_DIRIO
JPI$_IMAGECOUNT
JPI$_LAST_LOGIN_I
JPI$_LOGINTIM
JPI$_PAGEFLTS
JPI$_PID
JPI$_PPGCNT
JPI$_WSPEAK
JPI$_FINALEXC
So I'm looking for ideas, are there other items you'd recommend? Other sources? I've eyed through the JPI$_ macros, but am frankly getting a bit blurry eyed and don't feel I quite have a grasp of how variable the data for each item is (i.e. how good entropy it can give).
Cheers,
Richard
Hey,
The randomness machinery in OpenSSL has changed radically for upcoming version 1.1.1. The implementation is now a block cipher DRBG as specified in https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf, based on AES-256-CTR.
To get some kind of seeding, this implementation does like before, and relies on data from an entropy aqcuisition function (see below). It worked, by delivering just enough entropy (64 bytes at assumed 4 bits of entropy per byte => 256 bits, which is exactly enough for AES-256-CTR).
Now, we've run into a bit of trouble because of a very recent change that adds the possibility of a nonce to the pool. When no nonce is given, but there's still a nonce size specified, and I quote:
/*
* NIST SP800-90Ar1 section 9.1 says you can combine getting the entropy
* and nonce in 1 call by increasing the entropy with 50% and increasing
* the minimum length to accomadate the length of the nonce.
* We do this in case a nonce is require and get_nonce is NULL.
*/
So we suddenly find ourselves in a situation where 256 bits of entropy isn't enough, the requirement has been elevated to 384 bits (96 bytes à 4 bits of entropy per byte).
The VMS entropy acquisition function can be seen here:
https://github.com/openssl/openssl/blob/master/crypto/rand/rand_vms.c
It relies almost entirely on the data returned by sys$getjpiw for these item:
JPI$_BUFIO
JPI$_CPUTIM
JPI$_DIRIO
JPI$_IMAGECOUNT
JPI$_LAST_LOGIN_I
JPI$_LOGINTIM
JPI$_PAGEFLTS
JPI$_PID
JPI$_PPGCNT
JPI$_WSPEAK
JPI$_FINALEXC
So I'm looking for ideas, are there other items you'd recommend? Other sources? I've eyed through the JPI$_ macros, but am frankly getting a bit blurry eyed and don't feel I quite have a grasp of how variable the data for each item is (i.e. how good entropy it can give).
Cheers,
Richard
abrsvc
Apr 3, 2018, 1:06:19 PM4/3/18
to
How about using a system call to get a system resource count? For example, the number of disk operations for the system disk or the total number of pagefaults system wide. This would vary based not only by time but also from the level of activity on the system. This should be sufficiently "unpredictable" to qualify. Even something as simple as the freelist size or the modify page count would work although the variance would be less for those values.
If you want to avoid increasing values, perhaps the current pagefault rate or a current disk read rate? Again, the numeric range is limited, but should be relatively "random" based upon system activity.
Along a similar line, look at network connection read rates, packet counts etc.
Dan
If you want to avoid increasing values, perhaps the current pagefault rate or a current disk read rate? Again, the numeric range is limited, but should be relatively "random" based upon system activity.
Along a similar line, look at network connection read rates, packet counts etc.
Dan
Simon Clubley
Apr 3, 2018, 1:36:00 PM4/3/18
to
On 2018-04-03, abrsvc <dansabr...@yahoo.com> wrote:
> How about using a system call to get a system resource count? For
> example, the number of disk operations for the system disk or the total
> number of pagefaults system wide. This would vary based not only by time
> but also from the level of activity on the system. This should be
> sufficiently "unpredictable" to qualify. Even something as simple as the
> freelist size or the modify page count would work although the variance
> would be less for those values.
>
You should also be able to get a few bits out of the free block count on
> How about using a system call to get a system resource count? For
> example, the number of disk operations for the system disk or the total
> number of pagefaults system wide. This would vary based not only by time
> but also from the level of activity on the system. This should be
> sufficiently "unpredictable" to qualify. Even something as simple as the
> freelist size or the modify page count would work although the variance
> would be less for those values.
>
the system disk as well as the CPU used times for processes which are
_guaranteed_ to be around (such as the job controller).
> If you want to avoid increasing values, perhaps the current pagefault
> rate or a current disk read rate? Again, the numeric range is limited, but
> should be relatively "random" based upon system activity.
>
> Along a similar line, look at network connection read rates, packet
> counts etc.
>
which is guaranteed to be network stack neutral. You need to use sources
which are guaranteed to exist on all VMS systems which is why I only
mentioned the system disk above.
Simon.
--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world
abrsvc
Apr 3, 2018, 1:40:23 PM4/3/18
to
I agree with the network devices. I should have been more specific and mentioned the actual network device (perhaps at the hardware level) which by definition would be stack independent.
Dan
Craig A. Berry
Apr 3, 2018, 2:46:45 PM4/3/18
to
There is sys$crete_uid, which is supposed to give you 16 bytes of stuff you'll never get any other way. But it's not available on ancient versions of VMS and I don't know how it's implemented, so its sources of randomness could overlap some of the other things being used.
Richard Levitte
Apr 3, 2018, 3:08:05 PM4/3/18
to
Note: ancient VMS versions aren't relevant, we don't support anything older than CRTL 7.1.
Craig A. Berry
Apr 3, 2018, 3:09:45 PM4/3/18
to
On Tuesday, April 3, 2018 at 1:46:45 PM UTC-5, Craig A. Berry wrote:
> There is sys$crete_uid,
Bah. "create" not "crete".
> There is sys$crete_uid,
Bah. "create" not "crete".
Steven Schweda
Apr 3, 2018, 3:13:58 PM4/3/18
to
> [...] the number of disk operations [...]
> [...] network devices [...]
In general, I'd look at DVI$_* as well as JPI$_*. Disks,
network interfaces, whatever. SYI$_*?
> [...] network devices [...]
In general, I'd look at DVI$_* as well as JPI$_*. Disks,
network interfaces, whatever. SYI$_*?
Craig A. Berry
Apr 3, 2018, 3:26:37 PM4/3/18
to
On Tuesday, April 3, 2018 at 2:08:05 PM UTC-5, Richard Levitte wrote:
> Note: ancient VMS versions aren't relevant, we don't support anything older than CRTL 7.1.
Which is ancient enough. I don't think sys$create_uid ever existed on VAX and was new in 7.3-1.
> Note: ancient VMS versions aren't relevant, we don't support anything older than CRTL 7.1.
Arne Vajhøj
Apr 3, 2018, 5:33:35 PM4/3/18
to
$GETDVIW DEVNAM=SYS$DISK or DEVNAM=SYS$SYSDEVICE DVI$_*
$GETSYIW SYI$_*
$GETTIM
concat it all and do two SHA2-512.
Arne
Jan-Erik Söderholm
Apr 3, 2018, 6:56:00 PM4/3/18
to
supports UUID version 1. That is the version that have parts locked to
some local system parametes like MAC address and such. That is fine if
the only thing you want is to have a "value" that noone else has.
To get a full 16 byte random value (apart from the 3 or 4 bit field that
has the UUID version no) you need support for UUID version 4.
Arne Vajhøj
Apr 3, 2018, 8:37:41 PM4/3/18
to
On 4/3/2018 6:55 PM, Jan-Erik Söderholm wrote:
> I'm sure you know that the VMS implementation to create an UUID/GUID only
> supports UUID version 1. That is the version that have parts locked to
> some local system parametes like MAC address and such. That is fine if
> the only thing you want is to have a "value" that noone else has.
>
> To get a full 16 byte random value (apart from the 3 or 4 bit field that
> has the UUID version no) you need support for UUID version 4.
Note that RFC 4122 states:
> I'm sure you know that the VMS implementation to create an UUID/GUID only
> supports UUID version 1. That is the version that have parts locked to
> some local system parametes like MAC address and such. That is fine if
> the only thing you want is to have a "value" that noone else has.
>
> To get a full 16 byte random value (apart from the 3 or 4 bit field that
> has the UUID version no) you need support for UUID version 4.
<quote>
Do not assume that UUIDs are hard to guess; they should not be used
as security capabilities (identifiers whose mere possession grants
access), for example. A predictable random number source will
exacerbate the situation.
</quote>
Arne
Stephen Hoffman
Apr 4, 2018, 9:49:24 AM4/4/18
to
On 2018-04-03 16:00:34 +0000, Richard Levitte said:
> The randomness machinery in OpenSSL has changed radically for upcoming
> version 1.1.1. The implementation is now a block cipher DRBG as
> specified in
> https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf,
> based on AES-256-CTR.
> ...
> The randomness machinery in OpenSSL has changed radically for upcoming
> version 1.1.1. The implementation is now a block cipher DRBG as
> specified in
> https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf,
> based on AES-256-CTR.
> So I'm looking for ideas, are there other items you'd recommend? Other
> sources? I've eyed through the JPI$_ macros, but am frankly getting a
> bit blurry eyed and don't feel I quite have a grasp of how variable the
> data for each item is (i.e. how good entropy it can give).
Here's an interesting approach for collecting entropy based on the
> sources? I've eyed through the JPI$_ macros, but am frankly getting a
> bit blurry eyed and don't feel I quite have a grasp of how variable the
> data for each item is (i.e. how good entropy it can give).
complexity of current software designs:
http://www.acclab.helsinki.fi/~knordlun/mc/mc4nc.pdf
https://www.irisa.fr/caps/projects/hipsor/misc.php
https://www.irisa.fr/caps/projects/hipsor/old/HAVEGE1.0.html
Toss some thousands of various system calls in a loop, and you'll get
different values for the processing.
On OpenVMS, the process and system cycle counters are available for
these sorts of shenanigans.
OpenVMS hides its collection of correctable memory errors, which is
unfortunate for this application.
Networking activity is another potential source of entropy, as has been
mentioned.
Does seem there's a missing feature here, though. "The most secure
operating system on the planet" should not force apps to collect their
own entropy, after all.
--
Pure Personal Opinion | HoffmanLabs LLC
dgordo...@gmail.com
Apr 4, 2018, 12:44:33 PM4/4/18
to
The OS uses select system counters in some places as partial entropy sources:
pms$gl_lognam,
pms$gl_npagdynreq,
pms$gl_faults,
pms$gl_pagdynreq
Richard Levitte
Apr 4, 2018, 4:51:24 PM4/4/18
to
This is something I've never played with, and while I understand what this is about, I do not know how to get to those counters. Of course, should anyone like to lead me down this path, I'm all ears!
Cheers,
Richard
Richard Levitte
Apr 4, 2018, 4:53:55 PM4/4/18
to
Den onsdag 4 april 2018 kl. 15:49:24 UTC+2 skrev Stephen Hoffman:
> On 2018-04-03 16:00:34 +0000, Richard Levitte said:
>
> > The randomness machinery in OpenSSL has changed radically for upcoming
> > version 1.1.1. The implementation is now a block cipher DRBG as
> > specified in
> > https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf,
> > based on AES-256-CTR.
> > ...
> > So I'm looking for ideas, are there other items you'd recommend? Other
> > sources? I've eyed through the JPI$_ macros, but am frankly getting a
> > bit blurry eyed and don't feel I quite have a grasp of how variable the
> > data for each item is (i.e. how good entropy it can give).
>
> Here's an interesting approach for collecting entropy based on the
> complexity of current software designs:
>
> http://www.acclab.helsinki.fi/~knordlun/mc/mc4nc.pdf
> https://www.irisa.fr/caps/projects/hipsor/misc.php
> https://www.irisa.fr/caps/projects/hipsor/old/HAVEGE1.0.html
>
> Toss some thousands of various system calls in a loop, and you'll get
> different values for the processing.
Certainly interesting!
> On 2018-04-03 16:00:34 +0000, Richard Levitte said:
>
> > The randomness machinery in OpenSSL has changed radically for upcoming
> > version 1.1.1. The implementation is now a block cipher DRBG as
> > specified in
> > https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf,
> > based on AES-256-CTR.
> > ...
> > So I'm looking for ideas, are there other items you'd recommend? Other
> > sources? I've eyed through the JPI$_ macros, but am frankly getting a
> > bit blurry eyed and don't feel I quite have a grasp of how variable the
> > data for each item is (i.e. how good entropy it can give).
>
> Here's an interesting approach for collecting entropy based on the
> complexity of current software designs:
>
> http://www.acclab.helsinki.fi/~knordlun/mc/mc4nc.pdf
> https://www.irisa.fr/caps/projects/hipsor/misc.php
> https://www.irisa.fr/caps/projects/hipsor/old/HAVEGE1.0.html
>
> Toss some thousands of various system calls in a loop, and you'll get
> different values for the processing.
> Does seem there's a missing feature here, though. "The most secure
> operating system on the planet" should not force apps to collect their
> own entropy, after all.
Cheers,
Richard
Richard Levitte
Apr 4, 2018, 5:02:47 PM4/4/18
to
It's probably better to add just the few that have a chance of actually having changed enough between reseeds.
Cheers,
Richard
Simon Clubley
Apr 4, 2018, 5:15:59 PM4/4/18
to
what header file these are in, but the following works for me if you
want to play with them until someone tells you what header to include:
------------------------------------------------------------------------
$ cc /warn=warn=all richard.c
$ link/sysexe richard
$ r richard
pms$gl_lognam = 60352
pms$gl_npagdynreq = 2809
pms$gl_faults = 31807
pms$gl_pagdynreq = 2410
$ r richard
pms$gl_lognam = 60516
pms$gl_npagdynreq = 2809
pms$gl_faults = 31874
pms$gl_pagdynreq = 2410
$ type richard.c
#include <stdio.h>
extern unsigned long int pms$gl_lognam;
extern unsigned long int pms$gl_npagdynreq;
extern unsigned long int pms$gl_faults;
extern unsigned long int pms$gl_pagdynreq;
int main(int argc, char *argv[])
{
printf("pms$gl_lognam = %lu\n", pms$gl_lognam);
printf("pms$gl_npagdynreq = %lu\n", pms$gl_npagdynreq);
printf("pms$gl_faults = %lu\n", pms$gl_faults);
printf("pms$gl_pagdynreq = %lu\n", pms$gl_pagdynreq);
return(1);
}
$
------------------------------------------------------------------------
Richard Levitte
Apr 4, 2018, 5:24:05 PM4/4/18
to
Den onsdag 4 april 2018 kl. 23:15:59 UTC+2 skrev Simon Clubley:
> On 2018-04-04, Richard Levitte <ric...@levitte.org> wrote:
> > Den onsdag 4 april 2018 kl. 18:44:33 UTC+2 skrev dgordo...@gmail.com:
> >> On Tuesday, April 3, 2018 at 3:26:37 PM UTC-4, Craig A. Berry wrote:
> >> > On Tuesday, April 3, 2018 at 2:08:05 PM UTC-5, Richard Levitte wrote:
> >> > > Note: ancient VMS versions aren't relevant, we don't support anything older than CRTL 7.1.
> >> >
> >> > Which is ancient enough. I don't think sys$create_uid ever existed on VAX and was new in 7.3-1.
> >>
> >> SYS$CREATE_UID is a *very* poor source of entropy.
> >>
> >> The OS uses select system counters in some places as partial entropy sources:
> >>
> >> pms$gl_lognam,
> >> pms$gl_npagdynreq,
> >> pms$gl_faults,
> >> pms$gl_pagdynreq
> >
> > And this is when I feel like a complete rookie (and probably am).
> > This is something I've never played with, and while I understand what this is about, I do not know how to get to those counters. Of course, should anyone like to lead me down this path, I'm all ears!
> >
>
> I didn't know about these until I read Doug's message so I don't know
> what header file these are in, but the following works for me if you
> want to play with them until someone tells you what header to include:
>
> ------------------------------------------------------------------------
> $ cc /warn=warn=all richard.c
> $ link/sysexe richard
[...]
> On 2018-04-04, Richard Levitte <ric...@levitte.org> wrote:
> > Den onsdag 4 april 2018 kl. 18:44:33 UTC+2 skrev dgordo...@gmail.com:
> >> On Tuesday, April 3, 2018 at 3:26:37 PM UTC-4, Craig A. Berry wrote:
> >> > On Tuesday, April 3, 2018 at 2:08:05 PM UTC-5, Richard Levitte wrote:
> >> > > Note: ancient VMS versions aren't relevant, we don't support anything older than CRTL 7.1.
> >> >
> >> > Which is ancient enough. I don't think sys$create_uid ever existed on VAX and was new in 7.3-1.
> >>
> >> SYS$CREATE_UID is a *very* poor source of entropy.
> >>
> >> The OS uses select system counters in some places as partial entropy sources:
> >>
> >> pms$gl_lognam,
> >> pms$gl_npagdynreq,
> >> pms$gl_faults,
> >> pms$gl_pagdynreq
> >
> > And this is when I feel like a complete rookie (and probably am).
> > This is something I've never played with, and while I understand what this is about, I do not know how to get to those counters. Of course, should anyone like to lead me down this path, I'm all ears!
> >
>
> I didn't know about these until I read Doug's message so I don't know
> what header file these are in, but the following works for me if you
> want to play with them until someone tells you what header to include:
>
> ------------------------------------------------------------------------
> $ cc /warn=warn=all richard.c
> $ link/sysexe richard
That was almost too easy, I was expecting some darker magic ;-)
(I was actually kinda sorta aware of /sysexe, but had never used it)
Thanks!
Cheers,
Richard
Craig A. Berry
Apr 4, 2018, 5:32:34 PM4/4/18
to
Simon Clubley
Apr 4, 2018, 5:42:42 PM4/4/18
to
On 2018-04-04, Craig A. Berry <craig....@gmail.com> wrote:
> On Wednesday, April 4, 2018 at 4:24:05 PM UTC-5, Richard Levitte wrote:
>> Den onsdag 4 april 2018 kl. 23:15:59 UTC+2 skrev Simon Clubley:
>> > $ cc /warn=warn=all richard.c
>> > $ link/sysexe richard
>> [...]
>>
>> That was almost too easy, I was expecting some darker magic ;-)
>> (I was actually kinda sorta aware of /sysexe, but had never used it)
You are welcome; I'll introduce you to link/system sometime. :-)
> On Wednesday, April 4, 2018 at 4:24:05 PM UTC-5, Richard Levitte wrote:
>> Den onsdag 4 april 2018 kl. 23:15:59 UTC+2 skrev Simon Clubley:
>> > $ cc /warn=warn=all richard.c
>> > $ link/sysexe richard
>> [...]
>>
>> That was almost too easy, I was expecting some darker magic ;-)
>> (I was actually kinda sorta aware of /sysexe, but had never used it)
On a more serious note, I have not been able to find the header
definition for these fields so I must be looking in the wrong place.
The nearest I have got on Alpha V8.4 is in this file:
$ lib/extract=ewdatadef/output=ewdatadef.h sys$library:sys$lib_c.tlb
but I couldn't see anything with the above names.
>
> Some of these counters or similar are probably available via SYS$GETRMI
> if you want to avoid having to link /sysexe. I'd be surprised if these
> things can be peeked at directly without privileges but there are a lot of
> things I don't know.
my NOPRIV test account (on Alpha V8.4).
Jim
Apr 4, 2018, 5:46:40 PM4/4/18
to
http://www.eight-cubed.com/examples/framework.php?file=sys_getrmi.c
examining either rmidef.h or googling should reveal additional items to return.
Robert A. Brooks
Apr 4, 2018, 6:01:18 PM4/4/18
to
On 4/4/2018 5:42 PM, Simon Clubley wrote:
> On 2018-04-04, Craig A. Berry <craig....@gmail.com> wrote:
>> On Wednesday, April 4, 2018 at 4:24:05 PM UTC-5, Richard Levitte wrote:
>>> Den onsdag 4 april 2018 kl. 23:15:59 UTC+2 skrev Simon Clubley:
>>>> $ cc /warn=warn=all richard.c
>>>> $ link/sysexe richard
>>> [...]
>>>
>>> That was almost too easy, I was expecting some darker magic ;-)
>>> (I was actually kinda sorta aware of /sysexe, but had never used it)
>
> You are welcome; I'll introduce you to link/system sometime. :-)
>
> On a more serious note, I have not been able to find the header
> definition for these fields so I must be looking in the wrong place.
There is no publicly available definition for the cells that are part of
> On 2018-04-04, Craig A. Berry <craig....@gmail.com> wrote:
>> On Wednesday, April 4, 2018 at 4:24:05 PM UTC-5, Richard Levitte wrote:
>>> Den onsdag 4 april 2018 kl. 23:15:59 UTC+2 skrev Simon Clubley:
>>>> $ cc /warn=warn=all richard.c
>>>> $ link/sysexe richard
>>> [...]
>>>
>>> That was almost too easy, I was expecting some darker magic ;-)
>>> (I was actually kinda sorta aware of /sysexe, but had never used it)
>
> You are welcome; I'll introduce you to link/system sometime. :-)
>
> On a more serious note, I have not been able to find the header
> definition for these fields so I must be looking in the wrong place.
the kernel. It just so happens that those are protected as kernel-write
but user-read, which is why a user-mode program can read them.
Their location can vary from VMS version to VMS version, which is why you need
to link /SYSEXE; those vectors are defined in sys$loadable_images:sys$base_image.exe.
$ help link /sysexe
LINK
/SYSEXE
I64 and Alpha systems.
Directs the linker to process the system shareable image,
SYS$BASE_IMAGE.EXE, in a link operation. The linker looks for
SYS$BASE_IMAGE.EXE in the directory pointed to by the logical
name IA64$LOADABLE_IMAGES (I64) or ALPHA$LOADABLE_IMAGES (Alpha).
--
-- Rob
dgordo...@gmail.com
Apr 4, 2018, 6:21:04 PM4/4/18
to
On Wednesday, April 4, 2018 at 5:32:34 PM UTC-4, Craig A. Berry wrote:
>
> Some of these counters or similar are probably available via SYS$GETRMI if you want to avoid having to link /sysexe. I'd be surprised if these things can be peeked at directly without privileges but there are a lot of things I don't know.
$GETRMI offers lots of interesting counters to play with. It's even documented in the V8.4 (and earlier) version of the HP OpenVMS System Services Reference Manual.
>
> Some of these counters or similar are probably available via SYS$GETRMI if you want to avoid having to link /sysexe. I'd be surprised if these things can be peeked at directly without privileges but there are a lot of things I don't know.
As an OS person, I just link against the base image.
Arne Vajhøj
Apr 4, 2018, 6:48:15 PM4/4/18
to
But I think that will require relinking at VMS upgrade, which
I do not like as a concept.
Arne
Arne Vajhøj
Apr 4, 2018, 7:44:12 PM4/4/18
to
Not changing between reseeds: yes in that case you only need those that
change.
Protect against outsiders brute forcing: the values that do not change
frequently still provide value because their values can not be
easily guessed.
And the hashing ensures that all bits change at reseed even with several
values not changing.
Well - it is just some ideas. As they say: free advice is worth
every cent you pay. :-)
Arne
Richard Levitte
Apr 5, 2018, 2:20:41 AM4/5/18
to
But it still gave me a lead on stuff to look for :-)
Cheers,
Richard
Richard Levitte
Apr 5, 2018, 3:37:49 AM4/5/18
to
Den torsdag 5 april 2018 kl. 01:44:12 UTC+2 skrev Arne Vajhøj:
> On 4/4/2018 5:02 PM, Richard Levitte wrote:
> > Den tisdag 3 april 2018 kl. 23:33:35 UTC+2 skrev Arne Vajhøj:
> >> $GETJPIW PID=0 JPI$_*
> >> $GETDVIW DEVNAM=SYS$DISK or DEVNAM=SYS$SYSDEVICE DVI$_*
> >> $GETSYIW SYI$_*
> >> $GETTIM
> >>
> >> concat it all and do two SHA2-512.
> >
> > Mm, nah... I suspect the majority of those numbers are static or
> > near static, which only results in very little worth of entropy per byte of data.
> >
> > It's probably better to add just the few that have a chance of actually having changed enough between reseeds.
>
> That depends a little bit on what issue you want to protect against.
>
> Not changing between reseeds: yes in that case you only need those that
> change.
>
> Protect against outsiders brute forcing: the values that do not change
> frequently still provide value because their values can not be
> easily guessed.
Well, the DRBG depends on a seed with good "strength" (measured in bits of entropy, and never changing bits don't add much value in this regard, right?). It is up the the seed generator (i.e. what I'm dealing with) to estimate how much entropy a given set of bytes contains. Right now, with those JPI$_ items I mentioned, it's estimated that they give 4 bits of entropy per byte (which I suspect is highly over-valued on a quiet machine, specifically because of bits that never change over time).
> On 4/4/2018 5:02 PM, Richard Levitte wrote:
> > Den tisdag 3 april 2018 kl. 23:33:35 UTC+2 skrev Arne Vajhøj:
> >> $GETJPIW PID=0 JPI$_*
> >> $GETDVIW DEVNAM=SYS$DISK or DEVNAM=SYS$SYSDEVICE DVI$_*
> >> $GETSYIW SYI$_*
> >> $GETTIM
> >>
> >> concat it all and do two SHA2-512.
> >
> > Mm, nah... I suspect the majority of those numbers are static or
> > near static, which only results in very little worth of entropy per byte of data.
> >
> > It's probably better to add just the few that have a chance of actually having changed enough between reseeds.
>
> That depends a little bit on what issue you want to protect against.
>
> Not changing between reseeds: yes in that case you only need those that
> change.
>
> Protect against outsiders brute forcing: the values that do not change
> frequently still provide value because their values can not be
> easily guessed.
Incidently, I took a look at Linux' /dev/urandom driver, and they have an interesting take on evaluating the amount of entropy in any given counter value, by calculating a third order delta compared to a previous value. That's a method that could answer my concerns.
> And the hashing ensures that all bits change at reseed even with several
> values not changing.
> Well - it is just some ideas. As they say: free advice is worth
> every cent you pay. :-)
Cheers,
Richard
DaveFroble
Apr 5, 2018, 1:02:14 PM4/5/18
to
Arne Vajhøj wrote:
> But I think that will require relinking at VMS upgrade, which
> I do not like as a concept.
Wow! That's an interesting statement. I don't think I agree with it.
> But I think that will require relinking at VMS upgrade, which
> I do not like as a concept.
Our Codis applications have build procedures to allow a compile and line
whenever desired. Just queueing up the job is all that's required. We can, and
have at times, rebuild the entire app nightly. Not normally, but, the
capability is there.
Wouldn't have it any other way.
--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: da...@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
Jan-Erik Söderholm
Apr 5, 2018, 1:16:54 PM4/5/18
to
Den 2018-04-05 kl. 19:02, skrev DaveFroble:
> Arne Vajhøj wrote:
>
>> But I think that will require relinking at VMS upgrade, which
>> I do not like as a concept.
>
> Wow! That's an interesting statement. I don't think I agree with it.
>
> Our Codis applications have build procedures to allow a compile and line
> whenever desired. Just queueing up the job is all that's required. We
> can, and have at times, rebuild the entire app nightly. Not normally, but,
> the capability is there.
>
> Wouldn't have it any other way.
>
Would be a royal PITA if a relink would be *required* at each upgrade.
> Arne Vajhøj wrote:
>
>> But I think that will require relinking at VMS upgrade, which
>> I do not like as a concept.
>
> Wow! That's an interesting statement. I don't think I agree with it.
>
> Our Codis applications have build procedures to allow a compile and line
> whenever desired. Just queueing up the job is all that's required. We
> can, and have at times, rebuild the entire app nightly. Not normally, but,
> the capability is there.
>
> Wouldn't have it any other way.
>
B.t.w, do you require from all customers that they license and install
all compilers used by the application?
DaveFroble
Apr 5, 2018, 1:28:13 PM4/5/18
to
Jan-Erik Söderholm wrote:
> Den 2018-04-05 kl. 19:02, skrev DaveFroble:
>> Arne Vajhøj wrote:
>>
>>> But I think that will require relinking at VMS upgrade, which
>>> I do not like as a concept.
>>
>> Wow! That's an interesting statement. I don't think I agree with it.
>>
>> Our Codis applications have build procedures to allow a compile and
>> line whenever desired. Just queueing up the job is all that's
>> required. We can, and have at times, rebuild the entire app nightly.
>> Not normally, but, the capability is there.
>>
>> Wouldn't have it any other way.
>>
>
> Would be a royal PITA if a relink would be *required* at each upgrade.
OS upgrades are not a daily occurrence.
> Den 2018-04-05 kl. 19:02, skrev DaveFroble:
>> Arne Vajhøj wrote:
>>
>>> But I think that will require relinking at VMS upgrade, which
>>> I do not like as a concept.
>>
>> Wow! That's an interesting statement. I don't think I agree with it.
>>
>> Our Codis applications have build procedures to allow a compile and
>> line whenever desired. Just queueing up the job is all that's
>> required. We can, and have at times, rebuild the entire app nightly.
>> Not normally, but, the capability is there.
>>
>> Wouldn't have it any other way.
>>
>
> Would be a royal PITA if a relink would be *required* at each upgrade.
> B.t.w, do you require from all customers that they license and install
> all compilers used by the application?
not have C compilers on every system, so the few C routines are distributed in
object libraries.
Application sources are on each customer system, and when any change, there is a
nightly procedure to push any changes down to every customer system, and then to
build new executables.
Jan-Erik Söderholm
Apr 5, 2018, 2:09:58 PM4/5/18
to
Den 2018-04-05 kl. 19:28, skrev DaveFroble:
> Jan-Erik Söderholm wrote:
>> Den 2018-04-05 kl. 19:02, skrev DaveFroble:
>>> Arne Vajhøj wrote:
>>>
>>>> But I think that will require relinking at VMS upgrade, which
>>>> I do not like as a concept.
>>>
>>> Wow! That's an interesting statement. I don't think I agree with it.
>>>
>>> Our Codis applications have build procedures to allow a compile and line
>>> whenever desired. Just queueing up the job is all that's required. We
>>> can, and have at times, rebuild the entire app nightly. Not normally,
>>> but, the capability is there.
>>>
>>> Wouldn't have it any other way.
>>>
>>
>> Would be a royal PITA if a relink would be *required* at each upgrade.
>
> OS upgrades are not a daily occurrence.
No, right. And sometimes relinkes aren't a daily thing either.
> Jan-Erik Söderholm wrote:
>> Den 2018-04-05 kl. 19:02, skrev DaveFroble:
>>> Arne Vajhøj wrote:
>>>
>>>> But I think that will require relinking at VMS upgrade, which
>>>> I do not like as a concept.
>>>
>>> Wow! That's an interesting statement. I don't think I agree with it.
>>>
>>> Our Codis applications have build procedures to allow a compile and line
>>> whenever desired. Just queueing up the job is all that's required. We
>>> can, and have at times, rebuild the entire app nightly. Not normally,
>>> but, the capability is there.
>>>
>>> Wouldn't have it any other way.
>>>
>>
>> Would be a royal PITA if a relink would be *required* at each upgrade.
>
> OS upgrades are not a daily occurrence.
Haven't really thought about it, but here I see many EXEs linked the
27-sep-1994 (I think that was when they moved from VAX to Alpha).
No one seems to have seen a need to relink them since then.
Anyway, works just fine. And I just compiled and linked one of them
and that also run just fine.
Current OBJ and EXE:
creation date/time: 28-SEP-1994 12:25
"DEC COBOL V2.0-271"
link date/time: 28-SEP-1994 12:26:00.07
linker identification: "A11-14"
OBJ and EXE created today:
creation date/time: 5-APR-2018 19:55
"VSI COBOL V3.1-0007"
link date/time: 5-APR-2018 19:55:23.48
linker identification: "A13-04"
And the source says "DATE-WRITTEN. MARS 1986.".
But yes, it s not a bad idea to recompile now and then... :-)
Richard Levitte
Apr 5, 2018, 3:49:36 PM4/5/18
to
Den torsdag 5 april 2018 kl. 19:28:13 UTC+2 skrev DaveFroble:
> Jan-Erik Söderholm wrote:
> > Den 2018-04-05 kl. 19:02, skrev DaveFroble:
> >> Arne Vajhøj wrote:
> >>
> >>> But I think that will require relinking at VMS upgrade, which
> >>> I do not like as a concept.
> >>
> >> Wow! That's an interesting statement. I don't think I agree with it.
[...]
> Jan-Erik Söderholm wrote:
> > Den 2018-04-05 kl. 19:02, skrev DaveFroble:
> >> Arne Vajhøj wrote:
> >>
> >>> But I think that will require relinking at VMS upgrade, which
> >>> I do not like as a concept.
> >>
> >> Wow! That's an interesting statement. I don't think I agree with it.
> >
> > Would be a royal PITA if a relink would be *required* at each upgrade.
>
> OS upgrades are not a daily occurrence.
I think that most people would be royally pissed if every application that uses OpenSSL libraries would fall apart after an OS upgrade. So while your application may require this, I don't think it's defensible to have every relink Apache, WASD and whatever else just because they make a 8.4 -> 9.0 upgrade some time in the future.
> > Would be a royal PITA if a relink would be *required* at each upgrade.
>
> OS upgrades are not a daily occurrence.
Cheers,
Richard
Arne Vajhøj
Apr 5, 2018, 7:08:57 PM4/5/18
to
On 4/5/2018 3:37 AM, Richard Levitte wrote:
> Den torsdag 5 april 2018 kl. 01:44:12 UTC+2 skrev Arne Vajhøj:
>> And the hashing ensures that all bits change at reseed even with several
>> values not changing.
>
> Does scrambling all the bits around (and compressing them) with
> SHA2-512 turn 37 bits of entropy (regardless of input length...) into
> 512 bits of entropy?
It does not. At least not with a classic definition of entropy.
> Den torsdag 5 april 2018 kl. 01:44:12 UTC+2 skrev Arne Vajhøj:
>> And the hashing ensures that all bits change at reseed even with several
>> values not changing.
>
> Does scrambling all the bits around (and compressing them) with
> SHA2-512 turn 37 bits of entropy (regardless of input length...) into
> 512 bits of entropy?
N possible values hashed is still only N possible values.
But there are some benefits.
1) It smooths out the entropy so that instead of having some bytes with
a high entropy content and some bytes with a low entropy content one
get bytes with same entropy content. That may or may not be
beneficial depending on how the result is used. But I think
it is either beneficial or neutral, so I consider it a good
thing.
2) It helps a lot to protect against knowledge about a value
can be used to reduce possible value for another generation.
Simplified if you have 2 longwords where one is incremented by
1 every few hundred milliseconds and the other is constant then
getting one value makes it easy to come up with a list of a few
hundred possible values for next value generated. Hashing makes
that very hard. And note that applying a PRNG to the data will
typical not achieve the same as PRNG's often are reversible.
And again this may or may not be beneficial, but I like that
characteristic.
Arne
Arne Vajhøj
Apr 5, 2018, 7:10:26 PM4/5/18
to
On 4/5/2018 1:02 PM, DaveFroble wrote:
> Arne Vajhøj wrote:
>> But I think that will require relinking at VMS upgrade, which
>> I do not like as a concept.
>
> Wow! That's an interesting statement. I don't think I agree with it.
>
> Our Codis applications have build procedures to allow a compile and line
> whenever desired. Just queueing up the job is all that's required. We
> can, and have at times, rebuild the entire app nightly. Not normally,
> but, the capability is there.
.OLB and LINK.COM is an option.
> Arne Vajhøj wrote:
>> But I think that will require relinking at VMS upgrade, which
>> I do not like as a concept.
>
> Wow! That's an interesting statement. I don't think I agree with it.
>
> Our Codis applications have build procedures to allow a compile and line
> whenever desired. Just queueing up the job is all that's required. We
> can, and have at times, rebuild the entire app nightly. Not normally,
> but, the capability is there.
But it is a complication.
A complication I would like to avoid - and I suspect that I
am not the only one.
Arne
DaveFroble
Apr 5, 2018, 11:06:59 PM4/5/18
to
libraries have to do similar things for OpenSSL version updates? And isn't that
how we got the abortion called SSL1?
Our applications do not require re-building. We as developers demand that such
labor intensive tasks be automated. Sort of like the Jenson that John Malmberg
is using, I believe. Having developed that automation, we are now free to
develop, not spend time manually building executables.
And when presented with some reason to re-build the applications, perhaps such
as moving to VMS on x86, it's a non-issue. We queue up the job and away it goes.
Isn't that the way all software should be?
DaveFroble
Apr 5, 2018, 11:10:49 PM4/5/18
to
to do whatever is necessary?
On Codis, every module that is compiled is placed in a OLB. Linking is done out
of the OLB. Thus, we don't have to re-do everything, if we don't want to do so.
Arne Vajhøj
Apr 6, 2018, 10:17:54 AM4/6/18
to
On 4/5/2018 11:07 PM, DaveFroble wrote:
> Depending upon the modifications, doesn't those same people who use
> OpenSSL libraries have to do similar things for OpenSSL version
> updates?
I don't know about OpenSSL specifically, but in general know.
> Depending upon the modifications, doesn't those same people who use
> OpenSSL libraries have to do similar things for OpenSSL version
> updates?
Apps linked against shareable image version N should work without
relink when version N+1 is installed.
> Our applications do not require re-building. We as developers demand
> that such labor intensive tasks be automated. Sort of like the Jenson
Which is a fork of now dead Hudson (or Hudson was a fork of Jenkins
depending on perspective).
> that John Malmberg is using, I believe. Having developed that
> automation, we are now free to develop, not spend time manually building
> executables.
Stephen Hoffman
Apr 6, 2018, 11:59:09 AM4/6/18
to
On 2018-04-05 06:20:39 +0000, Richard Levitte said:
> Ah, yeah, me neither. This goes into libcrypto, making it VMS version
> sensitive this way would be downright irresponsible.
Or you "hack-JIT" the references, with a newly-minted OpenSSL startup
> Ah, yeah, me neither. This goes into libcrypto, making it VMS version
> sensitive this way would be downright irresponsible.
procedure that checks the OpenVMS version and builds and caches a
"stub" when the OpenVMS system is booted whenever lacking a "stub" for
that release. If you try to use OpenSSL without having run the
startup, you get an error from the presence of the default redirect
"stub" telling the user to how to deal with not having run the startup
and probably not having added the startup to the system startup, and
then try the OpenSSL request again.
If you really want to get fancy, all of this can be done at run-time
and without the system startup having been run, using
lib$find_image_symbol or via dlopen et al. While preferable, there's
no need for the dynamic-built references to be stored system-wide
though that caching would save on overhead and avoid having each
disparate caller with a local cache of the stub and/or the overhead of
the JIT.
I don't know if it'll work for these particular entry-points, but I'd
also try using lib$find_image_symbol to directly reference the symbols
without the build and run-time merge of the stub, as that call can find
various system-defined entry points. It's not widely known that the
lib$find_image_symbol call can find system-defined symbols including
system service entry-points via the P1 vectors (SYS$P1_VECTORS)
references. But it can find system services. It couldn't do that for
a while, but that got fixed. What I don't know is whether that call
can find these specific symbols. But I'd try it, if for no other
reason than this approach would completely obviate the "stub" and the
rest of the JIT shenanigans above, too.
Best approach long-term is VSI adding an API that can return wads of
entropy, and making it unnecessary to reference the performance
management statistics data or any other constructs.
Hey VSI: there are instances of a typo in the OpenVMS documentation at
http://h41379.www4.hpe.com/doc/82final/5932/5932pro_018.html where
LIB$EOMWARN should be LIB$_EOMWARN.
--
Pure Personal Opinion | HoffmanLabs LLC
DaveFroble
Apr 6, 2018, 12:21:55 PM4/6/18
to
Arne Vajhøj wrote:
> On 4/5/2018 11:07 PM, DaveFroble wrote:
>> Depending upon the modifications, doesn't those same people who use
>> OpenSSL libraries have to do similar things for OpenSSL version updates?
>
> I don't know about OpenSSL specifically, but in general know.
>
> Apps linked against shareable image version N should work without
> relink when version N+1 is installed.
I agree 100%. But it's not always possible. In this case, it doesn't happen
> On 4/5/2018 11:07 PM, DaveFroble wrote:
>> Depending upon the modifications, doesn't those same people who use
>> OpenSSL libraries have to do similar things for OpenSSL version updates?
>
> I don't know about OpenSSL specifically, but in general know.
>
> Apps linked against shareable image version N should work without
> relink when version N+1 is installed.
with the software mentioned, OpenSSL. Talk about getting into a glass house and
then start chucking rocks ....
>> Our applications do not require re-building. We as developers demand
>> that such labor intensive tasks be automated. Sort of like the Jenson
>
> Jenkins
>
> Which is a fork of now dead Hudson (or Hudson was a fork of Jenkins
> depending on perspective).
>
> > that John Malmberg is using, I believe. Having developed that
> > automation, we are now free to develop, not spend time manually building
> > executables.
>
> Arne
>
>
>
Craig A. Berry
Apr 6, 2018, 3:43:09 PM4/6/18
to
necessarily mean they should be. And last I heard, lib$find_image_symbol
can only find symbols in images that actually exist, which for the case
being discussed they won't, at least not unless these smart stubs you
are talking about know how to locate the relevant object code, do a
LINK/SYSEXE on the fly and then load symbols from the image produced.
That's a lot of shenanigans in order to be able to use symbols from the
VMS base image that only entered the discussion because they were the
first thing that came to mind for a kernel engineer. For OpenSSL, doing
it this way instead of using a supported API like SYS$GETRMI would be a
layer violation and add a lot of unnecessary complexity.
Stephen Hoffman
Apr 6, 2018, 4:42:57 PM4/6/18
to
nor with dependency chains nearly as long, on various other platforms
I'm working with.
> And last I heard, lib$find_image_symbol can only find symbols in images
> that actually exist, which for the case being discussed they won't, at
> least not unless these smart stubs you are talking about know how to
> locate the relevant object code, do a LINK/SYSEXE on the fly and then
> load symbols from the image produced.
version-specific stub shareable at either system startup, or at
application startup, when a seed is required. That can start from
source for those that might want to see it, or maybe from an object
module. Cache the stubs where possible. Preferably, this build all
happens at system startup using a (new) OpenSSL startup procedure, and
the run-time only does a lib$find_image_symbol or dlopen call to
activate the entropy source. (This also has the advantage of allowing
folks with an actual hardware entropy source to implement and reference
that source, if there's not already some way to reference a hardware
entropy source in OpenSSL.)
Or potentially directly locating the necessary external references in
the kernel at run-time, as that approach does work for system service
calls and for at least some related public symbols. I don't know if
this case works for these references.
I do know that the stubs-based dynamic-activation approach works and
works well, as I've had to use similar approaches to work around
various OpenVMS limitations and misfeatures. It's also part of how
I've been dealing with cases of API skew in underlying OpenVMS code,
too.
> That's a lot of shenanigans in order to be able to use symbols from the
> VMS base image that only entered the discussion because they were the
> first thing that came to mind for a kernel engineer. For OpenSSL, doing
> it this way instead of using a supported API like SYS$GETRMI would be a
> layer violation and add a lot of unnecessary complexity.
or a system app from the kernel entropy pool? That one? Or are we
discussing a deep, dark, desperate, implementation-specific, variously
inconsistently implemented, not entirely certain way of accumulating
entropy from various unrelated sources?
Are any of the approaches I've mentioned here pretty? No. Definitely not.
Does this approach make the user experience better? Yes.
Does it avoid support calls, or product re-installations akin to what
can happen with compiler installation kits, or version-specific
releases, or other obvious sorts of work-arounds? Yes.
Would I prefer to have a source of entropy in the kernel? Yes.
And yes, VSI is working to address as many of the omissions and
limitations within OpenVMS as they can reasonably resolve, too. As
for this case, it also seems likely that there are or will soon be
backchannels established into VSI, to assist with work on the OpenVMS
port of OpenSSL, too.
Richard Levitte
Apr 7, 2018, 6:49:17 AM4/7/18
to
Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
> Would I prefer to have a source of entropy in the kernel? Yes.
I'm tempted by the idea to write a device driver... RND0: (or if I feel narcissistic, RLD0: ;-))
> Would I prefer to have a source of entropy in the kernel? Yes.
Trouble is, device drivers are currently deep dark magic to me, and all I find via google is references to that same book... yeahok, I can certainly buy a book, not a problem, but I was impatient and was trying to see if there were examples available. It might be that my google-fu was off when I tried...
Cheers,
Richard
hb
Apr 7, 2018, 10:15:53 AM4/7/18
to
On 04/06/2018 05:59 PM, Stephen Hoffman wrote:
> I don't know if it'll work for these particular entry-points, but I'd
> also try using lib$find_image_symbol to directly reference the symbols
> without the build and run-time merge of the stub, as that call can find
> various system-defined entry points. It's not widely known that the
> lib$find_image_symbol call can find system-defined symbols including
> system service entry-points via the P1 vectors (SYS$P1_VECTORS)
> references. But it can find system services. It couldn't do that for
> a while, but that got fixed. What I don't know is whether that call can
> find these specific symbols. But I'd try it, if for no other reason
> than this approach would completely obviate the "stub" and the rest of
> the JIT shenanigans above, too.
I'm not sure if I fully understand. What do you mean by "various
> I don't know if it'll work for these particular entry-points, but I'd
> also try using lib$find_image_symbol to directly reference the symbols
> without the build and run-time merge of the stub, as that call can find
> various system-defined entry points. It's not widely known that the
> lib$find_image_symbol call can find system-defined symbols including
> system service entry-points via the P1 vectors (SYS$P1_VECTORS)
> references. But it can find system services. It couldn't do that for
> a while, but that got fixed. What I don't know is whether that call can
> find these specific symbols. But I'd try it, if for no other reason
> than this approach would completely obviate the "stub" and the rest of
> the JIT shenanigans above, too.
system-defined entry points" and/or "system-defined symbols", which
lib$fis should be able to find?
sys$base_image is a sort of shareable image in sys$loadable_images. It
exports symbols like pms$gl_faults (and yes, link/sysexe is more or less
a shortcut link against a special shareable image). Analyze/image shows
that symbol. So lib$fis can find it in the base image. But a quick test
with pms$gl_faults shows that lib$fis can't handle the found symbol:
LIB-W-ILLSYMLEN. So I guess, it would not return the VA of any pms
symbol. It seems, that lib$fis doesn't have specific code for the
special symbols in the base image. But I haven't looked at the sources.
What do you mean by "SYS$P1_VECTORS"?
sys$public_vectors is a shareable image in sys$share. It exports symbols
for the system services like sys$asctim (and yes, the linker resolves
from public vectors by default). Analyze/image shows these symbols. So
lib$fis can find them in the image. I tried it with the above system
service: a VA was returned; I could call the service via that VA and the
returned data made sense to me. OK, I know, no proof, just an example.
Robert A. Brooks
Apr 7, 2018, 4:51:57 PM4/7/18
to
On 4/7/2018 6:49 AM, Richard Levitte wrote:
> Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
>> Would I prefer to have a source of entropy in the kernel? Yes.
>
> I'm tempted by the idea to write a device driver... RND0: (or if I feel
> narcissistic, RLD0: ;-)) Trouble is, device drivers are currently deep dark
> magic to me, and all I find via google is references to that same book...
> yeahok, I can certainly buy a book, not a problem, but I was impatient and
> was trying to see if there were examples available.
I've never tried to build LRDRIVER, but in response to a request from someone
> Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
>> Would I prefer to have a source of entropy in the kernel? Yes.
>
> I'm tempted by the idea to write a device driver... RND0: (or if I feel
> narcissistic, RLD0: ;-)) Trouble is, device drivers are currently deep dark
> magic to me, and all I find via google is references to that same book...
> yeahok, I can certainly buy a book, not a problem, but I was impatient and
> was trying to see if there were examples available.
in comp.os.vms 15 years ago, I did ensure it's the one that was used to build
the shipping driver.
* X-4 VMS002 VMS Engineering 31-Mar-2003
* Fold of a couple of changes, making this example driver
* an exact copy of the source that is used to build
* sys$loadable_images:sys$lrdriver.exe.
*
* X-10 VMS OpenVMS Alpha Drivers 25-March-1996
* Yet another fix for the timer polling code. We cannout use
* UCB$V_INT to signal that the output is done. If the printer
* stalls for some reason the device timeout code clears UCB$V_INT
* even though there is still data to output. The flag to use is
* UCB$V_BSY. The busy bit should only be cleared when there is
* nothing more to output.
*
* X-9 VMS OpenVMS Alpha Drivers 12-February-1996
* Fix IPL misuse in lt$timer_int routine. It saved starting IPL
* in the wrong place. Then when it restored IPL when releasing
* the fork lock it got a random IPL value.
$ dir sys$examples:*driv*/siz=all/date
Directory SYS$COMMON:[SYSHLP.EXAMPLES]
DISK_DRIVER.C;1 30/30 17-MAR-1993 17:33:33.31
LRDRIVER.C;1 134/135 1-APR-2003 15:05:52.17
LRDRIVER.COM;1 12/12 29-AUG-1995 08:28:03.00
LRDRIVER.H;1 20/21 29-JUN-1995 08:32:10.89
--
-- Rob
Stephen Hoffman
Apr 7, 2018, 6:48:27 PM4/7/18
to
the OpenVMS documentation set, and that material and the available
example device drivers and freeware device drivers around should be a
good start for writing a pseudo device driver. Pseudo device drivers
don't have associated hardware, which avoids a fair amount of the
complexity found in many device drivers. You're very likely working in
C for any new device driver and calling the kernel C RTL, too. You're
probably also going to be able to perform most or all of your work in
the FDT (Function Decision Table) context, which avoids much of the
"fun" of working at higher interrupt priority levels (IPLs) and
related. This because your code will executed while in kernel mode
with process context mapped, and likely performing your entropy
collection work in addition to probing and reading and writing the user
buffers. Little (no?) need to deal with fork processors or high IPL
context or the rest of the baggage that actual hardware and the
hardware interrupts usually involves. Device driver prefixes
beginning with J and Q are available for users. The rest of the
letters are supposedly reserved and registered, but whether the product
registrar stuff is up and running at HPE and VSI?
The older driver doc won't reference the tr_print routine (and it may
still be considered undocumented), but that can be quite handy for
debugging.
#include <vms_macros.h>
…
tr_print(( “hello, world %d”, i ));
Valid formatting directives:
%S and %s — is an ASCIZ null-terminated string
%A and %a — is an ASCII string; requires specification of address and a
length argument
%D and %d — displays an integer as a decimal string
%X and %x — displays an integer as a hexadecimal string
%L and %l — displays a quadword integer as a hexadecimal string
%W and %w — displays a word as a hexadecimal string (V7.3-2 and later)
%B and %b — displays a byte as a hexadecimal string (V7.3-2 and later)
Enabling, Sizing and Displaying The Tracing Ring
Load the tracing support into the kernel, size the tracing ring, and
display the contents, you can use the following commands within SDA:
$ ANALYZE/SYSTEM
SDA> tr
Debug Tracing Utility TR commands:
TR LOAD
TR UNLOAD
TR START TRACE [/BUFFER=pages]
TR STOP TRACE
TR SHOW TRACE
For some related materials, extract vms_macros.h from
sys$library:sys$lib_c.tlb. You can use a command such as:
$ library sys$share:sys$lib_c.tlb/extract=vms_macros/out=vms_macros.h
Edit the file, searching for tr_print.
If you're doing this in a process context (and not a driver) LINK
/SYSEXE is needed to resolve some symbols and you'll want to append
+sys$library:sys$lib_c.tlb/library to the parameter list for the
compilation command. For instance:
$ CC program+sys$library:sys$lib_c.tlb/library
You can either manually CONNECT the driver, or use the file-based stuff:
In more recent OpenVMS releases — particularly V7.1 and later — the
usual hardware device add-in autoconfiguration involves the use of the
site-specific SYS$SYSTEM:SYS$USER_CONFIG.DAT file.
SYS$SYSTEM:SYS$CONFIG.DAT contains the system-defined devices, and
should not be modified. This file is, however, a valuable source of
device examples.
This file-based construct is significantly simpler than the ICBM
mechanism that preceded it.
DEVICE = "Device Name"
NAME = JJ
DRIVER = FOO$JJDRIVER
ID = 0x0000000
ADAPTER = PCI
FLAGS = NOVECTOR
UNITS = 1
NUM_VECTORS = 1
END_DEVICE
Another approach that'll work here is a user-written system service
(UWSS), and also (and sometimes somewhat confusingly, given differences
in structures and design) known as a privileged shareable image. That
UWSS doesn't use the device driver and related constructs, though it
does retain full exec and kernel access.
Stephen Hoffman
Apr 7, 2018, 8:23:38 PM4/7/18
to
On 2018-04-07 14:15:48 +0000, hb said:
> On 04/06/2018 05:59 PM, Stephen Hoffman wrote:
>> I don't know if it'll work for these particular entry-points, but I'd
>> also try using lib$find_image_symbol to directly reference the symbols
>> without the build and run-time merge of the stub, as that call can find
>> various system-defined entry points. It's not widely known that the
>> lib$find_image_symbol call can find system-defined symbols including
>> system service entry-points via the P1 vectors (SYS$P1_VECTORS)
>> references. But it can find system services. It couldn't do that for
>> a while, but that got fixed. What I don't know is whether that call
>> can find these specific symbols. But I'd try it, if for no other
>> reason than this approach would completely obviate the "stub" and the
>> rest of the JIT shenanigans above, too.
>
> I'm not sure if I fully understand. What do you mean by "various
> system-defined entry points" and/or "system-defined symbols", which
> lib$fis should be able to find?
lib$find_image_symbol should be able to find system services, as well
> On 04/06/2018 05:59 PM, Stephen Hoffman wrote:
>> I don't know if it'll work for these particular entry-points, but I'd
>> also try using lib$find_image_symbol to directly reference the symbols
>> without the build and run-time merge of the stub, as that call can find
>> various system-defined entry points. It's not widely known that the
>> lib$find_image_symbol call can find system-defined symbols including
>> system service entry-points via the P1 vectors (SYS$P1_VECTORS)
>> references. But it can find system services. It couldn't do that for
>> a while, but that got fixed. What I don't know is whether that call
>> can find these specific symbols. But I'd try it, if for no other
>> reason than this approach would completely obviate the "stub" and the
>> rest of the JIT shenanigans above, too.
>
> I'm not sure if I fully understand. What do you mean by "various
> system-defined entry points" and/or "system-defined symbols", which
> lib$fis should be able to find?
as symbols in referenced shareable images.
It seems clear that it cannot find and correctly process the PMS
symbols, though that would have been handy here.
> What do you mean by "SYS$P1_VECTORS"?
reference.
Robert A. Brooks
Apr 7, 2018, 8:47:10 PM4/7/18
to
On 4/7/2018 6:48 PM, Stephen Hoffman wrote:
> The older driver doc won't reference the tr_print routine (and it may still be
> considered undocumented), but that can be quite handy for debugging.
Sort-of documented in sys$library:lib.req
> The older driver doc won't reference the tr_print routine (and it may still be
> considered undocumented), but that can be quite handy for debugging.
--
-- Rob
Simon Clubley
Apr 8, 2018, 7:48:06 PM4/8/18
to
On 2018-04-07, Richard Levitte <ric...@levitte.org> wrote:
> Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
>> Would I prefer to have a source of entropy in the kernel? Yes.
>
> I'm tempted by the idea to write a device driver... RND0: (or if I feel
> narcissistic, RLD0: ;-))
I actually made the same suggestion recently. More than one person told
> Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
>> Would I prefer to have a source of entropy in the kernel? Yes.
>
> I'm tempted by the idea to write a device driver... RND0: (or if I feel
> narcissistic, RLD0: ;-))
me in response that wasn't the VMS way of doing things... :-)
> Trouble is, device drivers are currently deep dark magic to me, and all I
> find via google is references to that same book... yeahok, I can certainly
> buy a book, not a problem, but I was impatient and was trying to see if
> there were examples available. It might be that my google-fu was off when
> I tried...
>
I got it into my head to drop a PCI based WinTV card into an Alpha so
that I could view teletext pages on a VTxxx terminal... :-)
I used the "Writing OpenVMS Alpha Device Drivers in C" book, the sources
to LRDRIVER and a good number of questions in comp.os.vms.
Between the 3 sources of information, I was able to get the driver
working.
BTW, the first thing you will discover is that you cannot unload
and reload VMS device drivers in the same way as you can on Linux.
Simon.
--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world
Simon Clubley
Apr 8, 2018, 7:57:15 PM4/8/18
to
On 2018-04-07, Robert A. Brooks <FIRST...@vmssoftware.com> wrote:
>
> I've never tried to build LRDRIVER, but in response to a request from someone
> in comp.os.vms 15 years ago, I did ensure it's the one that was used to build
> the shipping driver.
>
> * X-4 VMS002 VMS Engineering 31-Mar-2003
> * Fold of a couple of changes, making this example driver
> * an exact copy of the source that is used to build
> * sys$loadable_images:sys$lrdriver.exe.
> *
I wonder if that was me. It sounds about the right time...
>
> I've never tried to build LRDRIVER, but in response to a request from someone
> in comp.os.vms 15 years ago, I did ensure it's the one that was used to build
> the shipping driver.
>
> * X-4 VMS002 VMS Engineering 31-Mar-2003
> * Fold of a couple of changes, making this example driver
> * an exact copy of the source that is used to build
> * sys$loadable_images:sys$lrdriver.exe.
> *
It's a pity that VMS doesn't ship a range of C language driver sources.
It's been a very long time since I last looked at this, but don't
the disk drivers (for example) use functionality that isn't used
by LRDRIVER ?
Are any of the later disk device drivers written in C ?
Are there any additional C language device drivers which could be
dropped into sys$examples ?
Thanks,
Richard Levitte
Apr 9, 2018, 1:21:56 AM4/9/18
to
Den söndag 8 april 2018 kl. 00:48:27 UTC+2 skrev Stephen Hoffman:
> On 2018-04-07 10:49:15 +0000, Richard Levitte said:
> > I'm tempted by the idea to write a device driver... RND0: (or if I
> > feel narcissistic, RLD0: ;-))
> > Trouble is, device drivers are currently deep dark magic to me, and all
> > I find via google is references to that same book... yeahok, I can
> > certainly buy a book, not a problem, but I was impatient and was trying
> > to see if there were examples available. It might be that my google-fu
> > was off when I tried...
[...]
> On 2018-04-07 10:49:15 +0000, Richard Levitte said:
> > I'm tempted by the idea to write a device driver... RND0: (or if I
> > feel narcissistic, RLD0: ;-))
> > Trouble is, device drivers are currently deep dark magic to me, and all
> > I find via google is references to that same book... yeahok, I can
> > certainly buy a book, not a problem, but I was impatient and was trying
> > to see if there were examples available. It might be that my google-fu
> > was off when I tried...
>
> Another approach that'll work here is a user-written system service
> (UWSS), and also (and sometimes somewhat confusingly, given differences
> in structures and design) known as a privileged shareable image. That
> UWSS doesn't use the device driver and related constructs, though it
> does retain full exec and kernel access.
Tempting! And now that Robert reminded me of SYS$COMMON:[SYSHLP.EXAMPLES], I got something to get me started. I think that the hardest part (if it's even hard, I dunno yet) is to figure out a way to have an asynchronous part that does the collecting of "stuff". Knowing nothing else, I'm going to assume that any idea I may have is treading on thin ice, so if you have any advice, I'm all ears.
> Another approach that'll work here is a user-written system service
> (UWSS), and also (and sometimes somewhat confusingly, given differences
> in structures and design) known as a privileged shareable image. That
> UWSS doesn't use the device driver and related constructs, though it
> does retain full exec and kernel access.
Cheers,
Richard
Richard Levitte
Apr 9, 2018, 1:28:08 AM4/9/18
to
Den måndag 9 april 2018 kl. 01:48:06 UTC+2 skrev Simon Clubley:
> On 2018-04-07, Richard Levitte <ric...@levitte.org> wrote:
> > Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
> >> Would I prefer to have a source of entropy in the kernel? Yes.
> >
> > I'm tempted by the idea to write a device driver... RND0: (or if I feel
> > narcissistic, RLD0: ;-))
>
> I actually made the same suggestion recently. More than one person told
> me in response that wasn't the VMS way of doing things... :-)
I vaguely remember reading that thread. :-)
> On 2018-04-07, Richard Levitte <ric...@levitte.org> wrote:
> > Den fredag 6 april 2018 kl. 22:42:57 UTC+2 skrev Stephen Hoffman:
> >> Would I prefer to have a source of entropy in the kernel? Yes.
> >
> > I'm tempted by the idea to write a device driver... RND0: (or if I feel
> > narcissistic, RLD0: ;-))
>
> I actually made the same suggestion recently. More than one person told
> me in response that wasn't the VMS way of doing things... :-)
And yeah, Hoffman's reminder of the UWSS possibility gave me yet another angle that actually looks easier, provided I can solve the collection bit.
> BTW, the first thing you will discover is that you cannot unload
> and reload VMS device drivers in the same way as you can on Linux.
Cheers,
Richard
dgordo...@gmail.com
Apr 9, 2018, 10:29:29 AM4/9/18
to
On Friday, April 6, 2018 at 11:59:09 AM UTC-4, Stephen Hoffman wrote:
>> Hey VSI: there are instances of a typo in the OpenVMS documentation at
> http://h41379.www4.hpe.com/doc/82final/5932/5932pro_018.html where
> LIB$EOMWARN should be LIB$_EOMWARN.
>
Noted, but comp.os.vms is not a supported way to report bugs.
>> Hey VSI: there are instances of a typo in the OpenVMS documentation at
> http://h41379.www4.hpe.com/doc/82final/5932/5932pro_018.html where
> LIB$EOMWARN should be LIB$_EOMWARN.
>
Stephen Hoffman
Apr 9, 2018, 11:40:10 AM4/9/18
to
considered formally documented then, though comments in those libraries
and analogous comments in help file source code have been classic forms
of undocumentation over the years. Used to be that extracting the help
libraries and searching for ! comments was useful way to locate latent
features. That reconnaissance is harder within the definition
libraries, other than by reviewing the differences among the versions
of interest.
Stephen Hoffman
Apr 9, 2018, 12:07:51 PM4/9/18
to
On 2018-04-08 23:57:14 +0000, Simon Clubley said:
> On 2018-04-07, Robert A. Brooks <FIRST...@vmssoftware.com> wrote:
>>
>> I've never tried to build LRDRIVER, but in response to a request from someone
>> in comp.os.vms 15 years ago, I did ensure it's the one that was used to build
>> the shipping driver.
>>
>> * X-4 VMS002 VMS Engineering 31-Mar-2003
>> * Fold of a couple of changes, making this example driver
>> * an exact copy of the source that is used to build
>> * sys$loadable_images:sys$lrdriver.exe.
>> *
>
> I wonder if that was me. It sounds about the right time...
And now I'm wondering if the obvious associated build issue was ever
> On 2018-04-07, Robert A. Brooks <FIRST...@vmssoftware.com> wrote:
>>
>> I've never tried to build LRDRIVER, but in response to a request from someone
>> in comp.os.vms 15 years ago, I did ensure it's the one that was used to build
>> the shipping driver.
>>
>> * X-4 VMS002 VMS Engineering 31-Mar-2003
>> * Fold of a couple of changes, making this example driver
>> * an exact copy of the source that is used to build
>> * sys$loadable_images:sys$lrdriver.exe.
>> *
>
> I wonder if that was me. It sounds about the right time...
fixed. :-)
> It's a pity that VMS doesn't ship a range of C language driver sources.
with current expectations, unfortunately. On another platform I deal
with, I'd expect to find a template device driver and a cookbook
documenting this and related tasks. That, and I'd expect to also find
a driver code-signing procedure necessary for loading the driver into
the kernel, outside of a suitably open-configured test system.
> It's been a very long time since I last looked at this, but don't the
> disk drivers (for example) use functionality that isn't used by
> LRDRIVER ?
likely operate entirely in the FDT, so it'll likely be somewhat less
involved than LRDRIVER. I'd not expect to have to deal with device IP
or fork processes here.
> Are any of the later disk device drivers written in C ?
various pseudo disk device drivers around including LDDRIVER:
https://www.digiater.nl/lddriver.html
> Are there any additional C language device drivers which could be
> dropped into sys$examples ?
from the reported contractual constraints.
DaveFroble
Apr 9, 2018, 12:37:33 PM4/9/18
to
:-)
dgordo...@gmail.com
Apr 9, 2018, 1:35:05 PM4/9/18
to
On Monday, April 9, 2018 at 12:37:33 PM UTC-4, DaveFroble wrote:
> dgordonatvsi wrote:
> > On Friday, April 6, 2018 at 11:59:09 AM UTC-4, Stephen Hoffman wrote:
> >>> Hey VSI: there are instances of a typo in the OpenVMS documentation at
> >> http://h41379.www4.hpe.com/doc/82final/5932/5932pro_018.html where
> >> LIB$EOMWARN should be LIB$_EOMWARN.
> >>
> >
> > Noted, but comp.os.vms is not a supported way to report bugs.
>
> You saying it's not a bug unless properly reported?
>
> :-)
No. I'm saying there's no guarantee that a random post from Hoff (or anyone else) will be read by anyone in a position to do something about it. I logged a documentation bug report, but one should not expect that to alway be the case.
> > On Friday, April 6, 2018 at 11:59:09 AM UTC-4, Stephen Hoffman wrote:
> >>> Hey VSI: there are instances of a typo in the OpenVMS documentation at
> >> http://h41379.www4.hpe.com/doc/82final/5932/5932pro_018.html where
> >> LIB$EOMWARN should be LIB$_EOMWARN.
> >>
> >
> > Noted, but comp.os.vms is not a supported way to report bugs.
>
> You saying it's not a bug unless properly reported?
>
> :-)
DaveFroble
Apr 9, 2018, 2:51:57 PM4/9/18
to
Richard Levitte
May 1, 2018, 3:10:22 AM5/1/18
to
Den tisdag 3 april 2018 kl. 18:00:36 UTC+2 skrev Richard Levitte:
> [disclaimer: this discusses pseudo-random numbers, not true randomness, so any time you see "random" written, think "pseudo-random". Bill Gunshannon, please pay attention and don't derail]
>
> Hey,
>
> The randomness machinery in OpenSSL has changed radically for upcoming version 1.1.1. The implementation is now a block cipher DRBG as specified in https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf, based on AES-256-CTR.
>
> To get some kind of seeding, this implementation does like before, and relies on data from an entropy aqcuisition function (see below). It worked, by delivering just enough entropy (64 bytes at assumed 4 bits of entropy per byte => 256 bits, which is exactly enough for AES-256-CTR).
>
> Now, we've run into a bit of trouble because of a very recent change that adds the possibility of a nonce to the pool. When no nonce is given, but there's still a nonce size specified, and I quote:
>
> /*
> * NIST SP800-90Ar1 section 9.1 says you can combine getting the entropy
> * and nonce in 1 call by increasing the entropy with 50% and increasing
> * the minimum length to accomadate the length of the nonce.
> * We do this in case a nonce is require and get_nonce is NULL.
> */
>
> So we suddenly find ourselves in a situation where 256 bits of entropy isn't enough, the requirement has been elevated to 384 bits (96 bytes à 4 bits of entropy per byte).
>
> The VMS entropy acquisition function can be seen here:
> https://github.com/openssl/openssl/blob/master/crypto/rand/rand_vms.c
>
> It relies almost entirely on the data returned by sys$getjpiw for these item:
>
> JPI$_BUFIO
> JPI$_CPUTIM
> JPI$_DIRIO
> JPI$_IMAGECOUNT
> JPI$_LAST_LOGIN_I
> JPI$_LOGINTIM
> JPI$_PAGEFLTS
> JPI$_PID
> JPI$_PPGCNT
> JPI$_WSPEAK
> JPI$_FINALEXC
>
> So I'm looking for ideas, are there other items you'd recommend? Other sources? I've eyed through the JPI$_ macros, but am frankly getting a bit blurry eyed and don't feel I quite have a grasp of how variable the data for each item is (i.e. how good entropy it can give).
>
> Cheers,
> Richard
So I did a bit of experiment and collected more data, most of all quite a lot of the RMI$_ items but also a few more JPI$_ items, at a total of 736 bytes per gathering round and a "random" 1 to 10 second sleep between rounds, fed that into a file that I then fed NISTs minimum entropy estimation program (*)... and the result was pretty harsh, an estimate of 0.082 entropy bits per 8 bits of data (**), so roughly 1 bit of entropy per 100 bits of data.
The problem is that a DRBG using AES256-CTR (which is the current new OpenSSL implementation) demands a minimum of 256 bits of entropy, i.e. a little more than 3KiB of data gathered at a maximum pace of 1 736 byte chunk per second. Yay for waiting about 5 seconds before one can even get some randomness out of the DRBG (***)...
I'm not amused.
/Richard
(*) https://github.com/usnistgov/SP800-90B_EntropyAssessment
(**) Running the non-iid program, this is what I get:
: ; ./a.out ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin 8 -v
Opening file: ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin
Running non-IID tests...
Entropic statistic estimates:
Most Common Value Estimate = 0.975224
Collision Test Estimate = 0.902997
Markov Test Estimate = 0.410808
Compression Test Estimate = 0.811274
t-Tuple Test Estimate = 0.0818796
Longest Reapeated Substring Test Estimate = 0.0818772
Predictor estimates:
Multi Most Common in Window (MultiMCW) Test: 100% complete
Correct: 507351
P_avg (global): 0.508671
P_run (local): 0.587891
Multi Most Common in Window (Multi MCW) Test = 0.76638
Lag Test: 100% complete
Correct: 269907
P_avg (global): 0.271051
P_run (local): 0.347168
Lag Prediction Test = 1.52629
MultiMMC Test: 100% complete
Correct: 11700
P_avg (global): 0.011977
P_run (local): 0.444824
Multi Markov Model with Counting (MultiMMC) Prediction Test = 1.16869
LZ78Y Test: 99% complete
Correct: 572107
P_avg (global): 0.573391
P_run (local): 0.615723
LZ78Y Prediction Test = 0.699647
Min Entropy: 0.0818772
(***) There's a thought that libcrypto could start a separate thread upon initialization that would gather data in the background, say once a second... it will take about 5 seconds for it to gather enough entropy at a 736 bytes per second rate.
> [disclaimer: this discusses pseudo-random numbers, not true randomness, so any time you see "random" written, think "pseudo-random". Bill Gunshannon, please pay attention and don't derail]
>
> Hey,
>
> The randomness machinery in OpenSSL has changed radically for upcoming version 1.1.1. The implementation is now a block cipher DRBG as specified in https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf, based on AES-256-CTR.
>
> To get some kind of seeding, this implementation does like before, and relies on data from an entropy aqcuisition function (see below). It worked, by delivering just enough entropy (64 bytes at assumed 4 bits of entropy per byte => 256 bits, which is exactly enough for AES-256-CTR).
>
> Now, we've run into a bit of trouble because of a very recent change that adds the possibility of a nonce to the pool. When no nonce is given, but there's still a nonce size specified, and I quote:
>
> /*
> * NIST SP800-90Ar1 section 9.1 says you can combine getting the entropy
> * and nonce in 1 call by increasing the entropy with 50% and increasing
> * the minimum length to accomadate the length of the nonce.
> * We do this in case a nonce is require and get_nonce is NULL.
> */
>
> So we suddenly find ourselves in a situation where 256 bits of entropy isn't enough, the requirement has been elevated to 384 bits (96 bytes à 4 bits of entropy per byte).
>
> The VMS entropy acquisition function can be seen here:
> https://github.com/openssl/openssl/blob/master/crypto/rand/rand_vms.c
>
> It relies almost entirely on the data returned by sys$getjpiw for these item:
>
> JPI$_BUFIO
> JPI$_CPUTIM
> JPI$_DIRIO
> JPI$_IMAGECOUNT
> JPI$_LAST_LOGIN_I
> JPI$_LOGINTIM
> JPI$_PAGEFLTS
> JPI$_PID
> JPI$_PPGCNT
> JPI$_WSPEAK
> JPI$_FINALEXC
>
> So I'm looking for ideas, are there other items you'd recommend? Other sources? I've eyed through the JPI$_ macros, but am frankly getting a bit blurry eyed and don't feel I quite have a grasp of how variable the data for each item is (i.e. how good entropy it can give).
>
> Cheers,
> Richard
So I did a bit of experiment and collected more data, most of all quite a lot of the RMI$_ items but also a few more JPI$_ items, at a total of 736 bytes per gathering round and a "random" 1 to 10 second sleep between rounds, fed that into a file that I then fed NISTs minimum entropy estimation program (*)... and the result was pretty harsh, an estimate of 0.082 entropy bits per 8 bits of data (**), so roughly 1 bit of entropy per 100 bits of data.
The problem is that a DRBG using AES256-CTR (which is the current new OpenSSL implementation) demands a minimum of 256 bits of entropy, i.e. a little more than 3KiB of data gathered at a maximum pace of 1 736 byte chunk per second. Yay for waiting about 5 seconds before one can even get some randomness out of the DRBG (***)...
I'm not amused.
/Richard
(*) https://github.com/usnistgov/SP800-90B_EntropyAssessment
(**) Running the non-iid program, this is what I get:
: ; ./a.out ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin 8 -v
Opening file: ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin
Running non-IID tests...
Entropic statistic estimates:
Most Common Value Estimate = 0.975224
Collision Test Estimate = 0.902997
Markov Test Estimate = 0.410808
Compression Test Estimate = 0.811274
t-Tuple Test Estimate = 0.0818796
Longest Reapeated Substring Test Estimate = 0.0818772
Predictor estimates:
Multi Most Common in Window (MultiMCW) Test: 100% complete
Correct: 507351
P_avg (global): 0.508671
P_run (local): 0.587891
Multi Most Common in Window (Multi MCW) Test = 0.76638
Lag Test: 100% complete
Correct: 269907
P_avg (global): 0.271051
P_run (local): 0.347168
Lag Prediction Test = 1.52629
MultiMMC Test: 100% complete
Correct: 11700
P_avg (global): 0.011977
P_run (local): 0.444824
Multi Markov Model with Counting (MultiMMC) Prediction Test = 1.16869
LZ78Y Test: 99% complete
Correct: 572107
P_avg (global): 0.573391
P_run (local): 0.615723
LZ78Y Prediction Test = 0.699647
Min Entropy: 0.0818772
(***) There's a thought that libcrypto could start a separate thread upon initialization that would gather data in the background, say once a second... it will take about 5 seconds for it to gather enough entropy at a 736 bytes per second rate.
Richard Levitte
May 1, 2018, 3:35:21 AM5/1/18
to
https://github.com/levitte/vms-experiments/tree/master/entropy-gathering
Cheers,
Richard
Stephen Hoffman
May 1, 2018, 11:39:52 AM5/1/18
to
On 2018-05-01 07:10:19 +0000, Richard Levitte said:
> So I did a bit of experiment and collected more data, most of all quite
> a lot of the RMI$_ items but also a few more JPI$_ items, at a total of
> 736 bytes per gathering round and a "random" 1 to 10 second sleep
> between rounds, fed that into a file that I then fed NISTs minimum
> entropy estimation program (*)... and the result was pretty harsh, an
> estimate of 0.082 entropy bits per 8 bits of data (**), so roughly 1
> bit of entropy per 100 bits of data.
I'd liberally mix in $purge_ws (for 0x0-0xffffffffffffffffff) and
> So I did a bit of experiment and collected more data, most of all quite
> a lot of the RMI$_ items but also a few more JPI$_ items, at a total of
> 736 bytes per gathering round and a "random" 1 to 10 second sleep
> between rounds, fed that into a file that I then fed NISTs minimum
> entropy estimation program (*)... and the result was pretty harsh, an
> estimate of 0.082 entropy bits per 8 bits of data (**), so roughly 1
> bit of entropy per 100 bits of data.
$resched calls and maybe toss around some ASTs using some mix of those
into your existing code trying to churn up some entropy, and through
all that sprinkle multiple clock_gettime (V7.3-2 and later) or
sys$rpcc_64 calls or rpcc built-in calls or probably better rscc
built-in calls, all in the service of added entropy. Maybe add a few
briefly-present busy-work compute-bound subprocesses while all that's
running; invoking the subprocesses using some busy-work option embedded
in the main OpenSSL utility or otherwise. The Alpha rpcc and rscc
stuff has been around for a long while, and the APIs did get ported to
Itanium. http://h41379.www4.hpe.com/wizard/wiz_4004.html et al.
This'd all be far easier if, you know, there was an entropy pool in the
kernel and APIs, but that's fodder for some future OpenVMS release.
Snowshoe
May 1, 2018, 3:52:10 PM5/1/18
to
On 4/3/2018 12:00 PM, Richard Levitte wrote:
> [disclaimer: this discusses pseudo-random numbers, not true randomness, so any time you see "random" written, think "pseudo-random". Bill Gunshannon, please pay attention and don't derail]
>
> Hey,
>
> The randomness machinery in OpenSSL has changed radically for upcoming version 1.1.1. The implementation is now a block cipher DRBG as specified in https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf, based on AES-256-CTR.
>
> To get some kind of seeding, this implementation does like before, and relies on data from an entropy aqcuisition function (see below). It worked, by delivering just enough entropy (64 bytes at assumed 4 bits of entropy per byte => 256 bits, which is exactly enough for AES-256-CTR).
>
> Now, we've run into a bit of trouble because of a very recent change that adds the possibility of a nonce to the pool. When no nonce is given, but there's still a nonce size specified, and I quote:
>
> /*
> * NIST SP800-90Ar1 section 9.1 says you can combine getting the entropy
> * and nonce in 1 call by increasing the entropy with 50% and increasing
> * the minimum length to accomadate the length of the nonce.
> * We do this in case a nonce is require and get_nonce is NULL.
> */
>
> So we suddenly find ourselves in a situation where 256 bits of entropy isn't enough, the requirement has been elevated to 384 bits (96 bytes à 4 bits of entropy per byte).
Get the equivalent of MC LANCP SHOW DEVICE Exxx:/COUNTER to get some
> [disclaimer: this discusses pseudo-random numbers, not true randomness, so any time you see "random" written, think "pseudo-random". Bill Gunshannon, please pay attention and don't derail]
>
> Hey,
>
> The randomness machinery in OpenSSL has changed radically for upcoming version 1.1.1. The implementation is now a block cipher DRBG as specified in https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf, based on AES-256-CTR.
>
> To get some kind of seeding, this implementation does like before, and relies on data from an entropy aqcuisition function (see below). It worked, by delivering just enough entropy (64 bytes at assumed 4 bits of entropy per byte => 256 bits, which is exactly enough for AES-256-CTR).
>
> Now, we've run into a bit of trouble because of a very recent change that adds the possibility of a nonce to the pool. When no nonce is given, but there's still a nonce size specified, and I quote:
>
> /*
> * NIST SP800-90Ar1 section 9.1 says you can combine getting the entropy
> * and nonce in 1 call by increasing the entropy with 50% and increasing
> * the minimum length to accomadate the length of the nonce.
> * We do this in case a nonce is require and get_nonce is NULL.
> */
>
> So we suddenly find ourselves in a situation where 256 bits of entropy isn't enough, the requirement has been elevated to 384 bits (96 bytes à 4 bits of entropy per byte).
entropy related to the comings and goings of LAN traffic. Should be good
for a few bytes of entropy. The more devices you have, the merrier.
MC SCACP SHOW CHANNEL /COUNTER will get you some more but because this
goes on the lan the actual entropy will be less than the sum of the two.
Stephen Hoffman
May 2, 2018, 12:01:08 PM5/2/18
to
FWIW... Linux cryptographic pseudo-random number generator (CPRNG)
weaknesses; the details of security and of CPRNGs are harder and more
complex than might be realized...
https://bugs.chromium.org/p/project-zero/issues/detail?id=1559
What's been discussed in this newsgroup thread is a user-level
work-around for the lack of a CPRNG and a lack of system-level entropy
pool and related collection in OpenVMS.
0 new messages