"bad select 38" (OpenSSL on VMS)

[Neil Rieck]

Gents,

I finally found a few spare hours to solve the "bad select 38" problem
as seen after executing DCL command "openssl s_client -connect". I
have a fix but it is not yet "production quality" and will never never
be as cool as solutions previously produced by the OpenVMS engineers.

It is important to note that "bad select 38" is never seen on the HP-
sanctioned versions of OpenSSL, just the ports done by third party
programmers (a.k.a. we, the unwashed public).

###

First off, here's a few basics about "UNIX I/O" and "select()".

I'm sure most of you already know that the standard UNIX devices
(stdin, stdout, and stderr) correspond to FD (file-descriptors)
numbers: 0, 1, and 2. As you open new i/o channels (files or sockets),
you use up more file-descriptors. In the VMS world these UNIX devices
map directly to SYS$INPUT, SYS$OUTPUT, and SYS$ERROR.

In the UNIX world, if you want to stall (block) your program while
waiting for I/O to complete, you indicate channels of interest by
setting associated bits in one of two longs (one for read, one for
write), then pass them on to select(). So if you wanted to only wait
for "read readiness from stdin" you would set bit-0 in the read bit-
field then pass it on to select(). If you wanted to wait for any one
of several devices, you would just set more bits.

But here is where "our VMS problem" develops. In the UNIX world "you
can use select() to test all i/o channels" while in the VMS world "you
may only use select() to test network channels (sockets)". In fact,
error #38 is defined in file "UCX$INETDEF.H" with the label "ENOTSOCK"
which is mapped to VMS code SS$_NOTNETDEV (Socket operation on
nonsocket)

As is typical of my luck, I was working backwards by starting with the
code, but had I checked the UNIX portability document here:

UNIX, Linux to OpenVMS Porting Guidelines
http://h71000.www7.hp.com/portability/portingguidelines.html
http://h71000.www7.hp.com/portability/portingguidelines.html#_Toc262045735

...then I could have saved myself much time.

So what to do? There are two ways out of this debacle:
1) test for keyboard input without using select()
(this is easy to do but you might miss i/o from other channels if
not careful)
2) follow the advice of the portability guide then write some code to
employ a socket to connect to SYS$INPUT (or SYS$COMMAND).

If you want to see an example of how one programmer ("I Think" it
might have been a Compaq employee but have no proof because the
program changes were not documented) solved this problem, then click
here:
http://www3.sympatico.ca/n.rieck/demo_vms_html/openvms_demo_index.html#openssl

Neil Rieck
Kitchener / Waterloo / Cambridge,
Ontario, Canada.
http://www3.sympatico.ca/n.rieck/docs/openvms_notes_ssl.html#bad-select-38

[Stephen Hoffman]

On the way to another SSL1-related bug, I've noticed that the s_client
bug Neil reported (below) now arises within the SSL1 kit, circa HP
AXPVMS SSL1 V1.0-2C.

It appears that the fix from the old HP SSL ports has been dropped from
the SSL1 kit.

I'll be having a look at the work-around mentioned below, and (yes)
will likely also get the underlying bug reported to HPE and/or VSI.

Since the posting I'm replying to here is a few years old and might
have expired out of your local newsgroup caches, I'm including the full
text of the previous discussion...

--
Pure Personal Opinion | HoffmanLabs LLC

[doug...@vmssoftware.com]

The fix for this issue is included in VSI's version of SSL1 (1.0.2h) as released on OpenVMS V8.4-2L1.

[Stephen Hoffman]

On 2016-09-13 20:02:24 +0000, doug...@vmssoftware.com said:

> The fix for this issue is included in VSI's version of SSL1 (1.0.2h) as
> released on OpenVMS V8.4-2L1.

Yeah; looks like this s_client bug is fixed in HPE 1.0.2H, too.

Clearly missed that update on this server.

Underlying bugs and related omissions here include how the C select()
call implementation is more limited on OpenVMS than on Unix, and that
there's no mechanism in OpenVMS for update notifications. That, and
an administrator that didn't check for current versions of effected
software. All are longstanding issues.

[John Reagan]

There is a CRTL feature logical to ask select() to ignore file descriptors instead of causing an error. I don't have it on-hand, I'm at the hospital visiting my father.

[Stephen Hoffman]

On 2016-09-13 22:31:07 +0000, John Reagan said:

> There is a CRTL feature logical to ask select() to ignore file
> descriptors instead of causing an error. I don't have it on-hand, I'm
> at the hospital visiting my father.

That likely won't work here, as the s_client code uses standard input
as one of the sources for the select().

Was going to use the work-around listed, but that's not necessary as
upgrading to SSL1 1.0.1h has resolved this bug.

Still trying to sort out the underlying TLS problem that led to testing
connections with the s_client in the SSL1 kit, but that's fodder for
another discussion and another time, and more debugging.

Haven't seen whether the changes were fed back, but didn't look too
carefully at the github master for the file, and I know the changes
aren't in the s_client from 1.0.1 head.

I'd expect that enhancing select() to better implement the expected
behavior from other systems — removing the present OpenVMS limitations
— is on the list somewhere, but probably not near the top.

Hope things work out.

[Richard Levitte]

Den onsdag 21 mars 2012 kl. 01:55:30 UTC+1 skrev Neil Rieck:
> If you want to see an example of how one programmer ("I Think" it
> might have been a Compaq employee but have no proof because the
> program changes were not documented) solved this problem, then click
> here:
> http://www3.sympatico.ca/n.rieck/demo_vms_html/openvms_demo_index.html#openssl

I have received that exact code from VSI, with reference to HP SSL, that should be enough confirmation that this is indeed made by a DeCompHP employee.

Unfortunately, I ran out of time evaluating the code before the release of OpenSSL 1.1.0. I have the code and s_client / s_server changes safely tucked in a branch and am looking them over right now. I expect it to become part of the next OpenSSL 1.0.2 and 1.1.0 updates.

Cheers,
Richard

[John E. Malmberg]

Hidden in the source code for many of the updated GNV utilities,
including Bash and coreutils is a wrapper for select() and poll() that
handles terminals, sockets, pipes, and if you prepare things right with
dup2() to reserve the descriptor number, X11 (implemented in VMS as
event flags).

Regards,
-John
wb8...@qsl.net_work

[John Reagan]

What's the license on that code?

[Craig A. Berry]

From a very quick glance at:

<https://sourceforge.net/p/gnv/coreutils/ci/default/tree/vms_source/coreutils/vms/vms_terminal_io.c>

the license looks BSD-like, but John M. holds the copyright and could
release it to you under any license that is mutually agreeable.

[Neil Rieck]

Okay so here are two links documenting my work way back in 2012

http://www3.sympatico.ca/n.rieck/docs/openvms_notes_ssl.html#s_client-caveat
http://www3.sympatico.ca/n.rieck/docs/openvms_notes_ssl.html#bad-select-38

At that time, all the HP-published code for "OpenSSL on OpenVMS" was broken starting with the first release after the acquisition of Compaq. So I engaged in a bit of archeology to acquire all the OpenSSL source code published by Compaq.

http://www3.sympatico.ca/n.rieck/demo_vms_html/openvms_demo_index.html#openssl

I discovered a piece of code (term_sock.c) which appeared in the Compaq library which "did not include any author information" or "copyright info" which I republished here:

http://www3.sympatico.ca/n.rieck/demo_vms_html/term_sock_c.html

I believe Windows code needs to jump through a similar hoop because stdin on a Windows machine also doesn't employ select()

Neil Rieck
Waterloo, Ontario, Canada.
http://www3.sympatico.ca/n.rieck/

[John E. Malmberg]

On 9/13/2016 10:32 PM, Neil Rieck wrote:
>
> I discovered a piece of code (term_sock.c) which appeared in the
> Compaq library which "did not include any author information" or
> "copyright info" which I republished here:
>
> http://www3.sympatico.ca/n.rieck/demo_vms_html/term_sock_c.html >

That method uses ASTs to have terminal input feed a TCP/IP socket so
that the actual select() or poll() can be used. In this case, the
terminal input has already been read into an additional buffer.

It has an overhead of 3 extra channels for each terminal input monitored.

I think that the calling program would need a wrapper to main() or VMS
specific code to dup2 the new socket to stdin.


My wrapper method opens a channel to the devices so uses sys$qiow()
calls to test if input is available on a terminal or mailbox device. No
actual reading of the input is done.

It has an overhead of one extra channel per device monitored. It was
designed for use the now old port of the gtk+ components so that it
could with terminals, pipes, and X11.

The design allowed the extra channel to be created/deleted on each call,
or cached if more wrappers are needed for other I/O in the program, such
as simulating the termios API.

It allows an unmodified source program to be used, except for the case
of X11, where a wrapper to main() should be used to dup2() file
descriptors to /dev/null for the X11 eventflag to make sure nothing
grabs it. Since X11 uses lib$get_ef to get the event flag, and by
default only two are free, it is easy for a wrapper to make sure that
those file descriptors are not used for real I/O.

Regards,
-John
wb8tyw@qsl_network

[Stephen Hoffman]

On 2016-09-14 12:56:03 +0000, John E. Malmberg said:

> On 9/13/2016 10:32 PM, Neil Rieck wrote:
>>

>> I discovered a piece of code (term_sock.c) ...

>
> That method uses ASTs to have terminal input feed a TCP/IP socket so
> that the actual select() or poll() can be used. In this case, the
> terminal input has already been read into an additional buffer.
>

> ...


The select() call implementation has been broken — sorry,
"feature-constrained" — for ~decades. How long does a common C call
need to remain broken before it can be fixed?

The dependencies are increasing in length and complexities, the work
arounds more complex, the effort higher, the resulting end-user
application code more convoluted and tricky, the more glue code, and
the risks of bugs and security exposures increase, as does the
development and maintenance costs. From the vendor side, the scale
and scope and speed of the product requirements and the features and
enhancements is increasing, too. Massively.

Fun times.

But I'm really getting tired of jackets and workarounds and everybody
repeatedly porting over or reimplementing their own version of
available open source. Jackets all the way down. it all makes for
more costs and more work and more application source code and more
complexity and more instability and more insecurity and more
maintenance.

As for this "bad select 38" error, that's been hacked around (again!)
in SSL1 1.0.1h, and s_client is now working. Things are working
again, at least until some other application smashes into select() or
some other gap.

[Neil Rieck]

Yep, well, I didn't write that code. I suspect someone at Compaq did -AND- they may have intended to get it into the official OpenSSL distro.

My advice you you is to get your working code into the OpenSSL distro so that others (including HP/HPE/VSI) aren't required to engage in a coding ritual every time a new version of OpenSSL is published.

Neil Rieck
Waterloo, Ontario, Canada.

#########

[Richard Levitte]

Den fredag 16 september 2016 kl. 13:41:25 UTC+2 skrev Neil Rieck:
> My advice you you is to get your working code into the OpenSSL distro so that others (including HP/HPE/VSI) aren't required to engage in a coding ritual every time a new version of OpenSSL is published.

Done, actually. The changes finally got merged into the 1.1.0 and 1.0.2 branches, so whenever 1.1.0a and 1.0.2i are released, mainstream OpenSSL will finally be free of this particular issue.

Cheers,
Richard

[Stephen Hoffman]

"OpenSSL will finally be free of this particular issue" when OpenVMS
has a functional select() call.

When the hack-around you've just added into OpenSSL can be excised.

It's these incompatibilities and the associated #ifdef __vms morasses
that have helped establish the reputation of OpenVMS in the computing
community.

It's these misfeatures that deservedly and sometimes even gleefully get
any latent OpenVMS support and its associated complexity ripped out of
open source projects, too.

[John E. Malmberg]

Which flavor of the changes?

One that ignores everything but TCP/IP sockets?

One that works only for sockets and terminal I/O?

One that works for Sockets, Terminal I/O, Pipes, and X11?

Regards,
-John
wb8...@qsl.net_work

[Craig A. Berry]

The change is here:

<https://github.com/openssl/openssl/commit/0d3b65832c6fa94c1d1cfa2f99335f323e3227c1>

At first glance, I think it's only sockets and terminals. Also at first
glance, it's only including iosbdef.h on Alpha, and otherwise rolling
its own _iosb struct. Surely that should be #ifndef _VAX rather than
#ifdef __alpha?

[Stephen Hoffman]

On 2016-09-16 14:56:24 +0000, Craig A. Berry said:

> Also at first glance, it's only including iosbdef.h on Alpha, and
> otherwise rolling its own _iosb struct. Surely that should be #ifndef
> _VAX rather than #ifdef __alpha?

Yeah; that #ifdef __alpha sends Itanium down the wrong path. It'll
very likely work (which is why nobody noticed?), but OpenVMS I64 does
provide the iosbdef.h include file, as undoubtedly will future
platforms.

And (in all seriousness) does anybody still need __VAXC? Even as
early as OpenVMS VAX V5.5-2, DEC C is available if you load the
AACRT060 patch kit, or the stock CRTL on later releases. And I'm not
so sure that the VAXCRTL would even work here given the subset of
routines available back then, and given the VAX C compiler is known to
mis-compile the published MD5 code and undoubtedly has other issues.
I'd want to test the code that far back, and would certainly expect to
encounter either compiler or run-time problems, or both.

[Richard Levitte]

Den fredag 16 september 2016 kl. 15:02:39 UTC+2 skrev Stephen Hoffman:
> On 2016-09-16 12:48:18 +0000, Richard Levitte said:
>
> > Den fredag 16 september 2016 kl. 13:41:25 UTC+2 skrev Neil Rieck:
> >> My advice you you is to get your working code into the OpenSSL distro
> >> so that others (including HP/HPE/VSI) aren't required to engage in a
> >> coding ritual every time a new version of OpenSSL is published.
> >
> > Done, actually. The changes finally got merged into the 1.1.0 and
> > 1.0.2 branches, so whenever 1.1.0a and 1.0.2i are released, mainstream
> > OpenSSL will finally be free of this particular issue.
>
> "OpenSSL will finally be free of this particular issue" when OpenVMS
> has a functional select() call.

Errr, color me confused, but isn't this thread about the issue that OpenVMS currently has a select() that works with sockets only? Did I miss something?

> When the hack-around you've just added into OpenSSL can be excised.

Sure, as soon as we no longer get stuff like "bad select 38" for trying to use it on fd 0.

> It's these incompatibilities and the associated #ifdef __vms morasses
> that have helped establish the reputation of OpenVMS in the computing
> community.
>
> It's these misfeatures that deservedly and sometimes even gleefully get
> any latent OpenVMS support and its associated complexity ripped out of
> open source projects, too.

I agree with you there, partly... VMS is not Unix, and when someone tries to have Unix as the holy standard, things are sure to get confusing.

That being said, the socket-only select() is lame.

Cheers,
Richard

[Richard Levitte]

Den fredag 16 september 2016 kl. 17:11:48 UTC+2 skrev Stephen Hoffman:
> On 2016-09-16 14:56:24 +0000, Craig A. Berry said:
>
> > Also at first glance, it's only including iosbdef.h on Alpha, and
> > otherwise rolling its own _iosb struct. Surely that should be #ifndef
> > _VAX rather than #ifdef __alpha?
>
> Yeah; that #ifdef __alpha sends Itanium down the wrong path. It'll
> very likely work (which is why nobody noticed?), but OpenVMS I64 does
> provide the iosbdef.h include file, as undoubtedly will future
> platforms.

If someone has a suggested change, please send them to me. I haven't made any functional change to vms_term_sock yet...

> And (in all seriousness) does anybody still need __VAXC?

Well, the build scripts in the 1.0.2 branch still support VAX C (best effort, mind you, and quite frankly, *my* effort has been zero), and will stay there. From 1.1.0 and on, however, VAX C and GNU C support have been ripped out and only DEC C remains. In fact, there is no VAX support at all.

> Even as
> early as OpenVMS VAX V5.5-2, DEC C is available if you load the
> AACRT060 patch kit, or the stock CRTL on later releases. And I'm not
> so sure that the VAXCRTL would even work here given the subset of
> routines available back then, and given the VAX C compiler is known to
> mis-compile the published MD5 code and undoubtedly has other issues.
> I'd want to test the code that far back, and would certainly expect to
> encounter either compiler or run-time problems, or both.

I'm sure... It would be interesting to try building the latest 1.0.2 on VAX and see how it flies. Alas, I haven't had a VAX for a long time...

Cheers,
Richard

[John Reagan]

On Friday, September 16, 2016 at 11:11:48 AM UTC-4, Stephen Hoffman wrote:

>
> And (in all seriousness) does anybody still need __VAXC?

__VAXC is what is set with /STANDARD=VAXC. That doesn't mean "I'm a VAX compiler". That is __VAX.

That said, I'm all for tossing /STANDARD=VAXC over the side, but such things tend to hang on for a long time until you are forced.

Heck, even gcc and clang still support pre-standard C's preprocessor with "-traditional".

[Richard Levitte]

Also, there's nothing stopping anyone from creating a vmslib which gives a compatibility / porting layer. I recall starting on something like that longago, and I can see myself putting something like that together. Having a fd friendly select() would be one item which I bet would be popular, for example.

Of course, we can keep lamenting all the things that are lacking in the current DECompHP tools, but how does that help us in the short run? We could also do what the open source community is known for, helping each other by building the tools ourselves. Of course, I'm hopeful that it ends up inspiring VSI to make better tools ;-)

Cheers,
Richard

[Stephen Hoffman]

On 2016-09-17 17:54:55 +0000, John Reagan said:

> On Friday, September 16, 2016 at 11:11:48 AM UTC-4, Stephen Hoffman wrote:
>
>>
>> And (in all seriousness) does anybody still need __VAXC?
>
> __VAXC is what is set with /STANDARD=VAXC. That doesn't mean "I'm a
> VAX compiler". That is __VAX.

I'm well aware.

I find the use of VAX C an excellent flag to identify C source code
that tends to be unmaintained or ill-maintained, and of source code
that's often buggy, unstable and/or contains security vulnerabilities.

> That said, I'm all for tossing /STANDARD=VAXC over the side, but such
> things tend to hang on for a long time until you are forced.

Then go do it. Have the courage of your convictions.

Establish and then communicate your future plans. That and the
ensuing discussions are part of what boot camp is useful for to VSI and
to customers, after all.

In this case, announce the end of VAX C. That VAX C code won't make
it across to x86-64.

(I'm here assuming there'll be little new C work on OpenVMS I64 or earlier.)

If folks really need VAX C and if VSI has the cycles, make'm pay for it.

I'd likely also announce that specific identified existing (and
deprecated) APIs are also subject to removal.

Then start providing replacements for other problematic APIs, and then
deprecate those problematic APIs, and then remove them.

I'd be seriously tempted to announce the deprecation and eventual
removal of DECnet, for that matter.

This'll be painful and it'll be a shock to some existing sites, but so
is making meaningful advancements within the constraints of ~forty
years' accretion of immutable errors and security vulnerabilities.

We all know the only outcome of the current historic-preservation
strategy, after all.

[David Froble]

Booo! Hisssss!

Ok, we know it's not secure. Run at your own risk.

I'm guessing that DECnet users use it only in house, for FAL and such, so if the
in house environment is secure, then security isn't an issue for DECnet.

If it's not going to take up time and effort, then why kill it off?

I personally find it can be useful.

It sure is handy when you need to shutdown and re-start TCP/IP on a remote (but
in house) system.

[Simon Clubley]

On 2016-09-17, David Froble <da...@tsoft-inc.com> wrote:

> Stephen Hoffman wrote:
>>
>> I'd be seriously tempted to announce the deprecation and eventual
>> removal of DECnet, for that matter.

Phase IV only or Phase V as well ?

>
> Booo! Hisssss!
>
> Ok, we know it's not secure. Run at your own risk.
>
> I'm guessing that DECnet users use it only in house, for FAL and such, so if the
> in house environment is secure, then security isn't an issue for DECnet.
>

As Dirk has just reminded me, Phase V DECnet is still in active use in
certain quarters.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world

[John Reagan]

On Saturday, September 17, 2016 at 4:13:39 PM UTC-4, Stephen Hoffman wrote:
> On 2016-09-17 17:54:55 +0000, John Reagan said:
>
> > That said, I'm all for tossing /STANDARD=VAXC over the side, but such
> > things tend to hang on for a long time until you are forced.
>
> Then go do it. Have the courage of your convictions.
>
> Establish and then communicate your future plans. That and the
> ensuing discussions are part of what boot camp is useful for to VSI and
> to customers, after all.
>
> In this case, announce the end of VAX C. That VAX C code won't make
> it across to x86-64.

Since the /STANDARD=VAXC support is in the frontend (and headers to get the old-style prototypes), that comes for free with our strategy of hooking the DEC C frontend to LLVM. It actually isn't a big deal in the frontend. However, we will not add such support into clang. If you want to use THAT compiler to gain access to C11 features, etc then you will not see VAXC support (or many other things that exist in the DEC C frontend). However, that will not prevent you from bringing your current code from Itanium to x86.

[Johnny Billquist]

On 2016-09-17 14:07, Richard Levitte wrote:
>
> I agree with you there, partly... VMS is not Unix, and when someone tries to have Unix as the holy standard, things are sure to get confusing.

Agree.

> That being said, the socket-only select() is lame.

Isn't that true of select() in Windows as well, or was that restriction
lifted after I last touched Windows (XP era). (Or do I remember wrong?)

Johnny

[Johnny Billquist]

simh is your friend on that one... ;-)

Johnny

[Richard Levitte]

Right you are... I forgot.

Ref: https://msdn.microsoft.com/en-us/library/windows/desktop/ms740141(v=vs.85).aspx

Cheers,
Richard

[Richard Levitte]

True, thanks for the reminder (although, between you and me, I'm not entirely sure I really wanted to get a reminder ;-) )

Cheers,
Richard

[johnwa...@yahoo.co.uk]

You don't remember wrong, and I don't think it's
changed.

From memory, and after briefly trying to check the
world of Windows as it is today, calling select()
in a Win32 application will return the error
WSAENOTSOCK
if one of the specified fds ins anything other than
a socket. See e.g. (URL will wrap)
https://msdn.microsoft.com/en-gb/library/windows/desktop/ms740141(v=vs.85).aspx

But apparently it's only VMS that's got a problem
with this behaviour. Windows has other redeeming
features that make this behaviour a non-problem
on a Window box (presumably).

Those who care about proper POSIX-like (not MS-style)
select() behaviour on a Window box may wish to look
at Cygwin, who have this to say about their select()
implementation:

"Select

The UNIX select function is another call that does not map cleanly on top of the Win32 API. Much to our dismay, we discovered that the Win32 select in Winsock only worked on socket handles. Our implementation allows select to function normally when given different types of file descriptors (sockets, pipes, handles, and a custom /dev/windows Windows messages pseudo-device).

Upon entry into the select function, the first operation is to sort the file descriptors into the different types. There are then two cases to consider. The simple case is when at least one file descriptor is a type that is always known to be ready (such as a disk file). In that case, select returns immediately as soon as it has polled each of the other types to see if they are ready. The more complex case involves waiting for socket or pipe file descriptors to be ready. This is accomplished by the main thread suspending itself, after starting one thread for each type of file descriptor present. Each thread polls the file descriptors of its respective type with the appropriate Win32 API call. As soon as a thread identifies a ready descriptor, that thread signals the main thread to wake up. This case is now the same as the first one since we know at least one descriptor is ready. So select returns, after polling all of the file descriptors one last time."

from
https://cygwin.com/cygwin-ug-net/highlights.html

Have a lot of fun.

[Richard Levitte]

Den måndag 19 september 2016 kl. 14:38:27 UTC+2 skrev johnwa...@yahoo.co.uk:
> From memory, and after briefly trying to check the
> world of Windows as it is today, calling select()
> in a Win32 application will return the error
> WSAENOTSOCK
> if one of the specified fds ins anything other than
> a socket. See e.g. (URL will wrap)
> https://msdn.microsoft.com/en-gb/library/windows/desktop/ms740141(v=vs.85).aspx
>
> But apparently it's only VMS that's got a problem
> with this behaviour. Windows has other redeeming
> features that make this behaviour a non-problem
> on a Window box (presumably).

Not sure how you mean that VMS has a problem with that behaviour while Windows doesn't... if you're talking about OpenSSL, the problem has been worked around quite differently on Windows than on VMS, but either way, there had to be a workaround.

Cheers,
Richard

[Hunter Goatley]

On 9/17/2016 7:15 AM, Richard Levitte wrote:
>
> I'm sure... It would be interesting to try building the latest 1.0.2 on VAX and see how it flies. Alas, I haven't had a VAX for a long time...

I tried not too long ago.

I ended up sticking with the last 0.9.8 that I had built. (I don't even
remember what problems I encountered, just that I had enough that I
decided it wasn't worth the time and effort.)

--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
goath...@goatley.com http://hunter.goatley.com/

[Mark Daniel]

On 20/09/2016 3:34 AM, Hunter Goatley wrote:
> On 9/17/2016 7:15 AM, Richard Levitte wrote:
>>
>> I'm sure... It would be interesting to try building the latest 1.0.2
>> on VAX and see how it flies. Alas, I haven't had a VAX for a long
>> time...
>
>
> I tried not too long ago.
>
> I ended up sticking with the last 0.9.8 that I had built. (I don't even
> remember what problems I encountered, just that I had enough that I
> decided it wasn't worth the time and effort.)

No /long long/ available for v1.0.0f for starters.

[Richard Levitte]

Ah, right, that's a major bummer...
(I never understood why there never was any long long support for VAX. After all, quadwords?)

Cheers,
Richard

[John E. Malmberg]

It is on my Todo list to find a GCC/VAX install kit either on my of my
disk backups or on-line and see if it understood long long.

One problem with GCC/VAX 2.7.x (the last I used) is the source provided
for gcc.exe, which is just a wrapper program, does not match the binary
provided, so gcc.exe can not be rebuilt or updated with out some work.

Regards,
-John
wb8tyw@qsl_network

[John E. Malmberg]

On 9/17/2016 2:41 PM, Richard Levitte wrote:
> Den lördag 17 september 2016 kl. 14:07:32 UTC+2 skrev Richard Levitte:
>> Den fredag 16 september 2016 kl. 15:02:39 UTC+2 skrev Stephen Hoffman:
>>> It's these misfeatures that deservedly and sometimes even gleefully get
>>> any latent OpenVMS support and its associated complexity ripped out of
>>> open source projects, too.
>>
>> I agree with you there, partly... VMS is not Unix, and when
>> someone tries to have Unix as the holy standard, things are sure to get
>> confusing.
>>
>> That being said, the socket-only select() is lame.
>
> Also, there's nothing stopping anyone from creating a vmslib which
> gives a compatibility / porting layer.

A full library turns out to be so complex to use that only the author
ends up comfortable with it. And it also has issues with keeping track
of which version of VMS needs what replacement routine.

And there is also can be the problem where program "a" needs a different
and conflicting hack than program "b".

So I am trying to do things for GNV / VMS-PORTS to be more modularized,
where I select the workarounds needed.

Once I get all of the GNV components building on Alpha / IA64, I will
look to see if those modules could be used to build an add-on porting
library, which pretty much also will need an alternate set of header TLB
libraries to make it easy to use.

Right now, I do not have the cycles to do both the GNV component updates
and try to do a library project properly with documentation and unit tests.

> Of course, we can keep lamenting all the things that are lacking in
> the current DECompHP tools, but how does that help us in the short run?
> We could also do what the open source community is known for, helping
> each other by building the tools ourselves. Of course, I'm hopeful that
> it ends up inspiring VSI to make better tools ;-)

From a hobbyist viewpoint, Until VSI has a hobby license available, I
am stuck with workarounds to the existing HP code.

Regards,
-John
wb8...@qsl.net_work

[John E. Malmberg]

So if I run s_client through popen() or otherwise through a pipe such as
being launched by GNV Bash it will fail?

Regards,
-John
wb8...@qsl.net_work

[Richard Levitte]

Probably, but also probably just because it assigns a channel to SYS$COMMAND. When calling a program with popen or through a pipe launched by GNV Bash, is there a SYS$INPUT? What happens if that's used instead of SYS$COMMAND?

Gods, it's been way too long since I did this kind of stuff...

Cheers,
Richard

[RobertsonEricW]

On Monday, September 19, 2016 at 7:58:23 PM UTC-4, Richard Levitte wrote:
> > So if I run s_client through popen() or otherwise through a pipe such as
> > being launched by GNV Bash it will fail?
>
> Probably, but also probably just because it assigns a channel to SYS$COMMAND. When calling a program with popen or through a pipe launched by GNV Bash, is there a SYS$INPUT? What happens if that's used instead of SYS$COMMAND?
>
> Gods, it's been way too long since I did this kind of stuff...
>
> Cheers,
> Richard

Richard, you are correct. It does not work correctly with GNV Bash. some extraneous output appears in the piped output as a result of using SYS$COMMAND:


openssl s_client -connect evn.crm.em2.oraclecloud.com:443
CONNECTED(00000006)
depth=2 C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root
verify error:num=20:unable to get local issuer certificate
---
Certificate chain
0 s:/C=US/ST=CA/L=Redwood Shores/O=Oracle Corporation/OU=Content Management Services IT/CN=*.crm.em2.oraclecloud.com
i:/C=NL/L=Amsterdam/O=Verizon Enterprise Solutions/OU=Cybertrust/CN=Verizon Akamai SureServer CA G14-SHA2
1 s:/C=NL/L=Amsterdam/O=Verizon Enterprise Solutions/OU=Cybertrust/CN=Verizon Akamai SureServer CA G14-SHA2
i:/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Root
2 s:/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Root
i:/C=US/O=GTE Corporation/OU=GTE CyberTrust Solutions, Inc./CN=GTE CyberTrust Global Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFozCCBIugAwIBAgIUJqSsQcjAxLRBtHzCHsF449Qe14QwDQYJKoZIhvcNAQEL
BQAwgY0xCzAJBgNVBAYTAk5MMRIwEAYDVQQHEwlBbXN0ZXJkYW0xJTAjBgNVBAoT
HFZlcml6b24gRW50ZXJwcmlzZSBTb2x1dGlvbnMxEzARBgNVBAsTCkN5YmVydHJ1
c3QxLjAsBgNVBAMTJVZlcml6b24gQWthbWFpIFN1cmVTZXJ2ZXIgQ0EgRzE0LVNI
QTIwHhcNMTYwNTAzMDk0MTM5WhcNMTcwNTAzMDk0MTM5WjCBnTELMAkGA1UEBhMC
VVMxCzAJBgNVBAgTAkNBMRcwFQYDVQQHEw5SZWR3b29kIFNob3JlczEbMBkGA1UE
ChMST3JhY2xlIENvcnBvcmF0aW9uMScwJQYDVQQLEx5Db250ZW50IE1hbmFnZW1l
bnQgU2VydmljZXMgSVQxIjAgBgNVBAMMGSouY3JtLmVtMi5vcmFjbGVjbG91ZC5j
b20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDTNjum0DDy9YS1bqsy
rSA6kdN1lY5fkyFwpirY4ci/M5Gkd1kMdMqT664bscQ1xUhdoAf7cfTLHMhZ9m1B
PPlAoPS4AEDdGwc3ZHIB53PsSsvxWPJ3m6wWvhKB1M6LaX0/gQvmiQgF0BoNN9Km
G46tWaCrVWX3ytCFJ/q43c4RVMqpNNPs9wb1+mzMJsiMyFjKNUXxPlhEmOvuNy8e
NqTbOd9MpTLwfan7eUehwhFbQ3ejvKCEodR3D7sxrwg0V6T3kZ4UMAkbUAw8yG9E
EVhoL1vYotgodKnWoLkNkapf/Hk/czt+ocWUsNnyM2RtS1187KyfROC6Bsg9L6y1
IgV1AgMBAAGjggHnMIIB4zAMBgNVHRMBAf8EAjAAMEwGA1UdIARFMEMwQQYJKwYB
BAGxPgEyMDQwMgYIKwYBBQUHAgEWJmh0dHBzOi8vc2VjdXJlLm9tbmlyb290LmNv
bS9yZXBvc2l0b3J5MIGvBggrBgEFBQcBAQSBojCBnzAtBggrBgEFBQcwAYYhaHR0
cDovL3Zhc3NnMTQyLm9jc3Aub21uaXJvb3QuY29tMDYGCCsGAQUFBzAChipodHRw
czovL2NhY2VydC5hLm9tbmlyb290LmNvbS92YXNzZzE0Mi5jcnQwNgYIKwYBBQUH
MAKGKmh0dHBzOi8vY2FjZXJ0LmEub21uaXJvb3QuY29tL3Zhc3NnMTQyLmRlcjAk
BgNVHREEHTAbghkqLmNybS5lbTIub3JhY2xlY2xvdWQuY29tMA4GA1UdDwEB/wQE
AwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHwYDVR0jBBgwFoAU
+L36r3N3xscb+UtNEafRM6+vchEwPgYDVR0fBDcwNTAzoDGgL4YtaHR0cDovL3Zh
c3NnMTQyLmNybC5vbW5pcm9vdC5jb20vdmFzc2cxNDIuY3JsMB0GA1UdDgQWBBQ5
2ttvTVNcmHbgTUnjk9+qE//QWjANBgkqhkiG9w0BAQsFAAOCAQEARUSJX5f2UgHr
/WVwiEB5gbONg79MIduBjnKQDy1kREl1LdCvQIlqBP1BQNwe650GpQevnKwY/BnI
rC8nSZWVUcS6VCQrLbZ6+dnR/+kyzPiiL4uTfbvhjBguKD5wl/9+c98m/aYbaExy
tRoDofxqJj5Sx4eYKnWdCeZpUD7ZrS+rymqxCrfOPERcGeht+YnUGvsBTdgXxjYB
nW5xh2zjX9ElfLnECllHnjlMyVedLM9uzWxvfGsFKFw0Wda0b+4WAEzaaMar3uLn
NI/lOCT3wA6IXPaOa0O/Qfs8FoMHE+xWdyeMpkRkUBigjzJYudW92BUKv0ItG1BK
IeMWytKjUA==
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Redwood Shores/O=Oracle Corporation/OU=Content Management Services IT/CN=*.crm.em2.oraclecloud.com
issuer=/C=NL/L=Amsterdam/O=Verizon Enterprise Solutions/OU=Cybertrust/CN=Verizon Akamai SureServer CA G14-SHA2
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 4495 bytes and written 326 bytes
Verification error: unable to get local issuer certificate
---
New, SSLv3, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-SHA
Session-ID: 8FF534B3E4E16E9579815E8CAEC02B1002A504129DC351D22A47794C51C3FE4A
Session-ID-ctx:
Master-Key: 41413D09E6D1E5E44FFFC803388C62BC548CBC2C8B5BF373CC7015E2318AD6424C16E20B1C87DC4980517E0126D6D407
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
0000 - 3f df d4 3e 8d f5 43 ee-55 61 49 4c 39 7e 94 ed ?..>..C.UaIL9~..
0010 - c7 c5 f8 d7 a5 ee 47 c6-b3 0d 75 47 20 d4 96 e6 ......G...uG ...
0020 - a8 0c 63 3e 0c c7 92 32-c2 fb 29 c7 a4 fc a7 22 ..c>...2..)...."
0030 - f0 ad 61 14 ee 3b 30 2f-41 c4 11 6e 79 3b 8c f0 ..a..;0/A..ny;..
0040 - 98 72 3f 5b db 5c 7d c8-c0 f2 47 74 4b 69 a6 6a .r?[.\}...GtKi.j
0050 - 71 a6 80 22 3e 7b 04 10-ee 84 be c0 c3 c5 8a 4d q..">{.........M
0060 - 8e c7 b5 8d bb 12 37 8f-d9 6f f9 37 6d af 8b 20 ......7..o.7m..
0070 - 3c b9 98 bf 9c ce f8 11-9e b6 aa 23 61 18 2d 9b <..........#a.-.
0080 - 00 e3 aa 9f 70 00 fb 1f-5f 9c a8 ff 56 c7 47 76 ....p..._...V.Gv
0090 - c9 a2 bc 83 12 d2 04 17-b5 97 a0 b0 4c 98 d3 76 ............L..v

Start Time: 1474332985
Timeout : 7200 (sec)
Verify return code: 20 (unable to get local issuer certificate)
Extended master secret: no
---
closed
ROBERTSON@srvr1:~/openssl-1_1_0 > openssl s_client -connect evn.crm.em2.oraclecloud.com:443 | grep "Master-Key:"
depth=2 C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root
verify error:num=20:unable to get local issuer certificate
Master-Key: E24EAC787E42548EB97E002DED67AFB98A9D01807BC4297B66D4348B86FE67477E916CDC9ED5CB3F3FEF0649FDC32A94
ROBERTSON@srvr1:~/openssl-1_1_0 >

[Paul Sture]

Even VAX-COBOL ended up with quadwords.

--
It was untidy, so got unplugged.
It was unplugged, so got thrown away.

[Richard Levitte]

That's a different issue. 'openssl s_client' outputs to stdout using normal means. Why grep outputs several lines is a good question, but hopefully not an OpenSSL issue (if it is, I sure would like to know!).

The issue we're talking about here is about input to s_client. What happens if you, for example, try this in GNV bash?

(echo "HEAD / HTTP/1.0"; echo "Host: www.openssl.org"; echo) | openssl s_client -connect www.openssl.org:443

I'm sure it's going to fail... However, if you go into apps/vms_term_sock.c and change "SYS$COMMAND" to "SYS$INPUT", rebuild OpenSSL and try again, will you get a different behaviour?

(quite frankly, I don't know at all what to expect. I just tried the above command on Linux, and the result was frankly disappointing... the input was simply ignored. I'll have to check why)

Cheers,
Richard

[Richard Levitte]

Den tisdag 20 september 2016 kl. 13:37:31 UTC+2 skrev Richard Levitte:
> (quite frankly, I don't know at all what to expect. I just tried the above command on Linux, and the result was frankly disappointing... the input was simply ignored. I'll have to check why)

Forgot to have '-ign_eof' in the command... gets me every time *sigh*

Cheers,
Richard

[RobertsonEricW]

Using OpenVMS GNV Bash the piped input appears to be ignored:

ROBERTSON@srvr1:~/openssl-1_1_0 > (echo "HEAD / HTTP/1.0"; echo "Host: www.openssl.org"; echo) | openssl s_client -ign_eof -connect www.openssl.org:443
CONNECTED(00000006)
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3

verify error:num=20:unable to get local issuer certificate
---
Certificate chain

0 s:/CN=www.openssl.org
i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3

---
Server certificate
-----BEGIN CERTIFICATE-----

MIIFAjCCA+qgAwIBAgISA6XFka8ITuu3xsfLC9a43mFyMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNjA4MTIyMDIyMDBaFw0x
NjExMTAyMDIyMDBaMBoxGDAWBgNVBAMTD3d3dy5vcGVuc3NsLm9yZzCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAJ5V5OmbzKKuA5lDHKdHjaeZwETkk2yq
38zb4HBxS2K86qwTPAg6GVzHAKI4tPO+QKTmvYYcn1V+Wb7HS2nkpm7LyrloThxf
Jk6f1XMZRSymu3KjOI4rqhaEQOCJL1FrYi1OjCWVWrluWO6IoGa+HmBUrau2raeI
lUtLnrwMsKRBSCGEc+gB23/9pphcXoUVbJktkr9nWa1HSNnDhs3cyQfSxOhQGL5b
Q62NB/lV2Tn5idqy8+aEfD+i1ZrkovlXmtLtdSchAaA2obsTmUMcGbqasWDX6Iq6
Qz3daKjggJLgDv48wSRWcfdZq73opgh4r+ADGzHQQZVSwtwneDN0HU8CAwEAAaOC
AhAwggIMMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYB
BQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUZzWG3xa+UHDTU6UBSOlOC9ZD
XKMwHwYDVR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEwcAYIKwYBBQUHAQEE
ZDBiMC8GCCsGAQUFBzABhiNodHRwOi8vb2NzcC5pbnQteDMubGV0c2VuY3J5cHQu
b3JnLzAvBggrBgEFBQcwAoYjaHR0cDovL2NlcnQuaW50LXgzLmxldHNlbmNyeXB0
Lm9yZy8wGgYDVR0RBBMwEYIPd3d3Lm9wZW5zc2wub3JnMIH+BgNVHSAEgfYwgfMw
CAYGZ4EMAQIBMIHmBgsrBgEEAYLfEwEBATCB1jAmBggrBgEFBQcCARYaaHR0cDov
L2Nwcy5sZXRzZW5jcnlwdC5vcmcwgasGCCsGAQUFBwICMIGeDIGbVGhpcyBDZXJ0
aWZpY2F0ZSBtYXkgb25seSBiZSByZWxpZWQgdXBvbiBieSBSZWx5aW5nIFBhcnRp
ZXMgYW5kIG9ubHkgaW4gYWNjb3JkYW5jZSB3aXRoIHRoZSBDZXJ0aWZpY2F0ZSBQ
b2xpY3kgZm91bmQgYXQgaHR0cHM6Ly9sZXRzZW5jcnlwdC5vcmcvcmVwb3NpdG9y
eS8wDQYJKoZIhvcNAQELBQADggEBAF3geQxfZcGUQw7ejHyMMIhSQ2UnzerXzbEf
6+xHh/CkznnT51L4/l24QDBOEYD27AWDTdsF5WYG1MjtprVC2eq9P3wK8hwlvEIs
V7A9vq/fMZHU6Mu3yL4DIvPB+NIv2WYTmPPrziN6Ac90MYND21H7WjCdhmER3yZt
uPV9DW/QF+J2B4qG+SGFBhVEyiyPIt9pVFxb4erteD0XY4wYHNaNrvjLiM/y4AgM
ennukGLbDvf1oSuFKDseEDqXSHjFGYE4TxeXC4EKXidTkXfpVSd9JUobu5205erf
0p3mqRxCaNDIlIRL3tCXa+ItlR0RoahsZDJOm9SBCvmClV54joM=
-----END CERTIFICATE-----
subject=/CN=www.openssl.org
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---

SSL handshake has read 3117 bytes and written 302 bytes

Verification error: unable to get local issuer certificate
---

New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384

Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2

Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: A59E0BE0F49423A5A87D2486D69613BCC17430CB2CB040A9517E8B9FA1448625
Session-ID-ctx:
Master-Key: 60850FDEE1B64B809B2603C6F5D029FC94DEBAF758F5AAD467092B50B4D6C0AA30E46A581151F79D8A42C4FFBD80EB0C

PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:

0000 - 21 ee ff a6 25 d5 cd 8a-e2 5c 28 00 c9 0e e1 25 !...%....\(....%
0010 - 68 18 e6 12 47 cd b3 d9-95 ae 89 45 2a 2b e5 f8 h...G......E*+..
0020 - 37 a7 e9 cf f9 e1 e9 3c-46 da b9 be 7b 74 92 a5 7......<F...{t..
0030 - 14 41 cc 57 5f e3 c5 ce-78 98 50 f8 f1 6f f0 8e .A.W_...x.P..o..
0040 - 25 cf ce 73 27 f8 7c e2-55 12 cd 6b ae 40 92 99 %..s'.|.U..k.@..
0050 - aa e0 c9 4d cb 9f 63 8f-93 4a 85 a0 d8 08 6e b2 ...M..c..J....n.
0060 - ff 48 23 58 a4 c9 a4 71-d7 db 6c 16 da 57 7c fc .H#X...q..l..W|.
0070 - 2f 63 41 db f3 1d 73 82-a8 03 9b 33 bb dc 20 b9 /cA...s....3.. .
0080 - 51 b3 ec 92 67 7c 6f a6-09 a3 1f 9e ac 8c 9f 2a Q...g|o........*
0090 - e7 10 e0 2c 76 8f db 45-3c 0e 94 f5 ab a2 6a 2b ...,v..E<.....j+

Start Time: 1474372960

[Richard Levitte]

Den tisdag 20 september 2016 kl. 14:14:51 UTC+2 skrev RobertsonEricW:
> On Tuesday, September 20, 2016 at 7:51:42 AM UTC-4, Richard Levitte wrote:
> > Den tisdag 20 september 2016 kl. 13:37:31 UTC+2 skrev Richard Levitte:
> > > (quite frankly, I don't know at all what to expect. I just tried the above command on Linux, and the result was frankly disappointing... the input was simply ignored. I'll have to check why)
> >
> > Forgot to have '-ign_eof' in the command... gets me every time *sigh*
> >
> > Cheers,
> > Richard
>
> Using OpenVMS GNV Bash the piped input appears to be ignored:

Also with "SYS$COMMAND" changed to "SYS$INPUT" in apps/vms_term_sock.c?