ARGV routine in C

[Shael Richmond]

We are moving our Oracle database from OpenVMS to Linux and they turned on case sensitivity in Oracle. We started having problems logging into the database. It seems like the ARGV routine will lowercase all parameters in TRADITIONAL mode and not touch them in EXTENDED. Is this the correct behavior? When using a FORTRAN program with lib$get_foreign instead of a C program it acts normal.

ISPPRC_PZ30DDD> set process/parse=traditional
ISPPRC_PZ30DDD> prog TEST TEST 1 2
l100e: test 2estment 1: test
ISPPRC_PZ30DDD> prog test test 1 2
l100e: test 2estment 1: test
ISPPRC_PZ30DDD> set process/parse=extended
ISPPRC_PZ30DDD> prog TEST TEST 1 2
l100e: TEST 2ESTment 1: TEST
ISPPRC_PZ30DDD> prog test test 1 2
l100e: test 2estment 1: test

Fortran:
ISPPRC_PZ30DDD> set proc/parse=trad
ISPPRC_PZ30DDD> prog HEMES/HEMES TEST
arameter buffer: HEMES/HEMES TEST

arameter count: 16
ISPPRC_PZ30DDD> prog hemes/hemes test
arameter buffer: HEMES/HEMES TEST

arameter count: 16
ISPPRC_PZ30DDD> prog "hemes/hemes" test
arameter buffer: "hemes/hemes" TEST

arameter count: 18

ISPPRC_PZ30DDD> set proc/parse=ext
ISPPRC_PZ30DDD> prog HEMES/HEMES TEST
arameter buffer: HEMES/HEMES TEST

arameter count: 16
ISPPRC_PZ30DDD> prog hemes/hemes test
arameter buffer: hemes/hemes test

arameter count: 16
ISPPRC_PZ30DDD> prog "hemes/hemes" test
arameter buffer: "hemes/hemes" test

arameter count: 18

This is what I am seeing with Oracle - it lowercases everything unless in extended mode or in quotes.

Shael Richmond
International Paper

[Craig A. Berry]

On Tuesday, August 16, 2016 at 10:24:15 AM UTC-5, Shael Richmond wrote:
> We are moving our Oracle database from OpenVMS to Linux and they turned on case sensitivity in Oracle. We started having problems logging into the database. It seems like the ARGV routine will lowercase all parameters in TRADITIONAL mode and not touch them in EXTENDED.

$ help crtl feature decc$argv_parse_style

CRTL

Feature_Logical_Names

DECC$ARGV_PARSE_STYLE

With DECC$ARGV_PARSE_STYLE enabled, case is preserved in command-
line arguments when the process has been set up for extended DCL
parsing using SET PROCESS/PARSE_STYLE=EXTENDED.

DECC$ARGV_PARSE_STYLE must be defined externally as a logical
name or set in a function called using the LIB$INITIALIZE
mechanism because it is evaluated before function main is
called.

[Stephen Hoffman]

On 2016-08-16 15:24:14 +0000, Shael Richmond said:

> We are moving our Oracle database from OpenVMS to Linux

OpenVMS upcases everything in DCL when traditional parsing is selected.
If the OpenVMS application code involved here — either
locally-written or something inside the Oracle database code itself —
is using the OpenVMS C library routines for command parsing, then those
upcased commands are then downcased when traditional parsing is
enabled, and the C parsing should not alter the case when in extended
parsing mode. There's also some case-related and character-set
shenanigans when processing filenames through the C library, too.
There are logical name knobs to mess with those settings in C,
DECC$ARGV_PARSE_STYLE, DECC$EFS_CASE_PRESERVE, DECC$EFS_CHARSET,
DECC$EFS_CASE_SPECIAL, probably some others.
http://labs.hoffmanlabs.com/node/1513 etc.

What Oracle is doing here? No sé. Haven't tried this particular combination.

Probably semi-related to this:
https://docs.oracle.com/database/121/NLSPG/ch2charset.htm#NLSPG002
http://docs.oracle.com/cd/E11882_01/server.112/e10729/ch5lingsort.htm#NLSPG005

Given the choice, I'd probably normalize to lowercase, too.
http://www.unicode.org/faq/normalization.html

OpenVMS has an old version of ICU around, too. That's sometimes used
for normalization. See the C I18N and related pieces.
http://site.icu-project.org

I'd also suggest working to develop and port and test in the target
Linux environment. Not by trying to get OpenVMS and OpenVMS
applications and ports to do what it's not good at; at case-sensitive
file access and case-sensitive command syntax. Probably better too
to ask other folks about migrating to Linux and Oracle databases on
Linux than a forum filled with a bunch of OpenVMS folks, after all.
Maybe over in the Oracle discussion forums, or more directly with the
Oracle support folks?




--
Pure Personal Opinion | HoffmanLabs LLC

[Bob Koehler]

In article <e6e07cf0-96f1-42ca...@googlegroups.com>, Shael Richmond <shael.r...@gmail.com> writes:
> We are moving our Oracle database from OpenVMS to Linux and they turned on =
> case sensitivity in Oracle. We started having problems logging into the da=
> tabase. It seems like the ARGV routine will lowercase all parameters in TR=
> ADITIONAL mode and not touch them in EXTENDED. Is this the correct behavio=
> r? When using a FORTRAN program with lib$get_foreign instead of a C progra=
> m it acts normal.

OK, I'm reading between the lines a lot here, but I'm pretty sure
what you are seeing is the documented and expected behaviour.

[John Reagan]

BTW, I just spent some time looking at this. I had wondered why the RTL couldn't just look at the setting of parse style and go from there...

Apparently, the support for /PARSE_STYLE=EXTENDED was released before the C RTL added any support. There was a fear that switching to sense the setting might break some applications (there was an active discussion and if I was there, I would have argued against the chosen behavior).

One thing I did learn is that the DOC isn't exactly right. When that feature logical is enabled, the RTL does look at the setting of JPI$_PARSE_STYLE_PERM to decide whether it should flip the case. I've always thought the casing was controlled by the DECC$ARGV_PARSE_STYLE, but it just controls whether the RTL looks at the JPI setting.

[Craig A. Berry]

And apparently wherever it does that is before LIB$INITIALIZE, so you can't change your process setting in LIB$INITIALIZE and have it do any good.

[hb]

On 08/16/2016 08:42 PM, Craig A. Berry wrote:
> And apparently wherever it does that is before LIB$INITIALIZE, so you
> can't change your process setting in LIB$INITIALIZE and have it do
> any good.

... before LIB$INITIALIZE of your main (executable) image, ...

[John Reagan]

I think you should be able to do that. The code is in CCMAIN and should be run long after the various LIB$INITALIZE routines have been processed. I'll double check when I get time.

[Steven Schweda]

> And apparently wherever it does that is before
> LIB$INITIALIZE, so you can't change your process setting in
> LIB$INITIALIZE and have it do any good.

Which "that" is that? Setting DECC$ARGV_PARSE_STYLE in a
LIB$INITIALIZE is known to work. I'm confused.

[Craig A. Berry]

"That" is the CRTL code John R. referred to where it checks the process
setting for extended parse and only honors DECC$ARGV_PARSE_STYLE if the
process permanent setting is extended parse. As I said, and as you used
to know from previous discussions, you cannot change the process-level
parse style setting from within your LIB$INITIALIZE routine and then set
DECC$ARGV_PARSE_STYLE and have it honored for the same program.

Example of calling SYS$SET_PROCESS_PROPERTIESW with
PPROP$C_PARSE_STYLE_TEMP or PPROP$C_PARSE_STYLE_PERM available at:

<https://sourceforge.net/p/vms-ports/tickets/34/>

where there is also additional explanation of things that should
work but don't.

[Craig A. Berry]

So you're saying that if I do this:

status = sys$set_process_propertiesw( 0,
0,
0,
PPROP$C_PARSE_STYLE_TEMP,
PARSE_STYLE$C_EXTENDED,
0 );

in a LIB$INITIALIZE routine that is part of a shareable image I've
linked to rather than the main image, then set DECC$ARGV_PARSE_STYLE in
the same LIB$INITIALIZE routine, then the feature will be honored, even
if extended parse was not set in the process before starting the program?

I'm skeptical but would be delighted to see a working example.

Complete non-working example at:

<https://sourceforge.net/p/vms-ports/tickets/34/>

[John E. Malmberg]

I believe what Craig is referring to is that you can not set the
JPI$_PARSE_STYLE in the LIB$INITIALIZE section.

There is no way for DCL to query an image before it is activated to
determine how to parse the arguments to it.

Regards,
-John

[John Reagan]

Looking at internal history, the CRTL's main routine was going to look at the _IMAGE flag but there was some current bug with it not being propagated properly. So they decided to look at _PERM instead. Again, I wouldn't have chosen this solution.

[John Reagan]

DCL, no. But LIB$GET_FOREIGN would be affected, right? And it also propagates into LIB$SPAWN.

[Stephen Hoffman]

On 2016-08-17 02:42:33 +0000, John E. Malmberg said:

> There is no way for DCL to query an image before it is activated to
> determine how to parse the arguments to it.

Ayup. Though this can be fixed. There's a very similar existing
implementation within OpenVMS I64 executables with SET IMAGE /FLAGS,
too.

Whether something akin to moving to application bundles and info.plist
https://developer.apple.com/library/ios/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html
— you can put licensing and signatures, requested access entitlements
and such into an analogous file in a bundle, too — and toward better
support of packaged applications and targeting app stacking and
container/sandbox deployments, or something following the more
traditional SET IMAGE implementation and approach, maybe using an ELF
note section or a user section.

A look at the whole of the C RTL settings logical name morass would be
nice here too, maybe providing support for application-specific
settings files — even if OpenVMS doesn't go as far as bundles? Have
the image activator and application callbacks allow access to that,
with some settings for OpenVMS and C and such, and have the storage
also available for application-specific keys and values.

[John Reagan]

You mean like the old MacOS resource fork or the Windows resource compiler (RC) that comes with Visual Studio?

Besides trying to force the CRTL settings into the ELF dynamic section, you could also go the protected/hidden ACE route but that could be just as clumsy and awkward. I've NEVER been able to edit an ACL with EDIT/ACL even with the manual in front of me.

[Stephen Hoffman]

On 2016-08-17 16:28:38 +0000, John Reagan said:

> On Wednesday, August 17, 2016 at 10:49:31 AM UTC-4, Stephen Hoffman wrote:
>> On 2016-08-17 02:42:33 +0000, John E. Malmberg said:
>>
>>> There is no way for DCL to query an image before it is activated to
>>> determine how to parse the arguments to it.
>>
>> Ayup. Though this can be fixed. There's a very similar existing
>> implementation within OpenVMS I64 executables with SET IMAGE /FLAGS,
>> too.
>>
>> Whether something akin to moving to application bundles and info.plist
>> https://developer.apple.com/library/ios/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html
>> — you can put licensing and signatures, requested access entitlements
>> and such into an analogous file in a bundle, too — and toward better
>> support of packaged applications and targeting app stacking and
>> container/sandbox deployments, or something following the more
>> traditional SET IMAGE implementation and approach, maybe using an ELF
>> note section or a user section.
>>
>> A look at the whole of the C RTL settings logical name morass would be
>> nice here too, maybe providing support for application-specific
>> settings files — even if OpenVMS doesn't go as far as bundles? Have
>> the image activator and application callbacks allow access to that,
>> with some settings for OpenVMS and C and such, and have the storage
>> also available for application-specific keys and values.
>>
>
> You mean like the old MacOS resource fork or the Windows resource
> compiler (RC) that comes with Visual Studio?

Here, I was thinking more along the lines of using existing ELF data
structures intended for this sort of usage. Resource forks involve
lower-level work in the file system, and have some trade-offs.
Resource forks or other "outboard" or out-of-image mechanisms tends to
get skewed or get stripped off, and that tends to get ugly. That
skewage includes plist files and logical names, which is at the core of
various gripes — surprises from out-of-band. That outside of an
implementation that provides application bundles that is, as that would
reduce the likelihood of a skew between the application and its
settings, particularly if the bundles are signed.

Settings that are embedded within an ELF image will persist across most
typical file-related operations involving executable and shareable
images, after all.

With the ELF sections — and haven't looked at nor thought through how
feasible this inheritance might or would be — shareable images might
conceivably allow shareable-specific settings for the non-global the C
settings, though explaining and documenting that might get as messy as
implementing that.

I'm not familiar with Visual Studio and the resource compiler, but
looking at
https://msdn.microsoft.com/en-us/library/windows/desktop/aa381042(v=vs.85).aspx
and related, that looks very similar. The macOS equivalent to that is
NSUserDefaults
https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSUserDefaults_Class/index.html
and the users and preferences settings.
https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/UserDefaults/Introduction/Introduction.html#//apple_ref/doc/uid/10000059i


In any event, if there's data embedded somewhere, there needs to be
some related callbacks to access the data. SET IMAGE is the closest
analog to that, in the existing OpenVMS environment. It's too limited,
though.

OpenVMS lacks any sort of standardized mechanism for storing this sort
of configuration or installation or preferences data, whether it's in
the bundle or in an external file or logical name or other data blob...
The old X stuff — and the name of the routines involved escapes me at
present — is one of the closest analogs that is available in OpenVMS,
and understandably almost nobody ever uses that stuff outside of
X-based applications.

http://labs.hoffmanlabs.com/node/1923
http://labs.hoffmanlabs.com/node/1733

After a while, writing and debugging and troubleshooting Yet Another
Parser for Yet Another Configuration File gets... old. it's the sort
of mechanism that's increasingly built into platforms. Or grab one of
the license-compatible libraries or frameworks, and use that.

> Besides trying to force the CRTL settings into the ELF dynamic section,
> you could also go the protected/hidden ACE route but that could be just
> as clumsy and awkward.

I've extended that particular ACL mechanism for other uses. It works,
and you'll need to implement an ACL-specific parser and formatter and
some other giblets. Biggest issue with using ACLs as a data store
tends to be encountering BACKUP /EXCHANGE and similar tools and
operations; cases where the ACLs get stripped off. (That there's such
a small and overlapping space for UIC identifiers is another and
related issue. Using UUIDs for that avoids much of that mess and the
need for tools such as ACL_SCRUB http://labs.hoffmanlabs.com/node/426
and such. Fixing that and moving to UUIDs most everywhere would be an
overhaul bigger than V6.0, though. But I digress.)

> I've NEVER been able to edit an ACL with EDIT/ACL even with the manual
> in front of me.

That's at least partially the fault of the EDIT /ACL user interface,
and the help text in that tool and the use of the keypad certainly
don't led themselves to ease-of-use. EDIT /ACL tends toward vim or
emacs as a user interface design, and probably should have aimed rather
more toward nano / pico. Experienced users are probably going to use
SET SECURITY, after all. But then EDIT /ACL and SET SECURITY
commands, and components such as ACME and CDSA and Enterprise Directory
and other security-related pieces all tend to get pretty murky pretty
quickly, and pieces are not nearly as integrated nor as consistent as
anybody would like, and murkiness and inconsistencies around security
tends to be problematic for folks...

[hb]

On 08/17/2016 06:28 PM, John Reagan wrote:
> On Wednesday, August 17, 2016 at 10:49:31 AM UTC-4, Stephen Hoffman
> wrote:

>> Ayup. Though this can be fixed. There's a very similar existing
>> implementation within OpenVMS I64 executables with SET IMAGE
>> /FLAGS, too.
>>

>> ..., or something following the more traditional SET IMAGE

>> implementation and approach, maybe using an ELF note section or a
>> user section.

Today, SET IMAGE flips some bits, it is more or less a more convenient
tool than ANALYZE/IMAGE and PATCH. Adding a section or extending a
section is some more work. You may want to remove some sections as well,
for example the symbol table, if you don't want anybody to link against
a shareable image: can be more convenient than re-linking with /NOGST.
And there was a SET IMAGE on Alpha as well. It was limited to one
specific area. It was call THREADCP and lived/lives in SYS$UPDATE.

>> A look at the whole of the C RTL settings logical name morass would
>> be nice here too, maybe providing support for application-specific
>> settings files — even if OpenVMS doesn't go as far as bundles?
>> Have the image activator and application callbacks allow access to
>> that, with some settings for OpenVMS and C and such, and have the
>> storage also available for application-specific keys and values.

> Besides trying to force the CRTL settings into the ELF dynamic

> section, ...

The VMS linker only creates a dynamic segment: there is no section which
overlays it. The main reason is that the image activator only looks at
the ELF header and the Program Header Table. On VMS the dynamic segment
already holds more than the traditional dynamic array: the image
relocations and fixups, obviously needed by the image activator.
However, after processing the dynamic segment the image activator
usually throws it away: it is unmapped. That would need to be changed as
well, so that the CRTL can find its settings.

[hb]

John said, that the CRTL looks at JPI$_PARSE_STYLE_PERM, so you need to
use PPROP$C_PARSE_STYLE_PERM. He also mentioned LIB$GET_FOREIGN. It
seems the CRTL is using it. (What else could it do, getting the raw line
from the recall buffer?) LIB$GET_FOREIGN is not affected by any parse
style setting at run time. But it is too late, the command line was
already parsed, symbols were substituted, UPCASED and LIB$GET_FOREIGN
can only get that.

But yes, if there were init code in the CRTL, which checks
JPI$_PARSE_STYLE_PERM, you can have your own shareable image with init
code, activivated before the DECC$SHR and set the PARSE_STYLE with
PPROP$C_PARSE_STYLE_PERM and that setting will be honored in the CRTL's
init code.

John also said, "The code is in CCMAIN". It looks like you confirmed
that in the referenced page on sourceforge.net:

"
If we modify the program to make the extended parse setting persist
after image run-down: ...
(which is undesirable because it affects other programs that may have
different requirements), then we bizarrely get neither the new case
preserved behavior nor the old downcased behavior but DCL-style upcasing:

$ set process/parse=traditional
$ mcr []argtest A b C d
argv[1]: A
argv[2]: B
argv[3]: C
argv[4]: D
"

Which to me looks like DCL did it's upcasing, as expected and you set
parse style extended which together with DECC$ARGV_PARSE_STYLE instructs
the CRTL to keep the command line as is and not to lowercase everything
(expect what is within double quotes).

[Craig A. Berry]

The DCL-style upcasing wouldn't be "expected" by anyone who didn't know
lots of details they shouldn't have to know about the sequence of
command processing and image start-up. It's also almost certainly
emergent behavior, an accident of the implementation, and not anything
that was done intentionally.

The desired goal is to be able to write a program that can get the
command-line arguments as the user typed them without the user having to
know about and enable some arcane feature called "extended parse."
Apparently that's not possible.

[Bob Koehler]

In article <np2vai$o38$1...@dont-email.me>, "Craig A. Berry" <craig...@nospam.mac.com> writes:
>
> The DCL-style upcasing wouldn't be "expected" by anyone who didn't know
> lots of details they shouldn't have to know about the sequence of
> command processing and image start-up. It's also almost certainly
> emergent behavior, an accident of the implementation, and not anything
> that was done intentionally.

You might not expect it, but I'm quite sure it was doneintentionally.
VMS was designed to be case-insensitive. That was implemented by
upcasing everything.

> The desired goal is to be able to write a program that can get the
> command-line arguments as the user typed them without the user having to
> know about and enable some arcane feature called "extended parse."
> Apparently that's not possible.

I'd probably hide it in sylogin.com. Especially if it's easy to
figure out which user. Or maybe I'd stuff it in login.com.

[John Reagan]

On Thursday, August 18, 2016 at 8:49:31 AM UTC-4, Bob Koehler wrote:

I used to think that turning on /PARSE=EXTENDED was benign and had in my LOGIN.COM and then I had to start working on Fortran. The group login creates a logical name table and splices it into LNM$FILE_DEV.

Don't do the following with /PARSE=EXTENDED. You'll get an LNM$FILE_DEV with lower case values. Apparently, the processing of LNM$FILE_DEV inside of RMS is case-sensitive so you'll see some odd behavior. Once I changed it to uppercase, I was able to restore my /PARSE=EXTENDED.

$ create/name_table/parant_table=lnm$process_directory f90_logicals_table
$ define/table=lnm$process_directory f90_logicals f90_logicals_table
$ define/table=lnm$process_directory lnm$file_dev -
lnm$process, -
lnm$job, -
f90_logicals, -
lnm$group, -
lnm$system, -
decw$logical_names

I would argue that the logical name processing of LNM$FILE_DEV should be either case-blind or at least tied to the dreaded /CASE=BLIND process setting.

[hb]

... and then DEFINE is defined as
$ verb define
define verb DEFINE
synonym DEF
cliroutine DEFINE
prefix CLI$K_DEFI_
parameter P1, prompt="Log name"
value (required,type=$outlog)
parameter P2, prompt="Equ name"
value (required,list,type=$old_file)
...
$

Any type which matches the regexp '\$.*file.*' is very likely sensitive
to the set parse style.

If you change the CLD to something like
value (required,list,type=$inlog)
the value of P2 no longer depends on the parse style.
Agreed, I have no insight into other effects such a change will cause.

There is another interesting built-in type: $rest_of_line_noupcase.

You will find these in SYS$UPDATE:*.CLD. It seems these are not
documented in the manuals. But you have the "real documentation" :-)

[Stephen Hoffman]

On 2016-08-18 00:25:53 +0000, Craig A. Berry said:

> The desired goal is to be able to write a program that can get the
> command-line arguments as the user typed them without the user having
> to know about and enable some arcane feature called "extended parse."
> Apparently that's not possible.

Correct. DCL gets in the way of that and upcases. If you're
approaching this issue from 1970s programming and RAD50 and FIELDATA
and ASRs and such, it was all uppercase or easily uppercased (US ASCII)
and it all makes perfect sense. In the current era, this mess is
another cost of backward-compatibility for applications.

Flipping the processing to case-preserving would have broken various
older applications, which means you can either keep those working
transparently by preserving the old behavior as the default, or you
could force the folks to enable mechanisms to preserve the old behavior
for specific applications and to move the defaults and new programs
over to case-preserving. The former approach was chosen.

Rather than make things better for new work, old work and increasingly
problematic work is preserved, and new work is made more difficult and
more complex.

Application compatibility is certainly valuable. Absolute application
compatibiltiy, and particularly application compatibility that
increases the cost of updates and of new development, is a cost on
future development work for the platform.

[Craig A. Berry]

On 8/18/16 7:46 AM, Bob Koehler wrote:
> In article <np2vai$o38$1...@dont-email.me>, "Craig A. Berry" <craig...@nospam.mac.com> writes:
>>
>> The DCL-style upcasing wouldn't be "expected" by anyone who didn't know
>> lots of details they shouldn't have to know about the sequence of
>> command processing and image start-up. It's also almost certainly
>> emergent behavior, an accident of the implementation, and not anything
>> that was done intentionally.
>
> You might not expect it, but I'm quite sure it was doneintentionally.
> VMS was designed to be case-insensitive. That was implemented by
> upcasing everything.

You have got to be kidding. The emergent behavior I'm talking about is
what happens with the sequence of:

1.) Set traditional parse in DCL.
2.) Set extended parse in LIB$INITIALIZE
3.) Set DECC$ARGV_PARSE_STYLE in LIB$INITIALIZE

which makes your C program behave like DCL and upcase everything that's
not quoted. That behavior is not documented under DECC$ARGV_PARSE_STYLE,
and while it may fall out naturally from the details of the
implementation, I can't think of a use case where this would be a good
thing and I very much doubt it was intended.

>> The desired goal is to be able to write a program that can get the
>> command-line arguments as the user typed them without the user having to
>> know about and enable some arcane feature called "extended parse."
>> Apparently that's not possible.
>
> I'd probably hide it in sylogin.com. Especially if it's easy to
> figure out which user. Or maybe I'd stuff it in login.com.

Give me access to edit sylogin.com on all your systems... no, don't do
that; it's the wrong fix, even if you were willing, which I know you aren't.

This is not some pedantic point for me. It comes from real problems
maintaining Perl on VMS for the last fifteen years. Perl and its various
modules for testing and extension building for the most part try to
quote command-line arguments on VMS only.

Of course to quote the command line arguments, you have to remove any
quotes already there so you don't double them up, which means parsing
and re-parsing those arguments. And then some platforms need to quote
some arguments when those arguments contain file names that contain
spaces, so we have to superimpose that quoting mechanism on the VMS
quoting mechanism without breaking anything, and have people maintain
this complex, fragile code, people who have never seen a VMS system and
have no ability to test any changes they make.

Surprisingly, that all works well enough most of the time. But people
keep forgetting to use the quoting features, because on every system
other than VMS, "foo -A -b" means what it says and distinguishes A/a and
B/b and it's the most natural thing in the world for people to expect
simple things to work in a simple way.

So my choice is either to keep patching each and every one of the tests
that fails under traditional parse, or to give up. I gave up, and there
are now a steadily growing number of tests in the Perl core test suite
that will fail under traditional parse but pass under extended parse.
Yes, I can document that (can't remember offhand whether I have yet),
but that's a weak solution.

I wanted to fix this by enabling DECC$ARGV_PARSE_STYLE in an init
section, but, as I've detailed at some length, that doesn't work unless
you change the process setting before image start-up. Which may break
other programs you want to run in the same process.

Ah well, I was hoping extended parse would become the default in VMS
9.0, but apparently there are still important things that don't work
with it enabled, so I'm not holding my breath.

[hb]

On 08/18/2016 09:59 PM, Craig A. Berry wrote:
> I wanted to fix this by enabling DECC$ARGV_PARSE_STYLE in an init
> section, but, as I've detailed at some length, that doesn't work unless
> you change the process setting before image start-up. Which may break
> other programs you want to run in the same process.

Yes, it may look like nitpicking: ... unless you change the process
setting before DCL parses your command line.

From what I know and maybe didn't make it clear, LIB$DET_FOREIGN just
picks up the result of the DCL parsing. (Image startup-time is also too
late. So the image activator can't help and it's not the main purpose of
the image activator to handle command line arguments, anyway.)

If you can force DCL to re-parse the original command line with the
changed process parse style before LIB$DET_FOREIGN is called from the
CRTL, it may work.

[Stephen Hoffman]

On 2016-08-18 20:34:59 +0000, hb said:

> If you can force DCL to re-parse the original command line with the
> changed process parse style before LIB$DET_FOREIGN is called from the
> CRTL, it may work.

Or move the upcase and related fussing out of the CLI and over to
lib$get_foreign and to the CLI callback routines, and into whatever
parser is lurking inside DCL — if it's not the CLI routines.

Or give lib$get_foreign a path to aDCL-hasn't-messed-with-it command
input buffer in the CLI.

Keeping the command around and then giving lib$get_foreign a path via
sys$cli is probably the most isolated approach.

[Steven Schweda]

> > I'd probably hide it in sylogin.com. Especially if it's easy to
> > figure out which user. Or maybe I'd stuff it in login.com.
>
> Give me access to edit sylogin.com on all your systems... no,
> don't do that; it's the wrong fix, even if you were willing,
> which I know you aren't.

I seem to have made my change to [SY]LOGIN.COM (on non-VAX
systems) somewhere around 30-JUL-2004. I can remember some
case-related problems in MMS (long ago), but I can't say if
the PROCESS /PARSE_STYLE setting affected them. In any case,
it amazes me (at least a little) that anyone runs with
Traditional these days.

[John Reagan]

I did some looking... LIB$GET_FOREIGN calls SYS$CLI to get the command string. It then upcases the string with no option to skip that (perhaps we should add a flag to the 4th argument to skip that?). It upcases the whole thing without regard to quoted strings, etc.

3 603 !+
3 604 ! Upcase the string
3 605 !-
3 606
3 607 CH$TRANSLATE (LIB$AB_UPCASE, .CLI_DSC [DSC$W_LENGTH],
3 608 .CLI_DSC [DSC$A_POINTER], %C' ',
3 609 .CLI_DSC [DSC$W_LENGTH], .CLI_DSC [DSC$A_POINTER]);
3 610 END;


Also the CRTL seems to have an alternate scheme to call another routine besides LIB$GET_FOREIGN. There is an entry point in the symbol vector of the form:

$! introduce the routine:
$!
$! int decc$$set_get_foreign(
$! const char* image_name,
$! const char* image_location,
$! const char* routine_name
$! );
$!
$! which would instruct CRTL to use "alternative" LIB$GET_FOREIGN
$! routine specified by the decc$$set_get_foreign.
$!
$! The interface of "alternative" LIB$GET_FOREIGN routine will be exactly
$! the same that that of "standard" LIB$GET_FOREIGN routine.
$!
$! The parameters of decc$$set_get_foreign follow the rules for parameters
$! of LIB$FIND_IMAGE_SYMBOL except that they are represented as
$! null-terminated character string as opposite to string descriptors.
$!
$! For example:
$!
$! decc$$set_get_foreign("stubrtl","xxx:[lib].exe","my$get_foreign");
$!
$! decc$$set_get_foreign calls some LIB$FIND_IMAGE_SYMBOL-based routine
$! which does not signal error condition and returns status returned
$! from LIB$FIND_IMAGE_SYMBOL. The success status is SS$_NORMAL.
$!
$! If no call to decc$$set_get_foreign is made or decc$$set_get_foreign
$! fails, main C routine calls "standard" LIB$GET_FOREIGN routine.

Of course, if you want to write your own LIB$GET_FOREIGN, you'll have to call SYS$CLI with the right argument block to get the command line. But you'll be able to skip that forced upcase of the entire command line (which CCMAIN then wants to lowercase things that aren't in quoted strings).

[hb]

On 08/19/2016 12:39 AM, John Reagan wrote:
> Of course, if you want to write your own LIB$GET_FOREIGN, you'll have
> to call SYS$CLI with the right argument block to get the command
> line. But you'll be able to skip that forced upcase of the entire
> command line (which CCMAIN then wants to lowercase things that aren't
> in quoted strings).

I tried that, calling SYS$CLI and it already returns an upcased string.
Maybe I'm missing something, but I wouldn't expect LIB$GET_FOREIGN to
upcase anything other than what it gets when there is a prompt.

[hb]

On 08/19/2016 12:39 AM, John Reagan wrote:

> I did some looking... LIB$GET_FOREIGN calls SYS$CLI to get the
> command string. It then upcases the string with no option to skip
> that (perhaps we should add a flag to the 4th argument to skip
> that?). It upcases the whole thing without regard to quoted strings,
> etc.

I don't have the "real documentation". This is what I see on an
8.3/Alpha system:

$ ty FOREIGN.C
#define __NEW_STARLET 1

#include <stdio.h>
#include <ssdef.h>
#include <starlet.h>
#include <descrip.h>
#include <lib$routines.h>

int main (int argc, char *argv[]) {
int status;
static char input[255+1];
static struct dsc$descriptor_s input_d = { sizeof (input) - 1,
DSC$K_DTYPE_T,
DSC$K_CLASS_S, input };
static const $DESCRIPTOR (prompt_d, "Enter your input: ");
int i;
status = lib$get_foreign (&input_d, &prompt_d,
&input_d.dsc$w_length);
if (status != SS$_NORMAL) {
sys$exit(status);
} else {
printf ("Input was: \"%-.*s\"\n",
input_d.dsc$w_length,
input_d.dsc$a_pointer);
}
for (i=0; i<argc; i++)
printf("[%d]: %s\n",i,argv[i]);
}
$ cc foreign
$ link foreign
$
$ set proc/parse=ext
$ mcr []foreign
Enter your input: Huhu
Input was: "HUHU"
[0]: eisner$dra3:[decuserve_user.becker_h]foreign.exe;1
$ mcr []foreign Huhu
Input was: "Huhu"
[0]: eisner$dra3:[decuserve_user.becker_h]foreign.exe;1
[1]: huhu
$
$ def/user DECC$ARGV_PARSE_STYLE 1
$ mcr []foreign Huhu
Input was: "Huhu"
[0]: EISNER$DRA3:[DECUSERVE_USER.BECKER_H]FOREIGN.EXE;1
[1]: Huhu
$

In any case, in the first two runs I would expect to see the same input,
no matter whether it was from the prompt or the command line.

[Stephen Hoffman]

On 2016-08-18 22:39:53 +0000, John Reagan said:

> On Thursday, August 18, 2016 at 5:39:12 PM UTC-4, Stephen Hoffman wrote:

>> ... lib$get_foreign ...

> It upcases the whole thing without regard to quoted strings, etc.

That's rather broken.

> Of course, if you want to write your own LIB$GET_FOREIGN, you'll have
> to call SYS$CLI with the right argument block to get the command line.

Any chance you can post up enough of the argument block to allow
somebody to try that path?

> But you'll be able to skip that forced upcase of the entire command
> line (which CCMAIN then wants to lowercase things that aren't in quoted
> strings).

And hope that you can get that intercept call in place early enough to
matter, and also hope that DCL hasn't also upcased everything before
lib$get_foreign upcases even the quoted strings, etc.

[John Reagan]

I look more in the morning and try to reconcile what I think the code does and what Hartmut showed.

[John Reagan]

1) Looking again at LIB$GET_FOREIGN, it is the string read from the prompt that is always uppercased. I'm guessing nobody checked that behavior when /PARSE=EXTENDED was added. That needs to be enhanced.

2) Extending on Harmut's example and calling the SYS$CLI to get the command line, you'll get the same behavior as LIB$GET_FOREIGN but with the leading MCR name. LIB$GET_FOREIGN trims that off for you.

$ deassign decc$argv_parse_style
$ set proc/parse=trad
$ mcr []foreign "foo" BaR
SYS$CLI GETCMD was: "[]FOREIGN "foo" BAR"
Input was: ""foo" BAR"
[0]: _dsa78:[jreagan]foreign.exe;14
[1]: foo
[2]: bar
$ set proc/parse=extended
$ mcr []foreign "foo" BaR
SYS$CLI GETCMD was: "[]foreign "foo" BaR"
Input was: ""foo" BaR"
[0]: _dsa78:[jreagan]foreign.exe;14
[1]: foo
[2]: bar
$ define decc$argv_parse_style enable
$ mcr []foreign "foo" BaR
SYS$CLI GETCMD was: "[]foreign "foo" BaR"
Input was: ""foo" BaR"
[0]: _DSA78:[JREAGAN]FOREIGN.EXE;14
[1]: foo
[2]: BaR

#define __NEW_STARLET 1

#include <stdio.h>
#include <ssdef.h>
#include <starlet.h>
#include <descrip.h>
#include <lib$routines.h>

#include <clidef.h>
#include <string.h>
int sys$cli(CLIDEF1*);

int main (int argc, char *argv[]) {
int status;
static char input[255+1];
static struct dsc$descriptor_s input_d = { sizeof (input) - 1,
DSC$K_DTYPE_T,
DSC$K_CLASS_S, input };
static const $DESCRIPTOR (prompt_d, "Enter your input: ");
int i;

CLIDEF1 cli_req_block;

memset(&cli_req_block, 0, CLI$C_REQDESC);
cli_req_block.cli$b_rqtype = CLI$K_GETCMD;
status = sys$cli(&cli_req_block);
#define CLI$_NORMAL 0x30001
if (status != CLI$_NORMAL) {
sys$exit(status);
} else {
printf ("SYS$CLI GETCMD was: \"%-.*s\"\n",
cli_req_block.cli$w_rqsize,
(char*)cli_req_block.cli$a_rqaddr);

[Craig A. Berry]

On 8/18/16 11:10 PM, John Reagan wrote:

> I look more in the morning and try to reconcile what I think the

> codedoes and what Hartmut showed.

>
> 1) Looking again at LIB$GET_FOREIGN, it is the string read from the
> prompt that is always uppercased. I'm guessing nobody checked that
> behavior when /PARSE=EXTENDED was added. That needs to be enhanced.
>
> 2) Extending on Harmut's example and calling the SYS$CLI to get the
> command line, you'll get the same behavior as LIB$GET_FOREIGN but with
> the leading MCR name. LIB$GET_FOREIGN trims that off for you.

Thanks for digging into this and for listening to me whine. The fact
that someone is willing to look at the cruft and suss out what can be
improved will take a little getting used to, but I think I can adapt :-).