%PCSIUI-E-PRIVCLASS1 when using PRODUCT with PIPE

[Bob Koehler]

Here's a new one I just found. First let me say I think somewhere
there's a disclaimer that PIPE doesn't work with everything.
OpenVMS Alpha 7.2-1. Note that PCSIUI is POLYCENTER Software
Installation Utility, not PIPE.

So why does PCSIUI care about SYSLCK only when the output is a pipe?

$ product show history
---------------------------------- ----------- -------------------------------
PRODUCT KIT TYPE OPERATION DATE AND TIME
----------------------------------- ----------- -------------------------------
CPQ AXPVMS MOZILLA M1.0 Full LP Install 06-JUN-2002 13:09:15
...

$ pipe product show history
---------------------------------- ----------- -------------------------------
PRODUCT KIT TYPE OPERATION DATE AND TIME
----------------------------------- ----------- -------------------------------
CPQ AXPVMS MOZILLA M1.0 Full LP Install 06-JUN-2002 13:09:15
...

$ pipe product show history | type/page sys$input
%PCSIUI-E-PRIVCLASS1, operation requires SYSLCK privilege
%PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition

$ set process/privilege=syslck
$ pipe product show history | type/page sys$input
---------------------------------- ----------- -------------------------------
PRODUCT KIT TYPE OPERATION DATE AND TIME
----------------------------------- ----------- -------------------------------
CPQ AXPVMS MOZILLA M1.0 Full LP Install 06-JUN-2002 13:09:15
...

[Craig A. Berry]

In article <LXX+ht...@eisner.encompasserve.org>,

koe...@encompasserve.org (Bob Koehler) wrote:
>
> So why does PCSIUI care about SYSLCK only when the output is a pipe?

> $ pipe product show history | type/page sys$input

> %PCSIUI-E-PRIVCLASS1, operation requires SYSLCK privilege
> %PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition

Interesting. I suppose it's possible PCSI really does want a
system-wide lock on one of the PPFs but can't get it because PIPE
already has it open. Or it could just be a bug where it does something
that tries to set the terminal characteristics on SYS$PIPE but since
that's not a terminal it falls back into some error code that isn't
entirely applicable.

[Jairo Alves]

Yes,

I'm seeing the same behaviour.

$ pipe product show product > sys$login:prod_show_prod.txt

%PCSIUI-E-PRIVCLASS1, operation requires SYSLCK privilege
%PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition

One extra bit of information: if run under system or sysadmin accounts
the same pipe works. But this is not a pipe fault. product command is the
one asking for the privilega.

One possible way to "solve" this:
UAF> modi <user> /priv=SYSLCK

And then it didn't even had to be clearly "enabled", just
adding to the authorizes privs list was enough to make it work.


--

$ show proc /priv

7-DEC-2018 10:16:57.27 User: USER Process ID: 0004ED93
Node: NODE Process name: "NAME"

Authorized privileges:
NETMBX SYSLCK TMPMBX

Process privileges:
NETMBX may create network device
TMPMBX may create temporary mailbox

[John E. Malmberg]

On 12/7/2018 7:18 AM, Jairo Alves wrote:

This is shown as a reply, but does not seem to have a parent post so far
in the newsgroup, but sometimes things do not make it from the mailing
list to the newsgroup, and sometimes posts are out of order.

> Yes,
>
> I'm seeing the same behaviour.
>
> $ pipe product show product > sys$login:prod_show_prod.txt
>
> %PCSIUI-E-PRIVCLASS1, operation requires SYSLCK privilege
> %PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition
>
> One extra bit of information: if run under system or sysadmin accounts
> the same pipe works. But this is not a pipe fault. product command is the
> one asking for the privilega.
>
> One possible way to "solve" this:
> UAF> modi <user> /priv=SYSLCK
>
> And then it didn't even had to be clearly "enabled", just
> adding to the authorizes privs list was enough to make it work.

Some product operations require privileges unless the installation
target is a user privately mounted volume.

The privately mounted volume is specified by the /remote qualifier.

Regards,
-John

[Stephen Hoffman]

On 2018-12-07 13:18:03 +0000, Jairo Alves said:

Here is the thread from 2002 that this is in reply to:

https://groups.google.com/d/msg/comp.os.vms/1p-RuxToEQY/YXKd5FrTh_wJ

> I'm seeing the same behaviour.
>
> $ pipe product show product > sys$login:prod_show_prod.txt
>
> %PCSIUI-E-PRIVCLASS1, operation requires SYSLCK privilege
> %PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition
>
> One extra bit of information: if run under system or sysadmin accounts
> the same pipe works. But this is not a pipe fault. product command is
> the one asking for the privilega.
>
> One possible way to "solve" this:
> UAF> modi <user> /priv=SYSLCK
>
> And then it didn't even had to be clearly "enabled", just adding to the
> authorizes privs list was enough to make it work.

PIPE and various DCL commands don't always play well together.
(There's much missing around the implementation of piping on OpenVMS.
Many of the DCL commands don't optionally offer to produce
pipe-friendly output, akin to what's possible with various shell
commands on other platforms. That and object-based piping and UTF-8
support and such are all fodder for another discussion or three,
though.)

PCSI expects full privileges lit and—until fairly recently—accepted
only a subset of the valid filename and directory path syntax. Use of
the SYSTEM username or another fully-privileged user is expected. This
is one of those murky areas of the OpenVMS documentation, though. (The
concept of disabling access to SYSTEM akin to what's happening with
root user access on some other Unix systems is an interesting topic to
ponder for the future of OpenVMS, too. But I digress.)

OpenVMS has long lacked a standard, non-privileged package installer.
This has caused folks problems going back years, too.

If you're solely looking for output from the command as part of a
software inventory or equivalent, then the following longstanding DCL
hackery will get you output from commands—such as this POLYCENTER
PRODUCT command—that lack an /OUTPUT qualifier:

$ @TT:/OUTPUT=file
$ {command}
^Z

Wrapping commands in DCL, and using @procedure-name/OUTPUT=file also works.

But maybe PCSI eventually sprouts some additional qualifiers, and some
other updates?

Given the scale and scope involved with bringing the OpenVMS software
packaging and installation environment forward to what's common on
other platforms, I'd eventually expect a wholesale replacement for PCSI
will arrive. This for connecting to VSI for notifications, and for
acquiring kits and certificates and other updates from the hypothetical
VSI software server, among other details. In deference to
compatibility, any replacement installer would preferably also accept
and process most PCSI kits, and/or a way to migrate the PCSI product
description files (PDF, yes, really) and product text files (PTF) and
the kit contents into the replacement environment. Accepting VMSINSTAL
kits, not so much. Though I could see automating and translating a
subset of what VMSINSTAL kits can do, VMSINSTAL kits tend to get
"creative" with their contents.




--
Pure Personal Opinion | HoffmanLabs LLC

[Jairo Alves]

John, maybe the thread was too old. I just found useful to answer
because in my case adding the privilege worked and that could be
the solution to a future user.

@Hoff
Thanks

Regarding this part:

$ @TT:/OUTPUT=file
$ {command}
^Z

That works(!) even without granting an extra privilege, which is a bonus.

How does it do it, though?
What do the TT stands for?
Why the Ctrl+Z at the end?

[Dave Froble]

TT is the logical for the terminal assigned to your process.

$ sho log tt
"TT" = "TNA12:" (LNM$PROCESS_TABLE)
$ sho proc

7-DEC-2018 13:03:07.67 User: DFE Process ID: 000001C1
Node: DFE90A Process name: "DFE"

Terminal: TNA12: (Host: 192.168.7.122 Port: 1421)
User Identifier: [DFE]
Base priority: 4
Default file spec: SYS$SYSDEVICE:[DFE]

Devices allocated: DFE90A$TNA12:

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: da...@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

[Jairo Alves]

>
> TT is the logical for the terminal assigned to your process.
>
> $ sho log tt
> "TT" = "TNA12:" (LNM$PROCESS_TABLE)

Thanks, Dave.

So, is running @TT is like saying to the compiler:
run the input from this logical 'TT' (my terminal) and direct
its sys$output to this file.

The ctrl+z then serves as the EOL flag for the logical content?

Is it somewhat similar to when you put a $ sign to finish a
'type sys$input' string?

That means this is used interactively only? So to use inside a procedure,
the better way would be the other solution indicated, to create a file
and put the commands inside it, then calling @com /out=file.
Still a pretty nice solution.

[Stephen Hoffman]

On 2018-12-07 17:31:48 +0000, Jairo Alves said:

> $ @TT:/OUTPUT=file
> $ {command}
> ^Z
>
> That works(!) even without granting an extra privilege, which is a bonus.

It works? Oh, my. How odd.

> How does it do it, though?

That is a DCL command procedure invocation. That command procedure
specifies the input be from the TT device. That then creates the DCL
equivalent of a subshell. And the output from DCL procedure
invocations can be redirected.

> What do the TT stands for?

TT: is the command input device. It's an RSX-era device naming
construct, and is a shorthand name for the current terminal or terminal
emulation session.

The OpenVMS logical names SYS$INPUT, SYS$COMMAND, SYS$OUTPUT and
SYS$PIPE have some similar uses, and can also be useful for redirection
and for pipelines.

> Why the Ctrl+Z at the end?

That's the End Of File for the input into that DCL procedure
invocation. You're effectively writing a DCL command procedure
directly into the DCL interpreter, and the ^Z pops the equivalent of a
subshell and returns you to the interactive DCL prompt.

The syntax in this area is also related to the leading $ sign in DCL
procedures for commands and no leading $ for data, and the DECK command.

This area also has parallels to the SYSBOOT> SET/STARTUP=OPA0:
conversational environment; the OpenVMS analog to the Unix single-user
mode environment. In that context, you're effectively entering the
commands of the startup script.

DCL pipelines and DCL I/O redirection are all still somewhat of a tire
fire, unfortunately.

[Jairo Alves]

> That is a DCL command procedure invocation. That command procedure
> specifies the input be from the TT device. That then creates the DCL
> equivalent of a subshell. And the output from DCL procedure
> invocations can be redirected.
>

I see. It's a subshell, not a "pseudo-command file". I tried out of
curiosity to run a loop in that command, and sure enough, DCL throws an
exception, precisely because it is not emulating a command file, but
instead a sub shell, where loops are forbidden.

I> @TT:
_I> n = 0
_I> loop:
%DCL-W-NOLBLS, label ignored - use only within command procedures
\LOOP:\
_I> wr sys$output n
0
_> Exit

> The syntax in this area is also related to the leading $ sign in DCL
> procedures for commands and no leading $ for data, and the DECK command.

When I teach simple DCL to my students they always complain about
the dollar sign convention. They find it kinda old-fashioned.

>
> This area also has parallels to the SYSBOOT> SET/STARTUP=OPA0:
> conversational environment; the OpenVMS analog to the Unix single-user
> mode environment. In that context, you're effectively entering the
> commands of the startup script.

By the way, I've used a guide of Hoffman labs to "break-in" an old
Alpha that nobody knew the password for a couple years ago. Thanks :)

As a measure of precaution, I printed that out and still have this
guide in my desk's drawer.

[Simon Clubley]

On 2018-12-11, Jairo Alves <jairo...@gmail.com> wrote:
>
> When I teach simple DCL to my students they always complain about
> the dollar sign convention. They find it kinda old-fashioned.
>

You could always try teaching them JCL (a z/OS command language). :-)

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world

[Tim Sneddon]

Jairo Alves <jairo...@gmail.com> wrote:
>
>> That is a DCL command procedure invocation. That command procedure
>> specifies the input be from the TT device. That then creates the DCL
>> equivalent of a subshell. And the output from DCL procedure
>> invocations can be redirected.
>>
>
> I see. It's a subshell, not a "pseudo-command file". I tried out of
> curiosity to run a loop in that command, and sure enough, DCL throws an
> exception, precisely because it is not emulating a command file, but
> instead a sub shell, where loops are forbidden.
>
> I> @TT:
> _I> n = 0
> _I> loop:
> %DCL-W-NOLBLS, label ignored - use only within command procedures
> \LOOP:\
> _I> wr sys$output n
> 0
> _> Exit
>

It is being treated as a command procedure. Years ago I investigated
this as I was trying to figure a way to provide a better implementation
of $(SHELL ...) for MMK. DCL actually checks to see if the command
procedure is disk based or not. If it is disk based, then certain
constructs become possible, like multi-line IF-THEN-ELSE-ENDIF, loops,
etc. If not, then you get single-line DCL.

If I had more time I'd dig out the specific place in the listings
where this is checked. Maybe VAXman (a.k.a Brian) has it memorised,
he has done quite of bit in the DCL space over the years.

Regards, Tim.

[Tim Sneddon]

Simon Clubley <clubley@remove_me.eisner.decus.org-earth.ufp> wrote:
> On 2018-12-11, Jairo Alves <jairo...@gmail.com> wrote:
>>
>> When I teach simple DCL to my students they always complain about
>> the dollar sign convention. They find it kinda old-fashioned.
>>
>
> You could always try teaching them JCL (a z/OS command language). :-)

That is just cruel...

Regards, Tim.

[Bill Gunshannon]

@START
@ACOB,SR MYPROG.COB,MYPROG.REL,
@MAP MYPROG.REL
@MSG,S Mount my tape now, Please.
@XQT MYPROG
Put Data Here
@EOF


Take that JCL!!!!

bill

[Stephen Hoffman]

On 2018-12-11 13:19:47 +0000, Jairo Alves said:

> When I teach simple DCL to my students they always complain about the
> dollar sign convention. They find it kinda old-fashioned.

I usually conceptually equate the leading $ to a semicolon in C or C++,
among those folks that have learned and programmed in those and similar
languages; to a source code line delimiter. There are some syntactic
differences there, of course.

Pragmatically, DCL is very much syntactic aggregations and accretions
perched atop a 1970s-era design, itself inspired by interactive BASIC
shells and with a dollop or two of FORTRAN syntax that was found in a
jar in the back of a Tewksbury 'fridge. FORTRAN 77 was bleeding edge,
when DCL was designed. In short, your students are quite correct. DCL
itself is quite old-fashioned, and variously also quite limited. The
DCL design and design compromises also skew heavily toward interactive
command input, and away from scripting and away from system and data
access features. Away from modern string handling, too; no regex, no
UTF-8, string comparisons and sorting and weak case conversions with no
localization, etc. Works decently well for basic US ASCII and DEC MCS,
but beyond that, there'll be deep trouble. Characters and signed
integer 32-bit integers, etc.

If you're not already looking at it or similar tools, interactive
Python (IPython) is a different approach with different design
compromises and different strengths, and focuses more toward scripting
and data access: https://ipython.org For broader usage beyond Python
and ilk, bash and maybe fish or another interactive shell would be more
commonly encountered. Microsoft also wishes PowerShell to be more
widely installed and more commonly available, though the success of
that open-sourcing effort remains to be seen. For embedded scripting,
prolly Lua.

> As a measure of precaution, I printed that out and still have this
> guide in my desk's drawer.

It's utterly absurd that single-user mode is even remotely necessary
for basic system and cluster configuration maintenance work; for the
password-reset and for licensing-related "fun", and for user-startup
troubleshooting. The world has moved past the use of command line for
these and similar maintenance tasks.

[Jairo Alves]

> DCL design and design compromises also skew heavily toward interactive
> command input, and away from scripting and away from system and data
> access features. Away from modern string handling, too; no regex, no

Couldn't regex be added as a lexical function?

[Arne Vajhøj]

Yes.

F$REGEX_MATCH would not be that difficult.

Arne

[John Reagan]

Besides the "match"/"nomatch" result, I assume you'd also like capturing syntax like you see in Perl regex strings? (aka \1 and $1)

The biggest obstacle in shoving a regex into the middle of DCL is making sure the license is suitable. For instance, we can't/won't put a piece of GPL code into DCL.EXE.

[Arne Vajhøj]

On 12/11/2018 2:39 PM, John Reagan wrote:
> On Tuesday, December 11, 2018 at 2:15:41 PM UTC-5, Arne Vajhøj wrote:
>> On 12/11/2018 1:42 PM, Jairo Alves wrote:
>>>> DCL design and design compromises also skew heavily toward interactive
>>>> command input, and away from scripting and away from system and data
>>>> access features. Away from modern string handling, too; no regex, no
>>>
>>> Couldn't regex be added as a lexical function?
>>
>> Yes.
>>
>> F$REGEX_MATCH would not be that difficult.
>

> Besides the "match"/"nomatch" result, I assume you'd also like capturing syntax like you see in Perl regex strings? (aka \1 and $1)

F$REGEX_MATCH may need some context to allow multiple calls.

And we would also need F$REGEX_REPLACE.

But if there were a desire to invest in DCL then it is doable.

> The biggest obstacle in shoving a regex into the middle of DCL is making sure the license is suitable. For instance, we can't/won't put a piece of GPL code into DCL.EXE.

PCRE library is BSD licensed.

Arne

[Jairo Alves]

>
> F$REGEX_MATCH would not be that difficult.
>
> Arne

Suggestion:

F$REGEX(input_string, regex_query, output_rule, modifiers)

For example:

$ str = "11-DEC-2018 17:24"
$ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
"$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")

$ wr sys$output str
"2018_DEC_11"

[Chris Scheers]

IIRC, DCL records the RFA for labels and ELSE/ENDIFs statement so that
they can be directly skipped to.

If it is not an RMS disk file, records don't have an RFA.

Another way to put it: The statements are not recorded and you can't
rewind a terminal.

--
-----------------------------------------------------------------------
Chris Scheers, Applied Synergy, Inc.

Voice: 817-237-3360 Internet: ch...@applied-synergy.com
Fax: 817-237-3074

[Stephen Hoffman]

On 2018-12-11 19:39:41 +0000, John Reagan said:

> Besides the "match"/"nomatch" result, I assume you'd also like
> capturing syntax like you see in Perl regex strings? (aka \1 and $1)
> The biggest obstacle in shoving a regex into the middle of DCL is
> making sure the license is suitable. For instance, we can't/won't put
> a piece of GPL code into DCL.EXE.

Search and replace. Likely POSIX regex syntax.

VSI already has regex code with search and replace capabilities within
LSEDIT and/or TPU. That assuming sufficient rights to reuse parts of
DECset, which I assume y'all have.

As for alternative sources, the BSDs have a wealth of Apache and
BSD/MIT-licensed app code.

https://lists.freebsd.org/pipermail/freebsd-announce/2018-December/001856.html

And...

NetBSD has a regular expression implementation in sed, and that
licensing is very likely compatible with your requirements:
http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/sed/?only_with_tag=MAIN
(Hi, Myrkraverk!)
http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/sed/process.c?rev=1.52&content-type=text/x-cvsweb-markup&only_with_tag=MAIN
etc.

[Craig A. Berry]

On 12/11/18 2:11 PM, Jairo Alves wrote:

> $ str = "11-DEC-2018 17:24"
> $ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
> "$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")

I think your second parameter would have to be "''$3'_''$2'_''$1'"
because to be really useful, the captured matches would need to be
ordinary DCL symbols that would persist after the lexical call. And in
that case, you probably really couldn't call them $1, $2, $3, etc.
because that's going to conflict with existing usage somewhere. So,
sorry to say, but they would probably need to be something like
DCL$REGEX_MATCH_1, DCL$REGEX_MATCH_2, etc.

[Arne Vajhøj]

On 12/11/2018 11:30 AM, Stephen Hoffman wrote:
> On 2018-12-11 13:19:47 +0000, Jairo Alves said:
>
>> When I teach simple DCL to my students they always complain about the
>> dollar sign convention. They find it kinda old-fashioned.
>
> I usually conceptually equate the leading $ to a semicolon in C or C++,
> among those folks that have learned and programmed in those and similar
> languages; to a source code line delimiter.  There are some syntactic
> differences there, of course.
>
> Pragmatically, DCL is very much syntactic aggregations and accretions
> perched atop a 1970s-era design, itself inspired by interactive BASIC
> shells and with a dollop or two of FORTRAN syntax that was found in a
> jar in the back of a Tewksbury 'fridge.  FORTRAN 77 was bleeding edge,
> when DCL was designed.  In short, your students are quite correct.  DCL
> itself is quite old-fashioned, and variously also quite limited.

Free format and semicolon separation is certainly common today.

But there are still languages including new and popular ones with
more fixed formats.

I do not see the lack of free format as a major problem for
DCL (there are many other real problems).

I think I could explain what:

$ cc /obj=z sys$input
#include <stdio.h>

int main()
{
printf("Hello world!\n");
return 0;
}
$
$ link z
$ run z
$ exit

does to a non-VMS person without the person screaming weird.

I am not sure that I could do the same for:

$ cc /obj=z sys$input
$ deck/dollars="****THE END****"
#include <stdio.h>

int main()
{
printf("Hello world!\n");
return 0;
}
****THE END****
$ link z
$ run z
$ exit

though.

:-)

Arne

[Arne Vajhøj]

On 12/11/2018 3:11 PM, Jairo Alves wrote:
>> F$REGEX_MATCH would not be that difficult.
>

> Suggestion:
>
> F$REGEX(input_string, regex_query, output_rule, modifiers)
>
> For example:
>
> $ str = "11-DEC-2018 17:24"
> $ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
> "$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")
>
> $ wr sys$output str
> "2018_DEC_11"

I think this is really F$REGEX_REPLACE.

Arne

[Arne Vajhøj]

On 12/11/2018 7:55 PM, Craig A. Berry wrote:
> On 12/11/18 2:11 PM, Jairo Alves wrote:
>> $ str = "11-DEC-2018 17:24"
>> $ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
>>                  "$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")
>
> I think your second parameter would have to be "''$3'_''$2'_''$1'"
> because to be really useful, the captured matches would need to be
> ordinary DCL symbols that would persist after the lexical call. And in
> that case, you probably really couldn't call them $1, $2, $3, etc.
> because that's going to conflict with existing usage somewhere.

Returning a single string as function result make perfect sense to me.

This is not F$REGEX_MATCH but F$REGEX_REPLACE.

> So,
> sorry to say, but they would probably need to be something like
> DCL$REGEX_MATCH_1, DCL$REGEX_MATCH_2, etc.

F$REGEX_MATCH would either have to set a bunch of symbols like
this.

Or have a context variable and make multiple calls to the
lexical similar to various F$GETXXX functions.

Arne

[Dave Froble]

Not casting a handful of mud and such at anyone, but that is ugly, and
at least for me giberish. I don't want to even try to understand it.

[Jairo Alves]

> I think this is really F$REGEX_REPLACE.

You could divide in two functions, or just merge them, returning a
boolean when the output_rule is empty, and returning a string otherwise.

>
> Not casting a handful of mud and such at anyone, but that is ugly, and
> at least for me giberish. I don't want to even try to understand it.
>

I agree with you, It is rather ugly. As someone that didn't come from
a CS background, I remember finding the regex syntax very daunting
to say the least at first sight. In fact, I didn't get it at first.

That being said, though, the flip side is that those rather ugly codes
are pretty handy tools to have at hand from time to time.

[VAXman-]

Correct! If DCL encounters <label>: when processing the procedure, a
symbol is created, a label symbol, of name <label> and the RFa of the
source where it was encountered is the symbol's value. If a control
command uses <label> as a target, DCL can immediately transfer to the
record identified by the RFA. If the label specified in the control
command has not yet been encountered, (ie, there's no label symol) a
scan of the file ensues to locate <label>: and create the label symbol
before the transfer of control.

>If it is not an RMS disk file, records don't have an RFA.

Yup. It has to be a random access device.

>Another way to put it: The statements are not recorded and you can't
>rewind a terminal.

A terminal is not a random access device.



To answer Tim's assertion, I believe this is checked for in the module
INDIRECT.

--
VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG

I speak to machines with the voice of humanity.

[Simon Clubley]

It's a pity DCL doesn't have proper arrays as this is something that
would be ideal to be returned as a variable sized array.

[Simon Clubley]

On 2018-12-12, Dave Froble <da...@tsoft-inc.com> wrote:
> On 12/11/2018 9:20 PM, Arne Vajhøj wrote:
>> On 12/11/2018 3:11 PM, Jairo Alves wrote:
>>>> F$REGEX_MATCH would not be that difficult.
>>>
>>> Suggestion:
>>>
>>> F$REGEX(input_string, regex_query, output_rule, modifiers)
>>>
>>> For example:
>>>
>>> $ str = "11-DEC-2018 17:24"
>>> $ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
>>> "$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")
>>>
>>> $ wr sys$output str
>>> "2018_DEC_11"
>>
>> I think this is really F$REGEX_REPLACE.
>

> Not casting a handful of mud and such at anyone, but that is ugly, and
> at least for me giberish. I don't want to even try to understand it.
>

$ set response/mode=good_natured

Nice to see you are as predictable as always. :-)

I wondered what you would say when you saw the above example of a regex. :-)

However, once you get used to them, a regex is a _very_ powerful
pattern matching tool and I don't have any problems understanding
what the above example is trying to say.

[Simon Clubley]

On 2018-12-11, Arne Vajhøj <ar...@vajhoej.dk> wrote:

The biggest thing for me is that John doesn't seem to think there's
a major conceptual problem with inserting C code into DCL, even
given DCL's unique requirements.

If you can get C code into DCL, I wonder if that could be used to
fix up other DCL issues.

For example, we know from previous discussions the line editing code
in the terminal driver is an unmaintainable mess, so we can't edit
command lines which are longer than the terminal width. Could the problem
be worked around (at least for DCL) by placing line editing code into
DCL ?

This could also bring along other benefits such as a much nicer history
searching method, including incremental search.

[Jan-Erik Söderholm]

Den 2018-12-12 kl. 14:30, skrev Simon Clubley:
> On 2018-12-12, Dave Froble <da...@tsoft-inc.com> wrote:
>> On 12/11/2018 9:20 PM, Arne Vajhøj wrote:
>>> On 12/11/2018 3:11 PM, Jairo Alves wrote:
>>>>> F$REGEX_MATCH would not be that difficult.
>>>>
>>>> Suggestion:
>>>>
>>>> F$REGEX(input_string, regex_query, output_rule, modifiers)
>>>>
>>>> For example:
>>>>
>>>> $ str = "11-DEC-2018 17:24"
>>>> $ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
>>>> "$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")
>>>>
>>>> $ wr sys$output str
>>>> "2018_DEC_11"
>>>
>>> I think this is really F$REGEX_REPLACE.
>>
>> Not casting a handful of mud and such at anyone, but that is ugly, and
>> at least for me giberish. I don't want to even try to understand it.
>>

Even if it started as a tool on some Unix environment, today it has
become more of a standard.

On VMS it doesn't have to be part of DCL in itself, there is regex
support on tools like awk and perl, and in more modern scripting
languages like Python. If you are in the need of regex processing,
you are also doing some other more complex processing where a more
capable tool (than DCL) can be handy.

[John E. Malmberg]

On 12/11/2018 7:47 AM, Tim Sneddon wrote:
> Jairo Alves <jairo...@gmail.com> wrote:
>>

snip

>
> It is being treated as a command procedure. Years ago I investigated
> this as I was trying to figure a way to provide a better implementation
> of $(SHELL ...) for MMK. DCL actually checks to see if the command
> procedure is disk based or not. If it is disk based, then certain
> constructs become possible, like multi-line IF-THEN-ELSE-ENDIF, loops,
> etc. If not, then you get single-line DCL.

Is LD source available?

Make a variant that could produce a small memory disk in the users
virtual address space instead of a file.

Or use LD on a temp file with good RMS caching.

Have MMK (or other tools) detect if that tool is available and use it to
run small scripts.

Regards,
-John

[Simon Clubley]

On 2018-12-11, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>

> I am not sure that I could do the same for:
>
> $ cc /obj=z sys$input
> $ deck/dollars="****THE END****"
> #include <stdio.h>
>
> int main()
> {
> printf("Hello world!\n");
> return 0;
> }
> ****THE END****
> $ link z
> $ run z
> $ exit
>
> though.
>

Would the following be nicer ?

cc /obj=z sys$input << "****THE END****"

#include <stdio.h>

int main()
{
printf("Hello world!\n");
return 0;
}
****THE END****

link z
run z
exit

You can argue that if the terminating string is mistyped, all the
input would be consumed as compiler input. However, in that case,
the command procedure would go obviously wrong (and in this case
would cause a compiler error).

In the normal DCL method, if the following command doesn't have
a "$", it would be consumed as program input so the command
procedure could go wrong in a subtle way instead of an obvious way.

You could also do this (which would be my preferred option):

cc /obj=z sys$input
|#include <stdio.h>
|
|int main()
|{
| printf("Hello world!\n");
| return 0;
|}

link z
run z
exit

where you invert what happens in DCL at the moment and place the
special character in front of the inline data instead of in front
of the DCL commands. This is the option I prefer as it gets rid of
all the leading "$" characters and is just as robust.

[johnso...@gmail.com]

On Wednesday, December 12, 2018 at 8:42:47 AM UTC-5, Jan-Erik Söderholm wrote:

> Even if it started as a tool on some Unix environment, today it has
> become more of a standard.

It's even part of the C++-11 standard.

https://en.cppreference.com/w/cpp/regex

[John E. Malmberg]

TL;DR:

My vote would be to keep DCL the way it is, but to provide a good
interactive Python shell for scripting that needs to be 64+ bit aware,
with applications designed to handle serialized objects.

Imagine a SHOW DEVICE/TO_XML or /TO_YAML, etc command in VMS that could
be invoked by by interactive python or other scripting languages.

Or a set foobar/from_xml command etc.

On 12/11/2018 10:30 AM, Stephen Hoffman wrote:
<snip>

> If you're not already looking at it or similar tools, interactive Python
> (IPython) is a different approach with different design compromises and
> different strengths, and focuses more toward scripting and data access:
> https://ipython.org   For broader usage beyond Python and ilk, bash and
> maybe fish or another interactive shell would be more commonly
> encountered.   Microsoft also wishes PowerShell to be more widely
> installed and more commonly available, though the success of that
> open-sourcing effort remains to be seen.  For embedded scripting, prolly
> Lua.

You keep mentioning PowerShell.

Have you done any significant coding in PowerShell?

Last time I was using it, I was spending a lot of time fixing scripts
migrated to PowerShell to work around bugs and omissions in PowerShell.

There are massive easy ways to get your self into trouble. There were
significant gaps in the methods available for PowerShell to do simple
basic operations used for system configuration and testing.

For example, operations on Powershell on mapped drive letters are
incompatible with the DOS commands for doing the same thing and the
PowerShell API is incomplete. But there is also a COM interface to
Powershell, and a COM interface to the mapped drive letters.

It of course is totally incompatible with the documented PowerShell API
that should do the same function. That they are incompatible is not
documented of course.

Trying to use COM from PowerShell to fill in the gaps will result in
days spent trying to figure out why the simple scripts for concept
testing worked, but the long running tests that use them periodically
randomly go into the weeds until you find out that is simply a
feature/bug of the COM interface that can not be disabled, so everything
with the drive mapping has to be converted back to DOS commands.

And it would help if someone at Microsoft would issue an edict that an
application can not ship unless it can be installed and configured
completely by PowerShell over a network connection.

For some reason Microsoft wants the only installation for several
products to be through logged in user at the GUI for what they call the
"Out of the Box" experience, where you have to play 20 questions to get
the system setup. And they seem to be actively trying to break any
automated way of installing / configuring, even for testing purposes
with each new release.

Is the grass greener there?

And what is the big feature of Powershell? Passing objects.

How are the objects passed between independent applications? They are
serialized into a format like XML, or read in as XML. I Seem to
remember there being two options, one of them being XML. You can
specify them as options to the PowerShell commands.

Powershell aware applications can know in a pipeline they should default
to write the output in XML or other serialized format, and the same for
reading, but when run interactively to format the output in a human
readable form.

That feature could be added to DCL commands and system utilities on VMS.

> It's utterly absurd that single-user mode is even remotely necessary for
> basic system and cluster configuration maintenance work; for the
> password-reset and for licensing-related "fun", and for user-startup
> troubleshooting.  The world has moved past the use of command line for
> these and similar maintenance tasks.

Linux still needs single user mode for recovery of a system where you do
not know the root password, especially with the current practice of not
setting a root password at all.

Almost all automatic and large scale system setup and maintenance is via
the command line under the covers at the implementation point. Remote
GUI console access is way too expensive when a "small" collection of
systems is in the middle double digits and a typical collection is much
larger.

All debugging is over serial connections.

Linux is also covering a mass of backward [in]compatibility with layers
of symbolic links and scripts in an attempt to hide distro differences.
Each application of course uses a different method.

Regards,
-John

[John Reagan]

On Wednesday, December 12, 2018 at 8:38:43 AM UTC-5, Simon Clubley wrote:
> On 2018-12-11, Arne Vajhøj <ar...@vajhoej.dk> wrote:
> > On 12/11/2018 2:39 PM, John Reagan wrote:
> >> The biggest obstacle in shoving a regex into the middle of DCL is making sure the license is suitable. For instance, we can't/won't put a piece of GPL code into DCL.EXE.
> >
> > PCRE library is BSD licensed.
> >
>
> The biggest thing for me is that John doesn't seem to think there's
> a major conceptual problem with inserting C code into DCL, even
> given DCL's unique requirements.
>
> If you can get C code into DCL, I wonder if that could be used to
> fix up other DCL issues.
>

There is already C code in DCL, albeit in the minority (< 10%). We have discussed conversion of large portions of Macro-32 code into C code in DCL (think symbol table management and the like). The code that manages stack context, etc. needs to stay in Macro-32 (and even some target assembly) due to DCL's unique ability for "^Y", do a set of DCL commands, "CONTINUE". And yes, we have discussed modifying that for images installed with privs.

[Stephen Hoffman]

On 2018-12-12 13:38:41 +0000, Simon Clubley said:

> The biggest thing for me is that John doesn't seem to think there's a
> major conceptual problem with inserting C code into DCL, even given
> DCL's unique requirements.
>
> If you can get C code into DCL, I wonder if that could be used to fix
> up other DCL issues.

C is available in kernel mode, and using, the so-called kernel CRTL.
Some C library functions are and others can be implemented in the
kernel CRTL fairly easily, while other functions—typically thise
involving system calls or RTL calls not directly available in kernel
context—are much more difficult to provide in the kernel CRTL. And
those are usually omitted.

> For example, we know from previous discussions the line editing code in
> the terminal driver is an unmaintainable mess, so we can't edit command
> lines which are longer than the terminal width. Could the problem be
> worked around (at least for DCL) by placing line editing code into DCL ?

DCL can't fix a terminal driver limit.

The terminal driver keeps one (non-wrapped) line around. DCL already
keeps the complete command line in its history, and works around the
terminal driver limit.

As I've commented elsewhere, the changes permissible within the
existing class and port terminal driver stack are constrained by the
expectations of upward-compatibility. It's difficult to retrofit
changes into this and similar environments, as it's very easy to break
(some) existing code with changes to documented or even to undocumented
behavior.

What do I mean by this? I broke the test suite with a change to
SYLOGIN.COM years ago; to the login terminal driver settings. Somebody
somewhere was going to be dependent on the old behavior, even though
the new behavior made more sense in the then-current era. Similarly
and separately an aeon or two ago, OpenVMS changed the I/O completion
processing to be asynchronous across multiple processors in an SMP
configuration, and that change exposed latently-broken OpenVMS cpde and
app code all over the place. Both sets of changes was reverted.

The terminal driver environment could be well-served by a redesign and
replacement. Most systems have at most one or maybe two hardware
serial lines and those now primarily relegated to server bring-up tasks
and console tasks, and we can only expect traditional serial consoles
to be replaced. Other "serial" connections are USB, network or iLO.
Keep the old terminal driver stack around pending app migrations, but
get folks over to the new implementation.

Rewriting the terminal driver and/or the CLI environment? I'd
certainly like to see that and for various reasons, but I'd assume that
there are few existing apps that would care about that, and there are
other projects folks want more. There's a port to complete, and a
whole lot of other work pending.

[Simon Clubley]

On 2018-12-12, Stephen Hoffman <seao...@hoffmanlabs.invalid> wrote:
> On 2018-12-12 13:38:41 +0000, Simon Clubley said:
>
>> The biggest thing for me is that John doesn't seem to think there's a
>> major conceptual problem with inserting C code into DCL, even given
>> DCL's unique requirements.
>>
>> If you can get C code into DCL, I wonder if that could be used to fix
>> up other DCL issues.
>
> C is available in kernel mode, and using, the so-called kernel CRTL.

Yes, I know, but that API had to be added to VMS before it was
possible to use C in the kernel.

Likewise, the use of C within DCL was interesting because I didn't
know if the use of C within DCL was even possible without adding
support for it. As John has said, it is already in use which opens
up various additional functionality options in the future.

> Some C library functions are and others can be implemented in the

> kernel CRTL fairly easily, while other functions?typically thise

> involving system calls or RTL calls not directly available in kernel

> context?are much more difficult to provide in the kernel CRTL. And
> those are usually omitted.
>

I've written a C language kernel device driver so I have a feeling
for what you can and cannot do (although it was a long time ago).

>> For example, we know from previous discussions the line editing code in
>> the terminal driver is an unmaintainable mess, so we can't edit command
>> lines which are longer than the terminal width. Could the problem be
>> worked around (at least for DCL) by placing line editing code into DCL ?
>
> DCL can't fix a terminal driver limit.
>

No, but it can do what bash on VMS does and provide its own functionality.
Even better, it could be turned into a library so that everyone else
could use it as well in their own programs.

> The terminal driver keeps one (non-wrapped) line around. DCL already
> keeps the complete command line in its history, and works around the
> terminal driver limit.
>

But doesn't allow you to edit multiple lines on a long command without
deleting enough of the command to force it on to one line.

> As I've commented elsewhere, the changes permissible within the
> existing class and port terminal driver stack are constrained by the
> expectations of upward-compatibility. It's difficult to retrofit
> changes into this and similar environments, as it's very easy to break
> (some) existing code with changes to documented or even to undocumented
> behavior.
>

I've never seen the VMS source code so I don't know the answer to
this, but is the terminal driver line editing code unmaintainable
because someone tried to be "clever" when writing the code and hence
caused maintainence problems later on when changes were required,
or was it simply that someone tried to fit something into the kernel
that should never have been placed in the kernel in the first place ?

[Stephen Hoffman]

On 2018-12-12 18:41:18 +0000, Simon Clubley said:

> I've never seen the VMS source code so I don't know the answer to this,
> but is the terminal driver line editing code unmaintainable because
> someone tried to be "clever" when writing the code and hence caused
> maintainence problems later on when changes were required, or was it
> simply that someone tried to fit something into the kernel that should
> never have been placed in the kernel in the first place ?

Operating systems are replete with trade-offs. The associated designs
and the inevitable trade-offs in each operating system age. At
different rates.

The terminal driver stack and the storage driver stack are effectively
enormous input-parsing state tables. Running in kernel-mode. Written
in iterative, procedural Macro32, Bliss, or C code. Without the
benefit of state-table tooling.

Drivers aren't the only spot that's been late to adopt tooling. DCL
had its own process threading implementation up until the Itanium port.
Use of KP Threads was then implemented. More than a little of OpenVMS
doesn't leverage now-available libraries and tools. Or doesn't provide
the libraries and tools.

The terminal class and port design does remain a quite good and useful
approach, though I suspect the API might look very different if it were
rebuilt in this era. But with multiple port drivers present,
substantive changes to the class driver or the class-port interface
incurs large testing efforts. Or the deprecation of various
communications or storage hardware. Or both.

In the era that VAX/VMS arose, process creation and interprocess
communications and the associated designs lacked performance, and
buffer copies were ubiquitous in buffered-I/O device drivers and in
many of the app designs. Having everything in the kernel—spread across
three modes/rings—was appropriate and had (and has) acceptable
performance for most applications. There's been more than a little
hardware misbehavior fixed in software, too. Some of the chipsets
are... buggy.

The maintenance problems here are not unexpected, and arise from
decades of device-specific accretions and work-arounds, and of decades
of API assumptions lurking in vendor and third-party app code.

In a wholesale redesign in this area and in this era, I'd look to use
something closer to seL4 and process-based isolation, at a migration to
state machine designs, and with the intent of never copying a buffer
around. With much thinner "drivers". But that's not going to happen.
Well, this refactoring sorta did happen, and it was called DEC MICA.

[Robert A. Brooks]

On 12/12/2018 1:41 PM, Simon Clubley wrote:

> I've never seen the VMS source code so I don't know the answer to
> this, but is the terminal driver line editing code unmaintainable
> because someone tried to be "clever" when writing the code and hence
> caused maintainence problems later on when changes were required,
> or was it simply that someone tried to fit something into the kernel
> that should never have been placed in the kernel in the first place ?

The terminal driver was the first to be written in the separated
"class" and "port" driver model, where the higher layer protocols
are in the class driver, and the port drivers contain hardware-specific
stuff. This was back in the VAX days, where you had an array of terminal
muxes with varying hardware support control needs. That separation was an
attempt at making the class driver a bit more maintainable. This model has been
followed with other driver pairs, most notably the SCSI class driver DKDRIVER
and the various SCSI and fibre channel port drivers below it.

The terminal class driver is written in MACRO-32, and is a bit of a mess.

I would think that t gives the host-based shadowing driver a run for its money
in terms of driving the Macro compiler nuts, although I've not heard Reagan
complain about it, and he's quite capable of complaining about stuff.

We are making some changes to the terminal class driver for X86, although those
changes are specific to the changes imposed by the X86 memory management model
that impact I/O.

For those keeping score at home, there is no SVAPTE on X86, which is keeping
our driver maintainers quite busy.

--
-- Rob

[dgordo...@gmail.com]

On Wednesday, December 12, 2018 at 2:49:59 PM UTC-5, Stephen Hoffman wrote:
> On 2018-12-12 18:41:18 +0000, Simon Clubley said:
>
>
> Drivers aren't the only spot that's been late to adopt tooling. DCL
> had its own process threading implementation up until the Itanium port.
> Use of KP Threads was then implemented.
>

And at least half of the ways I know to crash DCL are as a result of grafting that on to the existing DCL model. (And now Simon is excited that there might be more ways to crash DCL...)

[Jan-Erik Söderholm]

Den 2018-12-12 kl. 19:41, skrev Simon Clubley:

> But doesn't allow you to edit multiple lines on a long command without
> deleting enough of the command to force it on to one line.

In the 80's, many users might have been working from the DCL prompt.
Not so these days, mostly a few sysmgr's who still see the $-prompt.

I'd say that almost any work on the CLI/DCL interface is wasted money.

Put the resources and money where they actually make some use instead.

[VAXman-]

In article <purkod$2ph$1...@dont-email.me>, Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> writes:
>On 2018-12-12, Stephen Hoffman <seao...@hoffmanlabs.invalid> wrote:

>> On 2018-12-12 13:38:41 +0000, Simon Clubley said:
>>
>>> The biggest thing for me is that John doesn't seem to think there's a
>>> major conceptual problem with inserting C code into DCL, even given
>>> DCL's unique requirements.
>>>
>>> If you can get C code into DCL, I wonder if that could be used to fix
>>> up other DCL issues.
>>
>> C is available in kernel mode, and using, the so-called kernel CRTL.
>

>Yes, I know, but that API had to be added to VMS before it was
>possible to use C in the kernel.
>
>Likewise, the use of C within DCL was interesting because I didn't
>know if the use of C within DCL was even possible without adding
>support for it. As John has said, it is already in use which opens
>up various additional functionality options in the future.

One of the C modules is for Posix. Another has a few unit conversion
routines. And, one more, isn't really C at all; it's ia64 assembly in
C.

[VAXman-]

In article <pursc2$mj4$1...@dont-email.me>, "Robert A. Brooks" <FIRST...@vmssoftware.com> writes:
>On 12/12/2018 1:41 PM, Simon Clubley wrote:
>

>> I've never seen the VMS source code so I don't know the answer to
>> this, but is the terminal driver line editing code unmaintainable
>> because someone tried to be "clever" when writing the code and hence
>> caused maintainence problems later on when changes were required,
>> or was it simply that someone tried to fit something into the kernel
>> that should never have been placed in the kernel in the first place ?
>

>The terminal driver was the first to be written in the separated
>"class" and "port" driver model, where the higher layer protocols
>are in the class driver, and the port drivers contain hardware-specific
>stuff. This was back in the VAX days, where you had an array of terminal
>muxes with varying hardware support control needs. That separation was an
>attempt at making the class driver a bit more maintainable. This model has been
>followed with other driver pairs, most notably the SCSI class driver DKDRIVER
>and the various SCSI and fibre channel port drivers below it.
>
>The terminal class driver is written in MACRO-32, and is a bit of a mess.
>
>I would think that t gives the host-based shadowing driver a run for its money
>in terms of driving the Macro compiler nuts, although I've not heard Reagan
>complain about it, and he's quite capable of complaining about stuff.
>
>We are making some changes to the terminal class driver for X86, although those
>changes are specific to the changes imposed by the X86 memory management model
>that impact I/O.
>
>For those keeping score at home, there is no SVAPTE on X86, which is keeping
>our driver maintainers quite busy.

^^
@@

That's gonna make things interesting. I can't wait to see this when it's all
available. I'll be needing to make changes in my code too.

[Stephen Hoffman]

On 2018-12-12 22:14:06 +0000, dgordo...@gmail.com said:

> On Wednesday, December 12, 2018 at 2:49:59 PM UTC-5, Stephen Hoffman wrote:
>>
>> Drivers aren't the only spot that's been late to adopt tooling. DCL
>> had its own process threading implementation up until the Itanium port.
>> Use of KP Threads was then implemented.
>
> And at least half of the ways I know to crash DCL are as a result of
> grafting that on to the existing DCL model.

prefer Would home-grown you the threading?

[Simon Clubley]

Unfortunately, Simon's excitement is constrained by the fact there
are no Itanium full system emulators and that real Itanium hardware
is too noisy (and probably bulky) to run at home.

Oh, well. :-)

[Arne Vajhøj]

On 12/12/2018 8:38 AM, Simon Clubley wrote:
> On 2018-12-11, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>> On 12/11/2018 2:39 PM, John Reagan wrote:
>>> The biggest obstacle in shoving a regex into the middle of DCL is making sure the license is suitable. For instance, we can't/won't put a piece of GPL code into DCL.EXE.
>>
>> PCRE library is BSD licensed.
>
> The biggest thing for me is that John doesn't seem to think there's
> a major conceptual problem with inserting C code into DCL, even
> given DCL's unique requirements.
>
> If you can get C code into DCL, I wonder if that could be used to
> fix up other DCL issues.

I am sure you can use C in DCL.

C is used for kernel mode code in many OS'es because it works
in privileged mode.

And I believe device drivers in C has been officially supported
on VMS for years.

Arne

[Arne Vajhøj]

True interactive command line work is relative rare.

But scripting is not.

Which makes fixing the terminal driver a low priority, but may
keep a better scripting language let us call it a medium priority.

Arne

[Arne Vajhøj]

On 12/12/2018 12:40 AM, Dave Froble wrote:
> On 12/11/2018 9:20 PM, Arne Vajhøj wrote:
>> On 12/11/2018 3:11 PM, Jairo Alves wrote:
>>>> F$REGEX_MATCH would not be that difficult.
>>>
>>> Suggestion:
>>>
>>>   F$REGEX(input_string, regex_query, output_rule, modifiers)
>>>
>>> For example:
>>>
>>> $ str = "11-DEC-2018 17:24"
>>> $ str = F$REGEX(str, "(\d{2})-(\w+)-(\d{4})", -
>>>                  "$3_$2_$1", "UNGREEDY,CASE_INSENSITIVE")
>>>
>>> $ wr sys$output str
>>> "2018_DEC_11"
>>
>> I think this is really F$REGEX_REPLACE.
>

> Not casting a handful of mud and such at anyone, but that is ugly, and
> at least for me giberish.  I don't want to even try to understand it.

There is an old joke: I had a problem, then I decided to solve it using
regex and now I have two problems.

:-)

Regex is a bit cryptic. But it can be useful.

But I will say that all programmers should know SQL, regex and XPath.

Arne

[Arne Vajhøj]

POSIX regex has been standardized in POSIX for many years.

But most implementations use Perl regex.

Arne

[Arne Vajhøj]

On 12/12/2018 8:52 AM, Simon Clubley wrote:
> On 2018-12-11, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>>
>> I am not sure that I could do the same for:
>>
>> $ cc /obj=z sys$input
>> $ deck/dollars="****THE END****"
>> #include <stdio.h>
>>
>> int main()
>> {
>> printf("Hello world!\n");
>> return 0;
>> }
>> ****THE END****
>> $ link z
>> $ run z
>> $ exit
>>
>> though.
>>
>
> Would the following be nicer ?
>
> cc /obj=z sys$input << "****THE END****"
> #include <stdio.h>
>
> int main()
> {
> printf("Hello world!\n");
> return 0;
> }
> ****THE END****
>
> link z
> run z
> exit

That is about as obfuscated as the deck version, but it has
one advantage over it - it is not required to bring punched
cards up in the explanation.

Arne

[Arne Vajhøj]

On 12/12/2018 9:47 AM, John E. Malmberg wrote:
> My vote would be to keep DCL the way it is, but to provide a good
> interactive Python shell for scripting that needs to be 64+ bit aware,
> with applications designed to handle serialized objects.

I think many agrees with that.

:-)

I have not used PS for anything serious.

But I do know a few things about it.

First one should note that PS is .NET based, so no surprise that
PS and COM can give some "interesting" problems.

Second an important benefit of PS is the ability to extend it
with cmdlets written in C# (or another .NET language). So even
though it out-of-the-box may have some limitations, then
its extendability provides some options.

Arne

[Dave Froble]

On 12/12/2018 7:41 PM, Simon Clubley wrote:
> On 2018-12-12, dgordo...@gmail.com <dgordo...@gmail.com> wrote:
>> On Wednesday, December 12, 2018 at 2:49:59 PM UTC-5, Stephen Hoffman wrote:
>>> On 2018-12-12 18:41:18 +0000, Simon Clubley said:
>>>
>>>
>>> Drivers aren't the only spot that's been late to adopt tooling. DCL
>>> had its own process threading implementation up until the Itanium port.
>>> Use of KP Threads was then implemented.
>>>
>>
>> And at least half of the ways I know to crash DCL are as a result of grafting that on to the existing DCL model. (And now Simon is excited that there might be more ways to crash DCL...)
>>
>
> Unfortunately, Simon's excitement is constrained by the fact there
> are no Itanium full system emulators and that real Itanium hardware
> is too noisy (and probably bulky) to run at home.
>
> Oh, well. :-)
>
> Simon.
>

Nonsense! The systems aren't all that bulky. Ear plugs are cheap.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: da...@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

[Dave Froble]

I know some SQL. I don't even know what regex and xpath is. I guess
according to you I should stick to flying. Oh well, much more fun anyway.

Now if it would just warm up ....

[Arne Vajhøj]

On 12/12/2018 8:34 PM, Dave Froble wrote:
> On 12/12/2018 7:49 PM, Arne Vajhøj wrote:
>> Regex is a bit cryptic. But it can be useful.
>>
>> But I will say that all programmers should know SQL, regex and XPath.
>

> I know some SQL.  I don't even know what regex and xpath is.

regex is a way to find patterns in text.

XPath is a way to select parts of an XML document.

>   I guess
> according to you I should stick to flying.  Oh well, much more fun anyway.
>
> Now if it would just warm up ....

Or maybe something to study during Christmas vacation.

:-)

Arne

[Arne Vajhøj]

Simple regex example in VB.NET:

Imports System
Imports System.Text.RegularExpressions

Namespace DF1
Public Class Program
Public Shared Sub Main(args As String())
Dim s As String = "bla bla bla 11-DEC-2018 bla bla bla
12-DEC-2018 bla bla bla 13-DEC-2018 bla bla bla"
For Each m As Match In Regex.Matches(s,
"[0-9]{2}-[A-Z]{3}-[0-9]{4}")
Console.WriteLine(m.Value)
Next
End Sub
End Class
End Namespace

Output:

11-DEC-2018
12-DEC-2018
13-DEC-2018

Arne

[Arne Vajhøj]

On 12/12/2018 8:58 PM, Arne Vajhøj wrote:

Simple XPath example in VB.NET:

Imports System
Imports System.Xml

Namespace DF2

Public Class Program
Public Shared Sub Main(args As String())

Dim xml As String = "<all>" & vbCrLf & _
" <one>" & vbCrLf & _
" <a id=""1""/>" & vbCrLf & _
" <b>A</b>" & vbCrLf & _
" </one>" & vbCrLf & _
" <one>" & vbCrLf & _
" <a id=""2""/>" & vbCrLf & _
" <b>BB</b>" & vbCrLf & _
" </one>" & vbCrLf & _
" <one>" & vbCrLf & _
" <a id=""3""/>" & vbCrLf & _
" <b>CCC</b>" & vbCrLf & _
" </one>" & vbCrLf & _
"</all>"
Console.WriteLine(xml)
Dim doc As New XmlDocument()
doc.LoadXml(xml)
For Each n As XmlNode In doc.SelectNodes("/all/one")
Console.WriteLine(n.SelectSingleNode("a/@id").Value & "
" & n.SelectSingleNode("b/text()").Value)

Next
End Sub
End Class
End Namespace

Output:

<all>
<one>
<a id="1"/>
<b>A</b>
</one>
<one>
<a id="2"/>
<b>BB</b>
</one>
<one>
<a id="3"/>
<b>CCC</b>
</one>
</all>
1 A
2 BB
3 CCC

Arne

[Robert A. Brooks]

New data structure -- I/O buffer descriptor (IOBD)

It'll be a requirement for X86, and will also work for IA64.

Alpha, maybe.

All the work is being done on IA64 now.

--
-- Rob

[Dave Froble]

Rob, you don't mention at what level this will be required. Device
drivers? Any I/O buffer?

[Dave Froble]

Well, look at what you've done. Now I seem to know more than I wanted
to know.

Ok, sorry, a bit, for being a wise ass. It just comes so natural.

I don't do much with text, and I seem to do quite well with Instr and
Parse with XML, which I really do not like, when I need to.

As I may have mentioned before, I like things to be rather easy to
understand, and I'll take that over complex and powerful many times.

But I believe you were trying to be helpful, and for that, thank you.

[Tim Sneddon]

VAXman- wrote:
>

[...snip...]

> To answer Tim's assertion, I believe this is checked for in the module
> INDIRECT.

Thanks for the indepth response.

Regards, Tim.

[Tim Sneddon]

John E. Malmberg <wb8...@qsl.net_work> wrote:
> On 12/11/2018 7:47 AM, Tim Sneddon wrote:
>> Jairo Alves <jairo...@gmail.com> wrote:
>>>
> snip
>>
>> It is being treated as a command procedure. Years ago I investigated
>> this as I was trying to figure a way to provide a better implementation
>> of $(SHELL ...) for MMK. DCL actually checks to see if the command
>> procedure is disk based or not. If it is disk based, then certain
>> constructs become possible, like multi-line IF-THEN-ELSE-ENDIF, loops,
>> etc. If not, then you get single-line DCL.
>
> Is LD source available?
>
> Make a variant that could produce a small memory disk in the users
> virtual address space instead of a file.
>
> Or use LD on a temp file with good RMS caching.
>
> Have MMK (or other tools) detect if that tool is available and use it to
> run small scripts.

Thanks, John. That is worth investigating. I think the problem I was
having was that MMK feeds a sub-process the DCL lines (not just in
$(SHELL...)), but also build commands, one line at a time as they
are read out of the MMK definition.

I was looking for a way to "feed" the lines to DCL in a way that
would allow for complex commands (like is possible under GNU make
and other UN*X makes).

I suspect that the best way is simply to dump all the commands to
a file and then execute it. However, then the issue is how to
determine errors and which line caused it, etc.

It was a while ago that I was looking at this, things got busy
with real work and, well you know how it is...

Regards, Tim.

[Simon Clubley]

I should also point out that I was suggesting how DCL could be
enhanced to make things cleaner. I wasn't suggesting some existing
functionality (before someone attempts to use it.)

My first example wasn't completely clear on that. Sorry.

I still prefer my second example as a way of getting rid of the
leading $ characters if DCL were to be enhanced. I wouldn't
mind structured loops and iterators as well... :-)

[Robert A. Brooks]

On 12/12/2018 11:24 PM, Dave Froble wrote:
> On 12/12/2018 10:04 PM, Robert A. Brooks wrote:

>> New data structure -- I/O buffer descriptor (IOBD)
>>
>> It'll be a requirement for X86, and will also work for IA64.
>>
>> Alpha, maybe.
>>
>> All the work is being done on IA64 now.
>>
>
> Rob, you don't mention at what level this will be required.  Device drivers?
> Any I/O buffer?

This is a device driver-level change.

--
-- Rob

[Bob Koehler]

In article <f1991d58-f109-4e21...@googlegroups.com>, Jairo Alves <jairo...@gmail.com> writes:
>
>> DCL design and design compromises also skew heavily toward interactive
>> command input, and away from scripting and away from system and data
>> access features. Away from modern string handling, too; no regex, no
>
> Couldn't regex be added as a lexical function?

Couldn't we have a name that resembles English, instead?

[Bob Koehler]

In article <pur38l$ads$1...@dont-email.me>, =?UTF-8?Q?Jan-Erik_S=c3=b6derholm?= <jan-erik....@telia.com> writes:
>
> On VMS it doesn't have to be part of DCL in itself, there is regex
> support on tools like awk and perl, and in more modern scripting
> languages like Python. If you are in the need of regex processing,
> you are also doing some other more complex processing where a more
> capable tool (than DCL) can be handy.
>

I'm in favor of leaving DCL alone, and replacing it when VSI has
time. I think it's had enough hacks as it is.

[Bob Koehler]

In article <purare$ugm$1...@dont-email.me>, Stephen Hoffman <seao...@hoffmanlabs.invalid> writes:
>
> DCL can't fix a terminal driver limit.
>

No, but SMG doesn't have all the same recall and editing limits as
the terminal driver. There's no technical reason a real editor couldn't
be added to DCL, but I understand it would be hard. I think, however,
it should instead be a part of some replacement.

[VAXman-]

In article <putqh3$hnd$1...@dont-email.me>, "Robert A. Brooks" <FIRST...@vmssoftware.com> writes:
>On 12/12/2018 11:24 PM, Dave Froble wrote:

>> On 12/12/2018 10:04 PM, Robert A. Brooks wrote:
>
>>> New data structure -- I/O buffer descriptor (IOBD)
>>>
>>> It'll be a requirement for X86, and will also work for IA64.
>>>
>>> Alpha, maybe.
>>>
>>> All the work is being done on IA64 now.
>>>
>>
>> Rob, you don't mention at what level this will be required.  Device drivers?
>> Any I/O buffer?
>

>This is a device driver-level change.

I'd be more interested in how to overcome other memory handling where
the SVAPTE was traditionally required/used -- double-mapping, knitting
ones own memory, etc.

[Stephen Hoffman]

On 2018-12-13 04:24:16 +0000, Dave Froble said:

> On 12/12/2018 10:04 PM, Robert A. Brooks wrote:
>> New data structure -- I/O buffer descriptor (IOBD)
>>
>> It'll be a requirement for X86, and will also work for IA64.

>> ...

>
> Rob, you don't mention at what level this will be required. Device
> drivers? Any I/O buffer?

What I've gathered from various discussions so far...

◆Most (and likely all) inner-mode code that directly references the
memory page tables and the page table entries will have to change.
(This involves some device drivers, some execlets and some UWSS code.
Unsure about PFN-mapping user-mode code.)
◆By all accounts, the page table design is very different, and I'm
inferring a whole lot of what amounts to double-mapping of memory pages
is being implemented.
◆There's a memory page size change inbound, though the new page size
value is exactly half the size of an existing memory page size and
which means that most existing Alpha and I64 allocations and mappings
will continue work.
◆Last I heard, VSI was undecided around presenting mixed memory page
sizes and particularly the yet-larger page sizes.
◆There are IP-stack-related changes also inbound, around the migration
away from the HPE TCP/IP Stack. That'll (mostly) hit TCP/IP-specific
supporting procedures and related tools, and app testing, as the C and
$qio APIs are intentionally compatible.
◆The 64-bit I/O sector size and addressing promotions, and those will
(eventually) hit anything that knows about an LBN or a VBN or the
related fields in the RMS and XQP data structures; that's the second
part of VAFS support. These addressing are presently optional for app
developers, as existing apps will only need these changes when the app
encounters VAFS use on storage devices past 2 TiB.
◆Migrating to the newer compilers and particularly those with newer
standards support is presently optional, though some those migrations
will likely require source changes with at least some of the
programming languages.

More than a few other details and changes beyond these undoubtedly
await. Most user-mode code will be compile-and-go.

VSI is accruing details for their app migration documentation, more
presentations will be available at future boot camps, etc.

[Stephen Hoffman]

On 2018-12-13 01:07:11 +0000, Arne Vajh j said:

> On 12/12/2018 9:47 AM, John E. Malmberg wrote:
>> My vote would be to keep DCL the way it is, but to provide a good
>> interactive Python shell for scripting that needs to be 64+ bit aware,
>> with applications designed to handle serialized objects.
>
> I think many agrees with that.

As would I. DCL is a mess as it is currently implemented, and it's not
feasible to extend it without corresponding increases to the miasma of
compromise and compatibility.

>> You keep mentioning PowerShell.
>>
>> Have you done any significant coding in PowerShell?

And as I seem to have to repeat myself here, around my making
references to different approaches and different designs, and not to
specific solutions... If working with students as Jairo is—or working
with folks that might know only DCL, for that matter—exposure to
different environments is a very good way to learn about the values and
the limitations of the more familiar environments. Those examples
might well be bash, zsh, fish or one of the other available Unix shells
for Unix scripting, Python and IPython for scripting, and PowerShell is
an example of adding OO into the environment. Some folks use generated
scripts for automation and these can be written in various languages
including JavaScript. On Unix, psh one of the few shells that does OO,
and it's not all that common AFAIK. Why do I keep mentioning
PowerShell? because OO can nicely abstract and encapsulate data and
APIs. It's not so good for gonzo-scale data processing where scripts
written in SQL or NoSQL (RMS or otherwise) or possibly wrapped SQL via
something akin to FMDB would likely provide better performance.

Why do I keep mentioning PowerShell? Because—in various and very
central details—I don't want any hypothetical replacement DCL to be
anything close to DCL. If the hypothetical replacement is much closer
to PowerShell or to a newer bespoke design influenced by what's now
available elsewhere, all the better. Any home-grown replacement can
and should move forward ~40 years. If the replacement happens to be
interactive Python, well, that'll work. And it'll solve various of the
immediate limitations of DCL.

The OpenVMS platform and community has historically been insular. The
boxes have been in back rooms running quietly for decades, and other
stories. That insularity can cause developers working on OpenVMS to
miss out on useful tools and abstractions and APIs and ideas from those
other platforms. The effects of that insularity also applies to
marketing OpenVMS, too. Folks that might be considering using or folks
new to OpenVMS will have very different experiences from longstanding
OpenVMS users and developers, and can perceive OpenVMS as being
somewhere out past arcane, or as being outdated. Which is what DCL is,
pragmatically. OpenVMS marketing could learn a few things, too.

[Arne Vajhøj]

On 12/12/2018 11:32 PM, Dave Froble wrote:
> On 12/12/2018 9:16 PM, Arne Vajhøj wrote:
>> On 12/12/2018 8:58 PM, Arne Vajhøj wrote:
>>> On 12/12/2018 8:34 PM, Dave Froble wrote:
>>>> On 12/12/2018 7:49 PM, Arne Vajhøj wrote:
>>>>> Regex is a bit cryptic. But it can be useful.
>>>>>
>>>>> But I will say that all programmers should know SQL, regex and XPath.
>>>>
>>>> I know some SQL.  I don't even know what regex and xpath is.
>>>
>>> regex is a way to find patterns in text.
>>>
>>> XPath is a way to select parts of an XML document.
>>>
>>>>                                                              I guess
>>>> according to you I should stick to flying.  Oh well, much more fun
>>>> anyway.
>>>>
>>>> Now if it would just warm up ....
>>>
>>> Or maybe something to study during Christmas vacation.
>>>
>>> :-)
>>
>> Simple XPath example in VB.NET:

[code omitted]

> Well, look at what you've done.  Now I seem to know more than I wanted
> to know.
>
> Ok, sorry, a bit, for being a wise ass.  It just comes so natural.
>
> I don't do much with text, and I seem to do quite well with Instr and
> Parse with XML, which I really do not like, when I need to.
>
> As I may have mentioned before, I like things to be rather easy to
> understand, and I'll take that over complex and powerful many times.

For both regex and XPath I think it works a bit like:

have learned about them => super simple and easy to read

have not learned about them => is this ancient Egyptian put through an
AES-256 encryption?

:-)

> But I believe you were trying to be helpful, and for that, thank you.

Arne

[Jairo Alves]

> > Couldn't regex be added as a lexical function?
>
> Couldn't we have a name that resembles English, instead?

With time, sometimes we get used to some rather involved names.

I mean, wouldn't F$REGEX be just like, say, F$GETSYI or F$GETJPI?

:)

[johnso...@gmail.com]

I believe it to be fair to say that the rest of the development
world (outside of VMS that is) - would expect it to be called regex
or even just re. It's a pretty well established name.

So in that sense I'd say - you did get a name that resembles English.

EJ