VMS process communication

[Arne Vajhøj]

I did a little writeup:

https://www.vajhoej.dk/arne/articles/vmsipc.html

Content:

Introduction
Shared Memory
Concept
Demo
System Service (C, Fortran, Pascal)
Memory Mapped File
Concept
Demo
System Service (C, Fortran, Pascal)
Writeable Shareable Image
Concept
Demo
Linker/installer (Fortran, Pascal, C, Basic)
TCP/IP Socket
Concept
Demo
Java API (Java)
C API (C)
Python API (Python, Jython)
Wrapper C API (Fortran, Pascal, Cobol, Basic)
Message Queue
Concept
Demo
JMS API (Java, Jython)
STOMP C library (C)
STOMP Python library (Python)
Wrapper C STOMP library (Fortran, Pascal, Cobol, Basic)
Index Sequential File
Concept
Demo
Language builtin (Pascal, Cobol, Basic, Fortran)
RMS API (C)
VMS Python IndexedFile (Python)
JVM ISAM library (Java, Jython)
SQLite Database
Concept
Demo
C API (C)
Python API (Python, Jython)
JDBC API (Java)
JPA API (Java, Jython)
Wrapper C API (Pascal)

Arne

[John Forkosh]

Thanks so much. That's really terrific. And ditto for your other
articles https://www.vajhoej.dk/arne/articles.html which I hadn't
even been aware of. Maybe you might arrange/aggregate them into one
or more books along the lines of Stallman's just released C tutorial
and reference https://git.savannah.gnu.org/git/c-intro-and-ref.git
--
John Forkosh ( mailto: j...@f.com where j=john and f=forkosh )

[Dave Froble]

Why no mention of mailboxs ?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: da...@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

[Single Stage to Orbit]

On Mon, 2022-09-12 at 23:23 -0400, Dave Froble wrote:
> Why no mention of mailboxs ?

I did wonder about that, too.

--
Tactical Nuclear Kittens

[Marc Van Dyck]

Dave Froble formulated on Tuesday :

Was going to ask the same. Simple and efficient, covers many of the
needs. Enough for me to motivate the development of a wrapper allowing
me to use mailboxes from DCL.

--
Marc Van Dyck

[Bob Gezelter]

Arne,

My compliments. I know all too well how much effort it takes to produce such a compendium.

As has been observed, mailboxes are not included.

DECnet Task-to-Task would also make a useful addition. While DECnet does not include encryption, an omission that I have been quietly outraged by for decades, when tunneled over an encrypted IP tunnel OR used locally, it is quite useful. DECnet logical links are message-based, unlike stream-based TCP sockets, which does simplify programming to an extent.

Both network-based techniques have a significant synchronization advantage over shared memory techniques. When either partner of a TCP socket or a DECnet logical link exits, the corresponding partner is automatically notified, allowing the appropriate steps to be taken. Detecting a failed correspondent is far more complex with shared memory techniques. Network links are also atomic. A message is sent or not, there are no partial messages. Shared memory approaches have the inherent danger that an update may be incomplete, corrupting the shared data structure.

My normal recommendation has been to implement using message-passing techniques, unless one can conclusively demonstrate that the performance increment achieved by shared memory can be justified.

- Bob Gezelter, http://www.rlgsc.com

[Single Stage to Orbit]

On Tue, 2022-09-13 at 10:02 +0200, Marc Van Dyck wrote:
> > Why no mention of mailboxs ?
>
> Was going to ask the same. Simple and efficient, covers many of the
> needs. Enough for me to motivate the development of a wrapper
> allowing me to use mailboxes from DCL.

Back in the early 90s at Portsmouth Uni, we put together a MUD game
using only mailboxes as the chaps running the VAX systems limited us to
only the TMPMBX privileges and a ridiculously small 200 block quota
--
Tactical Nuclear Kittens

[Craig A. Berry]

On 9/13/22 2:39 AM, Single Stage to Orbit wrote:
> On Mon, 2022-09-12 at 23:23 -0400, Dave Froble wrote:
>> Why no mention of mailboxs ?
>
> I did wonder about that, too.

My first thought was why no ICC. And then I actually looked at the
page, which says:

Some ways of process communication are not covered (yet) including:

mailboxes
sys$icc_* system services

[Lee Gleason]

And event flags and locks?

--
Lee K. Gleason N5ZMR
Control-G Consultants
lee.g...@comcast.net

[Simon Clubley]

On 2022-09-13, Bob Gezelter <geze...@rlgsc.com> wrote:
>
> My compliments. I know all too well how much effort it takes to produce such a compendium.
>
> As has been observed, mailboxes are not included.
>
> DECnet Task-to-Task would also make a useful addition. While DECnet does
> not include encryption, an omission that I have been quietly outraged by
> for decades, when tunneled over an encrypted IP tunnel OR used locally, it
> is quite useful. DECnet logical links are message-based, unlike
> stream-based TCP sockets, which does simplify programming to an extent.

However, encryption only protects the data while it is in transit. It does
nothing to stop attacks against the protocol stack in the target server
itself if the client has access to the target server.

>
> Both network-based techniques have a significant synchronization
> advantage over shared memory techniques. When either partner of a TCP
> socket or a DECnet logical link exits, the corresponding partner is
> automatically notified, allowing the appropriate steps to be taken.
> Detecting a failed correspondent is far more complex with shared memory
> techniques. Network links are also atomic. A message is sent or not, there
> are no partial messages. Shared memory approaches have the inherent danger
> that an update may be incomplete, corrupting the shared data structure.
>

The no-partial-message guarantee is only true for stream protocols if you
implement something on top of the stream protocol that guarantees you have
received a full message.

In a stream protocol, just because the transmitter sent 4000 bytes, there's
no guarantee that all 4000 bytes were delivered to the receiver before
some connection interruption occurred.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

[Bob Gezelter]

Simon,

I mentioned message framing, although may phrasing may have been very soft. Of course, when using a stream connection, there is no mapping between QIO on the sender and the receive at the recipient. The only guarantee is that the bytes will be delivered in order. If the connection is severed, the results are "undefined." [If memory serves, Gene Amdahl was credited with that phrasing, which appeared in the IBM Systems/360 Principles of Operation manual.]

[Stephen Hoffman]

On 2022-09-13 09:29:19 +0000, Bob Gezelter said:

> As has been observed, mailboxes are not included.
>
> DECnet Task-to-Task would also make a useful addition. While DECnet
> does not include encryption, an omission that I have been quietly
> outraged by for decades, when tunneled over an encrypted IP tunnel OR
> used locally, it is quite useful. DECnet logical links are
> message-based, unlike stream-based TCP sockets, which does simplify
> programming to an extent.
>
> Both network-based techniques have a significant synchronization
> advantage over shared memory techniques. When either partner of a TCP
> socket or a DECnet logical link exits, the corresponding partner is
> automatically notified, allowing the appropriate steps to be taken.
> Detecting a failed correspondent is far more complex with shared memory
> techniques. Network links are also atomic. A message is sent or not,
> there are no partial messages. Shared memory approaches have the
> inherent danger that an update may be incomplete, corrupting the shared
> data structure.
>
> My normal recommendation has been to implement using message-passing
> techniques, unless one can conclusively demonstrate that the
> performance increment achieved by shared memory can be justified.

Network message-passing and shared-file communcations code is usually a
win, as it's common code for either local operations or for local and
remote operations, and it avoids all the "fun" with shared memory cache
coherency. If the performance of shared memory is necessary, it's best
treated as an add-on or an addition to network transfers.

OpenVMS lacks any sort of consistent communications API, with a whole
box of easy-to-make-mistakes some-assembly-required parts including
mailboxes, DECnet and IP, shared memory, DCE RPC and (IIRC) SunRPC, the
ever-popular TLS, intracluster communications services, IRPs and ACPs,
shared files, a couple of different message-passing add-on frameworks
including MQTT, etc.

Missing features here include IP and TLS integration with DCL and with
OpenVMS more generally, and preferably with frameworks intended to deal
with communications including IPv4 and IPv6, DNS, and with UDP, TCP,
QUIC, etc., support for REST, and the rest. Writing this code for
OpenVMS really reminds me of writing assembler. It's absolutely
possible to write this stuff for OpenVMS (including writing that code
in now-compiled VAX assembler), but other platforms abstract all this
for the developer. Which makes this stuff easier and faster to deploy,
and with fewer mistakes.

And preferably with connection security, data encryption, logging, and
authentication—that might not be necessary entirely locally, but I've
seen more than a few of these local YOLO pre-Y2K designs re-used for
remote connections, and where sadness ensued.

I've seen more than a few mistakes made with mailbox and DECnet and IP
communications, too. Mailboxes and DECnet and IP all have a
backpressure design which ~everybody gets wrong when they first
encounter it in production, too. Voice Of Experience: do NOT leave a
session in the debugger and head out to lunch.

And eightcubed has a wealth of source code examples.




--
Pure Personal Opinion | HoffmanLabs LLC

[Dave Froble]

While some things can be used for simple communications, I'd expect much more
from "communication". But yeah, a simple True/False can be considered
communication.

[Dave Froble]

That's what "handshakes" are for.

Hey, I want to send some data, 4000 bytes ...
Ok, ready to receive data ...
Send the data, wait for a response ...
Ok, received 4000 bytes ...

Anything else would be considered a failure.

In addition, checksums and such could be included to insure the 4000 bytes sent
is the same 4000 bytes received.

[Stephen Hoffman]

On 2022-09-13 20:59:40 +0000, Dave Froble said:

> That's what "handshakes" are for.
>
> Hey, I want to send some data, 4000 bytes ...
> Ok, ready to receive data ...
> Send the data, wait for a response ...
> Ok, received 4000 bytes ...
>
> Anything else would be considered a failure.
>
> In addition, checksums and such could be included to insure the 4000
> bytes sent is the same 4000 bytes received.

4K is ~one (modern) disk block, but yeah.

More common to send 4K with a checksum, or 4K encrypted with a
checksum, and let the receiver figure it out, though. An
application-level TCP-style handshake is closer to what is described is
certainly workable, but the chatter greatly slows things. I'd probably
look at the QUIC handshake, or at a message-passing or transaction
framework, if I needed the extra chatter. DECdtm, or Paxos protocol,
or MQTT, and tracking the transaction state, etc. But more generally
with the use of TCP or QUIC, you will get notification of either a
successful send, partial send, or of a failure to send.

It's up to the app or the transaction manager to determine where the
failure occurred, just as it's up to app-local error processing the
trap shadow on an Alpha to determine where the error occurred, or other
sorts of errors.

Put somewhat differently, it's usually best to design the connection
for speed and simplicity and for the common cause, and to (potentially)
make the error path and error recovery path somewhat more ugly.

PS: The TCP send operation has four potential outcomes: all bytes
transferred, a partial transfer with a need to loop and send more, a
local failure with no transfer, or a connection failure from the remote
side with no data transferred. Handling this usually involves a state
machine.

PPS: one of the more common errors here with TCP and such is assuming a
1-to-1 mapping of writes to reads. TCP and QUIC don't (necessarily)
work that way. One big write can become many smaller reads, for
instance.

PPPS: Depending on the platform, QUIC can have a specific API:
https://developer.apple.com/documentation/network/quic_options — with
that, QUIC can be switched to datagram, for instance. And AFAIK,
OpenVMS TCP/IP Services has UDP, TCP, and SCTP, but lacks QUIC.

PPPPS: having a framework that deals with this makes this whole area
far easier to deal with, and with fewer mistakes, and with fewer
problems with encryption or authentication.

[Robert A. Brooks]

On 9/13/2022 4:52 PM, Dave Froble wrote:
> On 9/13/2022 12:08 PM, Lee Gleason wrote:

>> And event flags and locks?

>

> While some things can be used for simple communications, I'd expect
> much more from "communication". But yeah, a simple True/False can
> be considered communication.

Lock value blocks can pass up to 16 bytes of data, or 64 bytes if using extended lock value blocks.

Shadowing driver is a heavy consumer of locks and lock value blocks.

-- Rob

[Arne Vajhøj]

They will hopefully both show up. But they did not make
it for version 1.0.

I do not have any experience wth sys$icc_* so that will
need a little research.

I have actually started on mailboxes, I encountered some
problems with what I want to do, so it got postponed.

Arne

[Arne Vajhøj]

On 9/13/2022 5:29 AM, Bob Gezelter wrote:
> On Monday, September 12, 2022 at 6:58:44 PM UTC-4, Arne Vajhøj wrote:
>> I did a little writeup:
>>
>> https://www.vajhoej.dk/arne/articles/vmsipc.html

> DECnet Task-to-Task would also make a useful addition.

Agree.

I had forgotten about that one.

It will go on the TODO list.

> My normal recommendation has been to implement using message-passing
> techniques, unless one can conclusively demonstrate that the
> performance increment achieved by shared memory can be justified.

I have always liked message queues.

:-)

Arne

[Arne Vajhøj]

On 9/13/2022 12:08 PM, Lee Gleason wrote:

> On 9/12/2022 10:23 PM, Dave Froble wrote:
>> On 9/12/2022 6:58 PM, Arne Vajhøj wrote:
>>> I did a little writeup:
>>>
>>> https://www.vajhoej.dk/arne/articles/vmsipc.html

>> Why no mention of mailboxs ?
>
>   And event flags and locks?

To me those are synchronization techniques not communication
techniques - or at least not general communication techniques.

Arne

[Dave Froble]

There are all types of communication requirements. Some simple, some not so
simple. For example:

When I need to tell a detached posting program to process an incoming order,
There are multiple things required in the message.

My utility for such communications passes messages almost 512 bytes in length.
Don't always need that much, but when I do, it's there.

[Dave Froble]

As Rob pointed out, one can do a lot with the DLM.

Isn't synchronization itself a form of communication?

[Dave Froble]

Well, yeah, but I learned communications over a serial line ...

[Dave Froble]

Ask your questions.

Do you want some examples?

[Arne Vajhøj]

On 9/13/2022 8:56 PM, Dave Froble wrote:
> On 9/13/2022 8:38 PM, Arne Vajhøj wrote:
>> On 9/13/2022 12:08 PM, Lee Gleason wrote:
>>> On 9/12/2022 10:23 PM, Dave Froble wrote:
>>>> On 9/12/2022 6:58 PM, Arne Vajhøj wrote:
>>>>> I did a little writeup:
>>>>>
>>>>> https://www.vajhoej.dk/arne/articles/vmsipc.html
>>
>>>> Why no mention of mailboxs ?
>>>
>>>    And event flags and locks?
>>
>> To me those are synchronization techniques not communication
>> techniques - or at least not general communication techniques.
>
> As Rob pointed out, one can do a lot with the DLM.
>
> Isn't synchronization itself a form of communication?

Yes.

But not what I would call general communication.

It is not intended to send arbitrary messages - it is
intended to synchronize flows.

Arne

[Stephen Hoffman]

On 2022-09-14 00:59:57 +0000, Dave Froble said:

> Well, yeah, but I learned communications over a serial line ...

I did, too.

With home-grown protocols.

With X/Y/ZMODEM.

With a whole lot of serial wiring hand-soldered together, or later with
pin-insertion, and checked with a breakout box.

With loading code from audio tape cassettes, and I'm not referring to DDS here.

And variously with busted UARTs, or with PIO activity that saturated
the processor, or other such.

With a side-trip through SCADA and PLCs, too many of the now-older PLCs
and controllers having completely demented serial protocols.

Times change.

The newer Ethernet pass-through connectors and crimping tools are
better and faster than non-passthrough crimping, and all vastly faster
and more reliable than drilling 10BASE5 taps.

We're all slinging a whole lot more data than we did back then, too.

And with vastly better tooling and apps and networking protocols available.

[Stephen Hoffman]

Doorbell locks are common—albeit when last I checked, the documentation
on the technical was, well, missing—and locks and lock value blocks are
how an app passes around state even in the event of cluster host
failures.

If you want a primary-secondary design or anything similar with a
counter or similar app-local context or related failovers, locks are
how that's done on OpenVMS.

[Richard Maher]

And I believe with Oracle's DLM the LVB can incorporate an entire
database page which forms the infrastructure needed for Cache Fusion.

The rest of us just have to BLAST the holder and ping it to disk unless
you're a single node with global buffers.

Anyway, I'd just like to say well done @Arne an not bitch about "but you
didn't do this". If Arne's put it in GitHub then knock yourselves out.

[Richard Maher]

Example: Trade Matching and Clearing system

Half-trades would come in as they happen.
Supplier process simply cannot be write to an ISAM file quick enough to
keep up.
If Consumer process reads past eof of matching sequential queue file it
will have to close the SEQ file, open it again and re-read the many
thousands of already processed trades to get back up to dat.

Solution: Lock Value Block

The ID of the last txn written is stored in the LVBLK.
When the Consumer obtains the lock and discovers the last txn available
is the same as the last it just processed it goes to sleep and waits for
a BLAST from the producer.

Summary: I call that interprocess communication

[Dave Froble]

From that perspective then more on your list aren't what I'd consider easy
communications. Global sections for example are some serious work. Nor would I
stoop to using files or databases for inter-process communications. Sockets and
mailboxes might be what I'd consider "low hanging fruit".

[Dave Froble]

Not arguing any of that. All I was implying was that both sides of a
communication must insure a successful communication, or, it never happened.

[Marc Van Dyck]

Arne Vajhøj formulated on Tuesday :

> I did a little writeup:
>
> https://www.vajhoej.dk/arne/articles/vmsipc.html
>

Should layered/third party products like RTR, DEC MessageQ or even
MQ Series (and its Opensource equivalents) be mentioned in such a work
too ? Honest open question, not saying that they are missing...

--
Marc Van Dyck

[Simon Clubley]

On 2022-09-13, Dave Froble <da...@tsoft-inc.com> wrote:
>
> Not arguing any of that. All I was implying was that both sides of a
> communication must insure a successful communication, or, it never happened.
>

Not that simple (for performance reasons).

Protocols such as TCP, ZMODEM, and Kermit (with sliding windows enabled)
are all asynchronous protocols where the delivery of data to the end user
program is decoupled from the sending of ACKs back to the transmitter.

The receiver end user program can easily have received a good amount of
data from the transmitter before the transmitter has received the ACKs
from the receiver to confirm receipt of that data.

You need something that guarantees the receiver end user program and
the sender end user program are in sync at all times ? You can do that
with a layer on top of the underlying protocol, but you will take a
serious performace hit for doing so.

[Dave Froble]

On 9/14/2022 1:34 PM, Simon Clubley wrote:
> On 2022-09-13, Dave Froble <da...@tsoft-inc.com> wrote:
>>
>> Not arguing any of that. All I was implying was that both sides of a
>> communication must insure a successful communication, or, it never happened.
>>
>
> Not that simple (for performance reasons).
>
> Protocols such as TCP, ZMODEM, and Kermit (with sliding windows enabled)
> are all asynchronous protocols where the delivery of data to the end user
> program is decoupled from the sending of ACKs back to the transmitter.
>
> The receiver end user program can easily have received a good amount of
> data from the transmitter before the transmitter has received the ACKs
> from the receiver to confirm receipt of that data.
>
> You need something that guarantees the receiver end user program and
> the sender end user program are in sync at all times ? You can do that
> with a layer on top of the underlying protocol, but you will take a
> serious performace hit for doing so.
>
> Simon.
>

What type of hit might one experience if the customer billing is say $1000 and
the last 2 zeros are chopped off the message?

Not that that is anything real. You cannot convince me that if the
communication was not verified, it is anything but "it never happened".

Accuracy is a great deal more important than performance. Actually, accuracy
isn't everything, it's the ONLY thing.

[Arne Vajhøj]

On 9/14/2022 5:13 AM, Marc Van Dyck wrote:
> Should layered/third party products like RTR, DEC MessageQ or even
> MQ Series (and its Opensource equivalents) be mentioned in such a work
> too ?

ActiveMQ is already there. If I were to add another message queue then
I would probably pick RabbitMQ as I think ActiveMQ/ArtemisMQ and
RabbitMQ are the two big ones today. But RabbitMQ code would be the
same as ActiveMQ code, because STOMP protocol and JMS API would
still be used. On the other side MQSeries/WebSphere MQ/IBM MQ
has a separate API besides the standard protocols/API so it
could be relevant. I just don't have access to the software.

I don't know much about RTR, but I got the impression that
it is way than a message queue - and I would probably
consider it out of scope.

Arne

[Craig A. Berry]

On 9/14/22 4:24 PM, Arne Vajhøj wrote:
> On 9/14/2022 5:13 AM, Marc Van Dyck wrote:
>> Should layered/third party products like RTR

> I don't know much about RTR, but I got the impression that
> it is way than a message queue - and I would probably
> consider it out of scope.

I don't know much either but I think you are right. It does distributed
transactions like TransactionScope in .NET will do when more than one
server is involved. It may use some IPC for some of what it does, but
it's not really IPC.

[Dave Froble]

On 9/14/2022 5:24 PM, Arne Vajhøj wrote:
> On 9/14/2022 5:13 AM, Marc Van Dyck wrote:
>> Should layered/third party products like RTR, DEC MessageQ or even
>> MQ Series (and its Opensource equivalents) be mentioned in such a work
>> too ?
>
> ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

[Arne Vajhøj]

On 9/14/2022 6:30 PM, Dave Froble wrote:
> On 9/14/2022 5:24 PM, Arne Vajhøj wrote:
>> On 9/14/2022 5:13 AM, Marc Van Dyck wrote:
>>> Should layered/third party products like RTR, DEC MessageQ or even
>>> MQ Series (and its Opensource equivalents) be mentioned in such a work
>>> too ?
>>
>> ActiveMQ is already there.
>
> Am I missing something?  I thought all those were third party products?

I believe RTR is VSI, but DEC MessageQ must be Oracle today, MQSeries
is IBM and ActiveMQ is open source (but VSI does make it available
https://vmssoftware.com/products/activemq/ for Itanium).

But if it is usable from VMS applications then I may
consider it in scope.

Arne

[Jan-Erik Söderholm]

Den 2022-09-15 kl. 00:30, skrev Dave Froble:
> On 9/14/2022 5:24 PM, Arne Vajhøj wrote:
>> On 9/14/2022 5:13 AM, Marc Van Dyck wrote:
>>> Should layered/third party products like RTR, DEC MessageQ or even
>>> MQ Series (and its Opensource equivalents) be mentioned in such a work
>>> too ?
>>
>> ActiveMQ is already there.
>
> Am I missing something?  I thought all those were third party products?

So what? If they enable IPC on VMS they are of interest, of course.

[Richard Maher]

Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to load
my VMS Tier3 ASCII files to GitHub

RTR is dead
DMQ is dead
MSMQ is dead
TIBCO is dead
Rendezvous is dead
ESB is dead


*** GraphQL is still born ***

To be brutally honest
VMS is dead

If someone can show me an easy way of getting ASCII fils CR/LF into a
GitHub repository I am happy to do it

[Fred. Zwarts]

Op 14.sep..2022 om 19:40 schreef Dave Froble:

> On 9/14/2022 1:34 PM, Simon Clubley wrote:
>> On 2022-09-13, Dave Froble <da...@tsoft-inc.com> wrote:
>>>
>>> Not arguing any of that.  All I was implying was that both sides of a
>>> communication must insure a successful communication, or, it never
>>> happened.
>>>
>>
>> Not that simple (for performance reasons).
>>
>> Protocols such as TCP, ZMODEM, and Kermit (with sliding windows enabled)
>> are all asynchronous protocols where the delivery of data to the end user
>> program is decoupled from the sending of ACKs back to the transmitter.
>>
>> The receiver end user program can easily have received a good amount of
>> data from the transmitter before the transmitter has received the ACKs
>> from the receiver to confirm receipt of that data.
>>
>> You need something that guarantees the receiver end user program and
>> the sender end user program are in sync at all times ? You can do that
>> with a layer on top of the underlying protocol, but you will take a
>> serious performace hit for doing so.
>>
>> Simon.
>>
>
> What type of hit might one experience if the customer billing is say
> $1000 and the last 2 zeros are chopped off the message?
>
> Not that that is anything real.  You cannot convince me that if the
> communication was not verified, it is anything but "it never happened".
>
> Accuracy is a great deal more important than performance.  Actually,
> accuracy isn't everything, it's the ONLY thing.
>

If accuracy is the only thing, it will be impossible to do a
transaction. In theory it will need an endless series of acknowledging
the acknowledgements. In practice we usually have enough trust in the
underlying communication channels that a one or two level
acknowledgement is sufficient.
There is a trade-of between the needed accuracy and the performance. Not
all communications need the same accuracy. A damaged frame when viewing
a video is a smaller problem than a missing 0 in a bank transfer. So,
the latter will use a more accurate communication protocol. But even
then we accept an error once in a million years, because otherwise
transactions are not possible at all.

[Dave Froble]

So Richard, if you don't use it, then it''s dead. Have I got that right?

What about the rest of us?

[Arne Vajhøj]

On 9/15/2022 2:58 AM, Richard Maher wrote:

> On 15/09/2022 2:44 pm, Jan-Erik Söderholm wrote:
>> Den 2022-09-15 kl. 00:30, skrev Dave Froble:
>>> On 9/14/2022 5:24 PM, Arne Vajhøj wrote:
>>>> On 9/14/2022 5:13 AM, Marc Van Dyck wrote:
>>>>> Should layered/third party products like RTR, DEC MessageQ or
>>>>> even MQ Series (and its Opensource equivalents) be mentioned in
>>>>> such a work too ?
>>>>
>>>> ActiveMQ is already there.
>>>
>>> Am I missing something?  I thought all those were third party
>>> products?
>>
>> So what? If they enable IPC on VMS they are of interest, of course.
>>
>
> Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to load
> my VMS Tier3 ASCII files to GitHub

Redis should be available:

https://vmssoftware.com/products/redis/

BTW, Redis would fit quite nicely into this article, so I will put that
on the TODO list.

> RTR is dead
> DMQ is dead
> MSMQ is dead
> TIBCO is dead
> Rendezvous is dead
> ESB is dead

> To be brutally honest
> VMS is dead

VMS is still alive. And hopefully will be so for many many
years to come.

RTR is also still alive. Not sure how much usage there is.

DEC MessageQ was bought by BEA I believe and I suspect the
product was effectively EOL'd when Oracle bought BEA.

MSMQ is still alive. Although a lot of Windows shops supposedly
use RabbitMQ today.

TIBCO rendezvous and other ESB products also still exist
even though their importance has declined.

Arne

[Stephen Hoffman]

On 2022-09-15 06:58:17 +0000, Richard Maher said:

> If someone can show me an easy way of getting ASCII fils CR/LF into a
> GitHub repository I am happy to do it

I prefer to convert source files and text files to Stream LF files as
part of kitting a release, and that particular issue goes alway.

Mostly.

Most OpenVMS apps will read Stream LF, but some apps do prefer not to
create that file format by default.

Which means reworking the OpenVMS environment to work around that.

With some of the available source code control tools, it's possible to
convert the sequential file format during check-in, as well as running
a pretty-printer on source files.

It'd be nice if the OpenVMS vGit git client flagged VFC and other
~incompatible file formats or reported and converted those files before
upload, but—not having verified this—I'd tend to assume it does not.

[Richard Maher]

David you should really (re)watch the Sixth Sense. You've been (brain)
dead for years :-)

[Richard Maher]

Ok. But what tool (that comes standard on the VMS side) do you use? FTP?

And on the Windows client? Filezilla?

[Stephen Hoffman]

To convert a VFC or other RMS sequential format into RMS sequential
format Stream LF:

$ CONVERT /FDL="RECORD; FORMAT STREAM_LF" from.txt to.txt

I've used a DCL loop with an f$file to detect and convert sequential
files not Stream LF, or similar code in the build or release or kitting
or check-in procedures to convert the file format.

Once converted, most OpenVMS tools will continue to work with the
converted file, and will preserve the Stream LF.

There's also ANALYZE /FDL and CREATE /FDL, depending on local
requirements, and fdl$create() for creating files in OpenVMS apps and
languages that don't do Stream LF well, or the FAB/RAB/XAB slog.

I haven't looked at whether any check-in hooks or format-detection
features are available in vGit, as mentioned earlier.

> FTP?

FTP? FTP gets used grudgingly around here. And is not a tool I'd choose
to use for RMS file format conversions.

> And on the Windows client? Filezilla?

Windows? Best ask somebody else about that.

[Arne Vajhøj]

On 9/13/2022 8:34 PM, Arne Vajhøj wrote:
> On 9/13/2022 8:52 AM, Craig A. Berry wrote:
>> On 9/13/22 2:39 AM, Single Stage to Orbit wrote:
>>> On Mon, 2022-09-12 at 23:23 -0400, Dave Froble wrote:

>>>> Why no mention of mailboxs ?
>>>

>>> I did wonder about that, too.
>>
>> My first thought was why no ICC.  And then I actually looked at the
>> page, which says:
>>
>> Some ways of process communication are not covered (yet) including:
>>
>>      mailboxes
>>      sys$icc_* system services
>
> They will hopefully both show up. But they did not make
> it for version 1.0.
>
> I do not have any experience wth sys$icc_* so that will
> need a little research.
>
> I have actually started on mailboxes, I encountered some
> problems with what I want to do, so it got postponed.

Updated with mailboxes and DECnet task-to-task.

Mailboxes

Concept
Demo
Standard IO (C, Fortran, Pascal, Cobol, Basic, Java, Python)

DECnet Task-to-Task

Concept
Demo
RMS (DCL, C, Fortran, Pascal, Cobol, Basic, Python)

SYS$ICC_* and Redis on the TODO list.

Arne

[Jan-Erik Söderholm]

Interesting! Now, what was that link now... :-)

[Single Stage to Orbit]

On Mon, 2022-09-19 at 11:28 +0200, Jan-Erik Söderholm wrote:
> Den 2022-09-19 kl. 01:58, skrev Arne Vajhøj:
>
>
> Interesting! Now, what was that link now... :-)

I bookmarked it some time ago:
https://www.vajhoej.dk/arne/articles/vmsipc.html
--
Tactical Nuclear Kittens

[Arne Vajhøj]

On 9/19/2022 2:21 PM, Single Stage to Orbit wrote:
> On Mon, 2022-09-19 at 11:28 +0200, Jan-Erik Söderholm wrote:
>> Den 2022-09-19 kl. 01:58, skrev Arne Vajhøj:
>>
>> Interesting! Now, what was that link now... :-)
>
> I bookmarked it some time ago:
> https://www.vajhoej.dk/arne/articles/vmsipc.html

Yep. Sorry for not include the link.

Arne

[Jan-Erik Söderholm]

No problem... :-)

In the part about mailboxes, I miss one language, DCL.

$ create/mailbox/perm abc


$ open/write abc
$ write abc "Hello".
$ close abc

$ open/read abc
$ read abc inline
$ write sys$output 'inline'

Very short example, and it might not work...

[Arne Vajhøj]

On 9/19/2022 6:06 PM, Jan-Erik Söderholm wrote:
> Den 2022-09-19 kl. 22:41, skrev Arne Vajhøj:
>> On 9/19/2022 2:21 PM, Single Stage to Orbit wrote:
>>> On Mon, 2022-09-19 at 11:28 +0200, Jan-Erik Söderholm wrote:
>>>> Den 2022-09-19 kl. 01:58, skrev Arne Vajhøj:
>>>>
>>>> Interesting! Now, what was that link now... :-)
>>>
>>> I bookmarked it some time ago:
>>> https://www.vajhoej.dk/arne/articles/vmsipc.html
>>
>> Yep. Sorry for not include the link.
>

> No problem... :-)
>
> In the part about mailboxes, I miss one language, DCL.
>
> $ create/mailbox/perm abc
>
>
> $ open/write abc
> $ write abc "Hello".
> $ close abc
>
> $ open/read abc
> $ read abc inline
> $ write sys$output 'inline'
>
> Very short example, and it might not work...

It should be possible to make it work.

Thinking about it - index-sequential files are
also accessible from DCL.

More to go on the TODO list.

Arne

[Stephen Hoffman]

On 2022-09-19 22:06:21 +0000, Jan-Erik S derholm said:

> In the part about mailboxes, I miss one language, DCL.

> ...


Here's DCL DECnet task-to-task:

https://www.digiater.nl/openvms/freeware/v80/hoffman_examples/xqtype.com

Wouldn't suggest using any of this for anything particularly relevent
or sensitive, and it's dicy even on a trusted network.

Unfortunately, DCL lacks TCP, TLS, and ssh connectivity support, and
lacks async / full duplex support.

[Arne Vajhøj]

Added:
* mailboxes / DCL
* socket / PHP
* index-sequential file / DCL
* SQLite / PHP

Arne

[Arne Vajhøj]

On 9/12/2022 10:36 PM, John Forkosh wrote:

> Arne Vajh??j <ar...@vajhoej.dk> wrote:
>> I did a little writeup:
>> https://www.vajhoej.dk/arne/articles/vmsipc.html
>> Content:

..
> Thanks so much. That's really terrific. And ditto for your other
> articles https://www.vajhoej.dk/arne/articles.html which I hadn't
> even been aware of.

Thank you.

> Maybe you might arrange/aggregate them into one
> or more books along the lines of Stallman's just released C tutorial
> and reference https://git.savannah.gnu.org/git/c-intro-and-ref.git

I have actually at occasions played with the idea.

But it would require some effort.

Maybe.

Arne

[Arne Vajhøj]

On 9/15/2022 4:32 PM, Arne Vajhøj wrote:
> On 9/15/2022 2:58 AM, Richard Maher wrote:
>> Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
>> load my VMS Tier3 ASCII files to GitHub
>
> Redis should be available:
>
> https://vmssoftware.com/products/redis/
>
> BTW, Redis would fit quite nicely into this article, so I will put that
> on the TODO list.

Done.

https://www.vajhoej.dk/arne/articles/vmsipc.html#redis

Redis Cache
Concept
Demo
Hiredis (C)
Official VMS wrapper hiredis (Fortran, Pascal, Cobol, Basic)
Custom VMS wrapper hiredis (Fortran, Pascal, Cobol, Basic)
Jedis (Java, Jython)
Python module (Python)

And just to be clear: this is Redis server running on VMS and client
applications running on VMS.

Arne

[Simon Clubley]

On 2022-09-28, Arne Vajhøj <ar...@vajhoej.dk> wrote:
> On 9/15/2022 4:32 PM, Arne Vajhøj wrote:
>> On 9/15/2022 2:58 AM, Richard Maher wrote:
>>> Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
>>> load my VMS Tier3 ASCII files to GitHub
>>
>> Redis should be available:
>>
>> https://vmssoftware.com/products/redis/
>>
>> BTW, Redis would fit quite nicely into this article, so I will put that
>> on the TODO list.
>
> Done.
>
> https://www.vajhoej.dk/arne/articles/vmsipc.html#redis
>

$ set response/mode=good_natured

This is just a guess Arne, but have you by any chance currently got
a lot of spare time on your hands ?

:-)

[Arne Vajhøj]

On 9/29/2022 8:57 AM, Simon Clubley wrote:
> On 2022-09-28, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>> On 9/15/2022 4:32 PM, Arne Vajhøj wrote:
>>> On 9/15/2022 2:58 AM, Richard Maher wrote:
>>>> Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
>>>> load my VMS Tier3 ASCII files to GitHub
>>>
>>> Redis should be available:
>>>
>>> https://vmssoftware.com/products/redis/
>>>
>>> BTW, Redis would fit quite nicely into this article, so I will put that
>>> on the TODO list.
>>
>> Done.
>>
>> https://www.vajhoej.dk/arne/articles/vmsipc.html#redis
>
> $ set response/mode=good_natured
>
> This is just a guess Arne, but have you by any chance currently got
> a lot of spare time on your hands ?
>
> :-)

Not really.

I started writing stuff in 2004.

Recently I have just written a lot about VMS.

And even though is is obviously some work, then I reuse
stuff - for Redis I had most of the API's covered from
when I wrote about cache servers in general back in 2020 - all
the tests back then was done on Windows but the C, Java and
Python API's are the same on on VMS, so I did not have to
start from scratch.

Arne

[Arne Vajhøj]

On 9/13/2022 8:34 PM, Arne Vajhøj wrote:
> I do not have any experience wth sys$icc_* so that will
> need a little research.

Done.

https://www.vajhoej.dk/arne/articles/vmsipc.html#icc

ICC
Concept
Demo
System Service (C, Fortran, Pascal)

Arne

[Ian Miller]

the ICC routines are intriguing. Good to see an example. Did you report the issue with Pascal to VSI?

[John Reagan]

I have not heard about any issues with the ICC definitions being Pascal "unfriendly".

If you ever want to override/bypass the type checking between the actual parameter and the formal parameter, you can use an explicit %IMMED, %REF, or %STDESCR as appropriate on the actual parameter. In that case, the compiler will skip the check and just pass what you specified. You can even provide more arguments than in the format definition by just using explicit argument mechanisms on the "extra" arguments.

[Arne Vajhøj]

On 10/5/2022 3:17 PM, John Reagan wrote:
> On Wednesday, October 5, 2022 at 11:28:16 AM UTC+2, Ian Miller wrote:
>> On Wednesday, October 5, 2022 at 12:11:32 AM UTC+1, Arne Vajhøj wrote:
>>> On 9/13/2022 8:34 PM, Arne Vajhøj wrote:
>>>> I do not have any experience wth sys$icc_* so that will
>>>> need a little research.
>>> Done.
>>>
>>> https://www.vajhoej.dk/arne/articles/vmsipc.html#icc
>>>
>>> ICC
>>> Concept
>>> Demo
>>> System Service (C, Fortran, Pascal)

>> the ICC routines are intriguing. Good to see an example. Did you report the issue with Pascal to VSI?
>
> I have not heard about any issues with the ICC definitions being Pascal "unfriendly".

I think only a small fraction of VMS sites use the ICC stuff and only a
small fraction of those use Pascal as language (the ICC stuff is new
enough to C being most common language).

But the fact that I could not get it working does not prove there
is a problem in SYS$LIBRARY:STARLET.PAS - maybe the problem is 40 cm
(16 inches) in front of the screen.

:-)

But some things puzzles me.

The client programs all use:
sys$icc_connectw
sys$icc_transmitw
sys$icc_receivew
sys$icc_disconnectw

All of them have an IOSB argument.

Per the documentation those are:
8 byte IOSB
8 byte IOSB
16 byte IOSB
8 byte IOSB

C has a:
struct _ios_icc

And defines the arguments for the 4 system services as as:
struct _ios_icc *ios_icc
struct _ios_icc *ios_icc
struct _ios_icc *ios_icc
struct _iosb *iosb

Which works fine. It is not obvious to me why the the last one is
different, but no problem.

Pascal has a record IOS_ICC$TYPE and it works fine for me, so that is
all good.

But the argument to the functions are defined as:
VAR IOS_ICC : [VOLATILE] $DEFPTR;
VAR IOS_ICC : [VOLATILE] $UQUAD;
VAR IOS_ICC : [VOLATILE] $DEFPTR;
VAR IOSB : [VOLATILE] $UQUAD;

With:
$DEFPTR = [UNSAFE] ^$DEFTYP;
$DEFTYP = [UNSAFE] INTEGER;
$UQUAD = [QUAD,UNSAFE] RECORD L0,L1:UNSIGNED; END;

Which seems inconsistent and hard/impossible to use for me
($DEFPTR really gave me a hard time).

So I simply used other declarations with arguments:
VAR IOS_ICC : [VOLATILE] IOS_ICC$TYPE;
VAR IOS_ICC : [VOLATILE] IOS_ICC$TYPE;
VAR IOS_ICC : [VOLATILE] IOS_ICC$TYPE;
VAR IOSB : [VOLATILE] IOS_ICC$TYPE;

It works fine and seems a lot more logical to me.

I also had problems with the buffers. $ICC_TRANSMITW
has a SEND_BUF and $ICC_RECEIVEW has a RECV_BUF but they
are declared as $DEFPTR and UNSIGNED. That gave my problems with
transmit so I changed SEND_BUF to also be UNSIGNED.

Arne

[Arne Vajhøj]

On 10/5/2022 7:06 PM, Arne Vajhøj wrote:
> I also had problems with the buffers. $ICC_TRANSMITW
> has a SEND_BUF and $ICC_RECEIVEW has a RECV_BUF but they
> are declared as $DEFPTR and UNSIGNED. That gave my problems with
> transmit so I changed SEND_BUF to also be UNSIGNED.

Let me elaborate on this.

The sys$libary:starlet.pas definition is:
SEND_BUF : $DEFPTR;

And I cannot get that to work.

I am probably missing something very basic, but ...

Demo:

$ type zb.c
#include <stdio.h>

void f(unsigned char *s)
{
printf("%02X %02X %02X %02X %02X %02X %02X %02X\n", s[0], s[1],
s[2], s[3], s[4], s[5], s[6], s[7]);
}
$ cc zb
$ type za.pas
program za(input,output);

type
$DEFTYP = [UNSAFE] INTEGER;
$DEFPTR = [UNSAFE] ^$DEFTYP;

[external('F')] procedure f1(%IMMED s : unsigned); external;
[external('F')] procedure f2(s : $DEFPTR); external;
[external('F')] procedure f3(%IMMED s : $DEFPTR); external;

var
s : varying [255] of char;

begin
s := 'ABCDEF' + chr(0);
f1(iaddress(s.body));
f2(s.body);
f2(iaddress(s.body));
f3(s.body);
f3(iaddress(s.body));
end.
$ pas za
$ link za + zb
$ run za
41 42 43 44 45 46 00 00
41 42 43 44 00 00 00 00
CA 79 E2 7A 00 00 00 00
00 00 00 00 00 00 00 00
41 42 43 44 45 46 00 00

Especially:
f2(s.body);
output puzzles me.

Arne

[Arne Vajhøj]

On 9/12/2022 6:58 PM, Arne Vajhøj wrote:
> I did a little writeup:
>
> https://www.vajhoej.dk/arne/articles/vmsipc.html

>     TCP/IP Socket
>         Concept
>         Demo
>         Java API (Java)
>         C API (C)
>         Python API (Python, Jython)
>         Wrapper C API (Fortran, Pascal, Cobol, Basic)

Just added:

HTTP
Concept
Demo
Embedded server (Java, Python)
Java API (Java)
urllib3 (Python)
Curl (C, PHP)
Direct socket (C)
Wrapper direct socket (Fortran, Pascal, Cobol, Basic)

Arne

[Simon Clubley]

On 2022-10-30, Arne Vajhøj <ar...@vajhoej.dk> wrote:
> On 9/12/2022 6:58 PM, Arne Vajhøj wrote:
>> I did a little writeup:
>>
>> https://www.vajhoej.dk/arne/articles/vmsipc.html
>

A suggestion: you should add a link back to the main entry point on your
website to the top of each of these pages. There's no immediate obvious
way to get to all of your website when you go in at a specific page such
as this one.

>>     TCP/IP Socket
>>         Concept
>>         Demo
>>         Java API (Java)
>>         C API (C)
>>         Python API (Python, Jython)
>>         Wrapper C API (Fortran, Pascal, Cobol, Basic)
>
> Just added:
>
> HTTP
> Concept
> Demo
> Embedded server (Java, Python)
> Java API (Java)
> urllib3 (Python)
> Curl (C, PHP)
> Direct socket (C)
> Wrapper direct socket (Fortran, Pascal, Cobol, Basic)
>

Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware of
the issues involved here ?

[Arne Vajhøj]

On 10/31/2022 9:18 AM, Simon Clubley wrote:
> On 2022-10-30, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>> On 9/12/2022 6:58 PM, Arne Vajhøj wrote:
>>> I did a little writeup:
>>>
>>> https://www.vajhoej.dk/arne/articles/vmsipc.html
>
> A suggestion: you should add a link back to the main entry point on your
> website to the top of each of these pages. There's no immediate obvious
> way to get to all of your website when you go in at a specific page such
> as this one.

There is a link at the bottom.

>>>     TCP/IP Socket
>>>         Concept
>>>         Demo
>>>         Java API (Java)
>>>         C API (C)
>>>         Python API (Python, Jython)
>>>         Wrapper C API (Fortran, Pascal, Cobol, Basic)
>>
>> Just added:
>>
>> HTTP
>> Concept
>> Demo
>> Embedded server (Java, Python)
>> Java API (Java)
>> urllib3 (Python)
>> Curl (C, PHP)
>> Direct socket (C)
>> Wrapper direct socket (Fortran, Pascal, Cobol, Basic)
>>
>
> Have you considered adding articles about 64-bit memory access programming
> on VMS and its limits ? Or is everyone already (painfully :-)) aware of
> the issues involved here ?

I have not considered that.

Long opinionated writing about what DEC decided 30 years ago,
why, consequences of that and, discussions about whether DEC
made the right decision and discussions about what VSI should do
now are out of scope for the series.

And from the more practical perspective I don't think there
is enough to write about. One page with background,
one page with all the stuff one can do in C and one
page about the fun of Java calling C via JNI. That
must be about it.

Arne

[Arne Vajhøj]

I decided to do it anyway.

And it turned out to be more than 3 pages to write.

Pre-release for comments:

https://www.vajhoej.dk/arne/articles/vms64.html

Arne

[Craig A. Berry]

On 10/31/2022 9:06 PM, Arne Vajhøj wrote:
> On 10/31/2022 9:18 AM, Simon Clubley wrote:
>> Have you considered adding articles about 64-bit memory access
>> programming
>> on VMS and its limits ? Or is everyone already (painfully :-)) aware

> I decided to do it anyway.
>
> And it turned out to be more than 3 pages to write.
>
> Pre-release for comments:
>
> https://www.vajhoej.dk/arne/articles/vms64.html

Looks good. Thanks for doing that. You may be interested in the
following tidbits that showed up in the release notes for the new
clang++ compiler:

1.3 Differences Between C++ on OpenVMS Itanium and OpenVMS x86

The datatypes 'long', 'size_t', 'nullptr_t', 'ptrdiff_t' are 64-bits
wide on OpenVMS x86 but only 32-bits wide on OpenVMS Itanium.

The default size for pointers is 64-bits on OpenVMS x86 but only 32-bits
on OpenVMS Itanium.

[Arne Vajhøj]

So /POINTER=64 is default. I should make a note about that.

Curious about long being 64 bit.

So:
Alpha & Itanium x86-64
short 16 16
int 32 32
long 32 64
long long 64 ?

?

I guess it is way overdue to always use intxx_t!

:-)

Arne

[Andreas Gruhl]

Thanks for your effort.

I like to add, that you can successfully access the lower 2 GB of P2
(%X00000000 80000000..%X00000000 FFFFFFFF) with 32 bit
pointers and descriptors.
You only have to treat the pointers as unsigned instead of signed
avoiding them to be interpreted as S0/S1 addresses.
We successfully use this in Pascal, which is missing 64-Bit descriptors.
The caller of a routine can then simply pass a structure in lower P2
without any special adaptions (except having to use the compiler
qualifier /USAGE=64BIT_TO_DESCR).
The routine itself has to do some additional work. It must extract the
structure's address from the descriptor and copy it to a 64-Bit Pointer
WITHOUT sign extension.
This makes the structure directly accessible although it resides in P2.

I wished treating descriptor pointers as unsigned would be the default
behaviour. The reason is simple: You will find extremely few application
programmers who ever needed to access an S0 or S1 structure via
descriptor - but many who would like to access P2 data.

Treating 32-bit pointers as unsigned is probably possible in any language
available under VMS.

Andreas

[JOUKJ]

I maintain a web-page on 64-bit pointers when using VSI-Fortran. The
site contains many problems you can encounter and what the work-arounds are.

Jouk

[JOUKJ]

JOUKJ wrote:
> I maintain a web-page on 64-bit pointers when using VSI-Fortran. The
> site contains many problems you can encounter and what the work-arounds
> are.

I forgot to copy the link:

http://nchrem.tnw.tudelft.nl/openvms/fortran64.html

[Jan-Erik Söderholm]

Den 2023-03-13 kl. 11:22, skrev JOUKJ:

> Arne Vajhøj wrote:
>>
>> Pre-release for comments:
>>
>> https://www.vajhoej.dk/arne/articles/vms64.html
>>
>> Arne

A bit pre-dated also... :-)


Article history:
Version Date Description
0.9 March 10th 20223 Pre-release

[Simon Clubley]

Typo:

"reduce memoy usage"

Generally, a well-written article.

One comment is to consider what the average person used to other operating
systems may ask and that is "why wasn't the operating system and the RTLs
simply recompiled to use 64-bit pointers instead of 32-bit pointers" ?

You have done some work at explaining this, but it doesn't state clearly
enough the real answer, which is that on VMS, pointers in the languages
and structures used to implement VMS are not hidden behind an abstracted
pointer type (ie: unsigned char *ptr), but are really just directly visible
integers (ie: uint32_t ptr).

[Simon Clubley]

On 2023-03-10, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>

> So /POINTER=64 is default. I should make a note about that.
>
> Curious about long being 64 bit.
>
> So:
> Alpha & Itanium x86-64
> short 16 16
> int 32 32
> long 32 64
> long long 64 ?
>
> ?
>
> I guess it is way overdue to always use intxx_t!
>
>:-)
>

Even better is to use uintXX_t unless you really want a signed integer. :-)

A big mistake languages made (IMHO) was to make their integers signed
by default, instead of you needing to ask for a signed integer if you
really needed one.

[John Dallman]

In article <tugdbo$25f5e$1...@dont-email.me>, craig...@nospam.mac.com

(Craig A. Berry) wrote:

> Looks good. Thanks for doing that. You may be interested in the
> following tidbits that showed up in the release notes for the new
> clang++ compiler:
>
> 1.3 Differences Between C++ on OpenVMS Itanium and OpenVMS x86
>
> The datatypes 'long', 'size_t', 'nullptr_t', 'ptrdiff_t' are 64-bits
> wide on OpenVMS x86 but only 32-bits wide on OpenVMS Itanium.

Have you got a link for that?

John

[Arne Vajhøj]

On 3/13/2023 9:32 AM, Simon Clubley wrote:
> On 2023-03-10, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>>
>> So /POINTER=64 is default. I should make a note about that.
>>
>> Curious about long being 64 bit.
>>
>> So:
>> Alpha & Itanium x86-64
>> short 16 16
>> int 32 32
>> long 32 64
>> long long 64 ?
>>
>> ?
>>
>> I guess it is way overdue to always use intxx_t!
>

> Even better is to use uintXX_t unless you really want a signed integer. :-)
>
> A big mistake languages made (IMHO) was to make their integers signed
> by default, instead of you needing to ask for a signed integer if you
> really needed one.

That is not really C specific.

Fortran INTEGER is signed. Pascal INTEGER is signed (even though Pascal
got CARDINAL for unsigned).

Java does not have unsigned integers.

Arne

[Simon Clubley]

On 2023-03-13, Arne Vajhøj <ar...@vajhoej.dk> wrote:
> On 3/13/2023 9:32 AM, Simon Clubley wrote:
>>
>> Even better is to use uintXX_t unless you really want a signed integer. :-)
>>
>> A big mistake languages made (IMHO) was to make their integers signed
>> by default, instead of you needing to ask for a signed integer if you
>> really needed one.
>
> That is not really C specific.
>

Notice I didn't say C, but "languages". :-)

I consider it a general design flaw in languages, probably based on the
need to mirror underlying early architecture and then it just became
established as the default.

> Fortran INTEGER is signed. Pascal INTEGER is signed (even though Pascal
> got CARDINAL for unsigned).
>
> Java does not have unsigned integers.
>

[Arne Vajhøj]

On 3/13/2023 10:01 AM, Simon Clubley wrote:
> On 2023-03-13, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>> On 3/13/2023 9:32 AM, Simon Clubley wrote:
>>>
>>> Even better is to use uintXX_t unless you really want a signed integer. :-)
>>>
>>> A big mistake languages made (IMHO) was to make their integers signed
>>> by default, instead of you needing to ask for a signed integer if you
>>> really needed one.
>>
>> That is not really C specific.
>
> Notice I didn't say C, but "languages". :-)
>
> I consider it a general design flaw in languages, probably based on the
> need to mirror underlying early architecture and then it just became
> established as the default.

I consider it rather natural to have integers signed by default.

Except byte that I like unsigned.

I really like C# where short, int and long are signed while
byte is unsigned - and one can pick ushort, uint, ulong and
sbyte when one want the other way.

Arne

[Arne Vajhøj]

On 3/13/2023 5:56 AM, Andreas Gruhl wrote:
> Arne Vajhøj schrieb am Freitag, 10. März 2023 um 22:01:40 UTC+1:
>> Pre-release for comments:
>>
>> https://www.vajhoej.dk/arne/articles/vms64.html

> I like to add, that you can successfully access the lower 2 GB of P2
> (%X00000000 80000000..%X00000000 FFFFFFFF) with 32 bit
> pointers and descriptors.
> You only have to treat the pointers as unsigned instead of signed
> avoiding them to be interpreted as S0/S1 addresses.
> We successfully use this in Pascal, which is missing 64-Bit descriptors.
> The caller of a routine can then simply pass a structure in lower P2
> without any special adaptions (except having to use the compiler
> qualifier /USAGE=64BIT_TO_DESCR).
> The routine itself has to do some additional work. It must extract the
> structure's address from the descriptor and copy it to a 64-Bit Pointer
> WITHOUT sign extension.
> This makes the structure directly accessible although it resides in P2.

Is /USAGE=64BIT_TO_DESCR new for x86-64?

Arne

[Craig A. Berry]

As far as I know it's only available in the service portal for people
with a support contract. I assume it will become part of the standard,
publicly-available docs once that compiler is out of field test.

[John Dallman]

In article <tunp6s$3sc9i$1...@dont-email.me>, craig...@nospam.mac.com

Makes sense, thanks. It makes porting C/C++ applications to VMS a good
deal simpler.

John

[Arne Vajhøj]

I believe it always have been the message that:
* Fortran, Pascal, Cobol, Basic and C will get all the DCL command
qualifiers, VMS specific language extensions and VMS specific CRTL
functions needed to compile old existing code
* C++ would be clang ported to VMS close to "as is"

When such a high level idea meet reality, then things can change
a little.

But it makes sense. There are probably not that many old
VMS specific C++ applications out there. For the simple
reason that when C++ for VMS showed up and became
reasonable stable, then VMS was no longer a common
platform choice for new applications.

But there are a lot of stuff developed on *nix and Windows
that need a C++ compiler with support for newer C++ standards
to be able to build on VMS. Including latest versions of
MySQL/MariaDB and OpenJDK.

Arne

[Dave Froble]

On 3/13/2023 10:19 AM, Arne Vajhøj wrote:
> On 3/13/2023 10:01 AM, Simon Clubley wrote:
>> On 2023-03-13, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>>> On 3/13/2023 9:32 AM, Simon Clubley wrote:
>>>>
>>>> Even better is to use uintXX_t unless you really want a signed integer. :-)
>>>>
>>>> A big mistake languages made (IMHO) was to make their integers signed
>>>> by default, instead of you needing to ask for a signed integer if you
>>>> really needed one.
>>>
>>> That is not really C specific.
>>
>> Notice I didn't say C, but "languages". :-)
>>
>> I consider it a general design flaw in languages, probably based on the
>> need to mirror underlying early architecture and then it just became
>> established as the default.
>
> I consider it rather natural to have integers signed by default.

The way most/all people consider math, numbers can be either positive or
negative. Yes, "natural". Computer languages were developed using this
concept. While unsigned numbers are useful, they aren't something people
normally consider. Not sure where Simon is coming from with his assertions.

> Except byte that I like unsigned.
>
> I really like C# where short, int and long are signed while
> byte is unsigned - and one can pick ushort, uint, ulong and
> sbyte when one want the other way.

The problem with all of those is they are vague, and are defined differently on
different HW and software. Precision is to be desired.


--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: da...@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

[Dave Froble]

Ah, the extreme power and capability of a keystroke, even time travel.

:-)

[Arne Vajhøj]

In C the types short, int and long are vaguely defined.

Not in C# and Java - there they are 16, 32 and 64 bit two's complement.
Only aspect left optional is big vs little endian.

Arne

[John Reagan]

That /USAGE qualifier is a Pascal qualifier. It asks the Pascal compiler to violate
the Calling Standard and not sign-extend a 32-bit pointer to duplicate what you can
get (by accident) on Itanium.

It wouldn't be a C or C++ qualifier since neither of those compilers do anything with
descriptors behind your back.

[John Reagan]

On Monday, March 13, 2023 at 3:08:34 PM UTC-4, Arne Vajhøj wrote:
> On 3/13/2023 2:53 PM, John Dallman wrote:
> > In article <tunp6s$3sc9i$1...@dont-email.me>, craig...@nospam.mac.com
> > (Craig A. Berry) wrote:
> >> On 3/13/23 8:40 AM, John Dallman wrote:
> >>> Have you got a link for that?
> >>
> >> As far as I know it's only available in the service portal for
> >> people with a support contract. I assume it will become part of the
> >> standard, publicly-available docs once that compiler is out of field
> >> test.
> >
> > Makes sense, thanks. It makes porting C/C++ applications to VMS a good
> > deal simpler.
> I believe it always have been the message that:
> * Fortran, Pascal, Cobol, Basic and C will get all the DCL command
> qualifiers, VMS specific language extensions and VMS specific CRTL
> functions needed to compile old existing code
> * C++ would be clang ported to VMS close to "as is"
>

There has been a constant drumbeat from customers (including many here
on this forum) about our odd choice of long=32-bit; ptrdiff_t=32-bit;
size_t=32-bit all while you can say /POINTER=64. For C++/clang, we just
get what clang gives you for a 64-bit compiler.

We have yet to provide a similar setting for the C frontend as so far it hasn't been
necessary.

We have modified/enhanced some headers to replace "long" with "int" to make sure
that certain fields that are 32-bits are still 32-bits with both x86 C and x86 C++.

My desire is to limit the # of clang OpenVMS-isms to those actually needed and to
skip the ones that have been just dragged forward from VAX for no real reason other
than just the fear of pissing off a customer. I'm in the "if you need it, ask and we'll
talk about it" camp.

[Arne Vajhøj]

On 3/13/2023 6:08 PM, John Reagan wrote:
> That /USAGE qualifier is a Pascal qualifier. It asks the Pascal compiler to violate
> the Calling Standard and not sign-extend a 32-bit pointer to duplicate what you can
> get (by accident) on Itanium.

Is it x86-64 only?

The value is not accepted with Pascal 6.2 on neither Alpha nor Itanium.

Arne

[John Reagan]

There hasn't been a Alpha or Itanium release with it. Since neither Alpha or Itanium
put code into P2 space that first chunk of memory is where a 64-bit NEW would
get memory from. With code in P2 space on x86, that shoves the 64-bit heap
up more and you need to get help from the compiler. It has never been a good thing
to pass around P2 addresses in the 32-bit field of descriptors.

[John Dallman]

In article <8a6020e4-d49c-4f35...@googlegroups.com>,

xyzz...@gmail.com (John Reagan) wrote:

> For C++/clang, we just get what clang gives you for a 64-bit compiler.
>
> We have yet to provide a similar setting for the C frontend as so
> far it hasn't been necessary.

Makes sense. If I ever get to port my employers' product to x86-64
OpenVMS, I'll need the Clang C setup to match the Clang C++ setup.
However, there hasn't been any interest in VMS from customers since the
late 1990s, so I fear the odds are against this happening.

John

[Simon Clubley]

On 2023-03-13, Dave Froble <da...@tsoft-inc.com> wrote:
>
> The way most/all people consider math, numbers can be either positive or
> negative. Yes, "natural". Computer languages were developed using this
> concept. While unsigned numbers are useful, they aren't something people
> normally consider. Not sure where Simon is coming from with his assertions.
>

Much of the time you don't need signed integers and the problem you are
trying to solve is a better match with unsigned integers.

Let me put it this way: How many filesystems and file sizes and other
limits through computing history are half the maximum size they should
be because everyone used signed integers instead of unsigned integers ?

[Johnny Billquist]

On 2023-03-13 15:01, Simon Clubley wrote:
> On 2023-03-13, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>> On 3/13/2023 9:32 AM, Simon Clubley wrote:
>>>
>>> Even better is to use uintXX_t unless you really want a signed integer. :-)
>>>
>>> A big mistake languages made (IMHO) was to make their integers signed
>>> by default, instead of you needing to ask for a signed integer if you
>>> really needed one.
>>
>> That is not really C specific.
>>
>
> Notice I didn't say C, but "languages". :-)
>
> I consider it a general design flaw in languages, probably based on the
> need to mirror underlying early architecture and then it just became
> established as the default.

I don't think there is anything in the underlying architecture that
leads one way or another on this topic (no matter which architecture).

I think it's much more likely that the chose to have them signed as most
people when they work with integers expect them to be able to represent
both positive and negative numbers.

Johnny

[Dave Froble]

On 3/14/2023 9:08 AM, Simon Clubley wrote:
> On 2023-03-13, Dave Froble <da...@tsoft-inc.com> wrote:
>>
>> The way most/all people consider math, numbers can be either positive or
>> negative. Yes, "natural". Computer languages were developed using this
>> concept. While unsigned numbers are useful, they aren't something people
>> normally consider. Not sure where Simon is coming from with his assertions.
>>
>
> Much of the time you don't need signed integers and the problem you are
> trying to solve is a better match with unsigned integers.
>
> Let me put it this way: How many filesystems and file sizes and other
> limits through computing history are half the maximum size they should
> be because everyone used signed integers instead of unsigned integers ?
>
> Simon.
>

Understand, I think unsigned numbers would be very useful. I wish Basic support
such. But my point is, they are not so "natural" to humans, and after all,
humans are the users of computers. So development followed what people do.

[Arne Vajhøj]

On 3/14/2023 1:37 PM, Dave Froble wrote:
> On 3/14/2023 9:08 AM, Simon Clubley wrote:
>> On 2023-03-13, Dave Froble <da...@tsoft-inc.com> wrote:
>>> The way most/all people consider math, numbers can be either positive or
>>> negative.  Yes, "natural".  Computer languages were developed using this
>>> concept.  While unsigned numbers are useful, they aren't something
>>> people
>>> normally consider.
>>

>> Much of the time you don't need signed integers and the problem you are
>> trying to solve is a better match with unsigned integers.
>>
>> Let me put it this way: How many filesystems and file sizes and other
>> limits through computing history are half the maximum size they should
>> be because everyone used signed integers instead of unsigned integers ?

When it involves data size and the demand grow x2 every 2 years or so,
then that extra bit will not save they day for long.

> Understand, I think unsigned numbers would be very useful.  I wish Basic
> support such.

I miss them in Java as well.

>   But my point is, they are not so "natural" to humans, and
> after all, humans are the users of computers.  So development followed
> what people do.

If you stop 100 random people on the street and ask about integers, then
99 will include negative numbers, that is the common definition in
math/english/danish/whatever.

Those that remember their math will remember the term natural
numbers to exclude negative numbers.

Arne

[Arne Vajhøj]

Note that it is really highly unlikely that a given
problem domain actually need types:

-2147483648..2147483647
0..4294967295

It is way more likely to have a need for types like:

1..100
0..99
-10..10
etc.

But that is not in fashion today (Pascal, Modula-2,
Oberon, Ada etc. are rare today).

Arne

[Simon Clubley]

On 2023-03-14, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>
> It is way more likely to have a need for types like:
>
> 1..100
> 0..99
> -10..10
> etc.
>
> But that is not in fashion today (Pascal, Modula-2,
> Oberon, Ada etc. are rare today).
>

Unfortunately. :-(

Are there any languages other than the Wirth-inspired ones which do
have them as part of the core language ?

BTW, given what the Rust people claim about Rust's target markets,
I consider that to be a surprising omission from Rust, given that they
had all these languages to look at when they were designing Rust.

Such types are used to help model the underlying problem more accurately,
which is exactly the kind of thing the Rust people (and any other such
language designers) should have latched onto.

[Arne Vajhøj]

On 3/14/2023 2:31 PM, Simon Clubley wrote:
> On 2023-03-14, Arne Vajhøj <ar...@vajhoej.dk> wrote:
>>
>> It is way more likely to have a need for types like:
>>
>> 1..100
>> 0..99
>> -10..10
>> etc.
>>
>> But that is not in fashion today (Pascal, Modula-2,
>> Oberon, Ada etc. are rare today).
>>
>
> Unfortunately. :-(
>
> Are there any languages other than the Wirth-inspired ones which do
> have them as part of the core language ?

Not that I am aware of.

> BTW, given what the Rust people claim about Rust's target markets,
> I consider that to be a surprising omission from Rust, given that they
> had all these languages to look at when they were designing Rust.

A large chunk of Rust target market is the low level
near to the hardware market and there it may actually
be the requirement that there is N bits to fill out.

But in the much bigger business related market then there is
rarely a bit requirements - most likely those defining the
requirements do not know what a bit is.

Arne

[bill]

On 3/14/2023 2:05 PM, Arne Vajhøj wrote:
>
>

> If you stop 100 random people on the street and ask about integers, then
> 99 will include negative numbers,

If you stop 100 random people on the street and ask about integers, then

99 won't have a clue what you are talking about.

bill

[Arne Vajhøj]

Those that remember their math should think of Z.

The rest should vaguely remember that "numbers without decimals" are
called integers.

Arne