New to IBM's mainframe world -- *.tap files, SL tapes, PDS data sets

115 views
Skip to first unread message

Peter Lund

unread,
Dec 18, 2020, 6:14:28 AM12/18/20
to z390
I have never seen, touched, or used any machine in the S/360 family.  They are like strange creatures from an alien world to me.

I recently started writing an S/360 emulator (+ disassembler) because I want to understand the S/360 and I learn much faster when unavoidably confronted with all the little details that I don't understand.

My first problem is how to get hold of S/360 binaries so I have some code to disassemble/execute.

I found http://www.bitsavers.org/bits/IBM/360/IBM_BOS_16K_Apr1966.zip , which seems to be not quite the bare minimum possible operating system on the architecture but damn close.  The problem is that the zip file contains *.tap files that I don't know how to read.  They don't seem to be raw binaries (the disassembler complained about lots and lots of undefined instructions).

Does anybody know what format they are?  Possibly in the form of some keywords I can throw at google -- or a website or PDF somewhere that describes the format?

I have also downloaded http://www.ibiblio.org/jmaynard/k360s-ma20x-aws.zip which contains OS/360 binaries.  They are in .aws files, which I have been reading up on this morning.  They seem easy enough to work with, but the tapedump.c file linked to on the Hercules webpage is downright awful.  I will probably write my own analyzer/extracter for those files.

And then there is the rabbit hole of "standard labeled" tapes and that of "partitioned data sets".  As far as I can tell, I need to look at the header/trailer label "files" around the actual data files in AWS files to get names, timestamps, file formats -- and then figure out how PDS files are stored so I can extract members.  At least that seems to be necessary for some of the executables.

Does anybody know of a good and thorough specification of PDS (and PDS/E) files, sorry, "data sets"?  And a similar good and thorough specification of the VOL1/HDR1/HDR2/EOF1/EOF2/EOV1/EOV2/etc labels for tapes?

What little I have found on IBM's documentation site seems very fragmented and I don't yet have all the IBMese vocabulary I need to google properly.

-Peter

PS: How long does it take to get used to IBM's incorrect bit numbering?

christopher Lang

unread,
Dec 18, 2020, 9:24:10 AM12/18/20
to Peter Lund, z390

Hi Peter,

 

Perhaps this might be useful from the hercules faq? There is a lot of missing history and discussion about these distributions (since the yahoo groups are gone now). Perhaps someone archived the old groups?

 

I think below are a couple of options for a running system.

 

2.04 Where can I obtain OS/360 ?

1. Rick Fochtman's OS/360 archive CD is now obtainable by download from these locations:

2. If you are interested in obtaining a copy of the CD itself, contact Rick Fochtman at rfochtman@ync.net.

3. Alternatively, you can download the OS/360 Y2K Starter package from http://open360.copyleft.de/OS360/Download.html which contains a full featured MVT system on a 3330 image, with some minimal documentation. The configuration is that of a 370/158 with 4 megabytes of main storage, running OS/MVT Release 21.0. The same site also offers a MVTDBL volume and a Builder package for those who like to participate in OS/360 nucleus hacking.

4. Jay Maynard's "IBM Public Domain Software Collection" at http://www.ibiblio.org/jmaynard/ contains copies of the OS/360 Release 21.8 distribution tapes.

 

I’ve also attached the tape FAQ from Hercules. It might be useful in answering some of your questions about tape formats and multi-volume tapes.

 

If I recall correctly there are also some standalone windows utilities for reading tape media (awsbrowse?). Fish from the Hercules project might also have written some tools for tape.

 

Cheers

 

Christopher

--
You received this message because you are subscribed to the Google Groups "z390" group.
To unsubscribe from this group and stop receiving emails from it, send an email to z390+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/z390/685fc5d6-cd42-4268-8435-3039d2c1a3een%40googlegroups.com.

README.TAPE

Peter Lund

unread,
Dec 18, 2020, 9:42:14 AM12/18/20
to christopher Lang, z390
Hi, you are the fourth person to reply -- and the first to also reply to the group!  One of them has already mailed back and said it was meant to be a group response.  Maybe the other two also got fooled by the user interface?


On Fri, Dec 18, 2020 at 3:24 PM christopher Lang <christop...@gmail.com> wrote:

4. Jay Maynard's "IBM Public Domain Software Collection" at http://www.ibiblio.org/jmaynard/ contains copies of the OS/360 Release 21.8 distribution tapes.


That is precisely where the  http://www.ibiblio.org/jmaynard/k360s-ma20x-aws.zip link is from.  I have the zip file, I have the AWS files from inside it.  I can't (yet) read the AWS files or the PDS'es it probably contains.

I’ve also attached the tape FAQ from Hercules. It might be useful in answering some of your questions about tape formats and multi-volume tapes.


Maybe. Probably not. I know today what "VOL1-HDR1/EOV/EOF based" means (which I didn't yesterday) so that's at least something.  I don't intend to mount AWS files under Hercules.  I want to extract files, sorry, "data sets" from them so I can test my disassembler on them and so I can start writing the emulator in earnest.
 

If I recall correctly there are also some standalone windows utilities for reading tape media (awsbrowse?). Fish from the Hercules project might also have written some tools for tape.


I do intend to install Hercules (and z390 and sim360c) if for nothing else to get AWS browse.  My previous experience with installing Hercules hasn't exactly been positive so it hasn't been the first thing on my agenda.

I have already experimented with tapedump (from the link at the bottom of the page, under the 2008 January heading):
http://www.jaymoseley.com/hercules/awstape_utilities/mawstape.htm

It didn't really do anything useful and the code isn't particularly readable.  That's why I was asking for some pointers so I could write a file extractor myself.

-Peter

christopher Lang

unread,
Dec 18, 2020, 9:50:10 AM12/18/20
to Peter Lund, z390

Hi Peter

 

Here is a recipe that might be useful

 

http://www.conmicro.com/hercos360/

 

Hercules should be quite easy to get working. What problems did you encounter? Also I believe there were some windows tools for mainframe tape. I will try to look around one of my harddrives.

 

Cheers

 

Christopher

christopher Lang

unread,
Dec 18, 2020, 10:07:33 AM12/18/20
to Peter Lund, z390

Hi Peter,

 

The zip was blocked (perhaps just to the group?). Look online for

 

AWSBrowse-1.5.1.1805-bin.zip

 

If you don’t want to build your own system perhaps you can find some OS360 binaries on the CBT site. As mentioned I have quite a bit of material for Hercules and various distributions. I can see if I have anything else useful.

 

Cheers

 

Christopher

George Lausman

unread,
Dec 18, 2020, 1:15:41 PM12/18/20
to Peter Lund, z390

Hi Peter,

S/360 is a family of machines introduced in the 60s’. It evolved a lot since then. You may feel like the five blind men touching the different parts of an elephant and trying to build a mental picture of the beast.

Learning by doing is what works for me too. Your thought to start writing an emulator is a bit too ambitious for your first S/360 project. So I will concentrate on your desire to write a disassembler. Here are the pieces you would need for your disassembler: (1) S/360 hardware or emulator, (2) an operating system to manage the hardware, (3) development tools (editor, assembler, linkage-editor, perhaps utilities to copy files, etc.). All this is contained in Hercules. My suggestion is to get all the pieces and install them on your home machine (Linux or Windows).  

I know, you would like to do something else now, but bear with me, it is tried and true method. Once you overcome all the disappointments and failures of your installation you will have a “playground” of your own. Now you can edit, assemble, link-edit and run your first code all on S/360. If you stay on S/360, you will learn a lot.

If you succumb to doing parts of your development on Windows or Linux, you will have to learn all the ugly details of ASCII vs EBCDIC encoding, conversion programs and their quirks. All these screw-ups translate to one thing – extra knowledge. 

The one thing you may not be prepared for is the S/360 I/O “richness”. Windows and Linux have just one I/O concept “a byte stream” unbroken for binaries or broken into lines for text files. The files are written to fixed blocks on a pre-formatted disk The S/360 uses “CKD” disk organization (Count – Key – Data) for each physical block. COUNT is the sequential number of physical blocks from the start of the track, KEY is the highest key in the following Data segment (used for direct access), DATA is the user data and it can be any length (later you will learn that max length is about 32K).

To create a dataset (file) one has to ALLOCATE it. This is done by JCL (Job Control Language). The process of “allocate” (a) reserves a space on the disk (we call it a VOLUME), (b) assigns a lot of attributes to the new dataset. Currently a JCL manual has about 300 pages devoted to explaining of individual attribute of files. (c) file information is written to an area in the volume (disk) called VTOC – Volume Table Of Contents. You can also ask for a data set to be “cataloged” in which case to VTOC entry for a dataset is also copied to a catalog (so you can find the dataset by name without knowing what volume it is on).

A “volume” is organized into blocks, tracks and cylinders. There are different volume types – 2311, 2314, 3030, 3050, 3880, 3890 come to mind. They are of different “geometries” (track size, cylinder size) and capacities.

The dataset organization is Sequential, Index Sequential, Direct. The records are fixed length, variable length, Spanned (logical record is longer than a physical block).  

When you would like to run (execute) a program you create a JCL for it. JCL has the program name, program’s location, its input files, work files, output files. The success/failure is returned by a program back to JCL to be used in decision to run subsequent programs. JCL is itself interpreted by a program called Initiator/Terminator.   

Sorry to get carried away, there is quite a bit more

Good luck, in your endeavors

    George

P.S

I used dataset, data set and file interchangeably.

My guess .tap may stand for “tape”. What the format is is an interesting question.

I would stay away from BOS (Basic Operating System). People who used it are “very old” and the knowledge is “dying out”. 


Peter Lund

unread,
Dec 18, 2020, 1:35:45 PM12/18/20
to George Lausman, z390
On Fri, Dec 18, 2020 at 7:15 PM George Lausman <george....@gmail.com> wrote:

Hi Peter,

S/360 is a family of machines introduced in the 60s’. It evolved a lot since then. You may feel like the five blind men touching the different parts of an elephant and trying to build a mental picture of the beast.

Learning by doing is what works for me too. Your thought to start writing an emulator is a bit too ambitious for your first S/360 project.

No, it most likely isn't.  It's not my first emulator and the S/360 isn't actually all that complicated.  It's just *old* and surprisingly badly documented compared to Z80 and 6502 machines.  The CPU has some unusual aspects but it isn't that bad.  Hex floats and ED/EDMK seem to be the worst parts of the original S/360 CPU architecture.  Channel I/O + the integration with the CPU is the biggest unknown but I hope to get my hands on real binaries that use channel I/O in order to understand it and in order to debug my emulation thereof when I get that far.

Compared to a cycle accurate emulation of an 80's home computer or a not quite so accurate emulation of a late 80's/early 90's AMIGA, PlayStation, or N64, the S/360 should be a piece of cake.

I think of channel I/O as just another badly explained asynch coprocessor facility but less fancy than most graphics hardware of 30 years ago and tighter integrated with the CPU.  It is normal for harddisks, NICs, and GPUs to have (multiple) command/response rings and interrupts for when CPU interaction is needed.  Channel I/O is not substantially different.

So I will concentrate on your desire to write a disassembler.

I already *have* a disassembler.  It has a handful of (really minor) problems that need to be fixed + I need to test it for real against real binaries.
 

Here are the pieces you would need for your disassembler: (1) S/360 hardware or emulator, (2) an operating system to manage the hardware, (3) development tools (editor, assembler, linkage-editor, perhaps utilities to copy files, etc.). All this is contained in Hercules. My suggestion is to get all the pieces and install them on your home machine (Linux or Windows).  

I absolutely  definitely *will* install Hercules, z390, and sim360c.  But not today.  Maybe next week.

If you succumb to doing parts of your development on Windows or Linux, you will have to learn all the ugly details of ASCII vs EBCDIC encoding, conversion programs and their quirks. All these screw-ups translate to one thing – extra knowledge. 

ASCII/EBCDIC/UTF-8 is easy.  And I *want* to learn how the silly structured file systems of the day worked.  Well, maybe not ISAM and VSAM.  You don't truly appreciate how nice simple byte sequences are until you have suffered with an operating system that tries to be smart and helpful.

The one thing you may not be prepared for is the S/360 I/O “richness”. Windows and Linux have just one I/O concept “a byte stream” unbroken for binaries or broken into lines for text files. The files are written to fixed blocks on a pre-formatted disk The S/360 uses “CKD” disk organization (Count – Key – Data) for each physical block. COUNT is the sequential number of physical blocks from the start of the track, KEY is the highest key in the following Data segment (used for direct access), DATA is the user data and it can be any length (later you will learn that max length is about 32K).

To create a dataset (file) one has to ALLOCATE it. This is done by JCL (Job Control Language). The process of “allocate” (a) reserves a space on the disk (we call it a VOLUME), (b) assigns a lot of attributes to the new dataset. Currently a JCL manual has about 300 pages devoted to explaining of individual attribute of files. (c) file information is written to an area in the volume (disk) called VTOC – Volume Table Of Contents. You can also ask for a data set to be “cataloged” in which case to VTOC entry for a dataset is also copied to a catalog (so you can find the dataset by name without knowing what volume it is on).

I'll never learn JCL.  I'd much rather spend my time on learning General Relativity.  That should be both easier and more useful.

I am used to the structured file craziness from VAX/VMS.  I found it silly, pointless, and annoying there and I'm pretty sure I will find it silly, pointless, and annoying on S/360 as well.  I'd say that the 1984 Turing Award was well-deserved!
 

A “volume” is organized into blocks, tracks and cylinders. There are different volume types – 2311, 2314, 3030, 3050, 3880, 3890 come to mind. They are of different “geometries” (track size, cylinder size) and capacities.

The dataset organization is Sequential, Index Sequential, Direct. The records are fixed length, variable length, Spanned (logical record is longer than a physical block).  

When you would like to run (execute) a program you create a JCL for it. JCL has the program name, program’s location, its input files, work files, output files. The success/failure is returned by a program back to JCL to be used in decision to run subsequent programs. JCL is itself interpreted by a program called Initiator/Terminator.   

All which made perfect sense in the days of punched cards :)
 

My guess .tap may stand for “tape”. What the format is is an interesting question.

My guess too.  And I don't have a clue about the format.  I'd hoped somebody here would now.
 

I would stay away from BOS (Basic Operating System). People who used it are “very old” and the knowledge is “dying out”. 

But it is a *tiny* system and that alone seems like a huge benefit at this early stage.

-Peter

Christopher Lang

unread,
Dec 18, 2020, 2:18:14 PM12/18/20
to Peter Lund, George Lausman, z390
You will not be able to build any systems then, re: So think again

I'll never learn JCL.  I'd much rather spend my time on learning General Relativity.  That should be both easier and more useful.

Cheers

Christopher


Christopher Lang

unread,
Dec 18, 2020, 5:21:54 PM12/18/20
to Peter Lund, George Lausman, z390
Look, I sent you how to build n OS/360 MVT system…




On 18 Dec 2020, at 19:35, Peter Lund <peterf...@gmail.com> wrote:

dave....@gmail.com

unread,
Dec 18, 2020, 6:12:29 PM12/18/20
to z3...@googlegroups.com

Well comparing S/360 with a Z80 is like comparing a Tesla with a Ford-T.

Actually one issue with S/360 is that its not a single thing.

The processors were upward compatible, but each had its own quirks.

Then remember the channels also runs channel programs which are programs and can contain conditional branches.

 

As for “badly documented” well a major problem is that there is so much documentation, and you need to read it all….

 

.. There is the Principles of Operation, which needs to be read in conjunction with the model specific manuals, and all the devices manuals.

 

Look IBM could produce a system in 1966 where the devices still worked on 1990’s machines I think it got something right….

 

… and if documentation is short how come we have so many emulators? Look at this

 

https://www.youtube.com/watch?v=1gDULzhKozU

 

FPGA re-implementation driving an original front panel…

 

Dave

 

From: z3...@googlegroups.com <z3...@googlegroups.com> On Behalf Of Christopher Lang
Sent: 18 December 2020 22:22
To: Peter Lund <peterf...@gmail.com>
Cc: George Lausman <george....@gmail.com>; z390 <z3...@googlegroups.com>
Subject: Re: New to IBM's mainframe world -- *.tap files, SL tapes, PDS data sets

 

Look, I sent you how to build n OS/360 MVT system…

Tony Harminc

unread,
Dec 19, 2020, 3:12:18 AM12/19/20
to z3...@googlegroups.com
On Fri, 18 Dec 2020 at 06:14, Peter Lund <peterf...@gmail.com> wrote:
>
> I have never seen, touched, or used any machine in the S/360 family. They are like strange creatures from an alien world to me.

Welcome to the alien world! Where is your home world?

> I recently started writing an S/360 emulator (+ disassembler) because I want to understand the S/360 and I learn much faster when unavoidably confronted with all the little details that I don't understand.
>
> My first problem is how to get hold of S/360 binaries so I have some code to disassemble/execute.

First, let's be clear about the architecture we're talking about.
S/360 is the 1964 base for a number of upward compatible architectures
with the current (2020) version being z Architecture. Although there
is some historic and academic interest in S/360, I am not aware of any
emulators for it. Most emulators are for S/370 (and descendants),
which is a very similar and 99+% upward compatible extension to S/360.
I suggest that you play with it if you are interested, as you will
find a lot more material on S/370 and many more knowledgeable people
around.

There are many forms of "binaries" (though nobody calls them that)
used on S/360 and its descendants. For example, if you run Linux on
zArch you will typically have ELF files, or possibly some different
formats generated by the compilers which are then bound into ELF. If
you run z/OS you will have three or four different formats. All of the
above usually contain both code and data, and metadata as well. So if
you want to disassemble binaries, you'll need to understand at least
enough of the metadata to tell what's what. Of course as with any von
Neumann architecture there is no fundamental difference between code
and data, so deciding which is which is a bit of an AI problem.

> I found http://www.bitsavers.org/bits/IBM/360/IBM_BOS_16K_Apr1966.zip , which seems to be not quite the bare minimum possible operating system on the architecture but damn close. The problem is that the zip file contains *.tap files that I don't know how to read. They don't seem to be raw binaries (the disassembler complained about lots and lots of undefined instructions).

I'm not sure what you would expect a "raw binary" to be, or why such a
thing would be present on a tape. But in any case I don't at a glance
recognize the format of those .tap files. (Do keep in mind that any of
the several "tape" file formats have nothing to do with S/360 et al;
they are containers used by the emulators or other software to
represent a physical tape with all its details in a stream file as
used on UNIX, Windows, and so on.)

> I have also downloaded http://www.ibiblio.org/jmaynard/k360s-ma20x-aws.zip which contains OS/360 binaries. They are in .aws files, which I have been reading up on this morning. They seem easy enough to work with, but the tapedump.c file linked to on the Hercules webpage is downright awful. I will probably write my own analyzer/extracter for those files.

Again, you'll be writing a program to process one layer of these
things. You have a zip file containing AWS files containing some kind
of files or parts thereof used by OS/360. Of course if you don't like
other peoples' programs, you can write all the layers yourself, but in
most cases they exist already.

> And then there is the rabbit hole of "standard labeled" tapes and that of "partitioned data sets". As far as I can tell, I need to look at the header/trailer label "files" around the actual data files in AWS files to get names, timestamps, file formats -- and then figure out how PDS files are stored so I can extract members. At least that seems to be necessary for some of the executables.

There is an executable format called a "load module" that can be
stored only in a PDS. Other formats - notably the "object modules"
produced by the compilers can be stored as a sequential file or as PDS
members. Or of course on punched cards... :-)

> Does anybody know of a good and thorough specification of PDS (and PDS/E) files, sorry, "data sets"? And a similar good and thorough specification of the VOL1/HDR1/HDR2/EOF1/EOF2/EOV1/EOV2/etc labels for tapes?

The tape part is easy; there is a book called "Tape Labels", which
hasn't changed significantly for around 50 years. Any version you find
on bitsavers should do. You'll find that some of the files on
bitsavers have been OCR'd; others not, or not well. For something like
Tape Labels with multiple versions available it may pay to search for
an OCR'd version.

There is no published specification of the layout on disk of PDSEs.
This is an unfortunate trend at IBM, and several other data formats of
recent years are also undocumented. But PDSEs do not exist on S/360 or
S/370 OSs. The layout of PDS data has been documented for decades, and
hasn't changed much over that time. To really understand it requires
some knowledge of the disk drives used on S/360, which are not much
like the fixed-length-block disks you are probably familiar with.

In addition, PDSs are more general data storage structures than needed
for load modules; conversely load modules make use of some PDS
features not used by any other mainstream software.

I would start with almost any bitsavers version of the Data Management
Services Guide (read up on disks - called Direct Access Storage
Devices - DASD), followed by the Linkage Editor and Loader, or perhaps
better - the current IBM book z/OS MVS Program Management: Advanced
Facilities, which has an appendix describing the layout of records in
object and load modules.

> PS: How long does it take to get used to IBM's incorrect bit numbering?

With that attitude it'll take you roughly forever. But do tell me - do
you prefer to number your bytes and words and pages and such in the
same order you number your bits, or in some different order?

Regards,

Tony H.

Peter Lund

unread,
Dec 19, 2020, 6:24:31 AM12/19/20
to z390
On Saturday, December 19, 2020 at 9:12:18 AM UTC+1 thar...@gmail.com wrote:
On Fri, 18 Dec 2020 at 06:14, Peter Lund <peterf...@gmail.com> wrote:
>
> I have never seen, touched, or used any machine in the S/360 family. They are like strange creatures from an alien world to me.

Welcome to the alien world! Where is your home world?

Z80 (ZX81 followed by Memotech MTX512).  Later PC's and various embedded CPUs (8051, ARM, others).  DOS and CCP/M-86 early on, later some Windows and some Unix, Linux as desktop since Summer '99, Windows 10 since Summer '98 (because it has WSL so I can continue to use Linux for the things Linux are good at).  And this is not my first disassembler or my first emulator.
 
> I recently started writing an S/360 emulator (+ disassembler) because I want to understand the S/360 and I learn much faster when unavoidably confronted with all the little details that I don't understand.
>
> My first problem is how to get hold of S/360 binaries so I have some code to disassemble/execute.

First, let's be clear about the architecture we're talking about.
S/360 is the 1964 base for a number of upward compatible architectures

I meant S/360 (excluding Model 67 and other oddities).  I may write an S/370 or Model 67 emulator later.
 
There are many forms of "binaries" (though nobody calls them that)

But that's what everybody else calls them outside the alien world.
 
I'm not sure what you would expect a "raw binary" to be, or why such a
thing would be present on a tape.

I expect there to be something the machine can boot, sorry, do "initial program load of".  I expect that to be very close to a raw binary.
I also expect there to be other programs that are run under a *very* minimal monitor/executor program.  Those programs are most likely not in quite such a raw form, but they will be close enough -- there will be big sections of machine code I can run my disassembler against.  I also expect those programs to be wrapped in something very simple because BOP was made for machines with as little as 16KB RAM, so it should be quite easy to pick those files apart if I need to.
 
But in any case I don't at a glance
recognize the format of those .tap files. (Do keep in mind that any of
the several "tape" file formats have nothing to do with S/360 et al;
they are containers used by the emulators or other software to
represent a physical tape with all its details in a stream file as
used on UNIX, Windows, and so on.)

I know the .tap files aren't in an "S/360"  format.  I'd hoped somebody would know what the format was but none have so far :(
AWS *is* a true IBM format for the S/360+ family.


> And then there is the rabbit hole of "standard labeled" tapes and that of "partitioned data sets". As far as I can tell, I need to look at the header/trailer label "files" around the actual data files in AWS files to get names, timestamps, file formats -- and then figure out how PDS files are stored so I can extract members. At least that seems to be necessary for some of the executables.

There is an executable format called a "load module" that can be
stored only in a PDS. Other formats - notably the "object modules"
produced by the compilers can be stored as a sequential file or as PDS
members. Or of course on punched cards... :-)

> Does anybody know of a good and thorough specification of PDS (and PDS/E) files, sorry, "data sets"? And a similar good and thorough specification of the VOL1/HDR1/HDR2/EOF1/EOF2/EOV1/EOV2/etc labels for tapes?

The tape part is easy; there is a book called "Tape Labels", which
hasn't changed significantly for around 50 years. Any version you find
on bitsavers should do. You'll find that some of the files on
bitsavers have been OCR'd; others not, or not well. For something like
Tape Labels with multiple versions available it may pay to search for
an OCR'd version.


Thank you!  That sounds *extremely* useful :)
 
There is no published specification of the layout on disk of PDSEs.
This is an unfortunate trend at IBM, and several other data formats of
recent years are also undocumented. But PDSEs do not exist on S/360 or
S/370 OSs. The layout of PDS data has been documented for decades, and
hasn't changed much over that time. To really understand it requires
some knowledge of the disk drives used on S/360, which are not much
like the fixed-length-block disks you are probably familiar with.

Sounds good.  A bit surprising that the disk drive stuff has seeped into the PDS format, but ok.
 
I would start with almost any bitsavers version of the Data Management
Services Guide (read up on disks - called Direct Access Storage
Devices - DASD), followed by the Linkage Editor and Loader, or perhaps
better - the current IBM book z/OS MVS Program Management: Advanced
Facilities, which has an appendix describing the layout of records in
object and load modules.

Thank you!  Again, that sounds *extremely* useful :)
 

> PS: How long does it take to get used to IBM's incorrect bit numbering?

With that attitude it'll take you roughly forever. But do tell me - do
you prefer to number your bytes and words and pages and such in the
same order you number your bits, or in some different order?

I number both the bytes and the bits in what anybody sane would consider the correct order.  Bytes sequentially starting from 0, bits according to their weight -- that means both are stable, no matter the number of bytes or the bit width of a register.  The utility of numbering bits according to their weight becomes even more obvious when working with binary fractions (fixed point numbers).  I have no hard opinion on byte order, there are good arguments for both little-endian and big-endian -- and even for the weird PDP-11 mixed-endian format for 32-bit numbers.  I also have no hard opinion on the right bit order for bit serial transmission.  Little-endian bit order works much better for bit serial processing like people used to do when hardware was really expensive and the bits were stored on a drum or in mercury delay lines or as acoustic pulses on long loops of wire.
IBM made a mistake regarding the bit ordering, which is weird considering how many other, less obvious, things they got right.

-Peter

Tony Harminc

unread,
Dec 20, 2020, 6:01:08 PM12/20/20
to z3...@googlegroups.com
On Sat, 19 Dec 2020 at 06:24, Peter Lund <peterf...@gmail.com> wrote:

> On Saturday, December 19, 2020 at 9:12:18 AM UTC+1 thar...@gmail.com wrote:

>> First, let's be clear about the architecture we're talking about.
>> S/360 is the 1964 base for a number of upward compatible architectures
>
> I meant S/360 (excluding Model 67 and other oddities). I may write an S/370 or Model 67 emulator later.

Sure. Keep in mind that if you want to write a true S/360 emulator
you'll have to check for operand alignment that was relaxed or removed
in S/370+. I am not aware of any emulator out there that bothers with
this.

A couple of other things to consider are handling the timer (there is
just one in S/360; S/370+ has more), how to handle USASCII mode (bit
12 in the PSW), managing storage protection with all its related
parts, ensuring that your interrupt handling is in the right order.

I agree with you and Dave and everyone that the instruction emulation
for S/360 is indeed quite easy. The trick is that unless you want to
key in your programs by hand and then hit Start, you'll need to
support the IPL sequence, and that in turn means you'll need to
emulate at least a good chunk of the I/O system, and that means disks
or at least tapes or card readers. None of which is really complex or
nasty, but is perhaps more work than is obvious.

I am reminded of when I first got a copy of the Java Virtual Machine
Specification around 1993 or so. I spent an hour reading about the
instruction set and the nature of the stack machine and thought to
myself "I can write a working JVM and get it debugged in about a
week". (I was writing in S/370 assembler - not the C that they
provided, and there were difficulties like at the time not having IEEE
FP hardware on S/370, but I knew I had access to an emulator for
that.) And I was right - it was very easy. But what I failed to
understand is that instruction emulation for Java is about 10% of the
whole task of being able to take something like a JAR file and run it.
I began to realize this when I considered how to test the first
program, and discovered a fat chapter on class loading. Followed
shortly after by how Java I/O works.

I'm not suggesting that S/360 is complex in the same way that Java is,
but there is enough baggage involved beyond the instruction emulation
to keep you busy for a while. And naturally part of it depends on just
how faithful you want to be to the architecture in your emulation.
"Good enough" to run a Hello, World or good enough to pass a
regression test with all the corner cases?

>> There are many forms of "binaries" (though nobody calls them that)
>
> But that's what everybody else calls them outside the alien world.

How nice for them. Of course everyone understands what you mean. I think.

>> I'm not sure what you would expect a "raw binary" to be, or why such a
>> thing would be present on a tape.
>
> I expect there to be something the machine can boot, sorry, do "initial program load of". I expect that to be very close to a raw binary.

To a point. The IPL sequence is architected, and as you've perhaps
read, it involves the "hardware" reading from a device (disk, tape,
cards) a string of 24 bytes into low storage using a virtual channel
program, and then continuing that I/O operation by logically branching
to the first byte read in and treating it as an extension of that
virtual channel program. When the program ends (note that no S/360 CPU
instructions have yet been executed) then a first PSW is loaded from
location 0 and execution begins as defined by that PSW.

So yes, your tape (or whatever) will probably have a raw string of
instructions that get read in and then executed. But most commonly
there will be a tiny loader program that knows how to read card-image
records from the same device you IPL'd from - i.e. following the
records containing the loader itself - and most commonly those records
will be in the "object module" format that I gave you a pointer to
earlier. You can recognize these as beginning with X'02' (you can
learn to translate to 0x02 notation if you must) followed by one of
TXT, RLD, ESD, or END. There are others not worth going into here.

So there will probably *not* be large sections of machine code on a
tape that you set out to IPL (or to read with your disassembler).

> I know the .tap files aren't in an "S/360" format. I'd hoped somebody would know what the format was but none have so far :(
> AWS *is* a true IBM format for the S/360+ family.

Well no - AWS is no more related to S/360+ than is that .tap stuff.
AWS is a container format for magnetic tapes for any system. It can
just as well represent a tape from a non-IBM system from the 1950s or
the 1990s. Yes, there are tapes that are physically/magnetically
compatible with the IBM tapes that can't be represented by AWS, but
that's entirely because AWS doesn't handle very large blocks well. And
of course I'm not talking about bizarre things like DECTAPE which are
directory structured like disks.

>> > And then there is the rabbit hole of "standard labeled" tapes and that of "partitioned data sets". As far as I can tell, I need to look at the header/trailer label "files" around the actual data files in AWS files to get names, timestamps, file formats -- and then figure out how PDS files are stored so I can extract members. At least that seems to be necessary for some of the executables.

It's up to you of course, but I wouldn't spend my time on that stuff
to start with. I would find one of the utility programs such as a
dump/restore program from one of the IBM OSs, and get *that* running
on your emulator. Tape labels and PDSs are at two quite different
layers of abstraction, and neither is needed to IPL and run code..

A very good place to start is the program called "3CARD LOADER" from
VM/370. It is (duh) 3 IPLable card-image (80-byte) records that can be
followed by an object module. You can disassemble it by eye. For that
matter since you've figured out the .tap format, you can extract the
first 24 bytes, figure out what they do when treated as CCWs (it will
almost certainly be to read in the following record off the tape), and
then figure out what that loaded program does.

> Sounds good. A bit surprising that the disk drive stuff has seeped into the PDS format, but ok.

You said the landscape is alien... You'll find that OS/360 and its
descendants are a mix of device independence (no, UNIX did not invent
that notion), and device *architecture* dependence. Inevitably various
other OSs for S/360+ have *some* dependence on the nature of the
hardware of the day, but for example VM/370 and the later UNIXs treat
the very complex disk drives as simple fixed-block ones.

[Just before hitting send I realized that my reply was addressed to
you, and I have to fix it up to go to the group. Is this something you
did - specified Reply-To somehow - or is it just another Google Groups
annoyance? If you have a choice it would be best not to specify
reply-to at all. I know nothing of the Google Groups web interface, if
that's what you're using.]

Tony H.
Reply all
Reply to author
Forward
0 new messages