RSX-11M text files
123 views
Skip to first unread message
Eckie 63
Nov 27, 2021, 2:29:59 PM11/27/21
to [PiDP-11]
I am looking to do something like a dos2unix-type conversion of RSX-11M text files but I can't figure out how they terminate a line. I don't see a CR or LF character. I see space characters but haven't found any other ones. I seem to recall seeing some info about this at some point but can't for the life of me remember where I saw it. Does anyone know how this is done in RSX?
-Peter
Warner Losh
Nov 27, 2021, 2:44:15 PM11/27/21
to Eckie 63, [PiDP-11]
On Sat, Nov 27, 2021, 12:30 PM Eckie 63 <epek...@gmail.com> wrote:
I am looking to do something like a dos2unix-type conversion of RSX-11M text files but I can't figure out how they terminate a line. I don't see a CR or LF character. I see space characters but haven't found any other ones. I seem to recall seeing some info about this at some point but can't for the life of me remember where I saw it. Does anyone know how this is done in RSX?
Aren't these the same as on VMs where there is a two byte count, followed by that many characters with an implied crlf at the end?
Warner
-Peter--
You received this message because you are subscribed to the Google Groups "[PiDP-11]" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pidp-11+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-11/c3eee175-091d-4a02-8f34-6e9be7cd3a26n%40googlegroups.com.
Anton Lavrentiev
Nov 27, 2021, 2:56:46 PM11/27/21
to Eckie 63, [PiDP-11]
dos2unix merely converts '\r\n' into single '\n' at the end of the
lines. RSX11M can use these line separators for so-called stream text
files. "Regular" text files, though, have a variable-record format,
where each line is preceded by a word with character count (<= 32767)
that follow, and if that count is odd, there is a pad character (with
an undefined value) added past the last character, so that the next
record's word of the byte count always begins at the word boundary.
There's a modification to this, when the records (lines) are not
allowed to span the block boundaries (and thus, are obviously less
than 510 bytes long): in this case, the last record in the block is
marked with 0xFFFF, and the remainder of the block is considered
unused. The information about the actual record format can be found
by using the DMP utility (which can print out the file header):
> RUN $DMP
DMP> TI:=DDU:[DIR]FILENAME.EXT/BL:0/HD
DMP> ^Z
HTH,
Anton
On Sat, Nov 27, 2021 at 2:30 PM Eckie 63 <epek...@gmail.com> wrote:
>
> I am looking to do something like a dos2unix-type conversion of RSX-11M text files but I can't figure out how they terminate a line. I don't see a CR or LF character. I see space characters but haven't found any other ones. I seem to recall seeing some info about this at some point but can't for the life of me remember where I saw it. Does anyone know how this is done in RSX?
>
> -Peter
>
lines. RSX11M can use these line separators for so-called stream text
files. "Regular" text files, though, have a variable-record format,
where each line is preceded by a word with character count (<= 32767)
that follow, and if that count is odd, there is a pad character (with
an undefined value) added past the last character, so that the next
record's word of the byte count always begins at the word boundary.
There's a modification to this, when the records (lines) are not
allowed to span the block boundaries (and thus, are obviously less
than 510 bytes long): in this case, the last record in the block is
marked with 0xFFFF, and the remainder of the block is considered
unused. The information about the actual record format can be found
by using the DMP utility (which can print out the file header):
> RUN $DMP
DMP> TI:=DDU:[DIR]FILENAME.EXT/BL:0/HD
DMP> ^Z
HTH,
Anton
On Sat, Nov 27, 2021 at 2:30 PM Eckie 63 <epek...@gmail.com> wrote:
>
> I am looking to do something like a dos2unix-type conversion of RSX-11M text files but I can't figure out how they terminate a line. I don't see a CR or LF character. I see space characters but haven't found any other ones. I seem to recall seeing some info about this at some point but can't for the life of me remember where I saw it. Does anyone know how this is done in RSX?
>
> -Peter
>
Anton Lavrentiev
Nov 27, 2021, 3:01:35 PM11/27/21
to Eckie 63, [PiDP-11]
... Oh, and there can be arranged exotic text file formats like
fixed-size records with CR-implied, as well as variable-size records
with Fortran control character at the beginning of each record(line),
and extremely rare are variable-length records with print control
(VFC) -- includes information of what print control to use before and
after every line.
fixed-size records with CR-implied, as well as variable-size records
with Fortran control character at the beginning of each record(line),
and extremely rare are variable-length records with print control
(VFC) -- includes information of what print control to use before and
after every line.
Eckie 63
Nov 27, 2021, 7:40:21 PM11/27/21
to [PiDP-11]
Hi Anton and Warner,
Thank you for all the information. I see now what's happening. So, what I am doing is, I am transferring files from RSX to one of my Linux boxes using FTP. I was transferring them in binary format, because I wanted them just like they are stored in RSX. But the header information from RSX isn't coming across so none of the record info is coming across. Just the text data. Turns out if I transfer text files in ASCII mode, it seems to do the translation for me and I get perfectly readable text files in Linux.
So since I don't get any of the record info in the file in Linux, I can't really do much I can do once it is transferred. So it seems ASCII mode will be my solution. But now I have had an opportunity to use the DMP utility and see how useful it can be, so it was good!
Thank you again for all the information!
-Peter
Anton Lavrentiev
Nov 27, 2021, 8:01:17 PM11/27/21
to Eckie 63, [PiDP-11]
Binary format would only work if you need to create an image of the file, completely unchanged. For text files, you've got to use the text mode, because it takes care of the actual "lines".... BTW, you can use image mode to transfer MAP files created by the task builder (TKB) -- because they are of the "unspecified" record type, and all the carriage control information is already included in there (like in stream files, that I've mentioned earlier, so they are practically the Unix format) ... It's funny because it's inconsistent with the very operating system's conventions
To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com.
Johnny Billquist
Nov 27, 2021, 8:25:38 PM11/27/21
to pid...@googlegroups.com
Hi. Jumping in a bit late, but anyway...
On 2021-11-28 01:40, Eckie 63 wrote:
> Hi Anton and Warner,
>
> Thank you for all the information. I see now what's happening. So, what
> I am doing is, I am transferring files from RSX to one of my Linux boxes
> using FTP. I was transferring them in binary format, because I wanted
> them just like they are stored in RSX. But the header information from
> RSX isn't coming across so none of the record info is coming across.
> Just the text data. Turns out if I transfer text files in ASCII mode, it
> seems to do the translation for me and I get perfectly readable text
> files in Linux.
What Anton and Warner wrote is basically all correct. However, it does
depend on the file attributes, which also need to be examined in order
to get the file correctly. The implied CR+LF are only implied if you
actually have the attribute implied CR+LF. Otherwise you should not add
such characters at the end of each record.
And it's only variable length record files that have a record length at
the beginning of each records. You can also have fixed length record
files, in which case the length of each record is also actually in the
file attributes.
So in order to correctly convert the raw data on disk for an RSX file,
you also need the file attributes, which is up to 20 bytes of meta-data
for the file. It can be read by special calls, but are not a part of the
actual file content.
And yes, ftp in ASCII mode does it for you. That's the whole point of
ASCII mode. It is supposed to transfer text files between systems,
independent of what the representation of a text file is on different
systems.
Binary mode should only be used when you want to preserve a file exactly
how it is, without any processing of the content. Which might mean it's
a file that don't make any sense anywhere else.
> So since I don't get any of the record info in the file in Linux, I
> can't really do much I can do once it is transferred. So it seems ASCII
> mode will be my solution. But now I have had an opportunity to use the
> DMP utility and see how useful it can be, so it was good!
In addition to just using DMP, check the /HD switch to DMP, and also
RMSDSP. Or read the manuals, which describes it all. :-)
Johnny
> <https://groups.google.com/d/msgid/pidp-11/c3eee175-091d-4a02-8f34-6e9be7cd3a26n%40googlegroups.com>.
> <https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: b...@softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
On 2021-11-28 01:40, Eckie 63 wrote:
> Hi Anton and Warner,
>
> Thank you for all the information. I see now what's happening. So, what
> I am doing is, I am transferring files from RSX to one of my Linux boxes
> using FTP. I was transferring them in binary format, because I wanted
> them just like they are stored in RSX. But the header information from
> RSX isn't coming across so none of the record info is coming across.
> Just the text data. Turns out if I transfer text files in ASCII mode, it
> seems to do the translation for me and I get perfectly readable text
> files in Linux.
depend on the file attributes, which also need to be examined in order
to get the file correctly. The implied CR+LF are only implied if you
actually have the attribute implied CR+LF. Otherwise you should not add
such characters at the end of each record.
And it's only variable length record files that have a record length at
the beginning of each records. You can also have fixed length record
files, in which case the length of each record is also actually in the
file attributes.
So in order to correctly convert the raw data on disk for an RSX file,
you also need the file attributes, which is up to 20 bytes of meta-data
for the file. It can be read by special calls, but are not a part of the
actual file content.
And yes, ftp in ASCII mode does it for you. That's the whole point of
ASCII mode. It is supposed to transfer text files between systems,
independent of what the representation of a text file is on different
systems.
Binary mode should only be used when you want to preserve a file exactly
how it is, without any processing of the content. Which might mean it's
a file that don't make any sense anywhere else.
> So since I don't get any of the record info in the file in Linux, I
> can't really do much I can do once it is transferred. So it seems ASCII
> mode will be my solution. But now I have had an opportunity to use the
> DMP utility and see how useful it can be, so it was good!
RMSDSP. Or read the manuals, which describes it all. :-)
Johnny
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pidp-11+u...@googlegroups.com
> <mailto:pidp-11+u...@googlegroups.com>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pidp-11+u...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com
> <https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: b...@softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
Johnny Billquist
Nov 27, 2021, 8:31:24 PM11/27/21
to [PiDP-11]
MAP files are not in an "unspecified" format.
They are variable record files with no record attributes. Which just
means there are no implied CR+LF after each record. So, if you want to
look at the as text, just read all of it, and use it verbatim.
So if you want to transfer them with ftp, still use ASCII mode, and they
will come across just right.
Do not use binary mode. That will again just mess you up. And no, they
are not the same as stream mode files, which is yet again a different
format. (And yes, stream mode is the closes to Unix, but it's a format
almost nothing use or understand.)
Johnny
> <https://groups.google.com/d/msgid/pidp-11/c3eee175-091d-4a02-8f34-6e9be7cd3a26n%40googlegroups.com>.
> <https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com?utm_medium=email&utm_source=footer>.
> <https://groups.google.com/d/msgid/pidp-11/CAAo%3Dyr2Yk%2BvCrjWkWUjvUDTxRQZK4HrPCsEiaUx%2BxLZDj4U9rw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
They are variable record files with no record attributes. Which just
means there are no implied CR+LF after each record. So, if you want to
look at the as text, just read all of it, and use it verbatim.
So if you want to transfer them with ftp, still use ASCII mode, and they
will come across just right.
Do not use binary mode. That will again just mess you up. And no, they
are not the same as stream mode files, which is yet again a different
format. (And yes, stream mode is the closes to Unix, but it's a format
almost nothing use or understand.)
Johnny
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pidp-11+u...@googlegroups.com
> <mailto:pidp-11+u...@googlegroups.com>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pidp-11+u...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com
> <https://groups.google.com/d/msgid/pidp-11/2fae67bd-5943-488e-a67d-cc8665da15dan%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pidp-11+u...@googlegroups.com
> <mailto:pidp-11+u...@googlegroups.com>.
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pidp-11+u...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pidp-11/CAAo%3Dyr2Yk%2BvCrjWkWUjvUDTxRQZK4HrPCsEiaUx%2BxLZDj4U9rw%40mail.gmail.com
> <https://groups.google.com/d/msgid/pidp-11/CAAo%3Dyr2Yk%2BvCrjWkWUjvUDTxRQZK4HrPCsEiaUx%2BxLZDj4U9rw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages