MAX record size in a sequential file

[StGallen]

Dear All,

I need to read records from a sequential file using DEC BASIC (version 1.3).
The said sequential file has the below structure (from ANALYZE/RMS):

RMS FILE ATTRIBUTES

File Organization: sequential
Record Format: stream-LF
Record Attributes: carriage-return
Maximum Record Size: 0
Longest Record: 32767
Blocks Allocated: 192, Default Extend Size: 0
End-of-File VBN: 39, Offset: %X'0192'
File Monitoring: disabled
Global Buffer Count: 0

QUESTION:
How does RMS come up with a value of 32767 for the Longest Record?

I did write a DCL procedure to check the size of each record and the largest record is 300. The total length of all records is 19125. Is RMS reporting 32767 as the largest record RMS can possibly fit in a record?

QUESTION:
IN DEC BASIC, I use:

MAP (ABC) STRING all_in_all=2048%

OPEN FILE_NAME FOR INPUT AS FILE #118%, &
ORGANIZATION SEQUENTIAL STREAM, &
ACCESS READ, &
ALLOW NONE, &
MAP ABC

This does not work and I get a run time error of :
BADRECVAL, Bad RECORDSIZE value on OPEN (ERR=148)

I feel it would be inelegant to set the MAP to 32767?

QUESTION:
I am having problem understanding the below :
Record Format: stream-LF
Record Attributes: carriage-return
Maximum Record Size: 0
Longest Record: 32767

Record Format: The records in this case are seperated by a LF character.
What is the Record Attribute? What does carriage_return mean here?


Thanks for all your answers.
Best

[Johnny Billquist]

I think your basic problem is that the file do not have a fixed record
length, so how do you expect the map to work?
When dealing with variable record lengths, you need to be more clever.
(What's wrong with just opening the file and doing LINPUT on it?)

Johnny

[Dirk Munk]

StGallen wrote:
> Dear All,
>
> I need to read records from a sequential file using DEC BASIC (version 1.3).
> The said sequential file has the below structure (from ANALYZE/RMS):
>
> RMS FILE ATTRIBUTES
>
> File Organization: sequential
> Record Format: stream-LF
> Record Attributes: carriage-return
> Maximum Record Size: 0
> Longest Record: 32767
> Blocks Allocated: 192, Default Extend Size: 0
> End-of-File VBN: 39, Offset: %X'0192'
> File Monitoring: disabled
> Global Buffer Count: 0
>
> QUESTION:
> How does RMS come up with a value of 32767 for the Longest Record?
>
> I did write a DCL procedure to check the size of each record and the largest record is 300. The total length of all records is 19125. Is RMS reporting 32767 as the largest record RMS can possibly fit in a record?
>

Indeed, it is the maximum possible record size in such a file.

> QUESTION:
> IN DEC BASIC, I use:
>
> MAP (ABC) STRING all_in_all=2048%
>

> OPEN FILE_NAME FOR INPUT AS FILE #118%,&
> ORGANIZATION SEQUENTIAL STREAM,&
> ACCESS READ,&
> ALLOW NONE,&

> MAP ABC
>
> This does not work and I get a run time error of :
> BADRECVAL, Bad RECORDSIZE value on OPEN (ERR=148)
>
> I feel it would be inelegant to set the MAP to 32767?
>
> QUESTION:
> I am having problem understanding the below :
> Record Format: stream-LF
> Record Attributes: carriage-return
> Maximum Record Size: 0
> Longest Record: 32767
>
> Record Format: The records in this case are seperated by a LF character.
> What is the Record Attribute? What does carriage_return mean here?
>

It means that RMS will add a <cr> to the data when the record is
retrieved from the file.

[StGallen]

On Tuesday, May 15, 2012 12:39:42 PM UTC+5:30, Johnny Billquist wrote:
>
> I think your basic problem is that the file do not have a fixed record
> length, so how do you expect the map to work?
> When dealing with variable record lengths, you need to be more clever.
> (What's wrong with just opening the file and doing LINPUT on it?)
>
> Johnny

LINPUT will only work with terminal format files. BASIC run time error = 164.
OPEN ABC_FILE FOR INPUT AS FILE #158%, &
ORGANIZATION UNDEFINED, &
RECORDTYPE ANY, &

ACCESS READ, &
ALLOW NONE

LINPUT #158%, ABC_REC !==> 158 must be a Terminal Format file.

[Bob Koehler]

In article <14611490.1188.1337062727195.JavaMail.geo-discussion-forums@pbcge2>, StGallen <sengupt...@gmail.com> writes:
> Dear All,
>
> I need to read records from a sequential file using DEC BASIC (version 1.3).
> The said sequential file has the below structure (from ANALYZE/RMS):
>
> RMS FILE ATTRIBUTES
>
> File Organization: sequential
> Record Format: stream-LF
> Record Attributes: carriage-return
> Maximum Record Size: 0
> Longest Record: 32767
> Blocks Allocated: 192, Default Extend Size: 0
> End-of-File VBN: 39, Offset: %X'0192'
> File Monitoring: disabled
> Global Buffer Count: 0
>
> QUESTION:
> How does RMS come up with a value of 32767 for the Longest Record?

Nobody told it better. 32767 is the maximum signed number that fits
in the bits.

I think you said you're reading the file in BASIC, and showed us the
code you are trying to use. Show us the code that creates the file.

With a record format of smtlf, I don't think a record attribute of
CR means much other than the writer is being inconsistent.

[Johnny Billquist]

So why do you (in this example) open the file specifying "organization
undefined"?

Johnny

[jbriggs444]

On May 15, 10:15 am, koeh...@eisner.nospam.encompasserve.org (Bob
Koehler) wrote:

I do not understand what the objection is to a record format of stream-
lf
with record attributes of CR.

Stream-lf means that your on-disk record layout is controlled by
the presence of line feeds in the data. Record attributes of CR means
that your on-the-printer layout has each record printed on a
separate line. Seems quite standard to me. Exactly what one would
want when attempting to reproduce the semantics of a Unix text file.

But I am in the presence of one who knows his stuff. So it is
plausible
that I am missing something.

[George Cornelius]

jbriggs444 wrote:
> Stream-lf means that your on-disk record layout is controlled by
> the presence of line feeds in the data. Record attributes of CR means
> that your on-the-printer layout has each record printed on a
> separate line. Seems quite standard to me. Exactly what one would
> want when attempting to reproduce the semantics of a Unix text file.
>
> But I am in the presence of one who knows his stuff. So it is
> plausible that I am missing something.

No, you have it right.

As a minor nit, it can take a CR-LF pair to actually generate the
desired display format, but, historically, the old Underwood would
do a line advance along with the carriage movement.

[I don't remember how we managed to get a return with no linefeed
when overtyping, although of course backspaces would do it].

But the original question still stands. Why do some programs seem
to update the header to reflect the longest record size while others
leave it at max? For the C run-time library, which tends to bypass
RMS for much of what it does, I understand, but I believe I have
seen this in other contexts as well.

George

[Steven Schweda]

> [...] Why do some programs seem to update the header to

> reflect the longest record size while others leave it at max?

I know nothing, but I can imagine a file-reading program
which uses a maximum-record-length value to set a buffer
size. I can also imagine a file-writing program which does
not care about, or keep track of, the actual maximum record
length in its output file. In such a situation, I know which
value I'd want the file-writing program to store as the
maximum record length.

For a file with variable-length records, an update
operation might require completely reading the file to
determine the new maximum record length, so I can easily
imagine not wanting to maintain an accurate
maximum-record-length value for an output file.

> [I don't remember how we managed to get a return with no linefeed
> when overtyping, although of course backspaces would do it].

In the pre-Selectric era, pushing the carriage back would
do it. Pushing on the handle did the vertical advance.
Pushing on the end of the platen did not.

[JF Mezei]

StGallen wrote:
>
> QUESTION:
> How does RMS come up with a value of 32767 for the Longest Record?

It is a default value because RMS doesn't keep track of the longest
record length in the file as this is basically a raw data file which is
interpreted by RMS at run time to emulate records.

The real limit on record size will be defined by your bufquo (buffer
quota) which limits the size of any individual IO. If you have a record
in the file that is longer than your bugquo, the read will fail.

[Arne Vajhøj]

Probably because he thought it meant undefined (as in "this app does
not know") as opposed to undefined (as in FAT$C_UNDEFINED / FAB$C_UDF).

Arne

[Hein RMS van den Heuvel]

On Tuesday, May 15, 2012 1:21:07 PM UTC-6, JF Mezei wrote:
> StGallen wrote:
> >
> > QUESTION:
> > How does RMS come up with a value of 32767 for the Longest Record?

RMS doesn't. The C-RTL does as controlled by the logical name DECC$DEFAULT_LRL
http://h71000.www7.hp.com/doc/732final/5763/5763pro_004.html
That same C-RTL will also default the file attributes to a prior version of the file if that exists, to futher muddle the waters.

> It is a default value because RMS doesn't keep track of the longest

It does.

> record length in the file as this is basically a raw data file which is
> interpreted by RMS at run time to emulate records.

It is not.

> The real limit on record size will be defined by your bufquo (buffer
> quota) which limits the size of any individual IO.

It does not.

'bufquo' which is probably referring to the SYSGEN param MAXBUF only influenced BUFFERED IO. Notably QIO to the TERMINAL.

RMS _was_ limited to an IO buffer size of 127 blocks, 'recently' bumped tp 255 blocks.
The record size is limited by the maximum signed 16 bit value (32767)

> If you have a record in the file that is longer than your bugquo, the read will fail.

No it will not. RMS will read it just fine.
Trying to write the read record to a buffered IO device will fail ($ TYPE ...)


StGallen> I feel it would be inelegant to set the MAP to 32767?
Why? Nothing wrong with that other than a 'wasted' VM.
It will nicely avoid future failures.

Alternatively you can
- create the file with RMS proper,
- Use that logical
- 'fix' the file with $ ANALYZE/RMS/UPDATE
- 'fix' the file with $ SET RMS/FILE=ATTRIB=LRL=xxx

Good luck!
Hein

[JF Mezei]

Hein RMS van den Heuvel wrote:

> No it will not. RMS will read it just fine.
> Trying to write the read record to a buffered IO device will fail ($ TYPE ...)

Ahhh, intreesting. Hadn't thought about that. So all those years that I
had trouble typing certain files, it was actually failing at writing to
terminal instead of reading from file.

[Bob Koehler]

In article <0cfabdf3-f7a3-498f...@e3g2000yqm.googlegroups.com>, jbriggs444 <jbrig...@gmail.com> writes:
>
> Stream-lf means that your on-disk record layout is controlled by
> the presence of line feeds in the data. Record attributes of CR means
> that your on-the-printer layout has each record printed on a
> separate line. Seems quite standard to me. Exactly what one would
> want when attempting to reproduce the semantics of a Unix text file.

Could be. Not something I'm likely to attempt.

[Bob Koehler]

In article <4fb2a165$0$63204$815e...@news.qwest.net>, George Cornelius <corn...@eisner.decus.org> writes:
>
> As a minor nit, it can take a CR-LF pair to actually generate the
> desired display format, but, historically, the old Underwood would
> do a line advance along with the carriage movement.

I would expect a new line for every LF in a stmlf file, independent
of any or no CRs.

But our old Underwood would not advance the line until the carriage
was fully returned AND we continued to push the lever. One could
stop without the line advance.

[Bob Koehler]

In article <ef4e88ca-38c0-426d...@qg3g2000pbc.googlegroups.com>, Steven Schweda <sms.an...@gmail.com> writes:
>
> In the pre-Selectric era, pushing the carriage back would
> do it. Pushing on the handle did the vertical advance.
> Pushing on the end of the platen did not.

Prior to Selectric there were plenty of electric typewriters with
buttons for carriage return, line feed, or both. The big button
did both.

But generally when we wanted to overtype an entire line uaually to
add underscores) we pushed the lever hard enough to get the line
advance, or hit the wrong button, out of habit, the reached up and
rolled the line back. Worked on LA36 and LA120, too, although we
generally did it programmatically.

But those knobs were missing from our VTs. And overtyping with
underscores left the text on the VT hard to read.

[jbriggs444]

On May 16, 9:30 am, koeh...@eisner.nospam.encompasserve.org (Bob
Koehler) wrote:

> In article <0cfabdf3-f7a3-498f-8d62-14f9a3384...@e3g2000yqm.googlegroups.com>, jbriggs444 <jbriggs...@gmail.com> writes:
>
>
>
> > Stream-lf means that your on-disk record layout is controlled by
> > the presence of line feeds in the data.  Record attributes of CR means
> > that your on-the-printer layout has each record printed on a
> > separate line.  Seems quite standard to me.  Exactly what one would
> > want when attempting to reproduce the semantics of a Unix text file.
>
>    Could be.  Not something I'm likely to attempt.

For your information...

The carriage control record attributes on a VMS file do not modify the
data returned by RMS to the user. They are intended as metadata to
be used by applications that intend to print or otherwise display the
file.

CR means carriage-return or "implied" carriage control, i.e.

When output to a print device (e.g. terminal or printer), each record
in the
data file is to be preceded with a line feed and followed with a
carriage
return

[In practice, the leading line feed in front of the first record in a
file is
usually suppressed]


Fortran means Fortran carriage control, i.e.

The first character in each record is interpreted as carriage control
information.

A blank (" ") means single-spacing
A hyphen ("-") means double-spacing
A plus sign ("+") means overstrike this line on top of the last one
A one ("1") means print this line on a new page


None means no carriage control, i.e

Carriage returns, line feeds and form feeds must be literally present
within the record. No carriage control is automatically supplied
between records. By default the entire file prints on one long
line.


Print-file carriage control requires that the file have VFC
organization
(variable length records with a fixed control region). The fixed
control
region is used to store the carriage control information. As I
recall,
this is basically a two byte field containing a count of newlines to
print
before the current record and a count of newlines to print after.


The standard expectation for a Stream-LF file that is printed would
be for each record to print on a line by itself. That means that
record attribute of CR is entirely appropriate.

The situation at hand involved basic and RMS. The paragraphs
below are not at all relevant to this situation.

The C run time library can make life difficult. It tries to arrange
things
so that the data stream that a user sees when reading from an RMS
file more or less matches the data stream that would result if that
file
were printed. It can look at the record attributes and synthesize
file
content accordingly.

http://h71000.www7.hp.com/doc/83final/5763/5763pro_006.html

[Note that the CRTL defaults to processing Stream-LF files in stream
mode. It disregards record attributes when operating in stream mode]

[Paul Sture]

I once worked on a project where you entered for example a start date and
end date before producing a printout. Needless to say it wasn't long
before customers with real data started complaining that this tied a
terminal up for too long. We devised a system of dumping the program's
working storage to disk and submitting a batch job to pick it up and
produce the report.

The next step of course was that the customer wanted to view those
reports from a VT and optionally manipulate print queue entries.

You wouldn't believe how many programmers had done "clever things" in
those reports, <CR>, reverse line feeds, backspace, the lot, and yours
truly had to unravel that lot.

It was actually quite fun seeing what I'd come across next :-)



--
Paul Sture