Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
record length in an unformatted (binary) file
93 views
Skip to first unread message
marcio.pmz
Oct 1, 2012, 9:21:34 PM10/1/12
to
Hello,
Is there somehow to know the lenght of records in an unformatted (binary) file?
Would 'inquire' statement be an option?
Of course I understand that with the record length I may use direct access at open statement. On the other hand, if I wish to use sequential access, how would I use the record length info?
Thanks!
Is there somehow to know the lenght of records in an unformatted (binary) file?
Would 'inquire' statement be an option?
Of course I understand that with the record length I may use direct access at open statement. On the other hand, if I wish to use sequential access, how would I use the record length info?
Thanks!
Richard Maine
Oct 1, 2012, 10:23:00 PM10/1/12
to
marcio.pmz <mtnobr...@gmail.com> wrote:
> Is there somehow to know the lenght of records in an unformatted (binary)
file?
Not anything portable or defined by the standard.
> Is there somehow to know the lenght of records in an unformatted (binary)
file?
I might also note that "binary" is not a term defined by the Fortran
standard, and it is not well-defined what it means. The literal meaning
("base 2") is almost never what anyone means when talking about computer
files. Although different people use the term with different meanings
(which is why I avoid it), perhaps the most common meaning is to
describe an unformatted file that has no record structure. That takes a
pretty long stretch to connect to the word "binary", but still that's
the most common usage. If that happens to be what you mean, then such a
file doesn't *HAVE* an inherent record length to be known.
With most implementations, you can open such a file with any record
length as long as you are using direct access. "Records" are arbitrarily
defined to be chunks of data of whatever size you specify in the open,
but these records have no physical embodiment in the file. You usually
can't open such a file for sequential acess at all. Exceptions exist,
but that's the way most implementations work.
I might note that unformatted stream access, introduced in f2003, and
supported by most current compilers, is a more natural fit to such
"binary" files.
> Would 'inquire' statement be an option?
> Of course I understand that with the record length I may use direct access
> at open statement. On the other hand, if I wish to use sequential
> access, how would I use the record length info?
actually the record length. It is the maximum record length. There is
usually little reason to specify it for sequential access. In the past,
some implementations have had fairly small default values for the
maximum record length. In that case, specifying a large value for recl
might tell the implementation to accomodate suffcient buffers or other
resources required to support that record length. However, to me
knowledge, most current implementations default to having the recl limit
as large as the implementation can support, which is often something
like 2GB (or sometimes more). If you are working with records long
enough that this is an issue, there are probably lots of other matters
that also need careful attention.
--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
Louisa
Oct 1, 2012, 10:44:22 PM10/1/12
to
a record of an unformatted file that you are about to write.
After using INQUIRE, you would then give the record length
in an OPEN statement.
Steve Lionel
Oct 2, 2012, 9:43:45 AM10/2/12
to
The method of storing those lengths is implementation-dependent, though
on operating systems that don't support a record length concept in the
file system (UNIX, Linux, Windows, OS X) the most common implementation
is to store a length before and after the data. The language does not
provide, however, a way to retrieve this length.
For a direct-access unformatted file, the most common implementation
does not store lengths in the file at all and it is up to the programmer
to know in advance what the length of each record is. Some file systems
(VMS, for example), do store this information as part of the file
attributes and you can typically use RECL= in an INQUIRE to retrieve it.
On file systems where the length is not stored, you can't.
--
Steve Lionel
Developer Products Division
Intel Corporation
Merrimack, NH
For email address, replace "invalid" with "com"
User communities for Intel Software Development Products
http://software.intel.com/en-us/forums/
Intel Software Development Products Support
http://software.intel.com/sites/support/
My Fortran blog
http://www.intel.com/software/drfortran
Refer to http://software.intel.com/en-us/articles/optimization-notice
for more information regarding performance and optimization choices in
Intel software products.
glen herrmannsfeldt
Oct 2, 2012, 9:47:28 AM10/2/12
to
Steve Lionel <steve....@intel.invalid> wrote:
> On 10/1/2012 9:21 PM, marcio.pmz wrote:
>> Hello,
>> Is there somehow to know the lenght of records in an unformatted
>> (binary) file?
>> Would 'inquire' statement be an option?
>> Of course I understand that with the record length I may use
>> direct access at open statement. On the other hand, if I wish to
>> use sequential access, how would I use the record length info?
> A sequential unformatted file may have records of differing lengths.
> The method of storing those lengths is implementation-dependent, though
> on operating systems that don't support a record length concept in the
> file system (UNIX, Linux, Windows, OS X) the most common implementation
> is to store a length before and after the data. The language does not
> provide, however, a way to retrieve this length.
Not easily, anyway. You are allowed to read less than a whole
> On 10/1/2012 9:21 PM, marcio.pmz wrote:
>> Hello,
>> Is there somehow to know the lenght of records in an unformatted
>> (binary) file?
>> Would 'inquire' statement be an option?
>> Of course I understand that with the record length I may use
>> direct access at open statement. On the other hand, if I wish to
>> use sequential access, how would I use the record length info?
> A sequential unformatted file may have records of differing lengths.
> The method of storing those lengths is implementation-dependent, though
> on operating systems that don't support a record length concept in the
> file system (UNIX, Linux, Windows, OS X) the most common implementation
> is to store a length before and after the data. The language does not
> provide, however, a way to retrieve this length.
record, and should get an error if you read more. It should, then,
be possible to do a binary search while checking for I/O error,
then BACKSPACE or REWIND to read it again. Likely slow and
inefficient, and I don't know quite what the standard says
about reporting of I/O errors.
-- glen
Richard Maine
Oct 2, 2012, 11:23:37 AM10/2/12
to
glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
> Steve Lionel <steve....@intel.invalid> wrote:
> > A sequential unformatted file may have records of differing lengths.
> > The method of storing those lengths is implementation-dependent, though
> > on operating systems that don't support a record length concept in the
> > file system (UNIX, Linux, Windows, OS X) the most common implementation
> > is to store a length before and after the data. The language does not
> > provide, however, a way to retrieve this length.
>
> Not easily, anyway. You are allowed to read less than a whole
> record, and should get an error if you read more. It should, then,
> be possible to do a binary search while checking for I/O error,
> then BACKSPACE or REWIND to read it again. Likely slow and
> inefficient, and I don't know quite what the standard says
> about reporting of I/O errors.
I do (know what the standard says about reporting I/O errors). And the
short version is that you can't count on anything.
The standard provides a mechanism (well, two of them) for the processor
to use for reporting I/O errors, but it does *NOT* require that the
processor do that. The methods are just there for the processor to use
if it chooses to do so. In particular, the standard explicitly says that
it does not define what constitutes an I/O error to be reported in those
ways. So any code that depends on getting I/O error indications has
portability problems. I have personally been bitten by that before I
finally learned the lesson (the quote my signature applies).
If your code does something like try to read a record longer than is on
the file, your code is just nonstandard in one of the ways that allows
the compiler to do whatever it likes. Most compilers will "nicely" (in
my opinion) report it via the iostat and err= specifiers, but the
standard does not require that. (We are talking unformatted here, so
pad= doesn't pertain).
For example, I used to have some code that determined whether a
particular formatted input field was for an integer by trying to read it
as an integer. That code failed to work as I intended for processors
that just "fixed" the error for me instead of reporting it as an error.
I also used to have code that used the method you describe above for
determining the record length of an existing unformatted sequential
file. I knew that all the records were supposed to be the same size for
these particular files, so I only needed to do this for the first
record, making the inefficiency was not much of an issue. The assumption
that it would work as intended was a bigger problem, though. I even used
that code for a long time. I'm having trouble recalling whether that
particular problem bit me, or whether I just fixed it after having been
bitten by other problems relating to I/O errors.
> Steve Lionel <steve....@intel.invalid> wrote:
> > A sequential unformatted file may have records of differing lengths.
> > The method of storing those lengths is implementation-dependent, though
> > on operating systems that don't support a record length concept in the
> > file system (UNIX, Linux, Windows, OS X) the most common implementation
> > is to store a length before and after the data. The language does not
> > provide, however, a way to retrieve this length.
>
> Not easily, anyway. You are allowed to read less than a whole
> record, and should get an error if you read more. It should, then,
> be possible to do a binary search while checking for I/O error,
> then BACKSPACE or REWIND to read it again. Likely slow and
> inefficient, and I don't know quite what the standard says
> about reporting of I/O errors.
short version is that you can't count on anything.
The standard provides a mechanism (well, two of them) for the processor
to use for reporting I/O errors, but it does *NOT* require that the
processor do that. The methods are just there for the processor to use
if it chooses to do so. In particular, the standard explicitly says that
it does not define what constitutes an I/O error to be reported in those
ways. So any code that depends on getting I/O error indications has
portability problems. I have personally been bitten by that before I
finally learned the lesson (the quote my signature applies).
If your code does something like try to read a record longer than is on
the file, your code is just nonstandard in one of the ways that allows
the compiler to do whatever it likes. Most compilers will "nicely" (in
my opinion) report it via the iostat and err= specifiers, but the
standard does not require that. (We are talking unformatted here, so
pad= doesn't pertain).
For example, I used to have some code that determined whether a
particular formatted input field was for an integer by trying to read it
as an integer. That code failed to work as I intended for processors
that just "fixed" the error for me instead of reporting it as an error.
I also used to have code that used the method you describe above for
determining the record length of an existing unformatted sequential
file. I knew that all the records were supposed to be the same size for
these particular files, so I only needed to do this for the first
record, making the inefficiency was not much of an issue. The assumption
that it would work as intended was a bigger problem, though. I even used
that code for a long time. I'm having trouble recalling whether that
particular problem bit me, or whether I just fixed it after having been
bitten by other problems relating to I/O errors.
marcio.pmz
Oct 4, 2012, 2:54:37 PM10/4/12
to
Maine,
I have followed your suggestion sugestion and have elaborated a routine to read the first record (or whatever we understand by this in an unformatted file), opening the file for sequential access, catching the err on a label to determine the length of that record. Thank you!
subroutine ledata
open(unit=1, file='data.dat',status='old', & access='sequential',form='unformatted')
ireg=0
do while(ireg<=icont)
ireg=ireg+1
read(1,err=100)quhes(1:ireg,1,1)
rewind(1)
enddo
100 write(*,*)'leu quhes sequential ',ireg
end subroutine
I have followed your suggestion sugestion and have elaborated a routine to read the first record (or whatever we understand by this in an unformatted file), opening the file for sequential access, catching the err on a label to determine the length of that record. Thank you!
subroutine ledata
open(unit=1, file='data.dat',status='old', & access='sequential',form='unformatted')
ireg=0
do while(ireg<=icont)
ireg=ireg+1
read(1,err=100)quhes(1:ireg,1,1)
rewind(1)
enddo
100 write(*,*)'leu quhes sequential ',ireg
end subroutine
marcio.pmz
Oct 4, 2012, 3:00:34 PM10/4/12
to
Glen,
As you had agreed with Lionel there is no easy way or a intrinsic. I followed your advice, but, as Richard pointed, being the whole file the same length in each record, I had just read the 1st one, which was enough for me.
Thank you!
subroutine ledata
open(unit=1,file='data.dat',status='old' & ,access='sequential',form='unformatted')
As you had agreed with Lionel there is no easy way or a intrinsic. I followed your advice, but, as Richard pointed, being the whole file the same length in each record, I had just read the 1st one, which was enough for me.
Thank you!
subroutine ledata
open(unit=1,file='data.dat',status='old' & ,access='sequential',form='unformatted')
Richard Maine
Oct 4, 2012, 3:23:59 PM10/4/12
to
I long ago stopped doing so because the standard does not actually
guarantee that it will work on all compilers. It probably will work on
most of them, perhaps even all that you might happen to encounter, but
it is not strictly guaranteed.
Admitedly, there isn't really any better way. The other alternatives are
quite a bit less portable. In recent decades (yes, decades; I did say it
was a long time ago that I used to use this), I have generally avoided
the need to do this by making sure that the needed size information was
available from some other source.
Also, it looks to me like you have an off-by-one error unless the code
compensates for it elsewhere. After exiting the loop, the value of ireg
is the one that failed rather than the last one that worked.
Gordon Sande
Oct 4, 2012, 4:14:03 PM10/4/12
to
some value then you could try the trick of setting a larger array to
all zeros and then reading the record in. Look at the array to see where
it is no longer zero. But you might be reading zeros so repeat with some
other value than zero. Look where the array is not that other value.
A variation on this is to have two arrays, one of zeros and one of (say)
ones and read the initial record into the two arrays. Scan for where the
two arrays agree and then become the zero and one you put there.
All of this assumes that you can recover from reading the short record.
And that there will be a reliable transfer of the values in the record.
It is a matter of picking your favorite nonstandard posion.
Richard Maine
Oct 4, 2012, 4:36:54 PM10/4/12
to
Gordon Sande <Gordon...@gmail.com> wrote:
> If you have some reason to believe that the records are shorter than
> some value then you could try the trick of setting a larger array to
> all zeros and then reading the record in. Look at the array to see where
> it is no longer zero...
> If you have some reason to believe that the records are shorter than
> some value then you could try the trick of setting a larger array to
> all zeros and then reading the record in. Look at the array to see where
> All of this assumes that you can recover from reading the short record.
> And that there will be a reliable transfer of the values in the record.
> It is a matter of picking your favorite nonstandard posion.
worse about portability. I have definitely been bitten in using that
trick. The standard says that the whole array becomes undefined. Yes,
there are implementations that read in the part of the array that
corresponds to data in the record. But there are also implementations
that leave the whole array unchanged.
As you say, it is a matter of picking your own nonstandard poison, but
I've got a pretty strong personal pick in that case. The standard at
least has what I might call a suggestion that checking iostat might be
useful; it just doesn't guarantee it. But the standard is pretty
explicit about the whole array becoming undefined; it gives not even a
hint that the processor might put data in part of it.
glen herrmannsfeldt
Oct 4, 2012, 8:31:28 PM10/4/12
to
Richard Maine <nos...@see.signature> wrote:
> Gordon Sande <Gordon...@gmail.com> wrote:
(snip)
>> All of this assumes that you can recover from reading the short record.
>> And that there will be a reliable transfer of the values in the record.
>> It is a matter of picking your favorite nonstandard posion.
> In my experience, although both are nonstandard, that method is far
> worse about portability. I have definitely been bitten in using that
> trick. The standard says that the whole array becomes undefined. Yes,
> there are implementations that read in the part of the array that
> corresponds to data in the record. But there are also implementations
> that leave the whole array unchanged.
I remember trying to do this on VAX with implied DO, and with END=
exit, then expecting the data to be read up to the end of the file,
and the appropriate value left in the implied-DO variable.
When it failed, I was ready to send in a bug report to DEC.
I believe I even had a form ready to be filled out.
Either I found out it wasn't a compiler bug, or just forgot,
but I never got around to filling it out and sending it.
That was before the web, and before I would have even thought
about seeing a copy of the standard. I didn't even have my own
copy of the VAX Fortran manual.
Changing to a non-implied DO fixed that one, and is probably
the better way to do it. I don't know if the data did or didn't
get stored, but the implied-DO variable didn't indicate that
any did. (That was for formatted input, but a related problem.)
-- glen
> Gordon Sande <Gordon...@gmail.com> wrote:
(snip)
>> All of this assumes that you can recover from reading the short record.
>> And that there will be a reliable transfer of the values in the record.
>> It is a matter of picking your favorite nonstandard posion.
> In my experience, although both are nonstandard, that method is far
> worse about portability. I have definitely been bitten in using that
> trick. The standard says that the whole array becomes undefined. Yes,
> there are implementations that read in the part of the array that
> corresponds to data in the record. But there are also implementations
> that leave the whole array unchanged.
exit, then expecting the data to be read up to the end of the file,
and the appropriate value left in the implied-DO variable.
When it failed, I was ready to send in a bug report to DEC.
I believe I even had a form ready to be filled out.
Either I found out it wasn't a compiler bug, or just forgot,
but I never got around to filling it out and sending it.
That was before the web, and before I would have even thought
about seeing a copy of the standard. I didn't even have my own
copy of the VAX Fortran manual.
Changing to a non-implied DO fixed that one, and is probably
the better way to do it. I don't know if the data did or didn't
get stored, but the implied-DO variable didn't indicate that
any did. (That was for formatted input, but a related problem.)
-- glen
glen herrmannsfeldt
Oct 4, 2012, 8:36:47 PM10/4/12
to
marcio.pmz <mtnobr...@gmail.com> wrote:
> As you had agreed with Lionel there is no easy way or a
> intrinsic. I followed your advice, but, as Richard pointed,
> being the whole file the same length in each record,
> I had just read the 1st one, which was enough for me.
I would have done a binary search, but this is fine, too.
> As you had agreed with Lionel there is no easy way or a
> intrinsic. I followed your advice, but, as Richard pointed,
> being the whole file the same length in each record,
> I had just read the 1st one, which was enough for me.
I would start with one, doubling each time until it failed,
then binary search, trying the size halfway between the last
one that worked and the last that didn't until the difference
is one.
But linear search also works. Do note that as Richard says,
you have the length that failed, not the length that worked.
> subroutine ledata
> open(unit=1,file='data.dat',status='old' & ,access='sequential',form='unformatted')
> do while(ireg<=icont)
> ireg=ireg+1
> read(1,err=100)quhes(1:ireg,1,1)
> rewind(1)
> enddo
> 100 write(*,*)'leu quhes sequential ',ireg
> end subroutine
Ron Shepard
Oct 5, 2012, 1:19:34 AM10/5/12
to
Someone commented in this thread that in order to support backspace
and the reading of partial records, the fortran i/o library must
know the lengths of unformatted records. Sometimes that information
is embedded within the file itself, and there are other ways to
store that information, but it must be stored.
Since this information is so useful to programmers for various
reasons, why hasn't there ever been a way to retrieve that
information defined by a standard? In the usual situation where the
record lengths are embedded within the file itself before and after
each record, this information could be obtained with almost no
effort.
$.02 -Ron Shepard
and the reading of partial records, the fortran i/o library must
know the lengths of unformatted records. Sometimes that information
is embedded within the file itself, and there are other ways to
store that information, but it must be stored.
Since this information is so useful to programmers for various
reasons, why hasn't there ever been a way to retrieve that
information defined by a standard? In the usual situation where the
record lengths are embedded within the file itself before and after
each record, this information could be obtained with almost no
effort.
$.02 -Ron Shepard
marcio.pmz
Oct 7, 2012, 2:50:29 PM10/7/12
to
> Also, it looks to me like you have an off-by-one error unless the code
> compensates for it elsewhere. After exiting the loop, the value of ireg
> is the one that failed rather than the last one that worked.
Surely there is this off-by-one error, which I haven't noticed and didn't know
> compensates for it elsewhere. After exiting the loop, the value of ireg
> is the one that failed rather than the last one that worked.
had a name, but I won't forget. I corrected it, thank you!
Still concern this issue of record length I am curious: what about the unit length of it? I know that it is 32 bits (word) by default, and it is supposed to work fine for integer kind 4 and real kind 4, in Intel Compiler, but if I wished to read a real kind 8 (64 bits)? Would I have to use double recl, multiply by 2? I mean, if I need to read one value from a file it would be like this?
real(8)::var
open(1,file='file.fil',access='direct',recl=2)
read(1,rec=1)var
glen herrmannsfeldt
Oct 7, 2012, 6:09:15 PM10/7/12
to
as possible to the actual case.
Though the standard requires (though implementations might not
comply) the default INTEGER, REAL, and LOGICAL to have the same
size, relying on it might cause problems. As far as I know, it
isn't usual to do alignment padding for UNFORMATTED I/O, but
you probably shouldn't count on it not being done.
Interestingly there is right now in anohter newsgroup discussion
on the RECFM=VBS data format used by OS/360 and successors,
including the current z/OS. It seems that the record length is
not allowed to be zero. Normally, that shouldn't cause any
problem, but if you tried to test for it you wouldn't be able
to distinguish 0 bytes from 1 byte. (An array of LOGICAL*1.)
I don't know that the standard prohibits any kind of padding, maybe
rounding to the next multiply of 8 or 16 bytes, or even 512 byte
block. (That, in addition to the previously mentioned lack of
requirement to report the error at all.)
I do remember using BASIC systems years ago that padded direct access
I/O to whole block boundaries. It was usual to carefully construct
records of 256 or 512 bytes.
-- glen
Richard Maine
Oct 7, 2012, 11:25:56 PM10/7/12
to
marcio.pmz <mtnobr...@gmail.com> wrote:
> > Also, it looks to me like you have an off-by-one error unless the code
> > compensates for it elsewhere. After exiting the loop, the value of ireg
> > is the one that failed rather than the last one that worked.
>
>
> Surely there is this off-by-one error, which I haven't noticed and didn't know
> had a name, but I won't forget. I corrected it, thank you!
Off-by-one errors are fairly common from many causes; that's why they
> > Also, it looks to me like you have an off-by-one error unless the code
> > compensates for it elsewhere. After exiting the loop, the value of ireg
> > is the one that failed rather than the last one that worked.
>
>
> Surely there is this off-by-one error, which I haven't noticed and didn't know
> had a name, but I won't forget. I corrected it, thank you!
have a name (not that the "name" isn't a pretty obvious one).
> Still concern this issue of record length I am curious: what about the
> unit length of it? I know that it is 32 bits (word) by default, and it
> is supposed to work fine for integer kind 4 and real kind 4, in Intel
> Compiler, but if I wished to read a real kind 8 (64 bits)? Would I have
> to use double recl, multiply by 2? I mean, if I need to read one value
> from a file it would be like this?
>
> real(8)::var
> open(1,file='file.fil',access='direct',recl=2)
> read(1,rec=1)var
that it is 32-bit words. That's what it is by default for some
compilers, but for other compilers the unit of measure is 8-bit bytes.
In fact, the f2003 standard recommends that the unit of measure be 8-bit
bytes. That is only a recommendation - not a requirement, but almost all
current compilers at least have an option to follow that recommendation.
For some compilers, it is the default, sometimes without any other
option.
The safest thing is to use the iolength option of the INQUIRE statement,
as in
inquire(iolength=whatever) var
Or substitute whatever iolist you might be using instead of var. The
record length required for that iolist will be returned in the variable
whatever, which you can then use to specify recl in an OPEN Statement.
That will work for any compiler regardless of the units of measure.
Yes, the units of measure work like any other units of measure. If you
want to read something that is twice as big as the unit of measure, you
specify the value 2. But it is safest not to assume a particular unit of
measure, particularly one that is different from the standard's
recommendation in f2003.
Phillip Helbig---undress to reply
Oct 9, 2012, 7:01:30 AM10/9/12
to
In article <k4suib$aq3$1...@speranza.aioe.org>, glen herrmannsfeldt
> the default INTEGER, REAL, and LOGICAL to have the same
> size, relying on it might cause problems.
And the reason for this is presumably so that one CAN rely on it,
assuming standard-conforming code and "processor".
<g...@ugcs.caltech.edu> writes:
> Though the standard requires (though implementations might not
> comply)
Which means that all standard-confroming implementations comply.
> Though the standard requires (though implementations might not
> comply)
> the default INTEGER, REAL, and LOGICAL to have the same
> size, relying on it might cause problems.
assuming standard-conforming code and "processor".
Clive Page
Oct 9, 2012, 7:22:45 AM10/9/12
to
the same size as far as storage is concerned - originally this was so
one could construct COMMON blocks using EQUIVALENCE, but this does not
necessarily apply as far as the effective number of bits that are used
in calculations. A LOGICAL scalar, after all, only has one bit that is
useful, the rest (often 31 of them) are normally just a waste of space.
The same storage requirements would seem to apply to data structures,
aka user-defined data types, though I think only if they contain a
SEQUENCE statement.
It's not clear to me whether the identical size requirement applies to
an I/O list - essentially that is why the INQUIRE by I/O list exists, so
one can find the effective length of the items of a given list, as far
as the current compiler is concerned. How it aligns the items, e.g.
whether each DOUBLE PRECISION real is aligned on an 8-byte boundary in
memory, is something that the programmer doesn't have to worry about any
more.
--
Clive Page
Richard Maine
Oct 9, 2012, 12:22:20 PM10/9/12
to
Clive Page <use...@page2.eu> wrote:
> On 09/10/2012 12:01, Phillip Helbig---undress to reply wrote:
> > In article<k4suib$aq3$1...@speranza.aioe.org>, glen herrmannsfeldt
> > <g...@ugcs.caltech.edu> writes:
> >
> >> the default INTEGER, REAL, and LOGICAL to have the same
> >> size, relying on it might cause problems.
> >
> > And the reason for this is presumably so that one CAN rely on it,
> > assuming standard-conforming code and "processor".
Yes. I found Glen's statement a bit confusing. I would say that you CAN
> On 09/10/2012 12:01, Phillip Helbig---undress to reply wrote:
> > In article<k4suib$aq3$1...@speranza.aioe.org>, glen herrmannsfeldt
> > <g...@ugcs.caltech.edu> writes:
> >
> >> the default INTEGER, REAL, and LOGICAL to have the same
> >> size, relying on it might cause problems.
> >
> > And the reason for this is presumably so that one CAN rely on it,
> > assuming standard-conforming code and "processor".
rely on any implementation you are likely to run into following the
standard in this regard. I recall some quite old f77 (or maybe even f66)
implementations that failed to follow the standard, at least by default.
The one I recall did have an option to follow the standard even though
that wasn't the default. But that doesn't count as a compiler that you
are likely to run into unless hearing some of us oldsters talk about it
on this forum counts as running into it.
However, you do have to be careful about where the standard's
requirement on this applies. It mostly applies "in a storage association
context." That's almost redundant in that almost anywhere that you can
directly see the effects of storage size turns out to be a storage
association context - either that or in code that is clearly nonstandard
for other reasons as well.
Looks like I/O is one area where the size can matter even though it does
not count as a storage association context. I hadn't thought of it like
that before now, but it seems to be so.
> It's not clear to me whether the identical size requirement applies to
> an I/O list - essentially that is why the INQUIRE by I/O list exists, so
> one can find the effective length of the items of a given list, as far
> as the current compiler is concerned. How it aligns the items, e.g.
> whether each DOUBLE PRECISION real is aligned on an 8-byte boundary in
> memory, is something that the programmer doesn't have to worry about any
> more.
different units of measure for the file storage unit. In fact, that's
much more the reason I've used it.
I was thinking that the standard did require data to be stored in an
unformatted file without padding, but on reading the relevant bits, I
don't see such a requirement. To the contrary, I see implications that
padding might be allowed in some cases, as you say. Good thing I didn't
post based on my guess without checking.
For unformatted stream, the standard (f2003, as that's when stream was
introduced) is pretty explicit on the subject of padding. One might
guess that this would extrapolate to unformatted record-oriented, but I
don't see that the standard requires that particular consistency. From
9.2.4 of f2003
"For a file connected for unformatted stream access, the processor
shall not have alignment restrictions that prevent a value of any
type from being stored at any positive integer file position."
Note 9.12 elaborates a bit on the implications of that requirement.
glen herrmannsfeldt
Oct 9, 2012, 1:18:11 PM10/9/12
to
Richard Maine <nos...@see.signature> wrote:
> Clive Page <use...@page2.eu> wrote:
(snip, I wrote)
without wasting 16 more bits on the INTEGER. (Then again, the 704
and successor 36 bit systems stored 16 bit signed integers in 36
bit words.)
But the reason for the statement was padding. In the Fortran 66,
and I believe Fortran 77 days, (at least for the implementations
that I knew about) padding wasn't allowed for COMMON. Mixing
DOUBLE PRECISION into a COMMON block, it wasn't hard to generate
alignment errors. Some systems would do a run-time fixup by
copying the data somewhere else, doing the operation, then copying
back.
> However, you do have to be careful about where the standard's
> requirement on this applies. It mostly applies "in a storage association
> context." That's almost redundant in that almost anywhere that you can
> directly see the effects of storage size turns out to be a storage
> association context - either that or in code that is clearly nonstandard
> for other reasons as well.
> Looks like I/O is one area where the size can matter even though it does
> not count as a storage association context. I hadn't thought of it like
> that before now, but it seems to be so.
I wasn't sure either way when I wrote that one. The systems that
I know about don't have alignment requirements for I/O, but I wouldn't
be sure that none did.
>> It's not clear to me whether the identical size requirement applies to
>> an I/O list - essentially that is why the INQUIRE by I/O list exists, so
>> one can find the effective length of the items of a given list, as far
>> as the current compiler is concerned. How it aligns the items, e.g.
>> whether each DOUBLE PRECISION real is aligned on an 8-byte boundary in
>> memory, is something that the programmer doesn't have to worry about any
>> more.
> Inquire by I/O list is also needed because of different processors using
> different units of measure for the file storage unit. In fact, that's
> much more the reason I've used it.
Yes. But even if you determine that INTEGER is four times the
size of CHARACTER*1, it might not be good to rely on that when
doing I/O. If you INQUIRE a whole I/O list, the system will include
any padding.
> I was thinking that the standard did require data to be stored in an
> unformatted file without padding, but on reading the relevant bits, I
> don't see such a requirement. To the contrary, I see implications that
> padding might be allowed in some cases, as you say. Good thing I didn't
> post based on my guess without checking.
Possibly along with the removal of the restriction on COMMON.
> For unformatted stream, the standard (f2003, as that's when stream was
> introduced) is pretty explicit on the subject of padding. One might
> guess that this would extrapolate to unformatted record-oriented, but I
> don't see that the standard requires that particular consistency. From
> 9.2.4 of f2003
In C, it isn't unusual to put data into a struct, and then use that
for the I/O, maybe even when there is no other need for the struct.
Pl/I RECORD I/O (the equivalent of UNFORMATTED) allows for only
one data item, usually a structure, per I/O operation.
(I will guess that was inherited from COBOL.)
In the C struct case, and the PL/I structure case, any data
padding will go into the I/O operation.
> "For a file connected for unformatted stream access, the processor
> shall not have alignment restrictions that prevent a value of any
> type from being stored at any positive integer file position."
But it can restrict for file position zero?
> Note 9.12 elaborates a bit on the implications of that requirement.
-- glen
> Clive Page <use...@page2.eu> wrote:
(snip, I wrote)
>> >> the default INTEGER, REAL, and LOGICAL to have the same
>> >> size, relying on it might cause problems.
>> > And the reason for this is presumably so that one CAN rely on it,
>> > assuming standard-conforming code and "processor".
> Yes. I found Glen's statement a bit confusing. I would say that you CAN
> rely on any implementation you are likely to run into following the
> standard in this regard. I recall some quite old f77 (or maybe even f66)
> implementations that failed to follow the standard, at least by default.
> The one I recall did have an option to follow the standard even though
> that wasn't the default. But that doesn't count as a compiler that you
> are likely to run into unless hearing some of us oldsters talk about it
> on this forum counts as running into it.
Yes, many 16 bit systems used 16 bit INTEGER and 32 bit REAL,
>> >> size, relying on it might cause problems.
>> > And the reason for this is presumably so that one CAN rely on it,
>> > assuming standard-conforming code and "processor".
> Yes. I found Glen's statement a bit confusing. I would say that you CAN
> rely on any implementation you are likely to run into following the
> standard in this regard. I recall some quite old f77 (or maybe even f66)
> implementations that failed to follow the standard, at least by default.
> The one I recall did have an option to follow the standard even though
> that wasn't the default. But that doesn't count as a compiler that you
> are likely to run into unless hearing some of us oldsters talk about it
> on this forum counts as running into it.
without wasting 16 more bits on the INTEGER. (Then again, the 704
and successor 36 bit systems stored 16 bit signed integers in 36
bit words.)
But the reason for the statement was padding. In the Fortran 66,
and I believe Fortran 77 days, (at least for the implementations
that I knew about) padding wasn't allowed for COMMON. Mixing
DOUBLE PRECISION into a COMMON block, it wasn't hard to generate
alignment errors. Some systems would do a run-time fixup by
copying the data somewhere else, doing the operation, then copying
back.
> However, you do have to be careful about where the standard's
> requirement on this applies. It mostly applies "in a storage association
> context." That's almost redundant in that almost anywhere that you can
> directly see the effects of storage size turns out to be a storage
> association context - either that or in code that is clearly nonstandard
> for other reasons as well.
> Looks like I/O is one area where the size can matter even though it does
> not count as a storage association context. I hadn't thought of it like
> that before now, but it seems to be so.
I know about don't have alignment requirements for I/O, but I wouldn't
be sure that none did.
>> It's not clear to me whether the identical size requirement applies to
>> an I/O list - essentially that is why the INQUIRE by I/O list exists, so
>> one can find the effective length of the items of a given list, as far
>> as the current compiler is concerned. How it aligns the items, e.g.
>> whether each DOUBLE PRECISION real is aligned on an 8-byte boundary in
>> memory, is something that the programmer doesn't have to worry about any
>> more.
> Inquire by I/O list is also needed because of different processors using
> different units of measure for the file storage unit. In fact, that's
> much more the reason I've used it.
size of CHARACTER*1, it might not be good to rely on that when
doing I/O. If you INQUIRE a whole I/O list, the system will include
any padding.
> I was thinking that the standard did require data to be stored in an
> unformatted file without padding, but on reading the relevant bits, I
> don't see such a requirement. To the contrary, I see implications that
> padding might be allowed in some cases, as you say. Good thing I didn't
> post based on my guess without checking.
> For unformatted stream, the standard (f2003, as that's when stream was
> introduced) is pretty explicit on the subject of padding. One might
> guess that this would extrapolate to unformatted record-oriented, but I
> don't see that the standard requires that particular consistency. From
> 9.2.4 of f2003
for the I/O, maybe even when there is no other need for the struct.
Pl/I RECORD I/O (the equivalent of UNFORMATTED) allows for only
one data item, usually a structure, per I/O operation.
(I will guess that was inherited from COBOL.)
In the C struct case, and the PL/I structure case, any data
padding will go into the I/O operation.
> "For a file connected for unformatted stream access, the processor
> shall not have alignment restrictions that prevent a value of any
> type from being stored at any positive integer file position."
> Note 9.12 elaborates a bit on the implications of that requirement.
Richard Maine
Oct 9, 2012, 1:53:34 PM10/9/12
to
glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
> > "For a file connected for unformatted stream access, the processor
> > shall not have alignment restrictions that prevent a value of any
> > type from being stored at any positive integer file position."
>
> But it can restrict for file position zero?
That would seem to be a bit of a moot question, as there is no file
> > "For a file connected for unformatted stream access, the processor
> > shall not have alignment restrictions that prevent a value of any
> > type from being stored at any positive integer file position."
>
> But it can restrict for file position zero?
position zero.
glen herrmannsfeldt
Oct 9, 2012, 2:44:46 PM10/9/12
to
Richard Maine <nos...@see.signature> wrote:
>> > "For a file connected for unformatted stream access, the processor
>> > shall not have alignment restrictions that prevent a value of any
>> > type from being stored at any positive integer file position."
(snip, then I wrote)
>> > "For a file connected for unformatted stream access, the processor
>> > shall not have alignment restrictions that prevent a value of any
>> > type from being stored at any positive integer file position."
>> But it can restrict for file position zero?
> That would seem to be a bit of a moot question, as there is no file
> position zero.
Oops. In C, files start at zero. I hadn't thought about that
> That would seem to be a bit of a moot question, as there is no file
> position zero.
being different in Fortran.
-- glen
0 new messages