STREAM I/O questions
John
! text-based utilities) that need stream I/O. C was the dominant
stream-I/O
! based language; so "When in Rome, do as the Romans do." seemed like
a
! good idea; so when available (as in gfortran/g95) I used the "C-
like"
! extensions:
!
! ftell, fseek, fputc, fgetc, fget, fput
! As described at
! http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gfortran/FTELL.html
!
! or called C from Fortran and make look-alikes for the routines.
There
! were a few bumps along the way (in some programming environments (in
! the past?) mixing C and Fortran I/O (heck, just mixing C and
Fortran)
! was fraught with peril.
!
! As g95(1) and other compilers have now implemented the F2003 stream
! I/O interface, I thought I could now make standard-based routines
with
! the same functionality (just using Fortran 2003; not the
ISO_C_BINDING
! interface to call C).
!
! So if I open a NAMED file with ACCESS="STREAM" it is quite straight-
forward.
! I can use reads and writes with POS= to do what FSEEK(3F) does; I
can use
! INQUIRE(UNIT=nn,POS=itell)
! itell=itell-1
! to replace FTELL(3F); and (for NAMED files that I opened) a simple
! WRITE(nn,'(a)') char --> FPUTC(3F)
! READ(nn,'(a)') char --> FGETC(3F)
!
! But I can't figure out a standard method of making the files pre-
assigned
! to standard input and standard output (and stderr and other pre-
assigned
! files too, I suppose) use stream I/O.
!
!
! Here is a simple demo. program I have to kludge to write to standard
! output, for example:
!-------------------------------------------------------------------------------
! read entire file into memory as a stream and write it in reverse
byte order
program rev_file
!
! access computing environment
USE ISO_FORTRAN_ENV, ONLY : ERROR_UNIT,INPUT_UNIT,OUTPUT_UNIT
implicit none
character(len=1024) :: filein
character(len=1024) :: fileout
character(len=1),allocatable :: text(:)
integer :: ios
integer :: iputunit=16
if(command_argument_count() < 1 .or. command_argument_count() .gt.
2 )then
call stderr('usage: rev inputfile [outputfile]')
stop
elseif(command_argument_count() .eq. 2)then
! get output filename
call get_command_argument(2,fileout)
! open named file in stream mode positioned to append
open (unit=iputunit, &
& file=fileout(:len_trim(fileout)), &
& access='stream')
iputunit=16
else
iputunit=6
endif
! call with each token from the command line
call GET_COMMAND_ARGUMENT(1, filein)
! allocate character array and copy file into it
call juslurp(filein,text)
if(command_argument_count() .eq. 1)then
! KLUDGE
! write file reversed to stdout
write(*,'(2000000000000000a:)',advance='no')text(size(text):
1:-1)
else
! write file reversed to a non-preassigned file
write(iputunit)text(size(text):1:-1)
endif
! release memory
deallocate(text)
close(16, iostat=ios)
!-------------------------------------------------------------------------------
contains
! allocate text() and read file filename into it
subroutine juslurp(filename,text)
!-------------------------------------------------------------------------------
! read an entire file into memory as a stream
! comment: never casually read an entire file into memory if you can
! process it per line or in smaller units; as large files
can
! consume unreasonable amounts of memory. What if someone
feeds
! in a file ten times bigger than you system memory, for
example (even accidently?)
!
! consider what happens on machines where more than one character is
used
! for an end-of-line or a character string is used for an internal
file
! seperator (usually for "multi-file" files, which were once
common)!
!-------------------------------------------------------------------------------
implicit none
! filename to shlep
character(len=*),intent(in) :: filename
! array to hold file
character(len=1),allocatable,intent(out) :: text(:)
integer :: nchars
! use newunit=igetunit in f08
integer,parameter :: igetunit=15
integer :: ios
! open named file in stream mode positioned to append
open (unit=igetunit, &
& file=filename(:len_trim(filename)), &
& access='stream', &
& position='append')
! assuming not open, open ready to append past end of file
! get number of bytes in file plus one
inquire(unit=igetunit,pos=nchars)
! opened for append, so subtract one to get current length
nchars=nchars-1
if(nchars.le.0)call stderr('empty file '//filename(:len_trim
(filename)))
if(allocated(text))deallocate(text)
! make enough storage to hold file
allocate ( text(nchars) )
rewind(igetunit)
! input file -> text
read(igetunit,iostat=ios) text
if(ios.ne.0)then
call stderr(' bad read of '//filename(:len_trim(filename)))
endif
close(igetunit)
return
end subroutine juslurp
end program rev_file
!-------------------------------------------------------------------------------
! how to make stdin and stdout and stderr stream files ???
! how to tell how many units can be open and what unit numbers are
OK ???
!-------------------------------------------------------------------------------
subroutine stderr(message)
! access computing environment
USE ISO_FORTRAN_ENV, ONLY : ERROR_UNIT,INPUT_UNIT,OUTPUT_UNIT
implicit none
character(len=*) :: message
write(ERROR_UNIT,'(a)')'*E-R-R-O-R:'//message
return
end
!-------------------------------------------------------------------------------
!
! I had hoped that if I closed the files assigned by ISO_FORTRAN_ENV
! to INPUT_UNIT, OUTPUT_UNIT, and ERROR_UNIT; and then opened them as
! non-scratch files but gave them a null name that they would open
assigned
! to the "stdin, stdout, stderr" files; or that if I did an OPEN on
the open
! pre-assigned files with ACCESS="STREAM" they would allow STREAM I/O
and
! so on but nothing worked. Is it an actual (and intentional?)
limitation
! of Fortran 2003 to not be able to use stream I/O on the "std*" files
or
! did I miss something? I have a lot of "filter" programs that do
stream
! I/O on stdin and stdout that I can at least change to use
ISO_C_BINDING
! and C calls instead of the hodge-podge of solutions I have now; but
until
! I tried it I has assumed that the 2003 Fortran stream I/O features
would
! allow me to write a stream-based "filter" program ??
!
! I really don't want to hear someone complain about Fortran lacking
portable
! stream I/O features again!! Any answers?
!
! Actually, an interesting exercise would be to see how many
extensions
! found at
! http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gfortran/FTELL.html
!
! get be replaced with standard-conforming Fortran 2003 itself or
it's
! ISO_C_BINDING interface to C. Anyone have any Fortran students that
need
! a homework assignment? The result might make a useful library.
Arjen Markus
> contains
> ! allocate text() and read file filename into it
> subroutine juslurp(filename,text)
> ! read an entire file into memory as a stream
> ! comment: never casually read an entire file into memory if you can
> ! process it per line or in smaller units; as large files
> can
> ! consume unreasonable amounts of memory. What if someone
> feeds
> ! in a file ten times bigger than you system memory, for
> example (even accidently?)
> !
> ! consider what happens on machines where more than one character is
> used
> ! for an end-of-line or a character string is used for an internal
> file
> ! seperator (usually for "multi-file" files, which were once
> common)!
> ! how to make stdin and stdout and stderr stream files ???
> ! how to tell how many units can be open and what unit numbers are
> OK ???
> subroutine stderr(message)
> ! access computing environment
> USE ISO_FORTRAN_ENV, ONLY : ERROR_UNIT,INPUT_UNIT,OUTPUT_UNIT
> implicit none
> character(len=*) :: message
> write(ERROR_UNIT,'(a)')'*E-R-R-O-R:'//message
> return
> end
Have you tried using non-advancing I/O instead on stdin?
(I tried the re-opening trick myself, but that led to seek errors ...
Then later I thought it might be possible to achieve what you want
using ADVANCE='NO'.)
Regards,
Arjen
Arjen Markus
>
> - Tekst uit oorspronkelijk bericht weergeven -
Here is a "simple" example:
! filter.f90 --
! Use stdin as a filter ...
!
program filter
character(len=20) :: string
integer :: ierr
integer :: count
integer :: line
logical :: add_line
! open( 5, access = 'stream', form = 'formatted', action = 'read' )
line = 1
add_line = .true.
do
read( 5, '(a)', advance = 'no', iostat = ierr, size = count )
string
if ( ierr == -1 ) exit
if ( ierr /= -2 ) then
if ( add_line ) then
add_line = .false.
write( *, '(i5,2a)', advance = 'no' ) &
line, ' >', string(1:count)
else
write( *, '(a)', advance = 'no' ) &
string(1:count)
endif
else
if ( add_line ) then
add_line = .false.
write( *, '(i5,3a)', advance = 'yes' ) &
line, ' >', string(1:count), '<'
else
write( *, '(2a)', advance = 'yes' ) string(1:count),
'<'
endif
line = line + 1
add_line = .true.
endif
enddo
end program filter
Use it for instance like this:
cat filter.f90 |filter
Regards,
Arjen
Richard Maine
> ! But I can't figure out a standard method of making the files pre-
> assigned ! to standard input and standard output (and stderr and other
> pre- assigned ! files too, I suppose) use stream I/O.
There isn't a guaranteed way. It is compiler- and system-dependent
whether you can do anything like that to standard input and output.
Might work, or might not depending on the compiler.
In principle, it is equally compiler- and system-dependent what you can
do to any file. It is just that standard input/output are more likely
than most other "random" files to have limitations. In general, I advise
against trying to reopen standard input/output.
As Arjen notes, you can use nonadvancing I/O to get some simillar
effects. It won't do all of the same things, but then some of the things
that you can't do with nonadvancing I/O are some of the things likely to
be problematic on standard input and output anyway. For example, it is
reasonably common for standard input/output to be connected to files
that cannot be positioned.
> ! how to tell how many units can be open and what unit numbers are
> OK ???
For a general answer to that question, see the INQUIRE statement. It is
fairly common to have a short routine that loops through candidate unit
numbers to find one that is vaid and unused. In practice, I find it
pretty safe today to assume that unit numbers 11 through 99 are valid. I
avoid single-digit unit numbers as they are often used for special
syste-dependent purposes, and 99 is the highest valid unit number on
some compilers (though it seems that most compilers today allow higher
ones).
The f2008 draft (last time I checked) provides a standardized way to get
an unused unit number, with the additional advantage that it gives you
one that is also guaranteed not to be used by anything else in the
future (something you can't otherwise guarantee). That makes it a bit
more like most other current languages, where you don't have to worry
about selecting a number, but just open a file and let the system assign
a suitable "handle". In my opinion, the use of numbers as unit handles
is an undesirable holdover from older days, but the proposed new feature
at least ameliorates the problems.
None of this will tell you, however, things like whether you can
successfully reopen standard input/output. It isn't specific to stream
I/O.
--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
glen herrmannsfeldt
(snip)
< ! But I can't figure out a standard method of making the
< ! files pre-assigned to standard input and standard output
< ! (and stderr and other pre-assigned files too, I suppose) use
< ! stream I/O.
C has the freopen() function, similar to fopen(), except that
it closes an existing stream and opens the new file on the
same stream. The usual use is for opening a new file as
stdin, stdout, or stderr.
As far as I know, there is no similar operation in Fortran.
-- glen
John
Thanks to everyone for the replies. When I don't have to categorize my
open files
I do use a routine that finds an unused number already( I added it to
http://fortranwiki.org/fortran/show/notopen); but I often find I need
to maintain a table with a text "tag" so I can scan for related files
(find all input files currently open, ...) so I find I usually use it
for files opened and closed in the same routine ( a help file, a
configuration file, ...). I had seen the f2008
NEWUNIT= put no compiler I tried seem to impliment it yet; I guess
that means if I open all my files that way in the future I don't need
to know the upper limit allowed
(until I hit the limit, I guess). I THINK the f1977 standard
used to say the guaranteed range was 1 to 99; I could
not find anything in the f2008 standard that mentioned any guaranteed
minimum range. freread(3c)
is interesting; somehow I missed that (and I've used
C a decent amount); I guess I am dissapointed that I
still cannot do (in a portable way) what I can do with the common
Fortran stream I/O extensions (ftell,fseek,fputc,fgetc,..). The
example using non-advancing I/O is something I'll keep; but that's
what
I was hoping to become free of ( using non-standard
extensions, calling C routines, not-so-simple uses of
non-advancing I/O and direct access files ) when I
want to make "binary" filters. Everything seems to work just fine
except with pre-connected units; but
I want to do binary streams too often to switch to the newer
ACCESS="STREAM" methods. I could already handle clear text files as
filters with f1977 for the
most part; it's the binary files or times that I want
to hold a text file in memory as a stream that I'd like to do in a
standard way. Thanks again;
John S. Urban
Richard Maine
[about valid unit numbers]
> I THINK the f1977 standard
> used to say the guaranteed range was 1 to 99;
No, it did not. No version of the Fortran standard has ever guaranteed
any particular range. And there were f77 compilers that didn't allow
that large a range, though that was a fairly common choice.
jfh
A problem with unit numbers that hit me 30 or so years ago was that a
certain commercial graphics package of Fortran subroutines used an
assortment of undocumented unit numbers for its own purposes, and I
had various problems if I tried to write to or read from one of those
units. The source code was not available. My workaround was to write a
test program to INQUIRE of each unit number 0-99 whether it was
currently open, before and after each subroutine call from the
package. (I didn't strike a case where a unit was opened and closed
during one package subroutine call.) Later, my university stopped
paying for that package, and PGPLOT became available. I still use
that.
John Harper
Ron Shepard
<01a9e5aa-5d2e-4576...@x5g2000yqk.googlegroups.com>,
jfh <john....@vuw.ac.nz> wrote:
> A problem with unit numbers that hit me 30 or so years ago was that a
> certain commercial graphics package of Fortran subroutines used an
> assortment of undocumented unit numbers for its own purposes, and I
> had various problems if I tried to write to or read from one of those
> units. The source code was not available. My workaround was to write a
> test program to INQUIRE of each unit number 0-99 whether it was
> currently open, before and after each subroutine call from the
> package.
I think a lot of software did this 30 years ago back before
open/close/inquire statements were available using preconnected
units and/or some kind of JCL incantation. Open statements were
available on, for example, some of the DEC compilers and operating
systems before f77, but not on very many machines made by many other
vendors. F77 fixed this, of course, but that would not really be
popular until the mid 80's, and even then often along with the older
version of the compiler that supported only the preconnected units.
Most software libraries I used at least documented which file unit
numbers they used, although some of them kept the filenames
undocumented (or used random character strings that changed from run
to run).
$.02 -Ron Shepard
glen herrmannsfeldt
< jfh wrote:
<> A problem with unit numbers that hit me 30 or so years ago was that a
<> certain commercial graphics package of Fortran subroutines used an
<> assortment of undocumented unit numbers for its own purposes,
< I think a lot of software did this 30 years ago back before
< open/close/inquire statements were available using preconnected
< units and/or some kind of JCL incantation.
The DEC compilers would create a file named after the unit,
like FOR001.DAT
For IBM, you needed a JCL DD statement for each unit.
If closed source software used a unit, it would have to
be documented such that the appropriate DD statement would
be supplied. The system cataloged procedures normally
supplied DD statements for 5, (SYSIN), 6 (SYSOUT=A, line printer),
and 7 (SYSOUT=B, card punch). The number of available units
is a sysgen option, likely more on larger machines.
< Open statements were
< available on, for example, some of the DEC compilers and operating
< systems before f77, but not on very many machines made by many other
< vendors. F77 fixed this, of course, but that would not really be
< popular until the mid 80's, and even then often along with the older
< version of the compiler that supported only the preconnected units.
(snip)
-- glen