reading through a directory of files with no common denominator
Luka Djigas
they share some differences, but nothing that can be relied upon.
It is some measurement data for a base of models, with file names like
m1023.dat
4665.dat
4666-1.dat
and
184.dat
And then the variations ... m1024.dat, m1025.dat and so on ...
They all however share the same format, and I'm trying to read the first line from each file to
enable the user to see them all on screen before proceeding to the choice of model.
Is there a way to "go through" the directory and get all file names, in a loop ?
I don't know if that is even the right approach ... but cannot think of anything else right now.
-- Luka
Luka Djigas
Also, let not forget, merry Easter to all who it may concern :-))
Luka
Richard Maine
> Is there a way to "go through" the directory and get all file names, in a
loop ?
Standard Fortran doesn't even have a concept of a directory, so no,
there is nothing portable.
Depending on the particular OS and compiler, there might be vendor
routines for that, but as you didn't mention a particular OS or
compiler...
One approach is to spawn a shell command to list the file names,
redirect the output of that shell command to a file, and then read that
file. Of course, the exact shell command to use depends on the OS, and
the exact way to spawn a shell command depends on the compiler vendor.
So the details are non-portable, but the general idea ports well and can
be isolated.
--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
Luka Djigas
>Luka Djigas <ldigas@___gmail___.com> wrote:
>
>> Is there a way to "go through" the directory and get all file names, in a
>loop ?
>
>Standard Fortran doesn't even have a concept of a directory, so no,
>there is nothing portable.
>
>Depending on the particular OS and compiler, there might be vendor
>routines for that, but as you didn't mention a particular OS or
>compiler...
Mea culpa. And what's worse, after all those "inquire the existence of
a directory" discussions :)
The compilers in question are either gfortran or Intel's, running on Windows.
>
>One approach is to spawn a shell command to list the file names,
>redirect the output of that shell command to a file, and then read that
>file. Of course, the exact shell command to use depends on the OS, and
>the exact way to spawn a shell command depends on the compiler vendor.
>So the details are non-portable, but the general idea ports well and can
>be isolated.
Yes, that's one approach. But the idea of reading the output of a DIR command (for example, although
I could easily use LS from unix tools as well) seem like a good way to terribly complicate your life
(all that string handling).
Do you perhaps have any advices on the handling of that output ?
I'm thinking it may be easier to maybe redirect the output of a DIR command to a file, then post
process it through some regex (just a thought), and then read that file in a fortran program.
This is as close to portable, as the moon is to an ant, but if I'm handling it "dirty" I might as
well keep as much of it from the program.
-- Luka
mecej4
For operating systems with file trees, there are a number of file-tree
"walking" routines available; most of them will be C-oriented, but can
be called from Fortran sometimes.
For Windows, for example, look up FindFirstFile and FindNextFile on
MSDN. Here is a short example from there:
#include <windows.h>
#include <stdio.h>
int main(int argc, TCHAR *argv[])
{
WIN32_FIND_DATA FindFileData;
HANDLE hFind;
if( argc != 2 ){
printf("Usage: %s [target_file_pattern]\n", argv[0]);
return;
}
printf ("Target file is %s\n", argv[1]);
hFind = FindFirstFile(argv[1], &FindFileData);
if (hFind == INVALID_HANDLE_VALUE) {
printf ("FindFirstFile failed (%d)\n", GetLastError());
return;
}
else
printf ("The first file found is %s\n",
FindFileData.cFileName);
while(FindNextFile(hFind,&FindFileData))
printf("The next file found is %s\n",FindFileData.cFileName);
printf("That's all\n");
FindClose(hFind);
}
--mecej4
Luka Djigas
>Luka Djigas <ldigas@___gmail___.com> wrote:
>
>> Is there a way to "go through" the directory and get all file names, in a
>loop ?
I forgot to mention one thing ... I don't know if it matters, but the program is in the directory
in which the files need to be processed are.
So he's not walking through a tree of dir/subdirectories.
I guess it doesn't make that much of a difference, but still, to avoid misunderstandings.
-- Luka
Sjouke Burry
Just a clean list of filenames.
Richard Maine
> Luka Djigas wrote:
> > On Sat, 3 Apr 2010 19:14:56 -0700, nos...@see.signature (Richard Maine)
wrote:
> >> One approach is to spawn a shell command to list the file names,
> >> redirect the output of that shell command to a file, and then read that
> >> file.
> > Yes, that's one approach. But the idea of reading the output of a DIR
> > command (for example, although I could easily use LS from unix tools as
> > well) seem like a good way to terribly complicate your life (all that
> > string handling).
> >
> > Do you perhaps have any advices on the handling of that output ?
> Have you ever tried: dir /b /a-d [yourdirectorylocation] ??
> Just a clean list of filenames.
Yeah. I don't know the Windows switches for DIR (even when I'm on
Windows, I usually use JPSoft's 4NT), but there is bound to be a set
that makes it pretty easy. I'm not sure what "all that string handling"
you are referring to would be. Picking out blank-delimitted fields is
pretty easy (and there is probably a set of switches where you don't
even need to do that, but as Sjouke said, has just the filenames).
jwm
With Intel (Visual) Fortran, you have at least four choices:
*Use SYSTEM/SYSTEMQQ to invoke a shell command, redirected to some
file (those functions are included in the IFPORT module)
*Use GETFILEINFOQQ. The function is also included in the IFPORT
module; you might want to take a look at the documentation, to know
how to use the FILE$FIRST, FILE$LAST and FILE$ERROR handles.
*Use the POSIX subroutines PXFOPENDIR/PXFREADDIR/PXFCLOSEDIR (included
in the IFPOSIX module)
*If on Windows, use the FindFirstFile/FindNextFile functions (which
are part of the kernel32.dll; their interfaces are included in the
KERNEL32 module)
I don't know if gfortran offers something else besides the SYSTEM
function and subroutine.
Jason Blevins
> Is there a way to "go through" the directory and get all file names,
> in a loop ?
One option, which seems to me to be more portable than using
system calls, is to use POSIX routines. This would of course
require a POSIX library on Windows, DOS, etc. such as Cygwin,
but should work very well on Unix-like systems such as Linux,
BSDs, Solaris, OS X, etc.
Posix90 provides a very complete Fortran 90 POSIX interface:
http://savannah.nongnu.org/projects/posix90/
Unfortunately the kind parameters in this interface are defined
in a non-portable way (across compilers) through setting the kind
numbers directly, rather than using the kind inquiry intrinsics,
but I would guess that it should work as is in most cases.
The documentation will be built automatically upon running
make, which builds the library and produces and HTML and PDF
documentation in the /doc subdirectory.
The routines of interest to your case are in the f90_unix_dirent
module (opendir, closedir, readdir, etc.). Here is a simple
example:
program ls
use f90_unix_dirent
implicit none
type(DIR) :: dirp
integer :: errno, name_len
character(LEN=128) :: name
call opendir('/home/jrblevin', dirp)
do
call readdir(dirp, name, name_len)
if (name_len > 0) then
print *, name(1:name_len)
else
exit
end if
end do
call closedir(dirp)
end program ls
To compile and link it (for example):
$ export POSIX90=$HOME/src/posix90/src
$ gfortran --std=f95 -I$POSIX90 -o ls ls.f90 -L$POSIX90 -lposix90
I've added this information to the wiki, feel free to edit or
expand it:
http://fortranwiki.org/fortran/show/Posix90
There is another similar library called fortranposix, which seems
less mature and less well-documented than Posix90, but might still
be useful:
http://sourceforge.net/projects/fortranposix/
--
Jason Blevins
Ph.D. Candidate, Department of Economics, Duke University
http://jblevins.org/
aerogeek
>
> - Show quoted text -
System is a standard fortran 77 command so this is supported by
gfortran...
I think i will use the system('dir /b >output.txt') command and then
read in this file and get the file names.
This is perhaps the easiest way to get your results but it might not
be a good option if you need a portable code.
Richard Maine
> System is a standard fortran 77 command
Not in any standard published to date and not with that name. There is a
related intrinsic procedure (not "command") in the draft f2008 proposal,
but it is not named "system". It is something like execute_command_line
(I might have that slightly off, but it is at least close to that).
Ragu
using ifort and Windows. It uses the vendor specific extensions that
jwm had mentioned. The following code might help you to get started. I
modified it to remove some specific code but had not compiled it. You
can easily modify it to save the file names to an array of file names
instead of printing it to listingfile.
HTH. Ragu
!-------------------------------------------------------------------------------
! This subroutine lists the files in the current directory
subroutine prepare_filelist(listingfile, wildcards)
use ifport
use ifcore
implicit none
character(len = 255), intent(in) :: listingfile ! File that lists the
filenames
character(len = 255), intent(in), optional :: wildcards
! define local variables
integer :: u_listingfile ! File unit for listingfile
character(len = 255) :: c_wildcards ! current wildcard
logical :: success
integer(kind=INT_PTR_KIND( )) fhandle
integer(4) :: flength ! per ifort GETFILEINFOQQ
type (FILE$INFO) :: info ! File info type
!TYPE FILE$INFO
! INTEGER(4) CREATION ! CREATION TIME (-1 on FAT)
! INTEGER(4) LASTWRITE ! LAST WRITE TO FILE
! INTEGER(4) LASTACCESS ! LAST ACCESS (-1 on FAT)
! INTEGER(4) LENGTH ! LENGTH OF FILE
! INTEGER(4) PERMIT ! FILE ACCESS MODE
! CHARACTER(255) NAME ! FILE NAME
!END TYPE FILE$INFO
continue
! Open the listing file
u_listingfile = 20 ! unit number
open(unit = u_listingfile, file = TRIM(listingfile), status =
'replace')
! set the wildcards
if(.NOT. present(wildcards)) then
c_wildcards = '*' ! wildcard
else
c_wildcards = wildcards
endif
fhandle = FILE$FIRST
! Get all the files in the directory
! Omit 1) Hidden Files,
! 2) System Files,
! 3) ReadOnly Files,
! 4) Directory names,
! 5) System (used by OS / system), and
! 6) File Volume (logical volume, or partition, on a physical disk
drive)
! This type of file appears only in the root directory of a
physical device.
fileloop: do while (.TRUE.)
flength = GETFILEINFOQQ(c_wildcards, info, fhandle)
if ((fhandle .EQ. FILE$LAST) .OR. &
(fhandle .EQ. FILE$ERROR)) then
select case (GETLASTERRORQQ())
case (ERR$NOMEM)
write(u_stdout,*) 'Out of memory'
case (ERR$NOENT)
exit
case default
write(u_stdout,*) 'Invalid file or path name'
end select
end if
! Hidden File
if ((info%permit .AND. FILE$HIDDEN) .NE. 0) cycle fileloop
! System File
if ((info%permit .AND. FILE$SYSTEM) .NE. 0) cycle fileloop
! Read Only File
if ((info%permit .AND. FILE$READONLY) .NE. 0) cycle fileloop
! Directory
if ((info%permit .AND. FILE$DIR) .NE. 0) cycle fileloop
! System (Used by the operating system)
if ((info%permit .AND. FILE$SYSTEM) .NE. 0) cycle fileloop
! A logical volume, or partition, on a physical disk drive.
! This type of file appears only in the root directory of a
physical device.
if ((info%permit .AND. FILE$VOLUME) .NE. 0) cycle fileloop
!! All other files are files that we are interested in
!! This is a normal file. This is different from FILE$NORMAL, but
may include FILE$NORMAL
!!if ((info%permit .AND. FILE$ARCHIVE) .NE. 0) write(u_stdout,*)
'Normal File'
! write the filename to the file
write(u_listingfile,*) TRIM(info%name)
end do fileloop
! Close the file and free lun
close(u_listingfile)
end subroutine prepare_filelist
!-------------------------------------------------------------------------------
Terence
Just arrange for a pre-program execution batch file to enquire for a
list of the existence of the files you need, with the output sent to
another file with a very different name (or "type"). Where possible,
for any O/S to be used, try to get your or the user's file list to
have only what you need, in the best approximation to a common name
and date field size and order model.
This batch file can of course then call your execution program fo rthe
general case of any user.
Now your program only has to read this list, expecting a very small
number of format possibilities (with luck you need just the file name
and possibly a date).
Process the first record, determie the format, rewind file and read
the list in that established format.
frank
A very interesting read, Jason. I can't wait to be home again to fire
up some of this posix fortran.
With the Fortran 90 Interface Project, how is it not GNU?
--
and congrats for another Duke win
must be tough to be a hoops fan in Durham
Jason Blevins
> On Apr 4, 9:54 am, Jason Blevins <jrbl...@sdf.lonestar.org> wrote:
>> On 2010-04-04, Luka Djigas <ldigas@___gmail___.com> wrote:
>>
>> > Is there a way to "go through" the directory and get all file names,
>> > in a loop ?
>>
>> One option, which seems to me to be more portable than using
>> system calls, is to use POSIX routines. This would of course
>> require a POSIX library on Windows, DOS, etc. such as Cygwin,
>> but should work very well on Unix-like systems such as Linux,
>> BSDs, Solaris, OS X, etc.
>>
>> Posix90 provides a very complete Fortran 90 POSIX interface:
>>
>> http://savannah.nongnu.org/projects/posix90/
>>
>
> A very interesting read, Jason. I can't wait to be home again to fire
> up some of this posix fortran.
>
> With the Fortran 90 Interface Project, how is it not GNU?
Although it is GPL-licensed, to become an official GNU project I think
the author has to submit for review and turn over copyright to the FSF.
Either way, unfortunately it appears to have been abandoned: there is
a statement on the website about some features not being complete, and
the last commit was almost four years ago.
> and congrats for another Duke win
> must be tough to be a hoops fan in Durham
Yes, it's been hard to find a good basketball game in these
parts for the past few years!