Issue with Open(unit,STATUS='SCRATCH', iostat = ios)
cjps...@gmail.com
assignment of I/O fortran units.
I have extracted the part of source which reproduce the bug and added
write statement to see what is done at each step.
I use gfortran 4.5 from mingw under Windows Vista :
$ gfortran -v
Utilisation des specs internes.
COLLECT_GCC=c:\mingw\bin\gfortran.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/4.5.0/lto-
wrapper.exe
Target: mingw32
Configuré avec: ../gcc-4.5.0/configure --enable-languages=c,c+
+,ada,fortran,objc,obj-c++ --disable-sjlj-exceptions --with-dwarf2 --
enable-shared --enable-libgomp --disable-win32-registry --enable-libs
tdcxx-debug --enable-version-specific-runtime-libs --disable-werror --
build=mingw32 --prefix=/mingw
Modèle de thread: win32
gcc version 4.5.0 (GCC)
The code try in a loop to reserve free units for later use.
When I execute the code I get a Segmentation fault at some random
varying indice of the loop in the range of 1 to 27 numbers of units.
I it is not the unit value witch causes the bug, it's just the number
of reserved units or something else.
The unit reservation is done by :
open(unit,STATUS='SCRATCH', iostat = ios)
and it is that statement witch crash the program.
With the following compilation options (static fortran library)
gfortran -funderscoring -O0 -g3 -Wall -c -fmessage-length=0
-o"bug_assign.o" "../bug_assign.f90"
gfortran -funderscoring -O0 -g3 -Wall -c -fmessage-length=0
-o"bugs.o" "../bugs.f90"
gfortran -static-libgfortran -o"bugs" ./bug_assign.o ./bugs.o
The Backtrace done with gbd execution is :
Program received signal SIGSEGV, Segmentation fault.
0x0040fb69 in _gfortrani_fstrlen ()
(gdb) backtrace
#0 0x0040fb69 in _gfortrani_fstrlen ()
#1 0x004156d2 in _gfortrani_unpack_filename ()
#2 0x00411ddf in _gfortrani_new_unit ()
#3 0x0040d0df in _gfortran_st_open ()
#4 0x0040155a in assignunit (unit=36, codeer=0) at ../
bug_assign.f90:22
#5 0x00401807 in bugs () at ../bugs.f90:7
#6 0x00401962 in main (argc=1, argv=0x972b11 "c\\bugs\\debug/
bugs.exe") at ../bugs.f90:15
#7 0x004010db in __mingw_CRTStartup ()
#8 0x00401178 in mainCRTStartup ()
(gdb)
With the following compilation options (shared fortran library)
gfortran -funderscoring -O0 -g3 -Wall -c -fmessage-length=0
-o"bug_assign.o" "../bug_assign.f90"
gfortran -funderscoring -O0 -g3 -Wall -c -fmessage-length=0
-o"bugs.o" "../bugs.f90"
gfortran -o"bugs" ./bug_assign.o ./bugs.o
The Backtrace done with gbd execution is :
Program received signal SIGSEGV, Segmentation fault.
0x6f679b1d in libgfortran-3!_gfortran_unpack1_char4 () from c:\mingw
\bin\libgfortran-3.dll
(gdb) backtrace
#0 0x6f679b1d in libgfortran-3!_gfortran_unpack1_char4 () from c:
\mingw\bin\libgfortran-3.dll
#1 0x6f68ddce in libgfortran-3!_gfortran_unpack1_char4 () from c:
\mingw\bin\libgfortran-3.dll
#2 0x6f67d9a7 in libgfortran-3!_gfortran_unpack1_char4 () from c:
\mingw\bin\libgfortran-3.dll
#3 0x6f66f707 in libgfortran-3!_gfortran_st_open () from c:\mingw\bin
\libgfortran-3.dll
#4 0x0040155a in assignunit (unit=36, codeer=0) at ../
bug_assign.f90:22
#5 0x00401807 in bugs () at ../bugs.f90:7
#6 0x00401962 in main (argc=1, argv=0x8f3109 "c\\bugs\\debug/
bugs.exe") at ../bugs.f90:15
#7 0x004010db in __mingw_CRTStartup ()
#8 0x00401178 in mainCRTStartup ()
(gdb)
And more than that I have a runtime warning at eatch open :
warning: Invalid parameter passed to C runtime function.
Here are the source code files :
file bugs.f90 :
program bugs
implicit none
integer :: i, unit, codeEr
do i = 1, 100 ! the value 100 is for the test
call AssignUnit(unit,codeEr)
if (codeEr == 0) then
write(*,*) "Assigned unit = ", unit
else
write(*,*) "Assign Error= ", codeEr
end if
end do
end program bugs
file bug_assign.f90 :
subroutine AssignUnit(unit,codeEr)
implicit none
integer, intent(out) :: codeEr
integer, intent(out) :: unit
integer :: ios
integer, parameter :: unit_mini = 10
integer, parameter :: unit_maxi = 110
logical :: Is_Connected
codeEr = 0
unit = unit_mini
write(*,*) "Begin assignment unit"
do while((Is_Connected(unit)).AND.(unit .LT. unit_maxi))
unit = unit + 1
end Do
if(unit.GT.unit_maxi)Then
codeEr = 1
else
write(*,*) " ** try to connect chosen unit = ", unit
Open(unit,STATUS='SCRATCH', iostat = ios)
if (ios == 0) then
write(*,*) " ** chosen unit ", unit, " connected"
else
write(*,*) " ** chosen unit ", unit, " connection error =
", ios
end if
endif
write(*,*) "End assignment unit"
write(*,*) "----------------------------"
end Subroutine AssignUnit
function Is_Connected(unit) Result(connected)
integer, intent(in) :: unit
logical :: connected
inquire(UNIT = unit, OPENED = connected)
end Function Is_Connected
Here is one run of the program :
(gdb) run
Starting program: c:\bugs\debug/bugs.exe
[New Thread 1964.0x9c8]
Begin assignment unit
** try to connect chosen unit = 10
warning: Invalid parameter passed to C runtime function.
** chosen unit 10 connected
End assignment unit
----------------------------
Assigned unit = 10
Begin assignment unit
** try to connect chosen unit = 11
warning: Invalid parameter passed to C runtime function.
** chosen unit 11 connected
End assignment unit
----------------------------
Assigned unit = 11
Begin assignment unit
** try to connect chosen unit = 12
warning: Invalid parameter passed to C runtime function.
** chosen unit 12 connected
End assignment unit
----------------------------
Assigned unit = 12
Begin assignment unit
** try to connect chosen unit = 13
warning: Invalid parameter passed to C runtime function.
** chosen unit 13 connected
End assignment unit
----------------------------
Assigned unit = 13
Begin assignment unit
** try to connect chosen unit = 14
warning: Invalid parameter passed to C runtime function.
** chosen unit 14 connected
End assignment unit
----------------------------
Assigned unit = 14
Begin assignment unit
** try to connect chosen unit = 15
warning: Invalid parameter passed to C runtime function.
** chosen unit 15 connected
End assignment unit
----------------------------
Assigned unit = 15
Begin assignment unit
** try to connect chosen unit = 16
warning: Invalid parameter passed to C runtime function.
** chosen unit 16 connected
End assignment unit
----------------------------
Assigned unit = 16
Begin assignment unit
** try to connect chosen unit = 17
warning: Invalid parameter passed to C runtime function.
** chosen unit 17 connected
End assignment unit
----------------------------
Assigned unit = 17
Begin assignment unit
** try to connect chosen unit = 18
warning: Invalid parameter passed to C runtime function.
** chosen unit 18 connected
End assignment unit
----------------------------
Assigned unit = 18
Begin assignment unit
** try to connect chosen unit = 19
warning: Invalid parameter passed to C runtime function.
** chosen unit 19 connected
End assignment unit
----------------------------
Assigned unit = 19
Begin assignment unit
** try to connect chosen unit = 20
warning: Invalid parameter passed to C runtime function.
** chosen unit 20 connected
End assignment unit
----------------------------
Assigned unit = 20
Begin assignment unit
** try to connect chosen unit = 21
warning: Invalid parameter passed to C runtime function.
** chosen unit 21 connected
End assignment unit
----------------------------
Assigned unit = 21
Begin assignment unit
** try to connect chosen unit = 22
warning: Invalid parameter passed to C runtime function.
** chosen unit 22 connected
End assignment unit
----------------------------
Assigned unit = 22
Begin assignment unit
** try to connect chosen unit = 23
warning: Invalid parameter passed to C runtime function.
** chosen unit 23 connected
End assignment unit
----------------------------
Assigned unit = 23
Begin assignment unit
** try to connect chosen unit = 24
warning: Invalid parameter passed to C runtime function.
** chosen unit 24 connected
End assignment unit
----------------------------
Assigned unit = 24
Begin assignment unit
** try to connect chosen unit = 25
warning: Invalid parameter passed to C runtime function.
** chosen unit 25 connected
End assignment unit
----------------------------
Assigned unit = 25
Begin assignment unit
** try to connect chosen unit = 26
warning: Invalid parameter passed to C runtime function.
** chosen unit 26 connected
End assignment unit
----------------------------
Assigned unit = 26
Begin assignment unit
** try to connect chosen unit = 27
warning: Invalid parameter passed to C runtime function.
** chosen unit 27 connected
End assignment unit
----------------------------
Assigned unit = 27
Begin assignment unit
** try to connect chosen unit = 28
warning: Invalid parameter passed to C runtime function.
** chosen unit 28 connected
End assignment unit
----------------------------
Assigned unit = 28
Begin assignment unit
** try to connect chosen unit = 29
warning: Invalid parameter passed to C runtime function.
** chosen unit 29 connected
End assignment unit
----------------------------
Assigned unit = 29
Begin assignment unit
** try to connect chosen unit = 30
warning: Invalid parameter passed to C runtime function.
** chosen unit 30 connected
End assignment unit
----------------------------
Assigned unit = 30
Begin assignment unit
** try to connect chosen unit = 31
warning: Invalid parameter passed to C runtime function.
** chosen unit 31 connected
End assignment unit
----------------------------
Assigned unit = 31
Begin assignment unit
** try to connect chosen unit = 32
warning: Invalid parameter passed to C runtime function.
** chosen unit 32 connected
End assignment unit
----------------------------
Assigned unit = 32
Begin assignment unit
** try to connect chosen unit = 33
warning: Invalid parameter passed to C runtime function.
** chosen unit 33 connected
End assignment unit
----------------------------
Assigned unit = 33
Begin assignment unit
** try to connect chosen unit = 34
warning: Invalid parameter passed to C runtime function.
** chosen unit 34 connected
End assignment unit
----------------------------
Assigned unit = 34
Begin assignment unit
** try to connect chosen unit = 35
warning: Invalid parameter passed to C runtime function.
** chosen unit 35 connected
End assignment unit
----------------------------
Assigned unit = 35
Begin assignment unit
** try to connect chosen unit = 36
Program received signal SIGSEGV, Segmentation fault.
Is somebody at comp.lang.fortran got the same behavior ?
I have tested gfortran 4.5, 4.5.2 and 4.6 from www.equation.com with
the same result.
May be I am missing something but I tend to think that's a compiler
bug ???
Sorry for the message length :-(
James Van Buskirk
news:af3f9b3b-18aa-435a...@p38g2000vbn.googlegroups.com...
> Program received signal SIGSEGV, Segmentation fault.
> Is somebody at comp.lang.fortran got the same behavior ?
> I have tested gfortran 4.5, 4.5.2 and 4.6 from www.equation.com with
> the same result.
> May be I am missing something but I tend to think that's a compiler
> bug ???
You do have one coding error. Change from:
do while((Is_Connected(unit)).AND.(unit .LT. unit_maxi))
to:
do while((Is_Connected(unit)).AND.(unit .LE. unit_maxi))
but this doesn't seem to be the root cause of your problems.
For me, the program experiences difficulties at unit = 36.
The 32-bit version from www.equation.com crashes there and
the 64-bit version keeps connecting unit=36 until the outer
loop is satisfied. If I start at unit=20 instead of unit=10,
the offending unit advances to unit=46, so I think you may
be hitting some kind of upper limit at 26 open scratch files.
--
write(*,*) transfer((/17.392111325966148d0,6.5794487871554595D-85, &
6.0134700243160014d-154/),(/'x'/)); end
cjps...@gmail.com
> <cjpsi...@gmail.com> wrote in message
>
> news:af3f9b3b-18aa-435a...@p38g2000vbn.googlegroups.com...
>
> > Program received signal SIGSEGV, Segmentation fault.
> > Is somebody at comp.lang.fortran got the same behavior ?
> > I have tested gfortran 4.5, 4.5.2 and 4.6 fromwww.equation.comwith
> > the same result.
> > May be I am missing something but I tend to think that's a compiler
> > bug ???
>
> You do have one coding error. Change from:
>
> do while((Is_Connected(unit)).AND.(unit .LT. unit_maxi))
>
> to:
>
> do while((Is_Connected(unit)).AND.(unit .LE. unit_maxi))
>
> but this doesn't seem to be the root cause of your problems.
> For me, the program experiences difficulties at unit = 36.
> the 64-bit version keeps connecting unit=36 until the outer
> loop is satisfied. If I start at unit=20 instead of unit=10,
> the offending unit advances to unit=46, so I think you may
> be hitting some kind of upper limit at 26 open scratch files.
>
Thanks for your answer.
But, if I launch multiples runs I get crash at different units,
sometime at 10, 21, 31 or 36. I don't think that an upper limit
could have such a behavior :-(
And with gbd run I get the message :
warning: Invalid parameter passed to C runtime function.
at each open.
Claude Simon
Tobias Burnus
> In a software I am porting from DVF to GCC I have a problem with
> assignment of I/O fortran units.
>
> I have extracted the part of source which reproduce the bug and added
> write statement to see what is done at each step.
>
> I use gfortran 4.5 from mingw under Windows Vista :
>
>
> When I execute the code I get a Segmentation fault at some random
> varying indice of the loop in the range of 1 to 27 numbers of units.
I really wonder what the problem on your side is.
One possible issue could be that you try to open more files that Windows
allows. At least on Linux, the program crashes as soon as I reach the
number of maximally allowed open files, which is by default on my system:
$ ulimit -a|grep open
open files (-n) 1024
However, the number should be much larger than the ~30 files you see.
According to Microsoft's MSDN the number should be rather 512 or 2048,
cf. http://msdn.microsoft.com/en-us/library/6e3b887c%28VS.71%29.aspx
In any case, if gfortran cannot open - for whatever reason - a file, it
returns with an error message. In case of scratch files, the problem is
that the file name is not set - which causes a segmentation fault.
Thus, there seem to be two issue:
a) Why does it fail that early for you?
b) That gfortran segfaults when opening a scratch file fails.
The second one is now tracked at
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47296
Regarding the first one: The segmentation fault probably means that for
some reason (maximal number of open files, no space on the disk, wrong
permissions, ...) creating and opening the scratch file failed. I have
no idea why this happens that already after very few files (but not for
the first file) - and I have no idea how one can best debug it. Maybe a
better error message is printed when the gfortran bug is fixed - which
should not take more than a couple of days.
Tobias
cjps...@gmail.com
> I really wonder what the problem on your side is.
>
> One possible issue could be that you try to open more files that Windows
> allows. At least on Linux, the program crashes as soon as I reach the
> number of maximally allowed open files, which is by default on my system:
>
> $ ulimit -a|grep open
> open files (-n) 1024
>
> However, the number should be much larger than the ~30 files you see.
> According to Microsoft's MSDN the number should be rather 512 or 2048,
> cf.http://msdn.microsoft.com/en-us/library/6e3b887c%28VS.71%29.aspx
>
> In any case, if gfortran cannot open - for whatever reason - a file, it
> returns with an error message. In case of scratch files, the problem is
> that the file name is not set - which causes a segmentation fault.
>
> Thus, there seem to be two issue:
> a) Why does it fail that early for you?
> b) That gfortran segfaults when opening a scratch file fails.
>
> The second one is now tracked athttp://gcc.gnu.org/bugzilla/show_bug.cgi?id=47296
>
> Regarding the first one: The segmentation fault probably means that for
> some reason (maximal number of open files, no space on the disk, wrong
> permissions, ...) creating and opening the scratch file failed. I have
> no idea why this happens that already after very few files (but not for
> the first file) - and I have no idea how one can best debug it. Maybe a
> better error message is printed when the gfortran bug is fixed - which
> should not take more than a couple of days.
>
> Tobias
The fact that the program crashes randomly is a malfunction signal.
I don't know where (system or compiler) but it is.
The fact that there is no runtime error message when the files number
is
reached is another issue, not so bad if the limit is over 20 opened
files,
the crach random nature of is blocking my work
May be the two facts have the same cause. May be a variable not
assigned???
I am stuck with that. May be I must continue my work on Linux. I have
to
do that anyway but I wanted after getting done the job on Windows.
Thanks to your answer.
Claude Simon
Jerry DeLisle
> On 14 jan, 21:07, Tobias Burnus<bur...@net-b.de> wrote:
>
>> I really wonder what the problem on your side is.
>>
>> One possible issue could be that you try to open more files that Windows
>> allows. At least on Linux, the program crashes as soon as I reach the
>> number of maximally allowed open files, which is by default on my system:
>>
>> $ ulimit -a|grep open
>> open files (-n) 1024
>>
>> However, the number should be much larger than the ~30 files you see.
>> According to Microsoft's MSDN the number should be rather 512 or 2048,
>> cf.http://msdn.microsoft.com/en-us/library/6e3b887c%28VS.71%29.aspx
>>
>> In any case, if gfortran cannot open - for whatever reason - a file, it
>> returns with an error message. In case of scratch files, the problem is
>> that the file name is not set - which causes a segmentation fault.
>>
>> Thus, there seem to be two issue:
>> a) Why does it fail that early for you?
>> b) That gfortran segfaults when opening a scratch file fails.
>>
The issue of irregular "early" failure on windows may be related to running out
of available handles that may not have been "returned" to the OS after the
segfault. I am only speculating of course and will try to do some more testing.
Regards,
Jerry
cjps...@gmail.com
Hi Jerry,
Thanks for the correction.
I just downloaded gfortran new version from www.equation.com :
gcc-4.6-20110122-32.exe
and run my test program with a modified "open statement" :
Open(unit,STATUS='SCRATCH', iomsg = msg, iostat = ios)
----------------
with a modified "write statement" when ios is not equal to 0 :
write(*,*) " ** chosen unit ", unit, " connection error =
", ios, "msg = ", msg
-----------------
I see three things :
First, the test program don't crashes anymore.
Second, after that 26 "open statement" succeeded, an error message for
each next "open statement" :
** chosen unit 36 connection error = 17 msg = File 'C:\...\Temp
\gfortrantmpz04448' already exists
Third, after the test program terminates, when I go to the temp
directory I see that all the scratch files from gfortrantmpa04448 to
gfortrantmpz04448 are not deleted.
When I repeat the run, others scratch files are created and not
deleted.
I have three questions :
1) the number of scratch file is it voluntarily limited to 26 ? and
why ?
2) is that scratch files must be deleted by the runtime at program
termination ?
The program I am working on is a big one (400000 sloc) and I can't see
if it needs
more than 26 scratch files but I think I can work with that new
version of gfortran.
I will try this week.
Thanks.
Claude SIMON
JB
No idea. Gfortran itself has no limits, per se, so this seems to be
some kind of system limit. The name clashes above might suggest that
the equation.com provided mkstemp()/mktemp() function is spectacularly
bad.
> 2) is that scratch files must be deleted by the runtime at program
> termination ?
Windows AFAIK does not support the Unix-style delete on last close
semantics, so in this case cleanup must be done on file close; if the
program is aborted or crashes the file closing procedures are not
executed. On Unix platforms this is no problem since in that case
Gfortran relies on the delete on last close semantics and accordingly
deletes the scratch files immediately after creating them.
--
JB
Tobias Burnus
> First, the test program don't crashes anymore.
That's due to the bug fix :-)
> Second, after that 26 "open statement" succeeded, an error message for
> each next "open statement" :
>
> ** chosen unit 36 connection error = 17 msg = File 'C:\...\Temp
> \gfortrantmpz04448' already exists
>
> Third, after the test program terminates, when I go to the temp
> directory I see that all the scratch files from gfortrantmpa04448 to
> gfortrantmpz04448 are not deleted.
I found the reason for the second problem - thanks to the information
for the third. The algorithm used by Microsoft to generate temporary
files allows only 26 files ("a" to "z") - or at least only 26 files
which exist at any given time.
From: http://msdn.microsoft.com/en-us/library/34wc6k1f%28v=VS.80%29.aspx
"_mktemp preserves base and replaces the first trailing X with an
alphabetic character. _mktemp replaces the following trailing X's with a
five-digit value; this value is a unique number identifying the calling
process, or in multithreaded programs, the calling thread."
> 1) the number of scratch file is it voluntarily limited to 26 ? and
> why ?
First question: Yes. Second question: No idea - ask Microsoft. Educated
guess: One never needs more than 26 scratch files (really!) and the
implementation is easy as the one can fit the 5-digit process numbers
into the remaining space. (POSIX has basename + 6 "X", where the X are
replaced mktemp.)
I think gfortran has to work around this stupid implementation choice.
> 2) is that scratch files must be deleted by the runtime at program
> termination ?
Yes - and in principle that should happen. On Unix it is done by
immediately unlinking the file, which means that the file is gone as
soon as the file is closed or the program ends or crashes.
I think on Windows that is not possible; on those systems it is only
removed (unlinked) when CLOSE is explicitly called - or when the program
properly ends via the cleanup function of the library.
The cleanup function should be called automatically, unless the program
crashes. Thus, I do not understand why it is not happing. As I just
learnt: Newer Windows allow to open files with FILE_FLAG_DELETE_ON_CLOSE
- thus gfortran could be modified to open scratch files such that they
are also deleted when the program crashes.
In any case: One should investigate why the clean up function does not
work if the program ends (w/o crashing).
> The program I am working on is a big one (400000 sloc) and I can't see
> if it needs more than 26 scratch files
I would expect that it does not need more than 26 *simultaneously*
opened scratch files (per process) - but of course, I could be wrong.
Tobias
PS: I filled a gfortran PR to track the issues, cf.
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47439
cjps...@gmail.com
> On 01/24/2011 03:14 PM, cjpsi...@gmail.com wrote:
> > Third, after the test program terminates, when I go to the temp
> > directory I see that all the scratch files from gfortrantmpa04448 to
> > gfortrantmpz04448 are not deleted.
>
> I found the reason for the second problem - thanks to the information
> for the third. The algorithm used by Microsoft to generate temporary
> files allows only 26 files ("a" to "z") - or at least only 26 files
> which exist at any given time.
>
> From:http://msdn.microsoft.com/en-us/library/34wc6k1f%28v=VS.80%29.aspx
>
> "_mktemp preserves base and replaces the first trailing X with an
> alphabetic character. _mktemp replaces the following trailing X's with a
> five-digit value; this value is a unique number identifying the calling
> process, or in multithreaded programs, the calling thread."
>
Yes, what I saw is that the five next characters of the file name are
constants over one run of my test program.
The program I am working on opens a scratch file to reserve an unique
unit number and at some points opens (reopens) one or more files with
file names
on that one or more unit numbers.
It is doing this for several unit numbers before doing allocations
with
file names. And this in several places in the code.
At some point I was thinking of another method of reservation but I
have no idea of the impact of such a modification on the general
behavior of the program.
I never saw a program with such a method of unit number allocation.
May be the use of newunit in open statement is a possible
workaround.
I don't know. I have to investigate more before taking such a
decision.
But if the program needs less than 26 scratch file I have no need to
make such a modification...
Any idea is wellcome.
Thank you for you answers.
Claude Simon