gfortran write performance

[tho...@antispam.ham]

I have a program that was originally developed on a 32-bit Windows XP
machine using CVF 6.6c. Fairly simple. Works fine.

More recently, I ported that program to 64-bit Windows 7 using
gfortran 4.6.1 (the TDM installation package). If I remember
correctly, the only change necessary to the source code was
replacing all instances of FORM='BINARY' to ACCESS='STREAM' in
the various OPEN statements. Still works fine.

Yesterday, it became necessary to run that program on a
different 32-bit Windows XP machine. I installed the 32-bit
version of TDM's gfortran 4.6.1 package and compiled the more
recent version of the source code there. It compiled properly
without any changes to the source code. However, when running
this executable, writes to disk using stream I/O are PAINFULLY
slow. Write speed was less than 300 kilobytes per second. It
took over two HOURS to create 12 files of 177 MB size. It's
not a problem with the disk. Minutes earlier, Firefox had
written 8 GB worth of downloaded files as fast as the ethernet
transfer speeds would allow (multiple streams at 5 MB/sec).
The disk has plenty of space available.

Now that I'm aware of it, slow write performance is being seen
in all other programs that I built using gfortran on 32-bit
Windows XP. It's only the programs that write big files that
are noticeably slow, however. Those that write on a few kilobytes
don't write enough to notice the speed.

Any ideas what could be causing such slow write performance?

[Gordon Sande]

There is often a major performance difference between


real :: x(100000)

write ( 11 ) ( x(i), i = 1, 100000 )

and just

write ( 11 ) x

as the intereaction with the runtime library is very different
on most systems. Rather different overhead on filling the
final buffer is quite common.

The same sort of issues could be going on internally in Win/XP
vrs Win/7 as well as in the Fortran runtime. It could be triggered
by differing block sizes on the two systems. Have you checked the
phase of the moon lately? ;-)

[Tobias Burnus]

> Yesterday, it became necessary to run that program on a
> different 32-bit Windows XP machine. I installed the 32-bit
> version of TDM's gfortran 4.6.1 package and compiled the more
> recent version of the source code there. It compiled properly
> without any changes to the source code. However, when running
> this executable, writes to disk using stream I/O are PAINFULLY
> slow. Write speed was less than 300 kilobytes per second. It
> took over two HOURS to create 12 files of 177 MB size. It's
> not a problem with the disk. Minutes earlier, Firefox had
> written 8 GB worth of downloaded files as fast as the ethernet
> transfer speeds would allow (multiple streams at 5 MB/sec).
> The disk has plenty of space available.

I think you run into the bug
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=50016
which affects MinGW only (32 and 64 bit). Affected is GCC 4.6.0 and
4.6.1; in GCC 4.6.2 the problem is mitigated and in 4.7 the performance
problem is solved again.


Background: Many operating systems buffer I/O. While under Linux (at
least for local files), the buffering is across all processes, under
Windows, this data becomes only available to other processes when
closing the file - or when explicitly calling _commit.

Calling _commit (or on Unix/Linux systems: fsync) is an expensive
operation: One tells the operating system to ensure that the data is
written to the disk. For 4.6 a patch was added, which called _commit for
every Fortran buffer flush - and the Fortran buffer gets flushed a lot.

For 4.6.2 (just before the release), the issue was mitigated by only
calling _commit for FLUSH (and for INQUIRE). Additionally,
INQUIRE(unit=...) now uses the internally known file length instead of
asking the operating system via "stat". The latter returned the wrong
length if the file hadn't been either "_commit"ted or closed after the
last write.

For 4.7, the _commit call was also removed for FLUSH. Now, one can run
into inconsistencies on Windows, but as users can call _commit or fsync
themselves, that seemed to be the best solution.

See also
http://gcc.gnu.org/onlinedocs/gfortran/Data-consistency-and-durability.html


Sorry for the breakage.


The best way is to either upgrade to GCC 4.6.1, GCC 4.7.1 or the current
developer version 4.8 (or to downgrade to GCC 4.5.x).

See http://gcc.gnu.org/wiki/GFortranBinaries for some options you have.

Tobias

[Terence]

I've written a lot (certainly over 300 in commercial use; perhaps double
that) of Fortran programs using FORM='BINARY' in the open statement, in both
MS F77 and CVF/DVF F90/95 compilers.

I never saw a problem with anything in the file area. Stream methods ae
superb for parsing text and outputting pseudo code.

However, running some of my fortran programs on a computer using Microsoft's
XP o/s , I found really bad problems with writing anything to the screen.
The last screen line was always cut at the halfway point, finished if more
lines were output. Some of the tested screen control methods seemed to also
give problems. The reputation of XP among users and aquaintences became one
of 'Don't touch it! Go back to NT or wait for the next o/s!' (Not a good
idea either- that next one was Vista!

But now with Version 7, again I see no problems and don't hear on any from
clients.
I'm happy with Windows 2000 professional and NT and Version 7 . (Oh , and
MSDOS :o)> ).

So, I suggest, don't use XP (or Vista). I 've also suspected XP and I know
Vista was a disaster.

[Louisa]

On Aug 30, 3:21 pm, "Terence" <tbwri...@bigpond.net.au> wrote:

> I've written a lot (certainly over 300 in commercial use; perhaps double
> that) of Fortran programs using FORM='BINARY' in the open statement, in both
> MS F77 and CVF/DVF F90/95 compilers.
>
> I never saw a problem with anything in the file area. Stream methods ae
> superb for parsing text and outputting pseudo code.
>
> However, running some of my fortran programs on a computer using Microsoft's
> XP o/s , I found really bad problems with writing anything to the screen.
> The last screen line was always cut at the halfway point, finished if more
> lines were output. Some of the tested screen control methods seemed to also
> give problems. The reputation of XP among users and aquaintences became one
> of 'Don't touch it! Go back to NT or wait for the next o/s!' (Not a good
> idea either- that next one was Vista!
>
> But now with Version 7, again I see no problems and don't hear on any from
> clients.
> I'm happy with Windows 2000 professional and NT and Version 7 . (Oh , and
> MSDOS  :o)>  ).
>
> So, I suggest, don't use XP (or Vista).

No problem winning Fortran on XP.

[tho...@antispam.ham]

To the high-level Fortran programmer, the two constructs would
produce identical results, so one might expect that an optimizer
might generate identical low-level code for both cases. If it
doesn't, however, it's not immediately obvious which version
would be more efficient. The latter looks simpler, but if it
breaks the array up into manageable chunks and flushes the buffer
every now and then, the code might be just as complex as the former
version.

> The same sort of issues could be going on internally in Win/XP
> vrs Win/7 as well as in the Fortran runtime. It could be triggered
> by differing block sizes on the two systems. Have you checked the
> phase of the moon lately? ;-)

Constantly. But Tobias nailed the cause, and it has nothing to do
with the phase of the Moon.

[tho...@antispam.ham]

Tobias Burnus <bur...@net-b.de> writes:

>> Yesterday, it became necessary to run that program on a
>> different 32-bit Windows XP machine. I installed the 32-bit
>> version of TDM's gfortran 4.6.1 package and compiled the more
>> recent version of the source code there. It compiled properly
>> without any changes to the source code. However, when running
>> this executable, writes to disk using stream I/O are PAINFULLY
>> slow. Write speed was less than 300 kilobytes per second. It
>> took over two HOURS to create 12 files of 177 MB size. It's
>> not a problem with the disk. Minutes earlier, Firefox had
>> written 8 GB worth of downloaded files as fast as the ethernet
>> transfer speeds would allow (multiple streams at 5 MB/sec).
>> The disk has plenty of space available.

> I think you run into the bug
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=50016
> which affects MinGW only (32 and 64 bit). Affected is GCC 4.6.0 and
> 4.6.1; in GCC 4.6.2 the problem is mitigated and in 4.7 the performance
> problem is solved again.

That must be it, though if I understand the issue properly, it
might be more appropriate to call it a design decision rather
than a bug. The use of _commit apparently allows the data to be
seen by other processes, but the penalty is performance. Without
_commit, the performance is great, but presumably other processes
might not see what the writing process thinks has been written.

But does it really affect both 32- and 64-bit MinGW? The snippet
from the bug report could be interpreted as referring to just
32-bit Windows:

#ifdef _WIN32
_commit (s->fd);
#endif

or is "WIN32" simply shorthand for any modern Windows system (those
not built on top of DOS like Windows 3.1), either 32- or 64-bit?
As I mentioned in my initial posting, the program had been ported to
64-bit Windows 7, and performance didn't seem to be an issue. Or
maybe it's just that the Windows 7 machine's performance is so much
better (it has an internal SSD instead of a traditional spinning
disk) that the performance penalty wasn't as noticeable. But now that
attention is being called to the problem, I have noticed that writes
to an external USB 3.0 (spinning) disk have been quite a bit slower
than reads. I thought it had to do with the disk thrashing, one
process reading while another process was writing, but now I'll have
to go back and check with a different version of the compiler.

> Background: Many operating systems buffer I/O. While under Linux (at
> least for local files), the buffering is across all processes, under
> Windows, this data becomes only available to other processes when
> closing the file - or when explicitly calling _commit.
>
> Calling _commit (or on Unix/Linux systems: fsync) is an expensive
> operation: One tells the operating system to ensure that the data is
> written to the disk. For 4.6 a patch was added, which called _commit for
> every Fortran buffer flush - and the Fortran buffer gets flushed a lot.
>
> For 4.6.2 (just before the release), the issue was mitigated by only
> calling _commit for FLUSH (and for INQUIRE). Additionally,
> INQUIRE(unit=...) now uses the internally known file length instead of
> asking the operating system via "stat". The latter returned the wrong
> length if the file hadn't been either "_commit"ted or closed after the
> last write.
>
> For 4.7, the _commit call was also removed for FLUSH. Now, one can run
> into inconsistencies on Windows, but as users can call _commit or fsync
> themselves, that seemed to be the best solution.
>
> See also
> http://gcc.gnu.org/onlinedocs/gfortran/Data-consistency-and-durability.html
>
> Sorry for the breakage.

Many thanks for the explanation, and the rapid response. It all makes
sense!

> The best way is to either upgrade to GCC 4.6.1, GCC 4.7.1 or the current
> developer version 4.8 (or to downgrade to GCC 4.5.x).
>
> See http://gcc.gnu.org/wiki/GFortranBinaries for some options you have.

I had been using the TDM build, because they have a convenient
installer that makes life easy. However, TDM seems to be stuck on
version 4.6.1. Meanwhile, equation.com has builds for 4.6.3 and
4.7.1, both 32- and 64-bit, as well as a weekly snapshot of 4.8.
However, the "use at your own risk" statement had given me pause,
and equation.com calls it a "variant" of GCC with the parallel
processing support in the library replaced with something else,
so I had some concerns over compatibility with my library that had
been built with TDM-GCC 4.6.1.

I decided to try the link to the "installer" version on the above
web page, and the installation of 4.8 was even easier than the TDM
version. One hidden gotcha, however: the TDM installer modified the
system PATH environment variable, whereas the 4.8 installer modified
the user PATH environment variable, and Windows builds the path by
concatenating the system value first and then the user value, so the
old 4.6.1 binary directory occurred earlier in the path than the new
4.8 binary directory, thus typing "gfortran" at the command prompt
still used the old 4.6.1 compiler. I had to manually change the
environment variables to fix that problem.

I recompiled using 4.8, and what took 550 seconds to write using
the 4.6.1 executable now took only 20 seconds using the 4.8
executable. Just as fast as the CVF 6.6c executable. Next I need
to get my hands on that Windows 7 machine again and see how much
of a difference it makes on a 64-bit system.

I didn't see any links to the older 4.5.x builds. Is there an
archive somewhere not mentioned on the above-mentioned web page?
And are there the handy "installer" executables for other builds
that aren't TDM or equation.com? Or is installation really nothing
more involved than copying (or unzipping) the files to the right
directories and then adding the path to the binaries to the
environment?

Is it generally a good idea to rebuild libraries from scratch when
a new version of the compiler is released?

[Tobias Burnus]

On 08/30/2012 02:25 PM, tho...@antispam.ham wrote:
> That must be it, though if I understand the issue properly, it
> might be more appropriate to call it a design decision rather
> than a bug. The use of _commit apparently allows the data to be
> seen by other processes, but the penalty is performance. Without
> _commit, the performance is great, but presumably other processes
> might not see what the writing process thinks has been written.

Well, it is not a correctness bug but a quality of implementation bug.
The chance is much higher that one writes a lot of data than that one
wants to access a file (or its meta data) from the outside ... (Or from
the inside: "stat" also returns the wrong result from within the
process, which has been fixed for INQUIRE(unit=...,SIZE=...) by using
Fortran's knowledge about the file.)

Additionally, the user can still explicitly call _commit or fstat, if
needed.

> But does it really affect both 32- and 64-bit MinGW? The snippet
> from the bug report could be interpreted as referring to just
> 32-bit Windows:
>
> #ifdef _WIN32
> _commit (s->fd);
> #endif
>
> or is "WIN32" simply shorthand for any modern Windows system (those
> not built on top of DOS like Windows 3.1), either 32- or 64-bit?

My understanding is that _WIN32 is set by both 32 and 64 bit Windows.

>> The best way is to either upgrade to GCC 4.6.1, GCC 4.7.1 or the current
>> developer version 4.8 (or to downgrade to GCC 4.5.x).
>>
>> See http://gcc.gnu.org/wiki/GFortranBinaries for some options you have.
>
> I had been using the TDM build, because they have a convenient
> installer that makes life easy. However, TDM seems to be stuck on
> version 4.6.1.

I once tried to reach them, but without success.

> Meanwhile, equation.com has builds for 4.6.3 and
> 4.7.1, both 32- and 64-bit, as well as a weekly snapshot of 4.8.
> However, the "use at your own risk" statement had given me pause,
> and equation.com calls it a "variant" of GCC with the parallel
> processing support in the library replaced with something else,
> so I had some concerns over compatibility with my library that had
> been built with TDM-GCC 4.6.1.

Regarding the own risk part: The problem is that they do note state
which changes they did. I know that for some equation.com version, a
Polyhedron test case failed while it worked with the unmodified MinGW
(or was it MinGW64?) version. I do not recall whether the problem was an
internal compiler error or wrong code (I think it was the former), but
without knowing what they did, one cannot do anything to solve the problem.

Regarding the thread/OpenMP performance: I think MinGW64 comes with a
newly written pthread library which is much faster than the old one.
Though, I don't know in which MinGW64 version nor whether it is enabled
by default or not.

> I recompiled using 4.8, and what took 550 seconds to write using
> the 4.6.1 executable now took only 20 seconds using the 4.8
> executable. Just as fast as the CVF 6.6c executable.

Good!

> I didn't see any links to the older 4.5.x builds. Is there an
> archive somewhere not mentioned on the above-mentioned web page?

No idea. The official GCC policy is to not create binaries but only the
source code and leave the compilation to users or to vendors. Thus, it
depends on the latter whether they offer older versions or not. For
MinGW (32bit), I see 4.5.2, 4.5.2 and 4.7.0 at
http://sourceforge.net/projects/mingw/files/MinGW/Base/gcc/Version4/

I haven't checked the MinGW64 versions.

Note that 4.5 is also not officially supported any more. Usually, old
GCC versions are supported for two years such that there is the latest
version (currently 4.7.1), and it predecessor (4.6.3) plus the actively
developed version, which is usually but not always rather stable (4.8,
will be released around April 2013).

After a .0 release the branch is supposed to see only minor changes,
especially regression fixes, thus, newer .x releases should be more
stable than the .0 release.

In general, newer 4.y releases generate faster code and support more
features; in terms of stability, it is probably the same - some bugs are
found and get removed, but some new regressions creep in. Of course, if
you use new features, using a newer version makes sense as those need a
few versions to mature.

For an overview about the new features, see
http://gcc.gnu.org/wiki/GFortran#news

> And are there the handy "installer" executables for other builds
> that aren't TDM or equation.com? Or is installation really nothing
> more involved than copying (or unzipping) the files to the right
> directories and then adding the path to the binaries to the
> environment?

Actually, I do not know how the different versions are packed. I have
most experience with the Linux version and there unpacking the files
into a directory is essentially enough. On Linux, for convenience I also
add the directory with the binaries to the PATH and, for running
dynamically linked files, the library directory to LD_LIBRARY_PATH. I
don't know what's the Windows equivalent for the latter. (At link time,
the compiler should be able to find the libraries without further help.)

> Is it generally a good idea to rebuild libraries from scratch when
> a new version of the compiler is released?

That depends. The run-time library of GCC is backward compatible since
4.3, i.e. older programs work with newer run-time libraries - and in
many cases also reverse, but that also depends on the features you use.
(In terms of performance, it might make sense to recompile.)

What so far always changed with a new compiler version was the format of
the .mod file. Some new feature always required some change which wasn't
backward compatible.

Additionally, the .mod version number got occasionally bumped to force
recompilation when the ABI changed; I think since 4.1, the only ABI that
changed (and will change again) is the one for polymorphic data structures.

Thus, if you don't use modules, you don't have to recompile; if you do,
you may; and if you use polymorphism, you should - not only because the
ABI might have changed, but also because new features are implemented
and bugs got fixed.

Upcoming for 4.8 is an ABI change to support FINAL subroutines; probably
for 4.9 the heart of the ABI will change: The array descriptor - at that
point, also the ABI of the library will be changed. (The descriptor
change is required for several reasons, one is TS29113 but also some
special Fortran 95 cases aren't representable with the current descriptor.)

Tobias

[dpb]

On 8/30/2012 8:32 AM, Tobias Burnus wrote:
> On 08/30/2012 02:25 PM, tho...@antispam.ham wrote:

...

>> And are there the handy "installer" executables for other builds
>> that aren't TDM or equation.com? Or is installation really nothing
>> more involved than copying (or unzipping) the files to the right
>> directories and then adding the path to the binaries to the
>> environment?
>
> Actually, I do not know how the different versions are packed. I have
> most experience with the Linux version and there unpacking the files
> into a directory is essentially enough. On Linux, for convenience I also
> add the directory with the binaries to the PATH and, for running
> dynamically linked files, the library directory to LD_LIBRARY_PATH. I
> don't know what's the Windows equivalent for the latter. (At link time,
> the compiler should be able to find the libraries without further help.)

...

One of the reasons I hadn't done anything w/ gfortran here--I could
never figure out how w/o a lot of trouble to get it set up correctly
under Windows.

I tried the binary download last night from the Mingw installer link and
it did install and in the installed directory I was able to compile and
execute a sample program.

However, when I moved that created executable to another directory, it
couldn't find the runtime DLL and I've found nowhere that tells what
environment variables are expected/need to be set...is there somewhere
that is given? I saw several "README"s but all had things about they're
out of date; one says there's a description in a "INSTALL" file but
there is no such file on disk...so, I'm kinda' where I was. I'm sure
there must be some mother lode of info somewhere but they surely don't
seem to make it trivial to find it.

--

[Ivan D. Reid]

On Thu, 30 Aug 2012 09:03:33 -0500, dpb <no...@non.net>
wrote in <k1nrrl$n60$1...@speranza.aioe.org>:

> I tried the binary download last night from the Mingw installer link and
> it did install and in the installed directory I was able to compile and
> execute a sample program.

> However, when I moved that created executable to another directory, it
> couldn't find the runtime DLL and I've found nowhere that tells what
> environment variables are expected/need to be set...is there somewhere
> that is given? I saw several "README"s but all had things about they're
> out of date; one says there's a description in a "INSTALL" file but
> there is no such file on disk...so, I'm kinda' where I was. I'm sure
> there must be some mother lode of info somewhere but they surely don't
> seem to make it trivial to find it.

I've never bothered with Mingw, but use full cygwin instead
(www.cygwin.com). Then starting a cygwin window picks up all the right
environment. I usually start up an X display manager but the straight
cygwin window works OK.

$ gfortran -v
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-pc-cygwin/4.5.3/lto-wrapper.exe
Target: i686-pc-cygwin
Configured with:
/gnu/gcc/releases/respins/4.5.3-3/gcc4-4.5.3-3/src/gcc-4.5.3/configure
--srcdir=/gnu/gcc/releases/respins/4.5.3-3/gcc4-4.5.3-3/src/gcc-4.5.3
--prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
--libexecdir=/usr/lib --datadir=/usr/share --localstatedir=/var
--sysconfdir=/etc --datarootdir=/usr/share --docdir=/usr/share/doc/gcc4 -C
--datadir=/usr/share --infodir=/usr/share/info --mandir=/usr/share/man -v
--with-gmp=/usr --with-mpfr=/usr --enable-bootstrap
--enable-version-specific-runtime-libs --libexecdir=/usr/lib
--enable-static --enable-shared --enable-shared-libgcc
--disable-__cxa_atexit --with-gnu-ld --with-gnu-as --with-dwarf2
--disable-sjlj-exceptions
--enable-languages=ada,c,c++,fortran,java,lto,objc,obj-c++
--enable-graphite --enable-lto --enable-java-awt=gtk --disable-symvers
--enable-libjava --program-suffix=-4 --enable-libgomp --enable-libssp
--enable-libada --enable-threads=posix --with-arch=i686
--with-tune=generic --enable-libgcj-sublibs CC=gcc-4 CXX=g++-4
CC_FOR_TARGET=gcc-4 CXX_FOR_TARGET=g++-4 GNATMAKE_FOR_TARGET=gnatmake
GNATBIND_FOR_TARGET=gnatbind --with-ecj-jar=/usr/share/java/ecj.jar
Thread model: posix
gcc version 4.5.3 (GCC)



--
Ivan Reid, School of Engineering & Design, _____________ CMS Collaboration,
Brunel University. Ivan.Reid@[brunel.ac.uk|cern.ch] Room 40-1-B12, CERN
GSX600F, RG250WD "You Porsche. Me pass!" DoD #484 JKLO#003, 005
WP7# 3000 LC Unit #2368 (tinlc) UKMC#00009 BOTAFOT#16 UKRMMA#7 (Hon)
KotPT -- "for stupidity above and beyond the call of duty".

[Tobias Burnus]

On 08/30/2012 04:03 PM, dpb wrote:
> One of the reasons I hadn't done anything w/ gfortran here--I could
> never figure out how w/o a lot of trouble to get it set up correctly
> under Windows.
>
> I tried the binary download last night from the Mingw installer link and
> it did install and in the installed directory I was able to compile and
> execute a sample program.
>
> However, when I moved that created executable to another directory, it
> couldn't find the runtime DLL and I've found nowhere that tells what
> environment variables are expected/need to be set...is there somewhere
> that is given?

Seemingly, directories which are in the PATH are searched for DLLs
[1,3]. Hence, you have to include the directory with GCC/MinGW's DLL in
the PATH, cf. [2].

Disclaimer: I have never run GCC on Windows.

Tobias

[1] http://www.bisque.com/sc/forums/p/326/310.aspx
[2]
http://geekswithblogs.net/renso/archive/2009/10/21/how-to-set-the-windows-path-in-windows-7.aspx
[3] Search for "the search order is as follows:" in
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx

[James Van Buskirk]

"dpb" <no...@non.net> wrote in message news:k1nrrl$n60$1...@speranza.aioe.org...

> One of the reasons I hadn't done anything w/ gfortran here--I could never
> figure out how w/o a lot of trouble to get it set up correctly under
> Windows.

> I tried the binary download last night from the Mingw installer link and
> it did install and in the installed directory I was able to compile and
> execute a sample program.

> However, when I moved that created executable to another directory, it
> couldn't find the runtime DLL and I've found nowhere that tells what
> environment variables are expected/need to be set...is there somewhere
> that is given? I saw several "README"s but all had things about they're
> out of date; one says there's a description in a "INSTALL" file but there
> is no such file on disk...so, I'm kinda' where I was. I'm sure there must
> be some mother lode of info somewhere but they surely don't seem to make
> it trivial to find it.

I find it best to create a shortcut for each version of gfortran
installed on my computer. I have discussed this many times in the
past:

https://groups.google.com/d/msg/comp.lang.fortran/Oq9iZLZGPog/RAahQmxUPIkJ

http://groups.google.com/group/comp.lang.fortran/msg/48effede59b43a2a?hl=en

http://groups.google.com/group/comp.lang.fortran/msg/94d74f2a67307d0a?hl=en

--
write(*,*) transfer((/17.392111325966148d0,6.5794487871554595D-85, &
6.0134700243160014d-154/),(/'x'/)); end

[dpb]

On 8/30/2012 10:29 AM, Tobias Burnus wrote:
...

> Seemingly, directories which are in the PATH are searched for DLLs
> [1,3]. Hence, you have to include the directory with GCC/MinGW's DLL in
> the PATH, cf. [2].

...

Indeed, that does seem to resolve the problem...

Unfortunately (and it's nothing to do w/ gcc), the old editor I prefer
has a problem w/ a PATH of over about 225 characters in length when it
tries to spawn a process to compile inside the editor. The innumerable
number of subdirectories CVF thinks it must have uses up a sizable
fraction of that; when add OpenWatcom it extends it beyond so have to
keep mucking around instead of being able to transparently use an alias
for a given compiler :(

--

[Rafik Zurob]

>> There is often a major performance difference between
>>
>> real :: x(100000)
>>
>> write ( 11 ) ( x(i), i = 1, 100000 )
>>
>> and just
>>
>> write ( 11 ) x
>>
>> as the intereaction with the runtime library is very different
>> on most systems. Rather different overhead on filling the
>> final buffer is quite common.
>
> To the high-level Fortran programmer, the two constructs would
> produce identical results, so one might expect that an optimizer
> might generate identical low-level code for both cases. If it
> doesn't, however, it's not immediately obvious which version
> would be more efficient. The latter looks simpler, but if it
> breaks the array up into manageable chunks and flushes the buffer
> every now and then, the code might be just as complex as the former
> version.

My colleague Xing Xue and I wrote a blog entry last year about this issue:

https://www.ibm.com/developerworks/mydeveloperworks/blogs/b10932b4-0edd-4e61-89f2-6e478ccba9aa/entry/improving_i_o_performance_in_xl_fortran3?lang=en

Basically, I'd rate the various forms from fastest to slowest as follows:

1. One write statement with an output list containing a whole array or array
section
2. One write statement with an output list containing an io-implied-do
1000. Explicit DO loop containing a write statement with an output list
containing a scalar. (You don't have this case above since you're writing
to one unformatted record, but for the sake of argument pretend that you
didn't care about how many records are being written.)

The main slowdown (at least for us) in the explicit DO loop case is not due
to buffering, but to thread-safety concerns.

Also, note that it's generally hard for the optimizer to change I/O calls,
even unformatted ones. Writing a scalar, for example, involves different
I/O runtime function calls and control structures than writing an array.
This kind of language-specific knowledge is usually kept out of optimizers.

Regards

Rafik

[glen herrmannsfeldt]

Rafik Zurob <nos...@hotmail.com> wrote:
>>> There is often a major performance difference between

>>> real :: x(100000)

>>> write ( 11 ) ( x(i), i = 1, 100000 )

>>> and just

>>> write ( 11 ) x

>>> as the intereaction with the runtime library is very different
>>> on most systems. Rather different overhead on filling the
>>> final buffer is quite common.

(snip)

> Basically, I'd rate the various forms from fastest to slowest as follows:

> 1. One write statement with an output list containing a whole array
> or array section

> 2. One write statement with an output list containing an io-implied-do

You forgot:

3. Explicit DO loop containing a write statement with an implied-DO
(or array section) that writes one row (or some part) of the array.
(Best to still write it in the order that they are stored in,
but that isn't always possible.)

For formatted output, this sometimes makes it easier to format
a matrix in human readable form.

For unformatted, it blocks the data into nice sized chunks.
Some systems require a buffer large enough for the whole WRITE.

> 1000. Explicit DO loop containing a write statement with an
> output list containing a scalar. (You don't have this case
> above since you're writing to one unformatted record,
> but for the sake of argument pretend that you didn't care
> about how many records are being written.)

Yes, I don't like this case. Too much overhead keeping track
of record boundaries. (Or in inter-block gaps on some systems.)

> The main slowdown (at least for us) in the explicit DO loop case
> is not due to buffering, but to thread-safety concerns.

Reminds me, I remember when the OS/360 compilers and library
changed from doing implied-DO in the generated code to doing
the loops inside the library. That made new programs incompatible
with the old library. (Following the usual rule that libraries
are back compatible but not forward compatible.)

(snip on optimizers)

-- glen

[user1]

Terence wrote:

>
> So, I suggest, don't use XP (or Vista). I 've also suspected XP and I know
> Vista was a disaster.
>

I presume the XP problems were observed when using that ancient 16-bit
MS Fortran?