Asynchronous I/O now fully implemented in gfortran

[Thomas Koenig]

Hi,

the development version of gfortran (aka trunk) now has real
asynchronous I/O.

In order to use it, the pthread library has to be linked in.
If this is not done, I/O is always done synchronously.

This should work on most modern Unix-like systems.
The implementation uses pthread condition variables; if
"man pthread_cond_broadcast" does not return anything on
your system, chances are you will have to be content with
what was done previously.

Here is a small example program that you can try:

$ cat hi.f90
program main
real, dimension(10**7) :: a, b, c
call random_number(a)
call random_number(b)
call random_number(c)
open(10,file="a.dat", asynchronous="yes")
open(11,file="b.dat", asynchronous="yes")
open(12,file="c.dat", asynchronous="yes")
write (10,fmt="(F12.7)", asynchronous="yes") a
write (11,fmt="(F12.7)", asynchronous="yes") b
write (12,fmt="(F12.7)", asynchronous="yes") c
wait(10)
wait(11)
wait(12)
end program main
$ gfortran hi.f90
$ time ./a.out

real 0m15.174s
user 0m14.848s
sys 0m0.324s
$ gfortran hi.f90 -pthread
$ time ./a.out

real 0m5.524s
user 0m16.194s
sys 0m0.176s

Have fun!

[Stefano Zaghi]

Great news Thomas,

thank you very much for your work!

My best regards.

[Damian Rouson]

On Monday, August 27, 2018 at 8:19:31 AM UTC-7, Stefano Zaghi wrote:
> Il giorno domenica 26 agosto 2018 20:57:27 UTC+2, Thomas Koenig ha scritto:
> >
> > the development version of gfortran (aka trunk) now has real
> > asynchronous I/O.

Thanks for this great news.

> >
> > In order to use it, the pthread library has to be linked in.

Could you explain how to ensure this?

> > $ gfortran hi.f90

Given that there's no indication of linking in the pthread library above, I'm guessing that the linking happens when gfortran is built. If you could post your steps for building gfortran, that would be very useful.

Damian

[Thomas Koenig]

Damian Rouson <dam...@rouson.net> schrieb:

> On Monday, August 27, 2018 at 8:19:31 AM UTC-7, Stefano Zaghi wrote:
>> Il giorno domenica 26 agosto 2018 20:57:27 UTC+2, Thomas Koenig ha scritto:
>> >
>> > the development version of gfortran (aka trunk) now has real
>> > asynchronous I/O.
>
> Thanks for this great news.
>
>> >
>> > In order to use it, the pthread library has to be linked in.
>
> Could you explain how to ensure this?

You need to add the pthread library during the linking step of your program.
This is valid for Linux. On systems like MacOS or Solaris, this is not
needed because pthreads is part of the normal libc. On systems which
lack POSIX condition variables, like AIX, async I/O does not work
for gfortran.


Examples, with separate compilation and linking:

$ gfortran -c hi.f90
$ gfortran -pthread hi.o

or

$ gfortran -c hi.f90
$ gfortran hi.o -lpthread

or

$ gfortran -fopenmp -c hi.f90
$ gfortran -fopenmp hi.o

The last version is only recommended when you want to use OpenMP
anyway, because of the other things that -fopenmp does, like
enabling -frecursive by default. If you use -fopenmp anyway and
have asynchronous I/O statements in your code, you will also get
real asynchronous I/O.

Compilation and linking in one step is

>> > $ gfortran hi.f90
>
> Given that there's no indication of linking in the pthread library above,

It doesn't. If you don't link in pthreads, your code is still
accepted, but the behavior is the same as before the patch -
I/O will then be done sequentially.

You can check if this works by using the example program I posted
before. If you are on a multiprocessor system and time reports
something like

real 0m5.669s
user 0m16.484s
sys 0m0.292s

(User time approximately three times real time), then you know that
I/O was indeed done asynchronously. If what you get looks like

real 0m15.003s
user 0m14.826s
sys 0m0.176s

then the I/O ran synchronously.

By the way, remember to put in these WAIT statements - we found a
few bugs in the gfortran test suite where these hat been omitted :-)