Diagnostic Test Set - Comparison of Three Fortran Vendors

Herman D. Knoble

unread,

May 26, 1998, 3:00:00 AM5/26/98

to

A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler
Generated and Run-Time Diagnostics for Fundamental Human Errors

ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt

gives a brief report of how three Fortran Vendors did on a set of 10
test programs. These programs test diagnostic ability, not speed:
undefined (uninitialized) variables, subscripts and strings out of range,
invalid arguments to subprograms, calling subprograms as functions, etc.
In most cases, if the situation is not diagnosed wrong results would be
generated. It is the opinion of this author that compilers should offer
fundamental diagnostics that prevent a simple oversight or typographical
error from giving "wrong answers". No visual debuggers were used in
these tests. Basic batch mode run-time agnostics are reported because
when working with a 100,000 line program which has one of these errors,
use of a debugger to set stop points is too often too time consuming.
Passing diagnostic tests illustrated by the test programs does not
guarantee correct programs; nor does it take the place of careful
programming. But it does make the development process and the
process of "inheriting" large codes, easier to test for fundamental
human errors.

The file results.txt gives a table of comparisons, and brief descriptions of
each test program and the actual diagnostic, if any, given by each of
three vendors. The files test*.for are the actual programs.

It is hoped, by posting this comparison:
- each of the three vendors will be able to make their products give better
run-time diagnostics.

- other diagnostic test programs or suggestions for improving a test can be
offered by the larger (Internet) computing community.

Toon Moene

unread,

May 26, 1998, 3:00:00 AM5/26/98

to

h...@psu.edu (Herman D. Knoble) wrote:
> A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler
> Generated and Run-Time Diagnostics for Fundamental Human Errors
>
> ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt

Did you try g77 (or are the examples Fortran 90/95 - not clear from the
summaries) ?

I for one would be interested in the results ....

TIA
--
Toon Moene (mailto:to...@moene.indiv.nluug.nl)
Saturnushof 14, 3738 XG Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
g77 Support: mailto:for...@gnu.org; NWP: http://www.knmi.nl/hirlam

William F Mitchell

unread,

May 29, 1998, 3:00:00 AM5/29/98

to

In article <356acf7b...@news.cac.psu.edu>, h...@psu.edu (Herman D. Knoble) writes:
|> A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler
|> Generated and Run-Time Diagnostics for Fundamental Human Errors
|>
|> ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt
|>

I note that all three compilers are Fortran 90 or 95 compilers on
MS Windows machines. You should learn to use the features of Fortran 90
to improve the diagnostic capabilities of the compilers. I think you will
find nearly all of your tests will be passed if you (1) use explicit
interfaces (interface blocks, internal procedures or module procedures),
(2) specify INTENT for all your arguments, and (3) use assumed shape arrays
(:) instead of assumed size (*).

--
William F. Mitchell | william....@nist.gov
National Institute of Standards and Technology | na.wmi...@na-net.ornl.gov
If anything looks like an opinion, NIST does not necessarily agree with it.
If anything looks like an official NIST or government statement, it's not.

John Appleyard

unread,

Jun 1, 1998, 3:00:00 AM6/1/98

to

In article <6kn137$suq$1...@news.nist.gov>, William F Mitchell
<mitc...@cam.nist.gov> writes

>In article <356acf7b...@news.cac.psu.edu>, h...@psu.edu (Herman D. Knoble)
>writes:
>|> A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler
>|> Generated and Run-Time Diagnostics for Fundamental Human Errors
>|>
>|> ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt
>|>
>
>I note that all three compilers are Fortran 90 or 95 compilers on
>MS Windows machines. You should learn to use the features of Fortran 90
>to improve the diagnostic capabilities of the compilers. I think you will
>find nearly all of your tests will be passed if you (1) use explicit
>interfaces (interface blocks, internal procedures or module procedures),
>(2) specify INTENT for all your arguments, and (3) use assumed shape arrays
>(:) instead of assumed size (*).
>

Personally, I don't think Fortran 90 advanced the cause of program
diagnostics all that much (I don't say not at all! :>).

As far as I can see at least 3 of the 6 examples which gave problems
would have been unaffected by changes to the syntax.

All 10 problems could be identified by a suitably intelligent Fortran 77
or Fortran 90 compiler without ANY changes.

All 10 problems could be ignored by a suitably dumb Fortran 77 or
Fortran 90 compiler with or without the changes you suggest.

At best, we're just asking the programmer to do work which should be
done by the compiler.
--
John Appleyard Jo...@polyhedron.com Jo...@polyhedron.co.uk

Polyhedron Software Ltd.
Programs for Programmers - QA, Compilers, Graphics

************ Visit our Web site on http://www.polyhedron.co.uk/ ************

Herman D. Knoble

unread,

Jun 1, 1998, 3:00:00 AM6/1/98

to

This is a valid point. However INTERFACE blocks and INTENT declarations
do not necessarily mean that the problems would be diagnosed by the
compiler. Some are and some would be. However, some related to INTENT and
INTERFACE would NOT be. While it is possible to diagnose many problems
at compile time, quite a few problems, including undefined variables,
can ONLY be diagnosed at run-time. And for very large existing codes,
converting the code to use newer features of Fortran can cost more
than it comes to.

Thank you for your posting. It has motivated me to add an introductory
remark in ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt

For a case where INTENT and INTERFACE fail for all three compilers
see the fine work submitted by Arnaud Desitter, School of Geographical
Sciences, University of Bristol :
ftp://ftp.cac.psu.edu/pub/ger/fortran/test/test11.for

By the way, concerning these public Fortran diagnostic bench marks,
notice how silent the vendors have been? The purpose here is not
to criticize vendors or their reps, but to encourage them to make their
products better.

On 29 May 1998 19:08:23 GMT, mitc...@cam.nist.gov (William F Mitchell)
wrote:

-|In article <356acf7b...@news.cac.psu.edu>, h...@psu.edu (Herman D.
Knoble) writes:
-||> A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler
-||> Generated and Run-Time Diagnostics for Fundamental Human Errors
-||>
-||> ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt
-||>
-|
-|I note that all three compilers are Fortran 90 or 95 compilers on
-|MS Windows machines. You should learn to use the features of Fortran 90
-|to improve the diagnostic capabilities of the compilers. I think you will
-|find nearly all of your tests will be passed if you (1) use explicit
-|interfaces (interface blocks, internal procedures or module procedures),
-|(2) specify INTENT for all your arguments, and (3) use assumed shape arrays
-|(:) instead of assumed size (*).

Steve Lionel

unread,

Jun 1, 1998, 3:00:00 AM6/1/98

to

In article <6kn137$suq$1...@news.nist.gov>, mitc...@cam.nist.gov
(William F Mitchell) writes:

|>I note that all three compilers are Fortran 90 or 95 compilers on

|>MS Windows machines. You should learn to use the features of Fortran 90

|>to improve the diagnostic capabilities of the compilers. I think you will

|>find nearly all of your tests will be passed if you (1) use explicit

|>interfaces (interface blocks, internal procedures or module procedures),

|>(2) specify INTENT for all your arguments, and (3) use assumed shape arrays

|>(:) instead of assumed size (*).

Maybe not. I took a look at these test programs. One of them, the one with
the undefined arguments, DVF is capable of catching (and would if you
compiled with /recursive), but due to a particular implementation choice,
isn't by default. I'm looking into that.

However, I explored the INTENT issue and don't think it helps, but I'd
like to see a discussion of the issues. Consider the following:

PROGRAM TEST
INTEGER I
INTERFACE
SUBROUTINE SUB
INTEGER, INTENT(IN) :: ARG
END SUBROUTINE SUB
END INTERFACE

CALL SUB (I)
END

Now... Can a compiler legitimately give a "use before definition" warning
for argument I? I thought at first that it could, but have since concluded
otherwise. The standard says:

The INTENT(IN) attribute specifies that the dummy argument
must not be redefined or become undefined during execution of
the procedure.

Note that it says what the procedure can NOT do - it does not say "this
argument is guaranteed to be read"! It would be perfectly permissible for
the called procedure to not use I at all and therefore an "undefined" warning
would be inappropriate.

This annoys me, as it leaves the compiler relatively helpless to diagnose
this common error unless it can also see the code of the called procedure.
(DVF can do this if the called procedure is in the same compilation and
if it chooses to do inlining.)

Comments?
--
Please send DIGITAL Visual Fortran support requests to dvf-s...@digital.com

Steve Lionel mailto:Steve....@digital.com
Fortran Development http://www.digital.com/info/slionel.html
Digital Equipment Corporation
110 Spit Brook Road, ZKO2-3/N30
Nashua, NH 03062-2698 "Free advice is worth every cent"

For information on DIGITAL Fortran, see http://www.digital.com/fortran

Mark Stevens

unread,

Jun 1, 1998, 3:00:00 AM6/1/98

to

On Tue, 26 May 1998 10:35:52 -0400, h...@psu.edu (Herman D. Knoble)
wrote:

>A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler

>Generated and Run-Time Diagnostics for Fundamental Human Errors
>

>ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt

[snip]

>It is hoped, by posting this comparison:
>- each of the three vendors will be able to make their products give better
> run-time diagnostics.
>
>- other diagnostic test programs or suggestions for improving a test can be
> offered by the larger (Internet) computing community.

We have fixed the three test programs that generated errors that we
did not trap (tests 9, 10 and 11) and improved the error reporting in
test8. You will now find that all of the programs in the test suite
generate errors (with Salford FTN95) either at compile time or at
run-time.

It should also be noted that the Salford debuggers will take you to
the exact line that generates the error message. You do not need to
set break points. You can simply load the program and press the "Go"
button. The debugger will reappear when the program terminates
normally or when an error condition is encountered.

Regards,
Mark
Mark Stevens Salford Software Ltd
Adelphi House, Adelphi Street, Salford, M3 6EN, UK
Tel: +44 (0) 161 834 2454 Fax: +44 161 834 2148
Web: www.salford.co.uk email: ma...@salford-software.com

Mark Stevens

unread,

Jun 1, 1998, 3:00:00 AM6/1/98

to

On 29 May 1998 19:08:23 GMT, mitc...@cam.nist.gov (William F
Mitchell) wrote:

>In article <356acf7b...@news.cac.psu.edu>, h...@psu.edu (Herman D. Knoble) writes:

>|> A Diagnostic Test Set - Comparison of Three Fortran Vendors Compiler
>|> Generated and Run-Time Diagnostics for Fundamental Human Errors
>|>
>|> ftp://ftp.cac.psu.edu/pub/ger/fortran/test/results.txt
>|>
>

>I note that all three compilers are Fortran 90 or 95 compilers on
>MS Windows machines. You should learn to use the features of Fortran 90
>to improve the diagnostic capabilities of the compilers. I think you will
>find nearly all of your tests will be passed if you (1) use explicit
>interfaces (interface blocks, internal procedures or module procedures),
>(2) specify INTENT for all your arguments, and

This is fine if you are using Fortran 90/95 from scratch. What
happens if you are moving a large body of existing code? I do not
think that I would want to change all of your subroutine and function
definitions.

> (3) use assumed shape arrays(:) instead of assumed size (*).

If you run the following program you will get some interesting
results:

program test
interface
subroutine foo(x)
integer :: x(:,:)
end subroutine foo
end interface
integer a(5, 5)

call foo(a)
end

subroutine foo(x)
integer :: x(:,:), y(6, 6)
y = 1
x = y - 1
end subroutine

Not all compilers will actually trap this and I believe that it
follows your recommendations.

There are some other interesting problems awaiting novice Fortran
porgrammers who venture into the world of pointers. Have a look at
the following program:

program pointers
implicit none
integer, pointer :: p2(:) => null()

call sub
call sub1

contains
subroutine sub
integer, target :: p1(40)

p2 => p1
p1 = 32
end subroutine

subroutine sub1
call corrupt_stack
end subroutine sub1

!
! The following function can overwrite its return address.
!
subroutine corrupt_stack
p2 = 1234
end subroutine corrupt_stack
end

You could try using ASSOCIATED before tha array access but this will
not trap this will all compilers. The Salford FTN95 compiler actually
traps this error and gives a useful diagnostic at run-time.

I know that the above example uses global variables to store the
pointers (something that may be frowned upon in certain circles) but
these constructs are 1) legal and 2) being used.

Herman D. Knoble

unread,

Jun 1, 1998, 3:00:00 AM6/1/98

to

Test results,
ftp://ftp.cac.psu.edu/pub/ger/fortran/test/test4.for,
and ftp://ftp.cac.psu.edu/pub/ger/fortran/test/dfdebug.bat
have been updated to reflect this wisdom. Thanks Steve.

On 1 Jun 1998 14:16:13 GMT, Steve Lion el <Steve....@digital.com> wrote:

-|
-|In article <6kn137$suq$1...@news.nist.gov>, mitc...@cam.nist.gov
-|(William F Mitchell) writes:
-|
-||>I note that all three compilers are Fortran 90 or 95 compilers on
-||>MS Windows machines. You should learn to use the features of Fortran 90
-||>to improve the diagnostic capabilities of the compilers. I think you
will
-||>find nearly all of your tests will be passed if you (1) use explicit
-||>interfaces (interface blocks, internal procedures or module procedures),
-||>(2) specify INTENT for all your arguments, and (3) use assumed shape
arrays
-||>(:) instead of assumed size (*).
-|
-|Maybe not. I took a look at these test programs. One of them, the one
with
-|the undefined arguments, DVF is capable of catching (and would if you
-|compiled with /recursive), but due to a particular implementation choice,
-|isn't by default. I'm looking into that.
-|
-|However, I explored the INTENT issue and don't think it helps, but I'd
-|like to see a discussion of the issues. Consider the following:
-|
-|
-| PROGRAM TEST
-| INTEGER I
-| INTERFACE
-| SUBROUTINE SUB
-| INTEGER, INTENT(IN) :: ARG
-| END SUBROUTINE SUB
-| END INTERFACE
-|
-| CALL SUB (I)
-| END
-|
-|Now... Can a compiler legitimately give a "use before definition" warning
-|for argument I? I thought at first that it could, but have since concluded
-|otherwise. The standard says:
-|
-| The INTENT(IN) attribute specifies that the dummy argument
-| must not be redefined or become undefined during execution of
-| the procedure.
-|
-|Note that it says what the procedure can NOT do - it does not say "this
-|argument is guaranteed to be read"! It would be perfectly permissible for
-|the called procedure to not use I at all and therefore an "undefined"
warning
-|would be inappropriate.
-|
-|This annoys me, as it leaves the compiler relatively helpless to diagnose
-|this common error unless it can also see the code of the called procedure.
-|(DVF can do this if the called procedure is in the same compilation and
-|if it chooses to do inlining.)
-|
-|Comments?

Mark Stevens

unread,

Jun 2, 1998, 3:00:00 AM6/2/98

to

On 1 Jun 1998 14:16:13 GMT, Steve Lionel <Steve....@digital.com>
wrote:

>However, I explored the INTENT issue and don't think it helps, but I'd

>like to see a discussion of the issues. Consider the following:
>
>

> PROGRAM TEST
> INTEGER I
> INTERFACE
> SUBROUTINE SUB
> INTEGER, INTENT(IN) :: ARG
> END SUBROUTINE SUB
> END INTERFACE
>
> CALL SUB (I)
> END
>

>Now... Can a compiler legitimately give a "use before definition" warning

>for argument I? I thought at first that it could, but have since concluded

>otherwise. The standard says:

The compiler will find this difficult to deal with. It does not stop
the run-time system from checking the usage at run-time. The Salford
FTN95 compiler will trap this violation even if the subroutine is
located in a separately compiled module.

Ron Sverdlove x2517

unread,

Jun 4, 1998, 3:00:00 AM6/4/98

to

In article <3572cb1e...@news.demon.co.uk>, ma...@salford-software.com (Mark Stevens) writes:
>
>If you run the following program you will get some interesting
>results:
>
>program test
> interface
> subroutine foo(x)
> integer :: x(:,:)
> end subroutine foo
> end interface
> integer a(5, 5)
>
> call foo(a)
>end
>
>subroutine foo(x)
> integer :: x(:,:), y(6, 6)
> y = 1
> x = y - 1
>end subroutine
>
>Not all compilers will actually trap this and I believe that it
>follows your recommendations.
>

It is true that it is hard for compilers to recognize this error. I tried it
with two different compilers. In both cases, the code compiled and ran with
no messages, but I got two different sets of answers when I printed out x after
setting its value. As it stands, this code should produce a run-time error
since it is illegal to assign an array expression to an array of a different
shape.

There is, however, a way to write this code correctly. I assume
that y is supposed to have the same shape as x, in which case the
declarations should be as follows:

integer,dimension(:,:) :: x
integer,dimension(size(x,1),size(x,2)) :: y

--
Ronald Sverdlove Computing Systems Research
r...@sarnoff.com Sarnoff Corporation
Tel. 609-734-2517 CN 5300
FAX 609-734-2662 Princeton, NJ 08543-5300

Herman D. Knoble

unread,

Jun 5, 1998, 3:00:00 AM6/5/98

to

Thanks. I believe this is the file:
ftp://ftp.cac.psu.edu/pub/ger/fortran/test/test11.for

On 4 Jun 1998 20:44:48 GMT, r...@sarnoff.com (Ron Sverdlove x2517) wrote:

Mark Stevens

unread,

Jun 7, 1998, 3:00:00 AM6/7/98

to

On 4 Jun 1998 20:44:48 GMT, r...@sarnoff.com (Ron Sverdlove x2517)
wrote:

>It is true that it is hard for compilers to recognize this error. I tried it

>with two different compilers. In both cases, the code compiled and ran with

>no messages, but I got two different sets of answers when I printed out x after

>setting its value. As it stands, this code should produce a run-time error

>since it is illegal to assign an array expression to an array of a different

>shape.

I have tried several compilers and only the Salford FTN95 compiler
actually traps this error at run-time.

Regards,
Mark
--
Mark Stevens ma...@nevyn.demon.co.uk

The Oxford English Dictionary definition of software now reads "late."