After some years doing different stuff, I'm back again to my old coding ways (looks like it's "once a programmer, forever a programmer" :) Now I'm working in a big multinational company, and the focus is on developing code which must be modified very often, very quickly, and which has to be extremely robust (so a lot of exception handling). My questions:
0. which is the latest version of the Fortran standard? Last time I checked, I guess it was 2008. Are there compilers for the Intel processor which implement it?
1. is there some Fortran version where the "try...catch" construct of C++, C#, Java, etc. is available, or something similar? I'd find it quite useful for the stuff I'm writing.
2. The code versioning system used is CVS. There are a lot of test cases which were used in the past to check if a new version returned consistent results with an older one. However, the code results (text files) were compared "by hand" against the test database outputs, to see if they were consistent with the results from previous versions. It wasn't possible to do something like
diff oldout newout
because even very small differences, of the order of the machine precisions, would give rise to hundreds of modified lines.
Today, the development pace is so fast that people are basically skipping the tests. Surely, sooner or later (probably sooner), this will produce a version which fails on the test cases (or it would, if anybody cared about running them). I've heard about "nightly regressions", which I guess are automated sets of tests for a code. Are there tools to create an automated testing system for Fortran code? Preferably, it should be a Windows-enviroment solution, since in Windows I have access to the Intel compiler with the Visual Studio 2008 GUI, while under Linux I'm restricted to the f77 compiler (!). Thanks,
> Hi all,
...
> the focus is on developing code which must be modified very often,
>very quickly, and which has to be extremely robust (so a lot of exception handling).
This is easy. do like Microsoft did: All applications for the new
windows 8 OS and beyond will be developed in
HTML5 + Javascript
We all know that Microsoft can't be wrong when it comes to
software. Hence follow the leader. Write everything in HTML5 and
Javascript.
> while under Linux I'm restricted to the f77 compiler (!). Thanks,
http://gcc.gnu.org/fortran/ "The GFortran compiler is fully compliant with the Fortran 95
Standard and includes legacy F77 support. In addition, a significant
number of Fortran 2003 and Fortran 2008 features are implemented."
On 15 May 2012 19:43, deltaquattro wrote:
> My questions:
>
> 0. which is the latest version of the Fortran standard?
> Last time I checked, I guess it was 2008.
The latest standard is still Fortran 2008; it usually takes about 6 years to get a new standard. However, there exists also the Technical Specification (TS) 29113 "Further Interoperability of Fortran with C". To be precise: The draft TS (DTS) has been forward to ISO's SC22 for the ballot. It is nearly certain that it will pass.
> Are there compilers for the Intel processor which implement it?
Short answer: No. While there about 9 commercial and 3 OpenSource Fortran 90+ compilers available (on the Intel/AMD x86/x86-64 platform), none has the full standard implemented. I think it is relatively simple to find compilers which have most of Fortran 2003 and parts of Fortran 2008 implemented, but a full Fortran 2003+2008 coverage is difficult to find.
Recall that the compiler developers can only seriously start implementing the new features after the standard is finished. In case of Fortran 2008 that was 2008; the official ISO publication was even only end of 2009.
To get an overview about the implementation status, have a look at the ACM SIGPLAN Fortran Forum article (April 2012 issue) at http://dx.doi.org/10.1145/2179280.2179282
> 1. is there some Fortran version where the "try...catch" construct
> of C++, C#, Java, etc. is available, or something similar?
Not really. One can try to emulate something like that, but there is no real exception handling like in C++/Java defined in the standard. And I am not aware of any.
> I'd find it quite useful for the stuff I'm writing.
I concur that it can be useful, though sometimes other nearly as good or even better solutions exists in Fortran; it depends on the exact task.
> 2. The code versioning system used is CVS. There are a lot of test
> cases which were used in the past to check if a new version returned
> consistent results with an older one. However, the code results (text
> files) were compared "by hand" against the test database outputs, to
> see if they were consistent with the results from previous versions.
> It wasn't possible to do something like diff oldout newout
> because even very small differences, of the order of the machine
> precisions, would give rise to hundreds of modified lines.
You should use a different tool to compare the result. I know the Octopus code uses a Perl script to compare the result within a certain tolerance. The Polyhedron test suite does the same (but wrote their program in Fortran). I think you could do likewise. As the output from programs is vastly different, I think no general tool exists.
> Today, the development pace is so fast that people are basically
> skipping the tests. Surely, sooner or later (probably sooner), this
> will produce a version which fails on the test cases (or it would,
> if anybody cared about running them).
It is definitely better to have an automatic regression test suite.
> I've heard about "nightly regressions", which I guess are automated
> sets of tests for a code. Are there tools to create an automated
> testing system for Fortran code?
No idea. As written, it depends on the exact output. I am sure that several codes have written one for their needs, but I am not aware of a wider used general solution.
In case of the Octopus code, the test cases contain the expected result and there are build bots, which automatically build and test the program (on several systems) every time a commit was done - and once per night a more extensive and once per week a large test set. (http://www.tddft.org/trac/octopus/ -> "Buildbot")
Other programs like CP2K simply record the current result of a run - and then compare with that result the next time one runs the program. That finds changes on one system, but does not compare different systems as no known-to-be-good result exist. But one saves to write the test cases oneself.
Note that compiler vendors also have test suite. GCC for instance has the policy that one has to run test test suite for every patch before committal. Additionally, there it is regularly (often daily) run on various systems. All those tests are hand-written. Additionally, some other software is also regularly build and run (Polyhedron test suite, SPEC CPU, CP2K, ...). That reduces the chance of regressions. However, as there are infinite ways of writing a program, regressions do creep into compilers and other software …
> Preferably, it should be a Windows-enviroment solution, since in Windows
> I have access to the Intel compiler with the Visual Studio 2008 GUI,
> while under Linux I'm restricted to the f77 compiler (!).
Why are you restricted to "the f77 compiler"? (There existed many Fortran 77 compilers.)
And why can't you update? I think the obvious choice would be either the Intel Compiler or the GCC/gfortran compiler.
The Intel compiler would be a good choice as you have it also under Windows - thus the supported features and bugs are the same. You just need to buy another license. (I assume that your Linux runs on standard Intel hardware.)
If you don't want to spend the money, or your Linux system is a bit special such as an ARM system or Android, or just to check the program with another compiler,* I suggest that you install GCC/gfortran.
Like Intel's compiler, gfortran is supports a large chunk of Fortran 2003 and 2008 and produces rather good code. It should be available with your Linux distribution (packages are typically named gcc-fortran, gcc-gfortran, gcc47-gfortran or similar). See also http://gcc.gnu.org/wiki/GFortranBinaries
Especially if you want to use Fortran 2003/2008 features, make sure that you have a recent compiler. (For instance, RHES/CentOS 5.x ship with GCC 4.1 which is rather old - but RHES/CentOS ship with a slightly newer version as technical preview.) See http://gcc.gnu.org/wiki/GFortran#news for an overview about the newly implemened features per version.
(* Using multiple compilers during the development makes programs more standard conforming, it helps to find bugs - and sometimes, it can make debugging easier as well.)
Tobias
Disclaimer: I contribute to the gfortran compiler.
Tobias Burnus <bur...@net-b.de> wrote:
> programs is vastly different, I think no general tool exists.
(snip)
> It is definitely better to have an automatic regression test suite.
> > I've heard about "nightly regressions", which I guess are automated
> > sets of tests for a code. Are there tools to create an automated
> > testing system for Fortran code?
> No idea. As written, it depends on the exact output. I am sure that > several codes have written one for their needs, but I am not aware of a > wider used general solution.
One thing that is sometimes done is to do coverage testing.
That is, find an input file that will test as many parts of a program as possible. There are tools which will help, by showing
which parts are, and are not, executed with a given input.
In the Fortran 66 days, I used a program which would take
Fortran input, generate a new Fortran program that included usage
counts (normally not shown to the user), compile it and run it.
At the end, the program itself would print out the source listing
with, for each statement, the number of times executed, for loops
the number of loop iterations, and an approximate (unitless) cost
for that execution.
Otherwise, regression tests may not cover enough of a program to
reliably test anything.
> Tobias Burnus <bur...@net-b.de> wrote:
>> programs is vastly different, I think no general tool exists.
> (snip)
>> It is definitely better to have an automatic regression test suite.
>>> I've heard about "nightly regressions", which I guess are automated
>>> sets of tests for a code. Are there tools to create an automated
>>> testing system for Fortran code?
>> No idea. As written, it depends on the exact output. I am sure that
>> several codes have written one for their needs, but I am not aware of a
>> wider used general solution.
> One thing that is sometimes done is to do coverage testing.
> That is, find an input file that will test as many parts of a
> program as possible. There are tools which will help, by showing
> which parts are, and are not, executed with a given input.
> In the Fortran 66 days, I used a program which would take
> Fortran input, generate a new Fortran program that included usage
> counts (normally not shown to the user), compile it and run it.
> At the end, the program itself would print out the source listing
> with, for each statement, the number of times executed, for loops
> the number of loop iterations, and an approximate (unitless) cost
> for that execution.
You are describing execution profiles that have exact line counts.
Not to be confused with location counter samplers that often are
called profilers (that seem to have come later as a hack to mimic
the line counting profilers). One uses the profilers to make better
test programs. There is no need to find the line counts for compiler
testing. There is the possibility that the output could be the same even
under differing line counts as may happen with convergence based numerical
procedures.
Compiler regression testing does not depend on full coverage within the
various test cases. Many will be of an old error in an isolated part
of a larger program that was not miniaturized. The syntax testing portions
will process the full program so coverage is not an issue. For code
generation coverage is a help. The compiler should be run under a profiler
that checks for full coverage under the test cases but that is a different
issue for compiler writers. But then any quality testing of any program
should use a tool like a profilers.
>> One thing that is sometimes done is to do coverage testing.
>> That is, find an input file that will test as many parts of a
>> program as possible. There are tools which will help, by showing
>> which parts are, and are not, executed with a given input.
>> In the Fortran 66 days, I used a program which would take
>> Fortran input, generate a new Fortran program that included usage
>> counts (normally not shown to the user), compile it and run it.
(snip)
> You are describing execution profiles that have exact line counts.
Well, I described two different things.
> Not to be confused with location counter samplers that often are
> called profilers (that seem to have come later as a hack to mimic
> the line counting profilers).
Well, possibly when the mechanism to do it reasonably well existed.
> One uses the profilers to make better test programs. There is no > need to find the line counts for compiler testing.
That is true. Location counter sampling might be good enough to
do yes/no on whether a location is reached. Then again, it might
be reached rarely enough, and quickly enough not to be easily found.
Supposedly Knuth did enough profiling on TeX to find everything
except for fatal error processing when given a special input file.
I am not sure by now if that was done with sampling or counting.
> There is the possibility that the output could be the same even
> under differing line counts as may happen with convergence > based numerical procedures.
TeX does all calculations related to page positioning in fixed point.
Some results in the log file are done in floating point, and could
possibly be different with different systems.
> Compiler regression testing does not depend on full coverage > within the various test cases. Many will be of an old error in > an isolated part of a larger program that was not miniaturized.
Without full coverage, you can't be sure that a change doesn't affect something that you don't test for. That can be especially
hard in optimizing compilers, where something might be done only
for very specific input.
> The syntax testing portions will process the full program so > coverage is not an issue. For code generation coverage is a help.
Well, ok, the input needed to test the syntax analyzer is much
simpler to generate.
> The compiler should be run under a profiler that checks for > full coverage under the test cases but that is a different
> issue for compiler writers. But then any quality testing of > any program should use a tool like a profilers.
On Wednesday, 16 May 2012 03:43:19 UTC+10, deltaquattro wrote:
> Hi all,
> After some years doing different stuff, I'm back again to my old coding ways (looks like it's "once a programmer, forever a programmer" :) Now I'm working in a big multinational company, and the focus is on developing code which must be modified very often, very quickly, and which has to be extremely robust (so a lot of exception handling). My questions:
For something like that you need PL/I or even Ada.
On Wednesday, 16 May 2012 03:43:19 UTC+10, deltaquattro wrote:
> Hi all,
> After some years doing different stuff, I'm back again to my old coding ways (looks like it's "once a programmer, forever a programmer" :) Now I'm working in a big multinational company, and the focus is on developing code which must be modified very often, very quickly, and which has to be extremely robust (so a lot of exception handling). My questions:
> 0. which is the latest version of the Fortran standard? Last time I checked, I guess it was 2008. Are there compilers for the Intel processor which implement it?
> 1. is there some Fortran version where the "try...catch" construct of C++, C#, Java, etc. is available, or something similar? I'd find it quite useful for the stuff I'm writing.
For something like that, you need PL/I (or even Ada).
Computational and other errors are routinely trapped in PL/I,
and you can recover from such errors and continue processing.
> 2. The code versioning system used is CVS. There are a lot of test
> cases which were used in the past to check if a new version returned
> consistent results with an older one. However, the code results (text
> files) were compared "by hand" against the test database outputs, to
> see if they were consistent with the results from previous versions.
> It wasn't possible to do something like
> diff oldout newout
> because even very small differences, of the order of the machine
> precisions, would give rise to hundreds of modified lines.
> Today, the development pace is so fast that people are basically
> skipping the tests. Surely, sooner or later (probably sooner), this
> will produce a version which fails on the test cases (or it would, if
> anybody cared about running them).
You need to build a regression test system which is capable of comparing
floating-point results to a precision you specify. I have some experience with and can recommend the "Testcode" package written by James Spencer of Imperial College, London
> After some years doing different stuff, I'm back again to my old coding ways (looks like it's "once a programmer, forever a programmer" :) Now I'm working in a big multinational company, and the focus is on developing code which must be modified very often, very quickly, and which has to be extremely robust (so a lot of exception handling). My questions:
> 0. which is the latest version of the Fortran standard? Last time I checked, I guess it was 2008. Are there compilers for the Intel processor which implement it?
> 1. is there some Fortran version where the "try...catch" construct of C++, C#, Java, etc. is available, or something similar? I'd find it quite useful for the stuff I'm writing.
> 2. The code versioning system used is CVS. There are a lot of test cases which were used in the past to check if a new version returned consistent results with an older one. However, the code results (text files) were compared "by hand" against the test database outputs, to see if they were consistent with the results from previous versions. It wasn't possible to do something like
> diff oldout newout
> because even very small differences, of the order of the machine precisions, would give rise to hundreds of modified lines.
> Today, the development pace is so fast that people are basically skipping the tests. Surely, sooner or later (probably sooner), this will produce a version which fails on the test cases (or it would, if anybody cared about running them). > I've heard about "nightly regressions", which I guess are automated sets of tests for a code. Are there tools to create an automated testing system for Fortran code? Preferably, it should be a Windows-enviroment solution, since in Windows I have access to the Intel compiler with the Visual Studio 2008 GUI, while under Linux I'm restricted to the f77 compiler (!). Thanks,
> Best Regards
> Sergio Rossi
You may want to have a look at the various unit test frameworks for Fortran.
(See http://fortranwiki.org). The one I know best (as I wrote) is ftnunit in my
Flibs project (http://flibs.sf.net). This is a Fortran-only framework (well, with some batch files/shell scripts and some other stuff for ease of use).
Il giorno martedì 15 maggio 2012 22:09:28 UTC+2, Tobias Burnus ha scritto:
> Hi Sergio,
[..]
However, there exists also the Technical
> Specification (TS) 29113 "Further Interoperability of Fortran with C".
[..]
Interesting: I have a couple questions about Fortran/C interoperability, but I guess it's better to open a new thread.
[..]
> > I'd find it quite useful for the stuff I'm writing.
> I concur that it can be useful, though sometimes other nearly as good or > even better solutions exists in Fortran; it depends on the exact task.
Do you mean the following:
1. IOSTAT, which is useful for I/O related runtime errors. However, the integer returned is processor-dependent, and that's annoying. For example, if an OPEN statement fails because of a "file not found", you can't easily catch it. 2. the ieee_arithmetic module allows catching divide by zero and other related stuff.
I don't know....it seems to me that catching these errors requires bloating the code with a lot of logic: IF or CASE statements based on the integer returned, and a module which tests for the possible error values of IOSTAT and stores them in user-defined parameters. The try...catch clause, together with suitably defined classes for the various kinds of runtime error I need to catch, seems to me a simpler, more uniform approach. But it may be just a matter of habit.
> > 2. The code versioning system used is CVS. There are a lot of test
> > cases which were used in the past to check if a new version returned
> > consistent results with an older one. However, the code results (text
> > files) were compared "by hand" against the test database outputs, to
> > see if they were consistent with the results from previous versions.
> > It wasn't possible to do something like diff oldout newout
> > because even very small differences, of the order of the machine
> > precisions, would give rise to hundreds of modified lines.
> You should use a different tool to compare the result. [..]
Thanks for the info. As you say, I definitely need an automated test suite. I'll try the framework suggested by other posters.
> > Preferably, it should be a Windows-enviroment solution, since in Windows
> > I have access to the Intel compiler with the Visual Studio 2008 GUI,
> > while under Linux I'm restricted to the f77 compiler (!).
> Why are you restricted to "the f77 compiler"? (There existed many > Fortran 77 compilers.)
> And why can't you update? I think the obvious choice would be either the > Intel Compiler or the GCC/gfortran compiler.
Eheh :) I work in a big company...I don't decide what gets installed on the Linux clusters. I could manage to install a compiler in my local area, but only *if* the installation procedure is simple/fast (I'm no Linux guru and I'm overloaded as it is), *if* no additional libraries must be installed, possibly in reserved areas like usr/bin , usr/lib, etc. Also, on Linux I would need to create a Makefile, take care of the dependencies, etc. The easiest route to go for me is the Windows solution, since the project has been already created there. Anyway, thanks for the suggestion!
Il giorno mercoledì 16 maggio 2012 08:16:26 UTC+2, Keith Refson ha scritto:
[..]
> You need to build a regression test system which is capable of comparing
> floating-point results to a precision you specify. I have some > experience with and can recommend the "Testcode" package written by > James Spencer of Imperial College, London
Il giorno mercoledì 16 maggio 2012 10:34:58 UTC+2, Arjen Markus ha scritto:
[..]
> You may want to have a look at the various unit test frameworks for Fortran.
> (See http://fortranwiki.org). The one I know best (as I wrote) is ftnunit in my
> Flibs project (http://flibs.sf.net). This is a Fortran-only framework (well, > with some batch files/shell scripts and some other stuff for ease of use).
> Regards,
> Arjen
Hi Arjen,
thanks for the reply. Surely I'll look into your code. Correct me if I'm wrong, but I think unit testing may be a bit too much for me - with unit testing I develop and mantain tests for each procedure/module. That's surely useful, but right now the situation is so messy, that I'd be satisfied with ensuring that new code versions at least runs for all the cases for which the preceding version ran. With case, I mean a couple input files - output files. Thanks,
> After some years doing different stuff, I'm back again to my old coding ways (looks like it's "once a programmer, forever a programmer" :) Now I'm working in a big multinational company, and the focus is on developing code which must be modified very often, very quickly, and which has to be extremely robust (so a lot of exception handling). My questions:
> 0. which is the latest version of the Fortran standard? Last time I checked, I guess it was 2008. Are there compilers for the Intel processor which implement it?
> 1. is there some Fortran version where the "try...catch" construct of C++, C#, Java, etc. is available, or something similar? I'd find it quite useful for the stuff I'm writing.
> 2. The code versioning system used is CVS. There are a lot of test cases which were used in the past to check if a new version returned consistent results with an older one. However, the code results (text files) were compared "by hand" against the test database outputs, to see if they were consistent with the results from previous versions. It wasn't possible to do something like
> diff oldout newout
> because even very small differences, of the order of the machine precisions, would give rise to hundreds of modified lines.
For the record: why not do this in a few lines of Fortran. Read output and reference in parallel, line by line, split into tokens (use SCAN/VERIFY, easy if floats are always separated by whitespace), try to READ the reference token as real/double. If successful, compare after rounding. If not successful, it was text which you should compare literally.
This is even simpler with the standard ISO_VARYING_STRING module, which has no fixed size for strings, and comes with a convenient SPLIT procedure.
An existing package is likely a better option, but a Fortran-only solution may allow the program to test itself ...
> Il giorno mercoledì 16 maggio 2012 10:34:58 UTC+2, Arjen Markus ha scritto:
> [..]
> > You may want to have a look at the various unit test frameworks for Fortran.
> > (See http://fortranwiki.org). The one I know best (as I wrote) is ftnunit in my
> > Flibs project (http://flibs.sf.net). This is a Fortran-only framework (well, > > with some batch files/shell scripts and some other stuff for ease of use).
> > Regards,
> > Arjen
> Hi Arjen,
> thanks for the reply. Surely I'll look into your code. Correct me if I'm wrong, but I think unit testing may be a bit too much for me - with unit testing I develop and mantain tests for each procedure/module. That's surely useful, but right now the situation is so messy, that I'd be satisfied with ensuring that new code versions at least runs for all the cases for which the preceding version ran. With case, I mean a couple input files - output files. Thanks,
> Best
> Sergio
Sure, but it depends on what you consider a unit :). If, as you say, it
is a matter of input files and output files, the ftnunit framework does allow you to compare text files. You can specify a tolerance for comparing the numbers in these files.
Of course, it is meant for testing individual routines, but that does not
mean it is not suitable for whole computations. (Other frameworks may be
more suitable for that type of testing, I agree)
deltaquattro wrote:
>> > while under Linux I'm restricted to the f77 compiler (!).
>> Why are you restricted to "the f77 compiler"? (There existed many
>> Fortran 77 compilers.)
>> And why can't you update? I think the obvious choice would be either the
>> Intel Compiler or the GCC/gfortran compiler.
> Eheh :) I work in a big company...I don't decide what gets installed on the
> Linux clusters. I could manage to install a compiler in my local area, but
> only *if* the installation procedure is simple/fast (I'm no Linux guru and
> I'm overloaded as it is), *if* no additional libraries must be installed,
> possibly in reserved areas like usr/bin , usr/lib, etc. Also, on Linux I
> would need to create a Makefile, take care of the dependencies, etc. The
> easiest route to go for me is the Windows solution, since the project has
> been already created there. Anyway, thanks for the suggestion!
Well, convincing an admin to install one additional vendor package shouldn't be that difficult. That way all users get some (typically older) gfortran in their path.
I fetch the three packages: The pre-build xz package, the nightly build (because that's the version I want to use), and the gcc-infrastructure one.
First, I have to unpack the xz package. (If the program xzcat exists, you have a fairly recent Linux or a good admin you can skip it.):
tar xvfz xz.tar.gz
Then you can unpack the gcc package by running
xzcat gfortran-trunk*.tar.xz | tar xvf -
(you may need to add the path to "xzcat" if you unpacked it as written above)
and finally, you go into the created directory (e.g. gcc-trunk) and unpack there the gcc-infrastructure package.
Then it should work: Simply run "gcc-trunk/bin/gfortran -v"
If you want to run the program, you still have to set the library path:
On 2012-05-16, Tobias Burnus <bur...@net-b.de> wrote:
> But one can also install an the binary in some directory - as I do at > work and on one supercomputer. I have a x86-64 system and thus simply do > what's described at http://gcc.gnu.org/wiki/GFortranBinaries#GNU.2BAC8-Linux
> I fetch the three packages: The pre-build xz package, the nightly build > (because that's the version I want to use), and the gcc-infrastructure one.
> First, I have to unpack the xz package. (If the program xzcat exists, > you have a fairly recent Linux or a good admin you can skip it.):
> tar xvfz xz.tar.gz
> Then you can unpack the gcc package by running
> xzcat gfortran-trunk*.tar.xz | tar xvf -
FWIW, recent enough(?) versions of GNU tar support xz compressed
archives, then you could do
>> First, I have to unpack the xz package. (If the program xzcat exists, >> you have a fairly recent Linux or a good admin you can skip it.):
>> tar xvfz xz.tar.gz
>> Then you can unpack the gcc package by running
>> xzcat gfortran-trunk*.tar.xz | tar xvf -
> FWIW, recent enough(?) versions of GNU tar support xz compressed
> archives, then you could do
> tar xvJf gfortran-trunk*.tar.xz
I know that when it does gzip (the z flag) it actually runs gzcat.
It seems likely that it actually runs xzcat in this case, but
I haven't actually looked. It does save a little typing, though.
