Release of stand-alone dependency tracker

[Thomas Jahns]

As promised over 1 year ago[2] I've now made our dependency tracker completely
independent from autoconf. It's available[3] from the site of the library it was
originally developed in[1].

To quote from the original post:

> We use a dependency tracker, maintained by me, that currently does the following:
>
> 1. parses fortran source including cpp directives
> 2. evaluates cpp macros to determine effective USE statements,
> i.e. the tracker doesn't generate dependencies for stuff that
> is effectively inside #if 0
> 3. finds external .mod files
> 4. can handle multiple modules per source, independent of source name
> 5. only writes the dependency part of the makefile (because in our setup that
> is maintained by automake/autoconf)

For a simple experiment run it like:

perl -Iscripts scripts/makef90depends -- -- src/*.f90

if your sources are in files in subdirectory src. Feedback is very
much appreciated. To get everything right for compilers varying from
typical Linux x86_64 compilers, a number of options needs to be
set. For slow compilers (xlf, sxf90), the internal caching functions might prove
useful.

To parse preprocessor conditionals correctly, environment variables FC
and FPP should be set to appropriate commands. Compilers that need
help with that[4] because they can't write preprocessed input to
stdout need to be wrapped one of the provided wrapper scripts. How to identify a
working wrapper can be seen from the scales-ppm configure.ac script source (see
m4/starlink_fpp.m4, macro _ACX_SL_PROG_FPP for implementation details).
Generally the following holds:

- NAG and Sun/Oracle sunf95 compilers use sunf95preproc-wrapper
- the NEC SX compiler is wrapped with sxpreproc-wrapper
- IBM xlf compilers require the xlfpreproc-wrapper

Generally options that change the behaviour of the tracker go before
the first double dash and options to the compiler (in our case
$(AM_FCFLAGS) $(FCFLAGS)) go before the second double dash so that the
tracker knows how to correctly invoke the compiler and can see
include/module directories for itself.

The distribution[5] contains a very detailed setup, but understanding
probably requires reading fluency of autoconf/automake.

The manual page can be generated with pod2man. To read it immediately
try somethng like:

$ pod2man config/makef90depends |man -l -

on a Unix system. The script is untested on Windows but should be easy
to get to work there also.

Regards, Thomas

[1] https://www.dkrz.de/redmine/projects/scales-ppm
[2] Message-ID: <jj2do2$hhv$1...@gwdu112.gwdg.de>

<https://groups.google.com/group/comp.lang.fortran/msg/494c861ebdfa7f2e?dmode=source&output=gplain&noredirect>
[3]
<https://www.dkrz.de/redmine/attachments/download/405/makef90depends-1.0.0.tar.gz>
[4] AFAIK: xlf, nagfor, sunf95, sxf90
[5] <https://www.dkrz.de/redmine/attachments/download/382/ppm-1.0.2.tar.xz>

[Arjan]

Hurray!

At the moment I use my own homebrew utility for this, but it assumes a certain (=my) way of coding and therefore has limited portability. I'll go and try your utility to see if it can replace mine, which would allow for easy integration of my code with stuff from others.

Doesn't "cmake" include something like this since a few years, or am I mistaken?

Cheers!

Arjan

[Thomas Jahns]

On 05/13/2013 07:39 PM, Arjan wrote:
> Doesn't "cmake" include something like this since a few years, or am I mistaken?

it supposedly does, but the project I wrote the tracker for is using autotools
and I'm not too comfortable with cmake. Also one of the original platforms to
support was NEC SX and porting cmake to that would probably have been more work
than writing the tracker, perl 5.6+ seems to be available everywhere.

Regards, Thomas

[Wolfgang Kilian]

On 05/13/2013 12:05 PM, Thomas Jahns wrote:
> As promised over 1 year ago[2] I've now made our dependency tracker completely
> independent from autoconf. It's available[3] from the site of the library it was
> originally developed in[1].
>
> To quote from the original post:
>
>> We use a dependency tracker, maintained by me, that currently does the following:
>>
>> 1. parses fortran source including cpp directives
>> 2. evaluates cpp macros to determine effective USE statements,
>> i.e. the tracker doesn't generate dependencies for stuff that
>> is effectively inside #if 0
>> 3. finds external .mod files
>> 4. can handle multiple modules per source, independent of source name
>> 5. only writes the dependency part of the makefile (because in our setup that
>> is maintained by automake/autoconf)

That is indeed very welcome. In our automake/autoconf/libtool
environment, we use a few lines of Makefile code (sed script) to extract
dependencies from Fortran 90 sources. This works surprisingly well, but
we do keep simple conventions in the code. In particular, there is no
preprocessor involved. So, there are good reasons to switch to a more
versatile tool that cooperates with the autotools environment.

-- Wolfgang

--
E-mail: firstnameini...@domain.de
Domain: yahoo

[Nasser M. Abbasi]

I am surprised one has to use all these user supplied tools to
make sure the Fortran program is build correctly, even in
modern Fortran?

Fyi, in Ada, it is not even possible to build an inconsistent
program, since it is part of the language itself, and
not the outside environment, how to the program is
put together from all its parts.

http://www.xgc.com/manuals/m1750-ada/xgc-ada-ug/x4020.html

"The rules in Ada do not permit such an inconsistent
program to be built. For example, if two clients have
different versions of the same package, it is not
possible to build a program containing these two clients."

http://gcc.gnu.org/onlinedocs/gnat_ugn_unw/Binding-with-gnatbind.html#Binding-with-gnatbind

"Checks that a program is consistent, in accordance with
the rules in Chapter 10 of the Ada Reference Manual."

I wonder if Fortran language can be somehow modified/enhanced
add rules, etc.. so that it would work like Ada in this sense?
i.e. make this part as part of the language specification itself,
similar to Ada?

--Nasser

[Wolfgang Kilian]

The problem here is not to build things correctly, but to build them at
all. Handling the distribution of source code among files and over the
file system is a matter of the operating system, not of the language
specification. Depending on your source layout, whether you use a
preprocessor, etc., this may be a simple or complicated task. Fortran
has been (for good reasons) very liberal regarding its implementation.
It is not even required to run on a computer ...

Other languages include more parts of the implementation in their
specification, up to a complete build environment. This is not always a
good idea.

[Ian Harvey]

Well, you don't have to use "all these tools". I wrote my own
"dependency tool" in Fortran 2003. What all these script kiddies have
probably done with at least a hundred lines of perl, I've been able to
do with only several thousand lines of Fortran!

[Arjan]

> Well, you don't have to use "all these tools". I wrote my own
> "dependency tool" in Fortran 2003. What all these script kiddies have
> probably done with at least a hundred lines of perl, I've been able to
> do with only several thousand lines of Fortran!

My (probably less versatile) utility was written in F90 and only a few hunderd lines, but then I use system calls to "grep" to scan a source-file for USE-statements. Now that I come to think of it, this will not have saved much lines of code. I do share the opinion that with modern fortran versions insisting on knowing interfaces of lower level code at compile time, it would have been nice if some means to collect that info were provided, say, 23 years ago...

[Zaak]

I just started using CMake and am very pleased with its facilities including dependency resolution. It seems to have very good support for Fortran dependency resolution, although I have yet to throw some of the F2003+ USE statements at it (USE ,INTRINSIC :: my_mod... etc.) It also has quite a nice testing framework, concluding automatic test-drive/test-harness generation in C or C++. I write my Fortran tests as functins with ISO_C_BINDINGs and can very easily setup tests and builds with CMake.

Some compilers, ifort (version 12+ ?) and gfortran (version 4.7+ ?) have capabilities to output dependencies, but the problem that I have with them is that they require any .mod files USEd in the source file under consideration to be present. Resolution of those annoying .mod files is *exactly* why I want such a tool, and find it quite frustrating that the compiler vendor provided tools can't parse your source code well enough or translate it into dependencies without the dependent .mod files being present. Very frustrating.

To anyone using make or autoconf with their projects I would definitely recommend looking into CMake. I'm not thrilled with it's documentation, but between the book, the mailing list and the web resources I've been able to learn how to effectively use it, and it is very pleasant.

Best,
Zaak

[\"Vladimír Fuka <"name.surnameat]

I have great results with SCons. waf should be also great.

Vladimir