Modules and Make Dependencies

[Coleman Federal Special Programs Group]

Does anyone have suggestions for establishing module dependencies
in make files?

The problem we have is this:

If module B (B_M) USEs module A (A_M) and module A_M changes,
make will re-compile A_M because its source file is newer than
its object file but B_M will not re-compile to include the
changes to A_M.

Make depend doesn't recognize f90 files the way C dependencies
are recognized. We can put targets at the bottom of our
makefiles for each file to ensure that it gets its correct
dependencies, but that is difficult to manage and hard to
maintain. Does anyone have a better solution?

--
Bradford Marx
ma...@mail.crc.com
ma...@goodnet.com
ma...@master.tds-az.lmco.com

[Jack Saba]

You'll have to modify this for your purpose. I use it to catch include
dependencies for f77. For each input file a.f, it generates a file a.d
which has a single line with the appropriate format for inclusion in the
Make file as a dependency line. If you're using gmake, there's supposed
to be a way to get it to include these files automatically.

#!/bin/sh
# NAME: dependencies
#
# FUNCTION: Find INCLUDE file dependencies in FORTRAN modules.
#
# USAGE: dependencies <list of files>
#
# OUTPUT: Files a.d with line of the form
# a.o: a.f d1 d2
#
# a.f = element of <list of files>
# d1, d2, ... = INCLUDE file dependencies in a.f
#
# HISTORY:
# 1997 May 22 J.L.Saba Developed with awk help from Mark Torrence.
# 1997 Sept 19 J.L.Saba Enhanced to catch lines with trailing comment
# string.
#
#######################################################################

for file in $*
do

# Strip off the file extension and path

fstripped=`basename ${file} .f`

# grep: find all INCLUDE lines: these have the form
# ws include ws 'filename' ws
# where 'ws' = white space (spaces,tabs,none)
# and 'include' is case-insensitive.
# tr: remove apostrophes from around the included file name
# awk: save included file name to a file, one name to a line

grep -i -e "^[ ]*include[ ]*'.*'[ ]*$" \
-e "^[ ]*include[ ]*'.*'[ ]*!" \
$file \
| tr "'" " " | awk '{print $2}' > ${fstripped}.tmp

# Generate the output file: Read the file just written and generate
# required output line from it, then write to .d file.

awk 'BEGIN{ printf "'${fstripped}'.o: '${fstripped}'.f" }
{ printf " %s", $1 }
END{ printf "\n" }' \
${fstripped}.tmp > ${fstripped}.d

# Remove the intermediate file

rm ${fstripped}.tmp
done

--
Jack Saba <ja...@magus.stx.com>

[Jiri Furst]

Coleman Federal Special Programs Group wrote:
>
> Does anyone have suggestions for establishing module dependencies
> in make files?

I use the "makemake" perl script. It can be found somwhere at

http://www.fortran.com/fortran/free.html

Jiri Furst
fu...@marian.fsik.cvut.cz

[Swietanowski Artur]

Coleman Federal Special Programs Group wrote:
>
> Does anyone have suggestions for establishing module dependencies
> in make files?

A clean and simple solution: if source file x.f90 depends *directly*
on a module in y.f90, write a dependency line:
x.o: x.f90 y.o

The scripts that produce dependency info are typically themselves
dependent on the way a particular compiler treats the module files.
The above solution seems to be 100% portable.

This will not work if you plan to manually delete modules (but I can
see no good reason to do this).

Regards,
Artur Swietanowski
----------------------------------------------------------------------
Artur Swietanowski mailto:swieta...@smc.univie.ac.at
Institut fuer Statistik, Operations Research und Computerverfahren,
Universitaet Wien, Universitaetsstr. 5, A-1010 Wien, Austria
tel. +43 (1) 407 63 55 - 120 fax +43 (1) 406 41 59
----------------------------------------------------------------------

[Peter Shenkin]

Swietanowski Artur wrote:
>
> Coleman Federal Special Programs Group wrote:
> >
> > Does anyone have suggestions for establishing module dependencies
> > in make files?
>
> A clean and simple solution: if source file x.f90 depends *directly*
> on a module in y.f90, write a dependency line:
> x.o: x.f90 y.o
>
> The scripts that produce dependency info are typically themselves
> dependent on the way a particular compiler treats the module files.
> The above solution seems to be 100% portable.
>
> This will not work if you plan to manually delete modules (but I can
> see no good reason to do this).

It also won't work if you want to distribute binary libraries and
module files for users to link with. (I mean, it won't work in
your customer's Makefile.) But then again, F90 doesn't provide any
way to make this work. :-)

I use the method Artur mentions in my own Makefiles, however. There
are several gotchas, however. Suppose the machine in question
produces a y.mod file which x.f90 "actually" needs at compile time.

If the interfaces don't change, y.mod will stay the same even though
y.o changes. Then you are recompiling x.f90 when you don't have to.
For a big software package, where there might be 100 x's that depend
on y, this is a big problem.

Second, if the compiler generates y.mod after it generates y.o, then,
in a parallel make environment, one might recompile x.f90, but pick
up the wrong (old and obsolete) y.mod file.

-P.

--
****************** Sir Isaiah Berlin, 1909 - 1997, RIP
********************
* Peter S. Shenkin; Chemistry, Columbia U.; 3000 Broadway, Mail Code
3153 *
** NY, NY 10027; she...@columbia.edu; (212)854-5143; FAX: 678-9039
***
*MacroModel WWW page:
http://www.columbia.edu/cu/chemistry/mmod/mmod.html *

[P. Bernedo x???? A??? gmv/fcsd/oad]

I have written a small C++ program to extract dependencies from fortran
90 programs. It assumes that fortran source programs have the .f90
extension and the module information files have the .M extension. This
is the way our F90 compiler works under Solaris. If your compiler uses
other extensions change the definition of the extension in the source
code.

The program can locate used files from a list of directories specified
by the user or can assume that all *.M files are in a given directory.
It writes the list of dependencies to stdout. A way to use the program
in the make file is shown below. You would type make depfor to generate
the fortran dependencies, just like make depend generates the C++
dependencies.

My utility creates dependency files, but it does not solve the problem
of the compiler generating a new *.M file even if only the
implementation changes.

MAKEFILE:

compile=CC -c -o $@ -DUNIX -native -fast -O4 +p +w
link=CC -o $@ -fast

compilefor=f90 -c -o $@ -native
compileM=f90 -c -o $*.o -native
linkfor=f90 -o $@

#suffixes
.SUFFIXES: .cpp .f90 .M

.cpp.o:
$(compile) $*.cpp

.f90.o:
$(compilefor) $*.f90

.f90.M:
$(compileM) $*.f90

#executables
TestSetsObjs = OGcst_TestSets.o OGcs_ConnectedSets.o TestSets.o

TestSets: $(TestSetsObjs)
$(linkfor) $(TestSetsObjs)

connObjs = connected.o testconn.o

conn: $(connObjs)
$(link) $(connObjs)

#dependencies

include fordeps.txt

depfor:
fordeps >fordeps.txt *.f90

depend:
makedepend -DUNIX *.cpp *.h
# DO NOT DELETE THIS LINE -- make depend depends on it.

FORDEPS.CPP:

Compile with CC fordeps.cpp

// Copyright 1997 by P. Bernedo.
// Send comments and bug reports to pber...@esoc.esa.de

#include <string.h>
#include <stdio.h>
#include <ctype.h>

enum { NO, YES };

// A node in the list of directories to search.
struct fdep_SearchDirs
{
char name[200];
int length;
fdep_SearchDirs *next;
};

// Actual computation of dependencies.
class fdep_Deps
{
FILE * d_output; // Send the output here.
char d_deps[1000], *d_next;
int d_lowercase; // Output file names in lowercase?
fdep_SearchDirs *d_root; // List of directories to search.
char d_modulesDir[1000], *d_toWrite;
int d_useModulesDir; // Assume all *.M are one directory?
int d_lakosFormat; // Output for Lakos' idep?

static char *s_srcExt; // Source module extension. .f90 on
Sun.
static char *s_modExt; // Extension of the compiler
generated
// file. .M on Sun.
int d_srcExtLength; // Length of the source extension.

void ProcessLine(char *s);

// Convert the string s to lower case
void ToLowercase(char *s);

// Check if there is a '!' character between start and target.
// Return true if it is (it means that target is commented)
int IsCommentedBefore(const char *start, const char *target);

const char * QualifiedName(const char *base);
const char * QualifiedNameWithFixed( const char *base );
const char * QualifiedNameWithSearch( const char *base );
const char * FixedTarget( const char *name );

public:
fdep_Deps(int lowercase = NO);
~fdep_Deps();

void AddSearchDir(const char *dir);
void SetLowercase(int flag) { d_lowercase = flag; }
void SetModulesDir( const char *dir );
void SetLakosianFormat(int flag) { d_lakosFormat = flag; }

void ProcessFile(char *name);

// Set the stream where dependencies will be sent to.
// If you don't set it, the default is to use stdout
void SetOutput(FILE *f);
};

const char DirSep = '/'; // Change this if this is ever ported to DOS.
char * fdep_Deps::s_srcExt = ".f90";
char * fdep_Deps::s_modExt = ".M";

fdep_Deps::fdep_Deps(int lowercase)
: d_output(stdout)
, d_next(0)
, d_lowercase(lowercase)
, d_root(0)
, d_useModulesDir(NO)
, d_toWrite(NULL)
, d_lakosFormat(NO)
{
d_srcExtLength = strlen( s_srcExt );
}

fdep_Deps::~fdep_Deps()
{
// Clean the directories list
fdep_SearchDirs *temp;
while (d_root)
{
temp = d_root->next;
delete d_root;
d_root = temp;
}
}

void fdep_Deps::SetOutput(FILE *f)
{
d_output = f;
}

// Add one directory at the end of the search list. Keep the order in
// the list as in the argument list that the user typed.
void fdep_Deps::AddSearchDir(const char *dir)
{
// Create the new directory information.
fdep_SearchDirs *sdp = new fdep_SearchDirs;
strcpy(sdp->name, dir);
sdp->length = strlen(dir);

// Make sure that it ends with a slash.
if (sdp->name[sdp->length - 1] != DirSep)
{
sdp->name[sdp->length++] = DirSep;
sdp->name[sdp->length] = 0;
}

// This is the last in the current list.
sdp->next = 0;

// Put it in the list.
if (d_root)
{
fdep_SearchDirs *prev = d_root;
while (prev->next) prev = prev->next;
prev->next = sdp;
}
else
{
d_root = sdp;
}
}

// Set the directory where all modules are kept.
void fdep_Deps::SetModulesDir( const char *dir )
{
strcpy( d_modulesDir, dir );
d_toWrite = d_modulesDir + strlen( d_modulesDir );
if (d_toWrite[-1] != DirSep)
{
*d_toWrite++ = DirSep;
*d_toWrite = 0;
}
d_useModulesDir = YES;
}

// Assume that the target is in the modules directory. This function
// takes the given name and creates a name that is in the modules
// directory.
const char * fdep_Deps::FixedTarget( const char *name )
{
const char *last = name + strlen(name) - 1;

while ( last >= name && *last != DirSep )
{
last--;
}
strcpy( d_toWrite, last + 1 );

return d_modulesDir;
}

// Process a f90 file.
void fdep_Deps::ProcessFile(char *name)
{
FILE *f = fopen(name, "r");
if (f)
{
d_next = d_deps;

char s[200];
while (fgets(s, sizeof s, f) != 0)
{
ProcessLine(s);
}
fclose(f);

if (d_deps == d_next) return;

char *cp = name + strlen(name) - d_srcExtLength;
if (strcmp(cp, s_srcExt) == 0)
{
*cp = 0;
if (d_lowercase)
{
ToLowercase(name);
}
if (d_useModulesDir)
{
if (d_lakosFormat)
{
fprintf(d_output, "%s%s %s\n\n", FixedTarget(name),
s_modExt, d_deps);
}
else
{
fprintf(d_output, "%s%s: %s\n", FixedTarget(name),
s_modExt, d_deps);
}
}
else
{
if (d_lakosFormat)
{
fprintf(d_output, "%s%s %s\n\n", name,
s_modExt, d_deps);
}
else
{
fprintf(d_output, "%s%s: %s\n", name,
s_modExt, d_deps);
}
}
if (!d_lakosFormat)
{
fprintf(d_output, "%s.o: %s\n", name, d_deps);
}
}
}
else
{
printf("could not open file %s\n", name);
}
}

// Detect commented use statements.
int fdep_Deps::IsCommentedBefore(const char *start, const char *target)
{
while (start < target && *start != '!')
{
start++;
}

return start != target;
}

// Return the full name of the file, assuming that it is either in the
// modules directory (if the user says so), in the search directories
// or in the current directory.
const char * fdep_Deps::QualifiedName( const char *base )
{
if ( d_useModulesDir )
{
return QualifiedNameWithFixed( base );
}
else
{
return QualifiedNameWithSearch( base );
}
}

const char * fdep_Deps::QualifiedNameWithFixed( const char *base )
{
strcpy( d_toWrite, base );
return d_modulesDir;
}

// Search for files in the list of directories. The files are the name
// stated in the module, plus the *.f90 or *.M extension.
const char * fdep_Deps::QualifiedNameWithSearch(const char *base)
{
const char *result = 0;
fdep_SearchDirs *sdp = d_root;

while (sdp)
{
sprintf(sdp->name + sdp->length, "%s%s", base, s_modExt);

FILE *test = fopen(sdp->name, "r");
if (test == 0)
{
sprintf(sdp->name + sdp->length, "%s%s", base, s_srcExt);
test = fopen(sdp->name, "r");
}
if (test)
{
// the file exists
sprintf(sdp->name + sdp->length, "%s", base);
fclose(test);
result = sdp->name;
break;
}

sdp = sdp->next;
}

if (result == 0)
{
result = base;
}

return result;
}

void fdep_Deps::ProcessLine(char *s)
{
// skip white space
while (isspace(*s)) s++;

// did we find use or USE ?
if (strncmp(s, "use ", 4) == 0 || strncmp(s, "USE ", 4) == 0)
{
char *found = s + 4;

if (*found)
{
// skip white space in front of the module name
while (isspace(*found)) found++;
char *start = found;

// collect the name of the module
while (isalnum(*found) || *found == '_') found++;
*found++ = 0;

if (d_lowercase)
{
ToLowercase(start);
}

d_next += sprintf(d_next, "%s%s ", QualifiedName(start),
s_modExt);

if (d_next > d_deps + sizeof d_deps)
{
puts("too many dependencies");
return;
}
}
}
}

// You know, strlwr is not part of ANSI, although most PC compilers
// have it.
void fdep_Deps::ToLowercase(char *s)
{
while (*s)
{
*s = tolower(*s);
s++;
}
}

int main(int argc, char **argv)
{
fdep_Deps deps;
int i;

if (argc == 1)
{
// Bla, bla.
puts("This is fordeps, the fortran 90 dependencies generator");
puts("Written by P. Bernedo, x3497");
puts("\nUsage is fordeps -l -d -M<dir> files");
puts("or fordeps -l -d -D<moddir> files");
puts("\nThe program scans the files looking for USE
statements.");
puts("It generates the dependencies based on the modules
used.");
puts("\nThe -l option converts all files to lower case before
writing");
puts("the dependencies.");
puts("\nEach use of the -M option adds a directory to the search
list");
puts("\nThe following line:");
puts("\n fordeps -l -M/home/pbernedo/develop/hpt
-M/home/napeos/lib *f90");
puts("\ntells the program to:");
puts("\n 1.- convert all file names to lower case. For
instance, if there");
puts(" is a statement USE Abc, the name of the file will be
abc,");
puts(" not Abc.");
puts("\n 2.- For each of the module names specified in the use
statements");
puts(" seach in the directories /home/pbernedo,
/home/napeos/lib for");
puts(" a file with the same name as the module and the .f90
or .M");
puts(" extension.");
puts("\nIf neither the module.f90 or module.M files are found in
the list");
puts("of directories the module.M name is output (it assumes
that it belongs");
puts("to the current directory.)");
puts("\nIf the -D option is used, what follows the D is
considered to be");
puts("the directory where all .M files are stored. The target
and dependent");
puts(".M files will be assumed to be there.");
puts("\nThe dependencies are sent to stdout. Commented use
statements are");
puts("ignored. If relative path names are given with the -M
option, the");
puts("dependencies will be output with relative path names.");
puts("\nThe -d option tells fordeps to produce the output in");
puts("the format suitable for the ldep utility to compute the
CCD.");
puts("The ldep utility is part of the package dependencies
analyzer");
puts("described in John Lakos' book Large Scale C++ Software
Design.");
}

for (i = 1; i < argc; ++i)
{
if (argv[i][0] == '-')
{
if (argv[i][1] == 'l')
{
deps.SetLowercase(YES);
}
else if (argv[i][1] == 'M')
{
deps.AddSearchDir(argv[i] + 2);
}
else if (argv[i][1] == 'D')
{
deps.SetModulesDir(argv[i] + 2);
}
else if (argv[i][1] == 'd') // output for lakos' dependency
checker.
{
deps.SetLakosianFormat(YES);
}
}
}

for (i = 1; i < argc; ++i)
{
if (argv[i][0] != '-')
{
deps.ProcessFile(argv[i]);
}
}

return 0;
}

[P. Bernedo x???? A??? gmv/fcsd/oad]