Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Circular module dependency
37 views
Skip to first unread message
Qolin
Mar 1, 2012, 5:19:07 PM3/1/12
to
I have today spent some hours chasing a build problem that turned out to be
caused by a circular module dependency. The circle in question contained
just 3 files, but my Windows Developer Studio running IVF 12.1 gave me a
complete run-around, being unable to compile scores of files, complaining of
absense of one or modules. I eventually diagnosed the culprit files, but I
had to reverse a few hours worth of work to do so.
I have considered writing a module dependency checker to detect such
circles, but thought I'd check here first to see if anyone else knows where
I can get a ready-made one. Also, any intelligence on how DevStudio and/or
IVF does its dependency analysis, and why it can't diagnose this problem?
Qolin
qolin at domain dot com
domain is qomputing
caused by a circular module dependency. The circle in question contained
just 3 files, but my Windows Developer Studio running IVF 12.1 gave me a
complete run-around, being unable to compile scores of files, complaining of
absense of one or modules. I eventually diagnosed the culprit files, but I
had to reverse a few hours worth of work to do so.
I have considered writing a module dependency checker to detect such
circles, but thought I'd check here first to see if anyone else knows where
I can get a ready-made one. Also, any intelligence on how DevStudio and/or
IVF does its dependency analysis, and why it can't diagnose this problem?
Qolin
qolin at domain dot com
domain is qomputing
gmail-unlp
Mar 2, 2012, 8:28:37 AM3/2/12
to
Fernando.
Richard Maine
Mar 2, 2012, 11:32:54 AM3/2/12
to
gmail-unlp <ftin...@gmail.com> wrote:
> On Mar 1, 7:19 pm, "Qolin" <no...@nowhere.com> wrote:
> > I have today spent some hours chasing a build problem that turned out to be
> > caused by a circular module dependency.
> On Mar 1, 7:19 pm, "Qolin" <no...@nowhere.com> wrote:
> > I have today spent some hours chasing a build problem that turned out to be
> > caused by a circular module dependency.
> Does the (any) standard define any thing related to this issue?
Yes, related, but no, not what Qoin is looking for.
The standard specifies that circular module dependencies are
non-conforming. But that's among the many things that the standard says
nothing about diagnostic requirements for.
--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
Daniel H
Mar 3, 2012, 5:03:30 AM3/3/12
to
>> On Mar 1, 7:19 pm, "Qolin"<no...@nowhere.com> wrote:
>>> I have today spent some hours chasing a build problem that turned out to be
>>> caused by a circular module dependency.
I encountered a similar problem this week. I did get a message that
>>> I have today spent some hours chasing a build problem that turned out to be
>>> caused by a circular module dependency.
there was a circular dependency, but didn't know which files were
involved. So I would be very interested in any tool that helps to
diagnose this.
Richard Maine
Mar 3, 2012, 11:48:00 AM3/3/12
to
Richard Maine <nos...@see.signature> wrote:
> gmail-unlp <ftin...@gmail.com> wrote:
>
> > On Mar 1, 7:19 pm, "Qolin" <no...@nowhere.com> wrote:
> > > I have today spent some hours chasing a build problem that turned out
> > > to be caused by a circular module dependency.
>
> > Does the (any) standard define any thing related to this issue?
>
> Yes, related, but no, not what Qoin is looking for.
Oops. I do know Qolin's name (and even once met him briefly). That was a
> gmail-unlp <ftin...@gmail.com> wrote:
>
> > On Mar 1, 7:19 pm, "Qolin" <no...@nowhere.com> wrote:
> > > I have today spent some hours chasing a build problem that turned out
> > > to be caused by a circular module dependency.
>
> > Does the (any) standard define any thing related to this issue?
>
> Yes, related, but no, not what Qoin is looking for.
typo.
Ron Shepard
Mar 3, 2012, 11:48:46 AM3/3/12
to
In article <jisq4b$18j$1...@dont-email.me>,
Such a message would have to come from the make utility or whatever
development environment you were using at the time. It seems
impossible for the compiler itself to detect something like this.
All the compiler does at any point is to search for mod files that
match USE statements. Unless the source of a mod file is the file
it is processing at that time, it cannot know the source of those
mod files. Several mod files can be generated from a single source
file, and during development and debug stages, you might even have
several possible source files that produce a particular mod file. Or
in some cases, the mod files might not even be in the local build
directory. So to the compiler, it would be like like a many-to-many
mapping problem, impossible to solve.
The other problem also exists. You can have a circular dependence
among your files, but the compiler might not be able to detect it.
If an "old" version of a mod file exists, then that will satisfy the
compiler's search triggered by a USE statement. Then later when the
correct mod file is created, it might not be able to recognize that
it has already accessed the outdated one at some previous step. I
sometimes get in this situation during development when I am moving
modules around within a set of files, and I don't recognize my
problem until I do a "make clean" to start the build process from
scratch.
With f90 and later, this file dependency structure is an essential
part of the compilation process. Makefiles should always be
distributed along with source code in order to document this
dependency structure. With f77, this did not matter, the files
could be compiled in any order, and it was only the link step where
order might make a difference. With modern fortran, this is
essential knowledge for the compilation step itself.
I agree that a standard tool that determines module/file
dependencies, including detecting circular ones, would be useful
(and would have been useful over the past 20 years). I usually do
something like "grep -i use" to get a listing of modules that are
needed by a particular file, but there is still the step of
determining from which files these modules are created. If you
always develop codes from the bottom up, then this is all fairly
straightforward to document (e.g. with Makefiles) during the
process. The difficulty comes in when you want to mix in some
top-down development (using modules that don't yet really exist) or
to take an existing set of files and refactor them (moving modules
or other information around within the set of files).
$.02 -Ron Shepard
development environment you were using at the time. It seems
impossible for the compiler itself to detect something like this.
All the compiler does at any point is to search for mod files that
match USE statements. Unless the source of a mod file is the file
it is processing at that time, it cannot know the source of those
mod files. Several mod files can be generated from a single source
file, and during development and debug stages, you might even have
several possible source files that produce a particular mod file. Or
in some cases, the mod files might not even be in the local build
directory. So to the compiler, it would be like like a many-to-many
mapping problem, impossible to solve.
The other problem also exists. You can have a circular dependence
among your files, but the compiler might not be able to detect it.
If an "old" version of a mod file exists, then that will satisfy the
compiler's search triggered by a USE statement. Then later when the
correct mod file is created, it might not be able to recognize that
it has already accessed the outdated one at some previous step. I
sometimes get in this situation during development when I am moving
modules around within a set of files, and I don't recognize my
problem until I do a "make clean" to start the build process from
scratch.
With f90 and later, this file dependency structure is an essential
part of the compilation process. Makefiles should always be
distributed along with source code in order to document this
dependency structure. With f77, this did not matter, the files
could be compiled in any order, and it was only the link step where
order might make a difference. With modern fortran, this is
essential knowledge for the compilation step itself.
I agree that a standard tool that determines module/file
dependencies, including detecting circular ones, would be useful
(and would have been useful over the past 20 years). I usually do
something like "grep -i use" to get a listing of modules that are
needed by a particular file, but there is still the step of
determining from which files these modules are created. If you
always develop codes from the bottom up, then this is all fairly
straightforward to document (e.g. with Makefiles) during the
process. The difficulty comes in when you want to mix in some
top-down development (using modules that don't yet really exist) or
to take an existing set of files and refactor them (moving modules
or other information around within the set of files).
$.02 -Ron Shepard
Richard Maine
Mar 3, 2012, 12:17:14 PM3/3/12
to
Ron Shepard <ron-s...@NOSPAM.comcast.net> wrote:
> I agree that a standard tool that determines module/file
> dependencies, including detecting circular ones, would be useful
> (and would have been useful over the past 20 years). I usually do
> something like "grep -i use" to get a listing of modules that are
> needed by a particular file, but there is still the step of
> determining from which files these modules are created.
It has been a while, but I used to use a tool called makedepf90. A quick
> I agree that a standard tool that determines module/file
> dependencies, including detecting circular ones, would be useful
> (and would have been useful over the past 20 years). I usually do
> something like "grep -i use" to get a listing of modules that are
> needed by a particular file, but there is still the step of
> determining from which files these modules are created.
google shows it is still around, though not actively maintained.
It doesn't do the circular detection (or I'd have mentioned it right
off), and it does have its limitations, but I found it useful. It beats
manual use of grep.
Arjan
Mar 3, 2012, 12:35:22 PM3/3/12
to
I wrote my own utility "getdeps.f90" to generate a fresh list of
source file names in order of dependence. I generally include it
inside my call to the compiler, like:
gfortran `getdeps.exe myproject.f90`
The core of this utility is, as suggested by Ron Shepard, an external
call like "grep -i use *.f90" (I don't remember the exact details, it
works for me now ;-)) ). I imposed a restriction to my source files: I
use 1 source file per MODULE and the filename equals the module name:
MODULE libxmath can be found in source file libxmath.f90. This
convenstion was used by Borland Pascal some decades ago and I have
always found it quite intuitive. I regularly swap routines or
operators from 1 module to another, or I split some module into 2
different ones after a module has grown out of its clothes. Never a
problem when I re-compile a program! People interested can have my
utility. It's not fool-proof: it's only me who uses it...
Now that I see the problem of circular referencing, I'll update my
routine accordingly!
Alternatively, there is a Perl script named "mkmf.pl", that can be
found on the Internet. It generates dependency-sorted Makefiles by
checking the source code. Maybe that routine is smarter than mine?
Arjan
source file names in order of dependence. I generally include it
inside my call to the compiler, like:
gfortran `getdeps.exe myproject.f90`
The core of this utility is, as suggested by Ron Shepard, an external
call like "grep -i use *.f90" (I don't remember the exact details, it
works for me now ;-)) ). I imposed a restriction to my source files: I
use 1 source file per MODULE and the filename equals the module name:
MODULE libxmath can be found in source file libxmath.f90. This
convenstion was used by Borland Pascal some decades ago and I have
always found it quite intuitive. I regularly swap routines or
operators from 1 module to another, or I split some module into 2
different ones after a module has grown out of its clothes. Never a
problem when I re-compile a program! People interested can have my
utility. It's not fool-proof: it's only me who uses it...
Now that I see the problem of circular referencing, I'll update my
routine accordingly!
Alternatively, there is a Perl script named "mkmf.pl", that can be
found on the Internet. It generates dependency-sorted Makefiles by
checking the source code. Maybe that routine is smarter than mine?
Arjan
Jinsong Zhao
Mar 3, 2012, 9:49:17 PM3/3/12
to
On 2012-3-4 1:35, Arjan wrote:
<snip>
I organized my source file in the same way. However, I have encountered
a difficult problem using this way.
I use one source file per subroutine/function, and put the
subroutine/function into a MODULE, something like:
!mod_sub_a.f90
Module mod_sub_a
use mode_sub_b
contains
subroutine sub_a()
write(*,*) 'calling sub_b'
call sub_b()
end subroutine sub_a
end module mod_sub_a
However, if mod_sub_b is defined as following:
!mod_sub_b.f90
Module mod_sub_b
use mode_sub_a
contains
subroutine sub_b()
write(*,*) 'calling sub_a'
call sub_a()
end subroutine sub_b
end module mod_sub_b
Then, the program can't be compiled. How to break the cycle?
Thanks in advance.
Regards,
Jinsong
<snip>
> works for me now ;-)) ). I imposed a restriction to my source files: I
> use 1 source file per MODULE and the filename equals the module name:
> MODULE libxmath can be found in source file libxmath.f90. This
> convenstion was used by Borland Pascal some decades ago and I have
> always found it quite intuitive. I regularly swap routines or
> operators from 1 module to another, or I split some module into 2
> different ones after a module has grown out of its clothes. Never a
> problem when I re-compile a program! People interested can have my
> utility. It's not fool-proof: it's only me who uses it...
>
<snip>
> use 1 source file per MODULE and the filename equals the module name:
> MODULE libxmath can be found in source file libxmath.f90. This
> convenstion was used by Borland Pascal some decades ago and I have
> always found it quite intuitive. I regularly swap routines or
> operators from 1 module to another, or I split some module into 2
> different ones after a module has grown out of its clothes. Never a
> problem when I re-compile a program! People interested can have my
> utility. It's not fool-proof: it's only me who uses it...
>
I organized my source file in the same way. However, I have encountered
a difficult problem using this way.
I use one source file per subroutine/function, and put the
subroutine/function into a MODULE, something like:
!mod_sub_a.f90
Module mod_sub_a
use mode_sub_b
contains
subroutine sub_a()
write(*,*) 'calling sub_b'
call sub_b()
end subroutine sub_a
end module mod_sub_a
However, if mod_sub_b is defined as following:
!mod_sub_b.f90
Module mod_sub_b
use mode_sub_a
contains
subroutine sub_b()
write(*,*) 'calling sub_a'
call sub_a()
end subroutine sub_b
end module mod_sub_b
Then, the program can't be compiled. How to break the cycle?
Thanks in advance.
Regards,
Jinsong
Richard Maine
Mar 3, 2012, 10:34:37 PM3/3/12
to
Jinsong Zhao <jsz...@yeah.net> wrote:
> On 2012-3-4 1:35, Arjan wrote:
> <snip>
> > works for me now ;-)) ). I imposed a restriction to my source files: I
> > use 1 source file per MODULE and the filename equals the module name:
> > MODULE libxmath can be found in source file libxmath.f90. This
> > convenstion was used by Borland Pascal some decades ago and I have
> > always found it quite intuitive. I regularly swap routines or
> > operators from 1 module to another, or I split some module into 2
> > different ones after a module has grown out of its clothes. Never a
> > problem when I re-compile a program! People interested can have my
> > utility. It's not fool-proof: it's only me who uses it...
> >
> <snip>
>
> I organized my source file in the same way. However, I have encountered
> a difficult problem using this way.
>
> I use one source file per subroutine/function, and put the
> subroutine/function into a MODULE...
> On 2012-3-4 1:35, Arjan wrote:
> <snip>
> > works for me now ;-)) ). I imposed a restriction to my source files: I
> > use 1 source file per MODULE and the filename equals the module name:
> > MODULE libxmath can be found in source file libxmath.f90. This
> > convenstion was used by Borland Pascal some decades ago and I have
> > always found it quite intuitive. I regularly swap routines or
> > operators from 1 module to another, or I split some module into 2
> > different ones after a module has grown out of its clothes. Never a
> > problem when I re-compile a program! People interested can have my
> > utility. It's not fool-proof: it's only me who uses it...
> >
> <snip>
>
> I organized my source file in the same way. However, I have encountered
> a difficult problem using this way.
>
> I use one source file per subroutine/function, and put the
[simple example of circular dependence]
Actually, the code exactly as shown also has other problems. The
subroutines are invoked recursively without being declared recursive.
That would be easy to fix. But the recursion is also infinite, which is
a more basic problem. I'll assume that those problems are just artifacts
of attempting to simplify the exposition of the module circularity
problem, and that they have nothing to do with your actual problem. If
your actual code has infinite recursion, then it needs algorithmic
attention rather than just language features. Anyway, ignoring those
problems (which should not normally show up as compilation failures) ...
> Then, the program can't be compiled. How to break the cycle?
source file per module organization that Arjan describes.
It does, however, look to come from putting one procedure per module.
That is *NOT* "organized...in the same way" as Arjen described, unless I
badly misread his description.
I recommend against that as a general rule. Sure, there are cases where
a module naturally has only a single routine; when that's what is
natural for a particular module, do it. But if you try to make it a
general rule, you will just exacerbate things like the module
circularity problem. In many cases where a set of procedures have
extensive mutual references, those procedures logically go well together
in a single module, which will not then have any problem with circular
module dependency. That's not true in all cases, but it is so in many of
them. I think you are liklely just causing your own problems by forcing
a one-procedure-per-module straightjacket on your code.
There are certainly some cases where grouping multiple procedures into
the same module doesn't work out well as a way to avoid circular
dependency. There are then several approaches.
In my opinion, the best approach is often to take advantage of
submodules. Unfortunately, submodules are likely not supported yet in
many compilers, so that might not be an option for you. Other approaches
often seem like hackish workarounds, I'd agree.
One of the hackish workarounds is to make the relevant procedures
external procedures instead of module ones. Your modules can then just
have interface blocks with interface bodies for the procedures. I really
don't like that approach. But it will work to remove the circular module
dependency.
Arjan
Mar 4, 2012, 1:44:23 PM3/4/12
to
> It doesn't sound to me like your problem has anything to do with the one
> source file per module organization that Arjan describes.
>
> It does, however, look to come from putting one procedure per module.
> That is *NOT* "organized...in the same way" as Arjen described, unless I
> badly misread his description.
Richard, you're right: I put 1 MODULE per source file, but my modules
> source file per module organization that Arjan describes.
>
> It does, however, look to come from putting one procedure per module.
> That is *NOT* "organized...in the same way" as Arjen described, unless I
> badly misread his description.
generally contain a lot of (related!) stuff: types, operators,
parameters, functions, procedures and the occasional variable. One
detail: Arjan is not the same as Arjen! Both appear on this forum,
but apart from most letters in our names, our home-country and some
interest in fortran, we don't share anything! That is to say: we did
exchanged some source-files a while ago...
Arjan
Richard Maine
Mar 4, 2012, 4:10:58 PM3/4/12
to
Arjan <arjan.v...@rivm.nl> wrote:
> > It doesn't sound to me like your problem has anything to do with the one
> > source file per module organization that Arjan describes.
> >
> > It does, however, look to come from putting one procedure per module.
> > That is *NOT* "organized...in the same way" as Arjen described, unless I
> > badly misread his description.
>
> > It doesn't sound to me like your problem has anything to do with the one
> > source file per module organization that Arjan describes.
> >
> > It does, however, look to come from putting one procedure per module.
> > That is *NOT* "organized...in the same way" as Arjen described, unless I
> > badly misread his description.
>
> One
> detail: Arjan is not the same as Arjen! Both appear on this forum,
> but apart from most letters in our names, our home-country and some
> interest in fortran, we don't share anything! That is to say: we did
> exchanged some source-files a while ago...
Oops. Sorry. I do know about there being the two of you here, but I'm
> detail: Arjan is not the same as Arjen! Both appear on this forum,
> but apart from most letters in our names, our home-country and some
> interest in fortran, we don't share anything! That is to say: we did
> exchanged some source-files a while ago...
afraid I do get the names confused at times. I see that I did it both
ways above. That wasn't to cover both bases; I just messed up.
I'm used to name confusions applied to me because there are so many of
us Richards that it is often ambiguous. There was a time when there were
about 6 people on the Fortran committee named either Richard or Dick
(Dick being a common nickname for Richard for reasons that I don't
recall; it's not one I usually use, but I'm used to some people calling
me by that anyway). I recall one committee discussion when the chair
called out for Dick to speak, and about 3 of us started at once. One of
the women on the committee then commented that "one problem with this
committee is that there are too many Dicks on it." That got a lot of
laughs, most of them as much as anything about how red her face then
turned.
Thomas Jahns
Mar 5, 2012, 8:06:10 AM3/5/12
to
We use a dependency tracker, maintained by me[1], that currently does the following:
1. parses fortran source including cpp directives
2. evaluates cpp macros to determine effective USE statements,
i.e. the tracker doesn't generate dependencies for stuff that
is effectively inside #if 0
3. finds external .mod files
4. can handle multiple modules per source, independent of source name
5. only writes the dependency part of the makefile (because in our setup that
is maintained by automake/autoconf)
I'm currently in the process of making a release of the library[2], that the
dependency tracker is included in, but if you're interested, the development
version can be retrieved via:
$ git clone git://eyewall.dkrz.de/scales-ppm
The interesting parts are:
- the tracker in config/makef90depends.in
- an example of its use in lines 130-142 of src/Makefile.am
- some wrappers in directory util for compilers that cannot produce
pre-processed output easily (because 2. from above requires running
the compiler sometimes)
- some macros for autoconf in directory m4 to discover various flags and
module related properties of the compiler
In case there is sufficient interest, I could certainly make a stand-alone
version of the dependency tracker available.
Regards, Thomas
[1] so I'm obviously a little biased here
[2] https://www.dkrz.de/redmine/projects/scales-ppm
1. parses fortran source including cpp directives
2. evaluates cpp macros to determine effective USE statements,
i.e. the tracker doesn't generate dependencies for stuff that
is effectively inside #if 0
3. finds external .mod files
4. can handle multiple modules per source, independent of source name
5. only writes the dependency part of the makefile (because in our setup that
is maintained by automake/autoconf)
I'm currently in the process of making a release of the library[2], that the
dependency tracker is included in, but if you're interested, the development
version can be retrieved via:
$ git clone git://eyewall.dkrz.de/scales-ppm
The interesting parts are:
- the tracker in config/makef90depends.in
- an example of its use in lines 130-142 of src/Makefile.am
- some wrappers in directory util for compilers that cannot produce
pre-processed output easily (because 2. from above requires running
the compiler sometimes)
- some macros for autoconf in directory m4 to discover various flags and
module related properties of the compiler
In case there is sufficient interest, I could certainly make a stand-alone
version of the dependency tracker available.
Regards, Thomas
[1] so I'm obviously a little biased here
[2] https://www.dkrz.de/redmine/projects/scales-ppm
Arjan
Mar 7, 2012, 1:47:47 PM3/7/12
to
> In case there is sufficient interest, I could certainly make a stand-alone
> version of the dependency tracker available.
That would be a good idea, Thomas!
> version of the dependency tracker available.
Arjan
Daniel H
Mar 8, 2012, 2:15:39 AM3/8/12
to
Daniel
Qolin
Mar 9, 2012, 3:09:06 AM3/9/12
to
"Daniel H" wrote in message news:jj9m64$qnn$1...@dont-email.me...
Qolin
0 new messages