info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Distribution of Fortran libraries in 2018
317 views
Skip to first unread message
Dominik Gronkiewicz
Jul 7, 2018, 5:59:30 PM7/7/18
to
With introduction of Fortran modules, a problem arose how library developers should package and distribute their libraries. In C/C++, a typical way is to install a binary blob to /usr/lib and a header (.h or .hpp) to /usr/include which allows to compile applications with that library. In Fortran, .mod file is required to compile the code that uses a module, but that file is compiler and architecture dependent (not even portable against different versions of the same compiler). There are at least 4 approaches that I am aware of:
1. For code that uses some limited subset of language (no deferred shape arrays etc) only the binary file is needed.
2. One can distribute and install module files. The problem is that the user is stuck with the exact compiler that was used to build the library (usually gfortran). Another issue is that there is no standard location for those files. In Red Hat based systems (RHEL, Fedora, Scientific Linux), this directory is /usr/lib64/gfortran/modules (replace lib64 with lib for 32 bit systems), and this is the installation path I assume in my makefiles.
3. One can write procedure interfaces in a "header" file which then is "included" in the program. This is the way how FFTW does it. It's a bit ugly but if the library contains only subroutines (no constants, derived types etc), it does its job.
4. Submodules -- very similar to 3, but more "Fortran way", and allows for using all language features. The downside is that only very recent compilers support it (and compilers 2-3 years old have very buggy support). Also, it is not completely clear to me how to include that in the compilation process.
I only have experience with Linux, so I have no idea which of these approaches is the best portability wise.
What is your take on the subject? Thanks in advance for opinions and suggestions!
Dominik
1. For code that uses some limited subset of language (no deferred shape arrays etc) only the binary file is needed.
2. One can distribute and install module files. The problem is that the user is stuck with the exact compiler that was used to build the library (usually gfortran). Another issue is that there is no standard location for those files. In Red Hat based systems (RHEL, Fedora, Scientific Linux), this directory is /usr/lib64/gfortran/modules (replace lib64 with lib for 32 bit systems), and this is the installation path I assume in my makefiles.
3. One can write procedure interfaces in a "header" file which then is "included" in the program. This is the way how FFTW does it. It's a bit ugly but if the library contains only subroutines (no constants, derived types etc), it does its job.
4. Submodules -- very similar to 3, but more "Fortran way", and allows for using all language features. The downside is that only very recent compilers support it (and compilers 2-3 years old have very buggy support). Also, it is not completely clear to me how to include that in the compilation process.
I only have experience with Linux, so I have no idea which of these approaches is the best portability wise.
What is your take on the subject? Thanks in advance for opinions and suggestions!
Dominik
Ian Harvey
Jul 7, 2018, 6:54:48 PM7/7/18
to
On 2018-07-08 07:29, Dominik Gronkiewicz wrote:
> With introduction of Fortran modules, a problem arose how library
> developers should package and distribute their libraries. In C/C++, a
> typical way is to install a binary blob to /usr/lib and a header (.h
> or .hpp) to /usr/include which allows to compile applications with
> that library. In Fortran, .mod file is required to compile the code
> that uses a module, but that file is compiler and architecture
> dependent (not even portable against different versions of the same
> compiler)...
> With introduction of Fortran modules, a problem arose how library
> developers should package and distribute their libraries. In C/C++, a
> typical way is to install a binary blob to /usr/lib and a header (.h
> or .hpp) to /usr/include which allows to compile applications with
> that library. In Fortran, .mod file is required to compile the code
> that uses a module, but that file is compiler and architecture
> dependent (not even portable against different versions of the same
An operating system distribution, almost by definition, provides a set
of rules and conventions for how software that is compatible with that
distribution needs to be packaged. Different distributions will have
different requirements.
Binary compatibility is by no means guaranteed across compilers/compiler
versions and compile options. This is not Fortran specific. If you
compile something in a way that is incompatible with the rules and
conventions of the distribution, then you have the same set of problems,
regardless of language, with the binaries. Module files (if that's how
a compiler implements modules) are just a more visible manifestation of
this.
One of the rules and conventions of a distribution may well be that
there is *a* single system Fortran compiler/version/compile options
combination, and if you are using something that is incompatible with
that, then you are out of luck. This is certainly the approach taken
for for other languages.
> There are at least 4 approaches that I am aware of:
>
> 1. For code that uses some limited subset of language (no deferred
> shape arrays etc) only the binary file is needed.
specifics.
> 2. One can distribute and install module files. The problem is that
> the user is stuck with the exact compiler that was used to build the
> library (usually gfortran). Another issue is that there is no
> standard location for those files. In Red Hat based systems (RHEL,
> Fedora, Scientific Linux), this directory is
> /usr/lib64/gfortran/modules (replace lib64 with lib for 32 bit
> systems), and this is the installation path I assume in my
> makefiles.
distribution may be that you are stuck with that compiler.
(That directory seems silly - consider what would happen if packages had
conflicting names for modules. I would instead expect a package
specific directory under /usr/lib or similar. But then again, it is up
to the distribution, perhaps one of their rules and conventions is that
modules used in packages cannot have conflicting names.)
> 3. One can write procedure interfaces in a "header" file which then
> is "included" in the program. This is the way how FFTW does it. It's
> a bit ugly but if the library contains only subroutines (no
> constants, derived types etc), it does its job.
specifics.
> 4. Submodules -- very similar to 3, but more "Fortran way", and
> allows for using all language features. The downside is that only
> very recent compilers support it (and compilers 2-3 years old have
> very buggy support). Also, it is not completely clear to me how to
> include that in the compilation process.
is a non-issue.
> I only have experience with Linux, so I have no idea which of these
> approaches is the best portability wise.
>
> What is your take on the subject? Thanks in advance for opinions and
> suggestions! Dominik
FortranFan
Jul 8, 2018, 2:52:32 PM7/8/18
to
On Saturday, July 7, 2018 at 5:59:30 PM UTC-4, Dominik Gronkiewicz wrote:
> With introduction of Fortran modules, a problem arose how library developers should package and distribute their libraries. In C/C++, a typical way is to install a binary blob to /usr/lib and a header (.h or .hpp) to /usr/include which allows to compile applications with that library. In Fortran, .mod file is required to compile the code that uses a module, but that file is compiler and architecture dependent (not even portable against different versions of the same compiler). ..
More a question rather than anything else: how is the ever-present possibility of binary incompatibility of files generated using C compiler(s) handled on your system?
For compiled objects such as .o generated using C compilers (or their agglomeration in *.a on *ux systems or *.lib on Windows), one cannot rely on any compatibility across compiler versions (GCC/Intel/Microsoft/NAg) or OS targets (32 vs 64-bit) or OS versions (Linux vs Mac vs Windows or at times across OS versions of the same vendor - Microsoft Windows XP vs 10).
So how is this aspect addressed on your system? Therein may lie an option as to how to approach the distribution and consumption of Fortran modules files.
A few systems I know of resolve this by enforcing a *single* set of build chain tools to be used by all coders who are doing 'production' work such as a specific Fortran compiler and version XX, C/C++ compiler and version YY, linker and version ZZ, etc. Distribution packages then deploy Fortran compiled module (e.g., *.mod) files and C/C++ *.h/*.hpp files into a suitable system xx/include location, binary compiled objects *.a (*.lib) into xx/lib location, and shared libraries (*.so/*.dll) and program executables (*.exe) into xx/bin location. Build environments for Fortran and C/C++ are then made aware of the above INCLUDE location, linker of the above LIB location, and execution environments of the path to the BIN location.
By the way, it's unclear to me how the SUBMODULE facility introduced starting with Fortran 2008 is of any relevance with this aspect of packaging and distribution of Fortran module files? SUBMODULEs enable interface-driven implementation, a big help from a coding point of view particularly with teams. And SUBMODULEs can help avoid compilation cascades and in some cases, they can help resolve circular module dependencies. But otherwise, the compiled submodule files (usually *.smod) are in the same category of module files that have come into play since Fortran 90.
Anyways, see a somewhat related recent post:
https://groups.google.com/d/msg/comp.lang.fortran/p5DHMxvZ0To/W52DvlA4DQAJ
> With introduction of Fortran modules, a problem arose how library developers should package and distribute their libraries. In C/C++, a typical way is to install a binary blob to /usr/lib and a header (.h or .hpp) to /usr/include which allows to compile applications with that library. In Fortran, .mod file is required to compile the code that uses a module, but that file is compiler and architecture dependent (not even portable against different versions of the same compiler). ..
More a question rather than anything else: how is the ever-present possibility of binary incompatibility of files generated using C compiler(s) handled on your system?
For compiled objects such as .o generated using C compilers (or their agglomeration in *.a on *ux systems or *.lib on Windows), one cannot rely on any compatibility across compiler versions (GCC/Intel/Microsoft/NAg) or OS targets (32 vs 64-bit) or OS versions (Linux vs Mac vs Windows or at times across OS versions of the same vendor - Microsoft Windows XP vs 10).
So how is this aspect addressed on your system? Therein may lie an option as to how to approach the distribution and consumption of Fortran modules files.
A few systems I know of resolve this by enforcing a *single* set of build chain tools to be used by all coders who are doing 'production' work such as a specific Fortran compiler and version XX, C/C++ compiler and version YY, linker and version ZZ, etc. Distribution packages then deploy Fortran compiled module (e.g., *.mod) files and C/C++ *.h/*.hpp files into a suitable system xx/include location, binary compiled objects *.a (*.lib) into xx/lib location, and shared libraries (*.so/*.dll) and program executables (*.exe) into xx/bin location. Build environments for Fortran and C/C++ are then made aware of the above INCLUDE location, linker of the above LIB location, and execution environments of the path to the BIN location.
By the way, it's unclear to me how the SUBMODULE facility introduced starting with Fortran 2008 is of any relevance with this aspect of packaging and distribution of Fortran module files? SUBMODULEs enable interface-driven implementation, a big help from a coding point of view particularly with teams. And SUBMODULEs can help avoid compilation cascades and in some cases, they can help resolve circular module dependencies. But otherwise, the compiled submodule files (usually *.smod) are in the same category of module files that have come into play since Fortran 90.
Anyways, see a somewhat related recent post:
https://groups.google.com/d/msg/comp.lang.fortran/p5DHMxvZ0To/W52DvlA4DQAJ
Vishnu
Jul 10, 2018, 12:46:21 PM7/10/18
to
I happened to recently encounter this exact issue, and in looking for a resolution, happened upon the following "bug report":
Search /usr/local/include and /usr/include for .mod files
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35707
The solution described in it worked for me. I put the .mod files in:
/usr/{lib/lib64}/gcc/x86_64-<name-linux>/<gcc_version>/finclude/
This is indicative of the fact that these .mod files are only good for that particular version of gcc. Distributions will have to recompile it every time there is a major update to the compiler, and it can also accommodate having multiple versions of gcc.
Of course, all this is specific to linux as well as gcc. Many other compiler seem to look in /usr/include for .mod files, so you could also place them there, but this isn't as neat, and is misleading about there being a stable ABI.
Search /usr/local/include and /usr/include for .mod files
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35707
The solution described in it worked for me. I put the .mod files in:
/usr/{lib/lib64}/gcc/x86_64-<name-linux>/<gcc_version>/finclude/
This is indicative of the fact that these .mod files are only good for that particular version of gcc. Distributions will have to recompile it every time there is a major update to the compiler, and it can also accommodate having multiple versions of gcc.
Of course, all this is specific to linux as well as gcc. Many other compiler seem to look in /usr/include for .mod files, so you could also place them there, but this isn't as neat, and is misleading about there being a stable ABI.
Wolfgang Kilian
Jul 11, 2018, 3:49:39 AM7/11/18
to
between compiler vendors on this issue would be very welcome. The
bugzilla entry is 10 years old. This was a non-issue in the past since
there were virtually zero modern-Fortran projects that were
binary-packaged and distributed via the usual OS repositories, but the
situation may change.
In contrast to the bugzilla entry where /usr/include is recommended, I
would also vote for /usr/lib{64}. All .mod files are generated by a
compile process and can have any format, machine-readable, while C
include files are a special case of source code, in human-readable ASCII
format.
The solution above looks reasonable - the compiler vendor and compiler
version must be part of the path, or it will not work with several
compilers/versions installed in parallel.
The Fortran package may be compiled with the system compiler by default,
say gfortran-4.8. There could be add-on packages, say
my-package-gfortran-5, my-package-gfortran-6, etc., that install extra
.mod files and other binaries that work with another gfortran on the
same system. Analogously, my-package-ifort-16 etc. Installing any
add-on package should call for the matching compiler package as a
prerequisite.
-- Wolfgang
0 new messages