I think a scripting language such as Python is a good complement to a
compiled language such as Fortran or C++, but I would still prefer to
have the "guts" of a computationally intensive program in Fortran,
because of speed, finer control (Fortran KIND's for example), and the
greater number of programming errors that are typically detected by a
good compiler vs. an interpreter.

Here is his essay.

===

The established use of Fortran in continuum models such as climate
models has some benefits, including very high performance and
flexibility in dealing with regular arrays, backward compatibility with

the existing code base, and the familiarity with the language among the

modeling community. Fortran 90 and later versions have taken many of
the lessons of object oriented programming and adapted them so that
logical separation of modules is supported, allowing for more effective

development of large systems. However, there are many purposes to which

Fortran is ill-suited which are increasingly part of the modeling
environment.

These include: source and version control and audit trails for runs,
build system management, test specification, deployment testing (across

multiple platforms), post-processing analysis, run-time and
asynchronous visualization, distributed control and ensemble
management. To achieve these goals, a combination of shell scripts,
specialized build tools, specialized applications written in several
object-oriented languages, and various web and network deployment
strategies have been deployed in an ad hoc manner. Not only has much
duplication of effort occurred, a great deal of struggling up the
learning curves of various technologies has been required as one need
or another has been addressed in various ad hoc ways.

A new need arises as the ambitions of physical modeling increase; this
is the rapid prototyping and testing of new model components. As the
number of possible configurations of a model increases, the expense and

difficulty of both unit testing and integration testing becomes more
demanding.

Fortunately, there is Python. Python is a very flexible language that
has captured the enthusiasm of commercial and scientific programmers
alike. The perception of Python programmers coming from almost any
other language is that they are suddenly dramatically several times
more productive than previously, in terms of functionality delivered
per unit of programmer time.

One slogan of the Python community is that the language "fits your
brain". Why this might be the case is an interesting question. There
are no startling computer science breakthroughs original to the
language, Rather, Python afficionados will claim that the language
combines the best features of such various languages as Lisp, Perl,
Java, and Matlab. Eschewing allegiance to a specific theory of how to
program, Python's design instead offers the best practices from many
other software cultures.

The synergies among these programming modes is in some ways harder  to
explain than to experience. The Python novice may nevertheless observe
that a single language can take the place of shell scripts, makefiles,
desktop computation environments, compiled languages to build GUIs, and

scripting languages to build web interfaces. In addition,  Python is
useful as a wrapper for Fortran modules, facilitating the
implementation of true test-driven design processes in Fortran models.

Another Python advocacy slogan is "batteries included". The point here
is that (in part because Python is dramatically easier to write than
other languages) there is a very broad range of very powerful standard
libraries that make many tasks which are difficult in other languages
astonishingly easy in Python. For instance, drawing upon the standard
libraries (no additional download required)  a portable webserver
(runnable on both Microsoft and Unix-based platforms) can be
implemented in seven lines of code. (See
http://effbot.org/librarybook/simplehttpserver.htm ) Installation of
pure python packages is also very easy, and installation of mixed
language products with a Python component is generally not
significantly harder than a comparable product with no Python
component.

Among the Python components and Python bindings of special interest to
scientists are the elegant and powerful matplotlib plotting package,
which began by emulating and now surpasses the plotting features of
Matlab, SWIG, which allows for runtime interoperability with various
languages, f2py which specifically interoperates with Fortran, NetCDF
libraries (which cope with NetCDF files with dramatically less fuss
than the standard C or Fortran bindings), statistics packages including

bindings to the R language,  linear algebra packages,  various
platform-specific and portable GUI libraries, genetic algorithms,
optimization libraries, and bindings for high performance differential
equation solvers (notably, using the Argonne National Laboratory
package PetSC). An especially interesting Python trick for runtime
visualization in models that were not designed to support it, pioneered

by David Beazley's SWILL, embeds a web server in your model code.

See especially http://starship.python.net/~hinsen/ScientificPython/ and

http://scipy.org as good starting points to learn about scientific uses

He mentioned the things that Fortran is ill-suited for (not actually describing them for
non-CS folks. "distributed control and ensemble management" ?? What does that mean?) but I
didn't see much in his list that is particularly Fortran-specific. I mean, "source and
version control"? What does that have to do with Fortran? You'd have the same issue with
Python; or <insert language of choice>. My experience with source control has been that
the biggest weakness is the people using it (or not as the case may be), not the method
employed.

And there was no analysis of the balance between his mentioned pros (high performance and
flexibility in dealing with regular arrays, backward compatibility with the existing code
base, and the familiarity with the language among the modeling community) and cons (source
and version control and audit trails for runs, build system management, test
specification,etc..) Should we weight them all equally? Should all involved disciplines
weight them the same way?

As to the ad hoc deployment - when you (figuratively) throw 1000's of people together to
work on something (such as a climate model) and those people are spread across the globe,
work for instituitions that typically have very different goals (even *within* agencies),
and are funded based on who-knows-what criteria, and -- assuming the grand master plan
poobah is prescient enough to forsee all the issues that will crop up along the way --
then good luck trying to come up with a plan for "ad-hoc deployment prevention" upon which
everybody involved can agree. More often than not single viewpoints will dominate (Did
somebody say security?)

I absolutely agree with his comments about unit and integration testing. But no real info
was put forth on how Python can help solve this problem. Will python help design the test
parameters and write the test code?

A rather bland series of paragraphs, IMO. It reads like either a) a funding proposal
[hence all the interspersed buzzwords and jargon to make managers get all excited] or b)
an undergraduate CS assignment.

Now, you'll have to excuse me while I go back to studying PHP/MySQL. It's going to solve
all my problems I'm told.... :o)

cheers,

paulv

Paul Van Delst a écrit :

http://www.python.org/pycon/dc2004/papers/4/PyCon2004DocTestUnit.pdf

Far better than xUnit style unit testing IMHO and AFAIK, it has no
equivalent
outside the python world.

Cheers,

I think that the typical Fortran programmer will feel the same. Python
begins blocks but never ends them with words, only by un-indentation.
The only way to see how many blocks are ending is to find the matching
indent and count intervening indents.

Ruby should be a better fit for the typical Fortran programmer. I am
surprised Ruby is not more the standard scripting language for
scientists. At least some people who know both say that Ruby is better
even though Python is (at least for now) more popular.

However, I have come to the decision that scripts should be for small
tasks, and that any "Application" should be in a compiled language. All
of these multi-language packages a bloated and slow.

Now that Fortran is finally getting a proper C interface for utilizing
things like regex and GUI libraries, and variable-length strings, I
think it is much more worth while for the average Fortran programmer to
develop modern code using Fortran rather than trying to learn Python.

Also, how can the indentation rules make life easier? The idea of
forcing proper indentation is probably good for making code more
readable, but the how can idea of invisible end-of-block syntax ever be
good? Some people may get used to or even like it, but I think that
programmers who use Fortran a lot are the least likely to be comfortable
with that syntax.

What might be nice is for a language to offer syntax variations to allow
a programmer to pick different block-delimiter styles. I think there is
not much of a language syntax conflict, just a language developer conflict.

--
J. Giles

Let's face it - we write a lot of hierarchically structured documents.
Every time I write a piece of text longer than, say a page, it will be
hierarchically structured - hey, that fairly often even applies to e-mail.
Any F90 program that doesn't fit on a single screen matches as well. And
so on, and so on, ... So why are we still using editors that have no other
view of the document but the single-level sequential one?

And you don't need to use a different editor: emacs, for instance, has a
folding mode you can use.

In this case, I think it's the other way 'round: the traditional way of
doing things is so entrenched that an apporpriate tool - appropriate to
human cognition - is _not_ being used widely.

(In fact, I managed to get this injected into the new Pragmatic
Bookshelf book I recently reviewed: /Practices of an Agile Developer/,
http://www.pragmaticprogrammer.com/titles/pad/.)

I have found that learning at least one other language that
is significantly different than your "native" language reaps
many unexpected benefits.  You gain not only the new ability
provided by the new language, but also the many lessons and
idioms that you are able to take back to your original one.

  http://hdl.handle.net/2002/16096

I guess I'm looking for a more concrete example closer to my own situation as we're
dealing with these sorts of issues here (in the group I work with). Software updates are
getting further and further apart because the testing schedule (component + regression)
keeps growing. It would be really nice if the testing requirements could be formalised in
such as way so that the individual developers (i.e. scientists) of new stuff (new
algorithms for old data/instruments, or new algorithms to deal with new data/instruments)
are also tasked with testing, but in a way that eases integration.

cheers,

paulv

If you liked the style of that one, you might also enjoy:

  http://doi.ieeecomputersociety.org/10.1109/MS.2003.1196317

or, an earlier (free) version, http://hdl.handle.net/2002/12649

  http://www.devsource.com/article2/0,1759,1778699,00.asp
  http://www-128.ibm.com/developerworks/linux/library/l-oslab/
  http://www.rubygarden.org/ruby?WhatDoesNasaUseRubyFor
  http://www.rubyquiz.com/quiz46.html
  (see also the front page of our FUN3D website)

However, the tools have a long way to go before they can
beat what I can already do without them.  If some part of
the program's text is to be folded, an abstract needs to be
provided in its stead that describes something of what the
invisible code does and the principal variables it operates
on.  Well, I can already do that without any special purpose
tool or feature of the editor: I can give a properly nested
part of the code a mnemonic name, and list its principal
variables as its argument list and make it a called procedure.

Depending on how closely bound the code is to the caller, it
may be an internal procedure, a module procedure, or an
independent external procedure.  The fact that it's also reusable
is convenient since pure hierarchies are not always the best
way to sort things out after all.

--
J. Giles

Gary Scott
mailto:garysc...@ev1.net

Fortran Library:  http://www.fortranlib.com

Support the Original G95 Project:  http://www.g95.org
-OR-
Support the GNU GFortran Project:  http://gcc.gnu.org/fortran/index.html

Why are there two?  God only knows.

If you want to do the impossible, don't hire an expert because he knows
it can't be done.

I write fortran with indentation and the end statement is more or less
only needed because fortran requires it. However, I guess a more
popular python would allow the programmer to choose either significant
indentation or begin/end.

The variable length strings referred to are, I presume, the allocatable
string length stuff of f2003. There is also the ISO_VARYING_STRING
module, but it doesn't integrate nearly as cleanly with the rest of the
language as allocatable strings do. (And, in particular, it doesn't
integrate at all reasonably with C).

The odds of g77 getting the f2003 C interop stuff are zero, if not less.
First g77 would need modules, as most of the C interop stuff is in an
intrinsic module. That's too major a chunk of f90 to have any chance. I
consider modules to be the major dividing line between f77 and f90
compilers. Many f77 compilers do a few f90 features, but if they don't
do modules, then they miss out on most of f90; and once modules are
done, it ends up making sense to finish off f90 and have a full f90
compiler.

I don't think that gfortran has the C interop stuff yet (though I could
be wrong), but it is certainly plausible that it could come. G95 does
have at least some of it - I haven't sat down and looked to see what
parts might still be missing.

I don't think that any compilers yet do the allocatable string length
thing. It is a step beyond the f95 allocatable TR. It wouldn't be
shocking if some do soon, or even if one does already and I've missed
it.

I abstain on whether things like Fortran's distinction between : and *
counts as cryptic. :-) I'm just trying to interpret what was written
here.

That ain't standard, tho...          @ Although it *is* useful

¹ You may insert a comma wherever you like.

However, using a folding editor applies this at several below the
modularity you then imply:

 > Depending on how closely bound the code is to the caller, it
 > may be an internal procedure, a module procedure, or an
 > independent external procedure.

For instance, all declarations would be one fold, with subfolds
for the declaration of dummy arguments, local parameters, local
arrays, and local scalars (by way of example). Same principle
applies to the executable statements. This made it possible to
quickly navigate largish pieces of code even on a 24x80 screen.