Hi Mark, all
For the history, there were other open models and projects around at the outset. In some kind of order, Balmorel, GnuAE, and deeco predate OSeMOSYS. TEMOA was contemporaneous with OSeMOSYS. Some provided OSI‑approved software licenses, others tried hard to build communities, others not. Two are still active. I wrote up these early developments as best I could five years ago (doi:10.1016/j.esr.2017.12.010).
I think attempts to paint one project or another as somehow
"first" — and necessarily in some qualified sense: suitable
licensing, user engagement, technical scope, innovation,
subsequent success, official interest, and so forth — distract
somewhat from this whole journey. And I would like to see newer
projects — like CityLearn, to name just one —accorded space too.
Kudos to OSeMOSYS for its current profile and contribution. Clearly. Although I don't regard OSeMOSYS to be particularly unique at the outset. And OSeMOSYS was built on other open source tooling, including the GNU MathProg language and the GLPK solver.
Fast forward, and I do agree that we are on the cusp of open
source becoming the default in much of our domain — although the
big single‑institution in‑house models will hang on for as long as
their funders and public sector clients allow. The populated
models and supporting datasets necessarily following along well
behind — and presenting at least as significant open challenge as
the code. I won't mention names, but there are data visionaries
out there who are yet to see much in the way of fruition or
acknowledgment on that front. In addition, only very recently
have open energy system models had any resonance within the wider
open source world.
Being "early" is sufficient qualification in my view.
For disclosure, I was involved with the deeco codebase, now archived on
GitHub and Software Heritage.
with best wishes, Robbie
You received this message because you are subscribed to the Google Groups "openmod initiative" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openmod-initiat...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openmod-initiative/DB9PR04MB8235C1D79823194C95267851F814A%40DB9PR04MB8235.eurprd04.prod.outlook.com.
-- Robbie Morrison Address: Schillerstrasse 85, 10627 Berlin, Germany Phone: +49.30.612-87617
Hi Mark, all
Thanks for the follow up and context. And hopefully this discussion might still be informative for some on this list who are relatively new to the domain.
I am going to talk about deeco (that stylization being how the project name was usually typeset). And perhaps in a somewhat lucid fashion.
My involvement with high‑resolution contiguous‑time energy systems modeling at a national level began in 1995. I had left my job at a small energy NGO, the Sustainable Energy Forum (SEF), frustrated with the "arm‑waving" analysis that dominated the policy debate on all sides. And enrolled in a new masters course on energy management run by the physics department at Otago University, Dunedin, New Zealand (my roommate instead modeled Bose‑Einstein condensation). I was given free rein and in January 1995 stumbled across a paper in Energy describing the NEMESS framework. I emailed lead author Helmuth Groscurth, Würzburg University, Germany, and asked for a copy of the source. The deeco research team conferred and decided that (1) being based in New Zealand I posed no risk and (2) I would not get the C++ source — which had been developed on an expensive HP workstation — to run on Intel hardware in any case. I was subsequently told that if I had been a German researcher, I would not have been given access.
As it happened, I did port the C++ to a commercial UNIX which provided the necessary compile‑time dependencies and set about applying the framework to New Zealand (but didn't get very far). deeco had been conceived for municipal systems and this was new territory. I was also later told that the idea of using deeco for national systems analysis had never been considered by the original deeco team.
In 2003, I convinced lead author Thomas Bruckner at TU Berlin to open source the project and duly did under Gnu GPL‑2.0. The deeco project ran on a dedicated SCO UnixWare server. Those developing remotely could edit files locally but would need to build and run on a SSH terminal measuring 80 chars × 24 lines (just stop and think about that for a moment!). By 2004, I had provided UNIX accounts about eight people from all over the world, but no one did anything remotely substantial. The decommissioned server is sitting behind me as a type and should, in theory, boot up. The website I wrote to support the project was abandoned by TU Berlin but still have the hand‑coded HTML somewhere. I later heard that the university was distinctly unhappy that the IP in the project had been thus squandered — but given no further details.
It should be remembered that few programmers back then wrote serious software in interpreted languages. MATLAB was admittedly used and also PERL for housekeeping tasks. Version control was almost non‑existent, bar some file level support in the emacs text editor.
Regarding GNU MathProg and GLPK, I contributed to that project for about four years.
MARKAL (TIMES came later) was not an inspiration for deeco.
Indeed, I recall crafting carefully worded criticisms of MARKAL not
spanning contiguous time. deeco in contrast modeled
seasonal heat storage and related technologies quite well, for
example — and was genuinely high‑resolution in all respects. It
also leveraged, for better or worse, the doctrine of
object‑orientation (OOP), the dominant software development paradigm
I was always interested in systems integration, so initiated this soft conference back in 2000. Just for fun, here is the flier I drew:
That original reference I mentioned (also worth flicking through as much the same paradigm is still widely used for frameworks like PyPSA):
Are there lessons in all this? That sharing source is good, I
But in the context of public policy analysis, that was an idea
that took a long time to gain traction — as Mark also alluded to.
Helmuth and I meet the then New Zealand energy minister back in
1999, but that kind of analysis and planning was well out of style
and neoliberalism was instead big.
And just as a marker, the Open Energy Modelling Initiative was formed in Berlin in 2014.
On the question of the timing of projects in general, I will comment on the recent Linux Foundation report on open source and sustainability, for which all history prior to the advent of GitHub is absent. And I also have a video presentation to upload to YouTube, so more on that very soon.
best wishes to all, Robbie
PS: @Mark: I don't think there's anything to follow up off‑list. And my remarks about the timing of TEMOA were inaccurate as you rightly point out. I also support Erica Thompson's comments about the need for diversity and controversy in modeling.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openmod-initiative/DB9PR04MB8235AA44B2CA3C03B56A06B4F814A%40DB9PR04MB8235.eurprd04.prod.outlook.com.
Hi again all
Two small follow‑ups. The deeco project did not use a dynamically‑linked optimization solver — free or otherwise — as is invariably the case today. Rather it used the circa 30 lines of C code given in the numerical recipes series, more‑or‑less directly pasted in:
This code implemented the simplex method that crawls over your necessarily convex solution space to find a not‑necessarily unique optimal.
That series of books (which included FORTRAN code) was widely used back then and researchers were generally oblivious to the notion that the code provided was not suitably licensed. Thomas even submitted a patch back to the authors to fix a corner case.
If you head to wikipedia, you will find a discussion on the licensing terms used by Numerical Recipes, their workability, and the fact that the non‑open nature of Numerical Recipes drove the development of the GNU Scientific Library.
While I am here, a small shout‑out for the HiGHS project, aimed a providing an open‑licensed high‑performance solver engine. Please trial these tools and submit bug reports when you encounter issues.
And the second follow‑up. I regard this paper from the deeco team (for which I had no part) as seminal. It shows that the process of high‑resolution optimization is essentially the process of uncovering latent synergies in the design and operation of the systems under investigation. Which I think is a nice way of framing this endeavor.
Sad that so much of this material remains paywalled. And a little known fact, but for German researchers, Elsevier publications have been off‑limits for five years because the associated Projekt DEAL negotiations failed. That means both access and authorship are Verboten. Indeed, German researchers are not supposed to submit to Elsevier titles. More on wikipedia.
with best wishes as always, kia ora, Robbie
To view this discussion on the web, visit https://groups.google.com/d/msgid/openmod-initiative/04f60059-d914-4844-3f5b-f3373670fd5c%40posteo.de.