Huebner etal (2021) guidance on energy research practices concerning transparency, reproducibility, and quality

Skip to first unread message

Robbie Morrison

Jan 14, 2021, 12:10:21 PMJan 14
to openmod list

Hello all

For your interest, here is a recent paper and associated blog on improving transparency, reproducibility, and quality in energy system research:

The pivotal role of open licensing is still not adequately covered in these kind of discussions.  So I have some comments of course.

Open licensing

One thing that this paper (as well as some of the cited research) fails to recognize is that open licensing is a legal necessity for scientific reproducibility in the numerical sciences. Indeed there is no mention of public licensing (beyond one remark on unclear data licensing), nor of copyright, database rights, or open source development.  The paper only goes this far on the issue (page 1):

"making data and code publicly available"

These legal matters are not incidental: it is a copyright infringement to build and run a non‑trivial (more than a few source lines) third‑party computer program without a suitable license. It may be a copyright and/or a database right infringement to modify and distribute third‑party datasets without a suitable license. It is a copyright infringement to copy and distribute third‑party documentation without a suitable license. A suitable license need not necessarily be open, but in the context of scientific transparency and reproducibility, it is hard to imagine that licensing other than open could remotely suffice.

I guess this paper shows how much more progress on code and data licensing is required in our domain. I first published that policy‑oriented energy models be developed as open source in 2000, released the first open source energy system model in 2003, and tried (unsuccessfully) to build developer and user communities. So I am a bit disheartened that one can publish on scientific reproducibility in 2021 and not cover the legal context whatsoever.

On the upside, there is more interest these days in license selection for energy system models and I occasionally get asked for my thoughts. As I imagine others within this community do as well.

Improved research

To return to the paper, there are many valuable procedural recommendations that would be well worth pursuing, particularly around data management.  And in this context, Huebner et al (2021) is a welcome and useful contribution, well written and with cogent arguments otherwise.  Although I will express my doubts that preregistration of analysis makes much sense when the focus is on developing and assembling the models and datasets needed.

And I will take issue with the idea that the adoption of tools to improve reproducibility within the energy research domain "remain[s] largely a niche concern" (page 2).  Indeed I would argue the opposite — that such tools, together with the enabling open licensing, are foundational to our openmod community of several hundred energy system researchers (for instance, our mailing list has 800 subscribers, our forum 650 subscribers, our last physical workshop attracted 190 participants, and our three more recent mini‑workshops garnered something like unique 200 participants).

Worth noting too is that open source lawyer Till Jaeger (see below) spoke at the 2014 (first) and 2020 (eleventh) openmod workshops so software licensing and open source development have always been key themes in this community.

To close, open licensing is the primary necessary condition for reproducible research.  Without suitable licensing, everything else above is legally unsafe.  You as a researcher may be happy enough to disregard intellectual property rights, but your institutional lawyers, funders, publishers, co‑workers, and collaborating institutions are becoming increasingly aware of the need to actively comply with copyright and database law through suitable open licensing.

Further reading

Additional background on open source licensing:

  • Brock, Amanda (editor) (2021). Free and open source software: law, policy and practice (2nd ed). Oxford, United Kingdom: Oxford University Press. Due Spring 2021.

  • Jaeger, Till and Axel Metzger (6 February 2020). Open Source Software: Rechtliche Rahmenbedingungen der Freien Software (5. Ausgabe) [Open source software: legal framework for free software (5th ed)] (in German). Munich, Germany: CH Beck. ISBN 978-3-406-73497-7.

  • Meeker, Heather (26 March 2020). Open (source) for business: a practical guide to open source software licensing (3nd ed). South Carolina, USA: Kindle Direct Publishing Platform. ISBN 979-861820177-3. Paperback edition.

  • Shemtov, Noam and Ian Walden (editors) (2013). Free and open source software: law, policy and practice (1st ed). Oxford, United Kingdom: Oxford University Press. ISBN 978-0-19-968049-8. Paperback edition.

I may shift these comments to the forum, dependent on the ensuing discussion, if any.

with best wishes, Robbie

Robbie Morrison
Address: Schillerstrasse 85, 10627 Berlin, Germany
Phone: +49.30.612-87617
Reply all
Reply to author
0 new messages