Huebner etal (2021) guidance on energy research practices concerning transparency, reproducibility, and quality
Robbie Morrison
Hello all
For your interest, here is a recent paper and associated blog on improving transparency, reproducibility, and quality in energy system research:
-
Huebner, Gesche M, Michael J Fell, and Nicole E Watson (4 January 2021). "Improving energy research practices: guidance for transparency, reproducibility and quality". Buildings and Cities. 2 (1): 1–20. ISSN 2632-6655. doi:10.5334/bc.67. Creative Commons CC‑BY‑4.0 license. Landing page.
-
Huebner, Gesche M, Michael J Fell, and Nicole E Watson (13 January 2021). Opening up energy research: a practical guide. UK Energy Research Centre (UKERC). United Kingdom. Blog.
The pivotal role of open licensing is still not adequately
covered in these kind of discussions. So I have some comments of
course.
Open licensing
One thing that this paper (as well as some of the cited research)
fails to recognize is that open licensing is a legal necessity for
scientific reproducibility in the numerical sciences. Indeed there
is no mention of public licensing (beyond one remark on unclear
data licensing), nor of copyright, database rights, or open source
development. The paper only goes this far on the issue (page 1):
"making data and code publicly available"
These legal matters are not incidental: it is a copyright infringement to build and run a non‑trivial (more than a few source lines) third‑party computer program without a suitable license. It may be a copyright and/or a database right infringement to modify and distribute third‑party datasets without a suitable license. It is a copyright infringement to copy and distribute third‑party documentation without a suitable license. A suitable license need not necessarily be open, but in the context of scientific transparency and reproducibility, it is hard to imagine that licensing other than open could remotely suffice.
I guess this paper shows how much more progress on code and data licensing is required in our domain. I first published that policy‑oriented energy models be developed as open source in 2000, released the first open source energy system model in 2003, and tried (unsuccessfully) to build developer and user communities. So I am a bit disheartened that one can publish on scientific reproducibility in 2021 and not cover the legal context whatsoever.
On the upside, there is more interest these days in license selection for energy system models and I occasionally get asked for my thoughts. As I imagine others within this community do as well.
Improved research
To return to the paper, there are many valuable procedural
recommendations that would be well worth pursuing, particularly
around data management. And in this context, Huebner et al
(2021) is a welcome and useful contribution, well written and with
cogent arguments otherwise. Although I will express my doubts
that preregistration of analysis makes much sense when the focus
is on developing and assembling the models and datasets needed.
And I will take issue with the idea that the adoption of tools to
improve reproducibility within the energy research domain
"remain[s] largely a niche concern" (page 2). Indeed I would
argue the opposite — that such tools, together with the enabling
open licensing, are foundational
to our openmod community of several hundred energy system
researchers (for instance, our mailing list has 800 subscribers,
our forum 650 subscribers, our last physical workshop attracted
190 participants, and our three more recent mini‑workshops
garnered something like unique 200 participants).
Worth noting too is that open source lawyer Till Jaeger (see below) spoke at the 2014 (first) and 2020 (eleventh) openmod workshops so software licensing and open source development have always been key themes in this community.
To close, open licensing is the primary necessary condition
for reproducible research. Without suitable licensing,
everything else above is legally unsafe. You as a researcher may
be happy enough to disregard intellectual property rights, but
your institutional lawyers, funders, publishers, co‑workers, and
collaborating institutions are becoming increasingly aware of the
need to actively comply with copyright and database law through
suitable open licensing.
Further reading
Additional background on open source licensing:
-
Brock, Amanda (editor) (2021). Free and open source software: law, policy and practice (2nd ed). Oxford, United Kingdom: Oxford University Press. Due Spring 2021.
-
Jaeger, Till and Axel Metzger (6 February 2020). Open Source Software: Rechtliche Rahmenbedingungen der Freien Software (5. Ausgabe) [Open source software: legal framework for free software (5th ed)] (in German). Munich, Germany: CH Beck. ISBN 978-3-406-73497-7.
-
Meeker, Heather (26 March 2020). Open (source) for business: a practical guide to open source software licensing (3nd ed). South Carolina, USA: Kindle Direct Publishing Platform. ISBN 979-861820177-3. Paperback edition.
-
Shemtov, Noam and Ian Walden (editors) (2013). Free and open source software: law, policy and practice (1st ed). Oxford, United Kingdom: Oxford University Press. ISBN 978-0-19-968049-8. Paperback edition.
I may shift these comments to the forum, dependent on the ensuing
discussion, if any.
with best wishes, Robbie
-- Robbie Morrison Address: Schillerstrasse 85, 10627 Berlin, Germany Phone: +49.30.612-87617