pypsa-eur system requierements

[David Arango]

Dear colleagues!

I would like to have more details about the system requirements to run PyPSA-Eur in the documentation it says...requieres strong interior-point solver compatible with the modelling library PYOMO ..., I have checked the requirements of the solver and they fullfil PYOMO's requirements (like CPLEX academic version). But due to the model is quite big, I would like to know if there is a minimum hardware requirements (besides the 100Gbs for the whole model scenarios)?, I read that for PyPSA-ZA, it took even a week to generate insights for the whole scenarios, how about PyPSA-eur?

Is it possible to run it in google colab or amazon servers, or a personal workstation?.

Thank for your time, I really appreciate it.

Best regards,

David.

[Fabian Neumann]

Hi David,

usually it is no problem to run the preparatory steps of PyPSA-Eur
(until `prepare_network`) on a personal computer (>=8GB RAM) in about
15-20 minutes.

Unless you are solving really small problems, however, using cloud
computing services or your department's high-performance computing
resources is currently the way to go to solve the prepared networks.

Conservative estimates for memory usage of the `solve_network` rule
depending on temporal aggregation level and the number of clusters are
in the `Snakefile`

https://github.com/PyPSA/pypsa-eur/blob/7a18740cfb565b5d83d04387f3a7b35c28fa56cc/Snakefile#L208-L218

def memory(w):
factor = 3.
for o in w.opts.split('-'):
m = re.match(r'^(\d+)h$', o, re.IGNORECASE)
if m is not None:
factor /= int(m.group(1))
break
if w.clusters.endswith('m'):
return int(factor * (18000 + 180 * int(w.clusters[:-1])))
else:
return int(factor * (10000 + 195 * int(w.clusters)))

To give you a rough idea of solving times:
- 100 nodes, 2-hourly -> ~6 hours, 32 GB RAM
- 181 nodes, 3-hourly -> ~2.5 hours, 33 GB RAM
- 121 nodes, 3-hourly -> ~1 hour, 24 GB RAM

Tom Brown has recently been working on reducing the memory consumption
and computation times of PyPSA by not using Pyomo at
https://github.com/PyPSA/nomopyomo with extremely promising results.

Best wishes,
Fabian

On 02/10/2019 21:25, David Arango wrote:
> Dear colleagues!
>
> I would like to have more details about the system requirements to run

> PyPSA-Eur in the documentation it says.../requieres strong
> interior-point solver compatible with the modelling library PYOMO/ ...,

> I have checked the requirements of the solver and they fullfil PYOMO's
> requirements (like CPLEX academic version). But due to the model is
> quite big, I would like to know if there is a minimum hardware
> requirements (besides the 100Gbs for the whole model scenarios)?, I read
> that for PyPSA-ZA, it took even a week to generate insights for the
> whole scenarios, how about PyPSA-eur?
>
> Is it possible to run it in google colab or amazon servers, or a
> personal workstation?.
>
> Thank for your time, I really appreciate it.
> Best regards,
> David.
>
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pypsa/CACk%2ByXCQOJFCU71V049EqmDAdunNpKHX5rMKxXr2Jd9-utuFJQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/pypsa/CACk%2ByXCQOJFCU71V049EqmDAdunNpKHX5rMKxXr2Jd9-utuFJQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Karlsruhe Institute of Technology (KIT)
Institute for Automation and Applied Informatics (IAI)

Fabian Neumann, M.Sc.
PhD Student in Energy System Modelling

Phone: +49 721 608 25707
Mobile: +49 171 2943831
Fax: +49 721 608 22602
Group Website: www.iai.kit.edu/english/esm.php
Personal Website: www.neumann.fyi
Github: fneum
LinkedIn: fabian-neumann-renewables
ORCID: 0000 0001 8551 1480

Building 445, Office 304
Campus North
Hermann-von-Helmholtz-Platz 1
76344 Eggenstein-Leopoldshafen

KIT – the Research University in the Helmholtz Association