[PyPSA-Eur] Trouble replicating Tutorial

[Sebastián Rivero Equiza]

Hi all,

I am having trouble replicating the PyPSA-Eur Tutorial: Electricity-Only.

I have already cloned the pypsa-eur repository, created and activated the corresponding environment. 

When passing the following command in the Ubuntu Command Prompt:

snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile test/config.electricity.yaml

I get the following error:

Error in rule build_ship_raster:
    jobid: 15
    input: data/shipdensity_global.zip, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc
    output: resources/shipdensity_raster.nc
    log: logs/test-elec/build_ship_raster.log (check log file(s) for error details)
    conda-env: /home/sriveroe/pypsa-eur/.snakemake/conda/2e95ad1e3640ec7dfb44488631b1e892_

Could someone please help me to solve this issue?

I would also like to ask about the following line that pops up when I run the above mentioned command:

No parameters matching '_test' found
What does this mean?

Thank you in advance for your help.

Regards,

Sebastian

[Tessa Weiss]

Hi both, 

I'm getting a similar error to Sebastian when I try to run the tutorial. It happens when I use both 5 or 6 in the call "snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile test/config.electricity.yaml". My error looks to be that the output files necessary to run the rules are not in existence. I cloned the github repo to get all the files on my computer and the github repo does not appear to have them either. Am I missing a step in creating these output files? 

rule build_ship_raster:

    input: data/shipdensity_global.zip, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc
    output: resources/shipdensity_raster.nc
    log: logs/test-elec/build_ship_raster.log

    jobid: 15
    benchmark: benchmarks/test-elec/build_ship_raster
    reason: Missing output files: resources/shipdensity_raster.nc
    resources: tmpdir=C:\Users\TESSAW~1\AppData\Local\Temp, mem_mb=5000, mem_mib=4769

[Fri Apr  7 08:13:04 2023]
rule retrieve_databundle:
    output: data/bundle/ch_cantons.csv, data/bundle/je-e-21.03.02.xls, data/bundle/eez/World_EEZ_v8_2014.shp, data/bundle/hydro_capacities.csv, data/bundle/naturalearth/ne_10m_admin_0_countries.shp, data/bundle/NUTS_2013_60M_SH/data/NUTS_RG_60M_2013.shp, data/bundle/nama_10r_3popgdp.tsv.gz, data/bundle/nama_10r_3gdp.tsv.gz, data/bundle/corine/g250_clc06_V18_5.tif
    log: logs/test-elec/retrieve_databundle.log
    jobid: 9
    reason: Missing output files: data/bundle/ch_cantons.csv, data/bundle/hydro_capacities.csv, data/bundle/nama_10r_3popgdp.tsv.gz, data/bundle/NUTS_2013_60M_SH/data/NUTS_RG_60M_2013.shp, data/bundle/naturalearth/ne_10m_admin_0_countries.shp, data/bundle/eez/World_EEZ_v8_2014.shp, data/bundle/nama_10r_3gdp.tsv.gz, data/bundle/corine/g250_clc06_V18_5.tif, data/bundle/je-e-21.03.02.xls
    resources: tmpdir=C:\Users\TESSAW~1\AppData\Local\Temp, mem_mb=1000, mem_mib=954

[Fri Apr  7 08:13:04 2023]
rule build_load_data:
    input: data/load_raw.csv
    output: resources/load.csv
    log: logs/test-elec/build_load_data.log
    jobid: 20
    reason: Missing output files: resources/load.csv
    resources: tmpdir=C:\Users\TESSAW~1\AppData\Local\Temp, mem_mb=5000, mem_mib=4769

Thanks! 
Tessa

On Thursday, April 6, 2023 at 9:58:32 AM UTC-4 Martin Roach wrote:

Hi Sebastian,

I'm no expert, but the clusters in the snakemake call statement should match what's in the config.electricity.yaml file. You stated 6, but in the original file I see 5.

I don't know what the No parameters matching '_test' means, but it always appears and I've been working on non test-related config files.

Hope this is some help.

Best regards,

Martin

[Sebastián Rivero Equiza]

Hi Martin,

Thank you for your answers.

I get that same error using 5, so I would say that the error is related to something else.

Regards,

Sebastian

--
You received this message because you are subscribed to a topic in the Google Groups "pypsa" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pypsa/HPqbMScQTMs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pypsa+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pypsa/8dbdc761-aa00-427e-96f5-c78a609e3d47n%40googlegroups.com.

[Fabian Neumann]

Hi Sebastian and Tessa,

It should work directly after cloning the GitHub repository.

There should be further, more pythonic error messages in your consoles. Could you share them as well?

The warning "No parameters matching '_test' found" you can safely ignore.

Best wishes,

Fabian

-- 
Dr.-Ing.
Fabian Neumann (he/him)
Postdoctoral Researcher

Department of Digital Transformation in Energy Systems
Institute of Energy Technology
Technische Universität Berlin

Group website: https://www.tu.berlin/ensys
Personal website: https://neumann.fyi
Schedule Meeting: https://calendly.com/fneum

Einsteinufer 25 (TA 8)
10587 Berlin

[Sebastián Rivero Equiza]

Hi Fabian,

I deleted pypsa-eur and the created environment and started all over again, but the following error was still there:

[Thu Apr 20 10:30:21 2023]
rule build_ship_raster:

    input: data/shipdensity_global.zip, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc
    output: resources/shipdensity_raster.nc
    log: logs/test-elec/build_ship_raster.log

    jobid: 15
    benchmark: benchmarks/test-elec/build_ship_raster
    reason: Missing output files: resources/shipdensity_raster.nc; Input files updated by another job: cutouts/be-03-2013-era5.nc, data/shipdensity_global.zip
    resources: tmpdir=/tmp, mem_mb=5000, mem_mib=4769

Traceback (most recent call last):
  File "/home/sriveroe/pypsa-eur/.snakemake/scripts/tmp7q9de0ev.build_ship_raster.py", line 71, in <module>
    with xr.open_rasterio("shipdensity_global.tif") as ship_density:
AttributeError: module 'xarray' has no attribute 'open_rasterio'
[Thu Apr 20 10:30:45 2023]

Error in rule build_ship_raster:
    jobid: 15
    input: data/shipdensity_global.zip, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc, cutouts/be-03-2013-era5.nc
    output: resources/shipdensity_raster.nc
    log: logs/test-elec/build_ship_raster.log (check log file(s) for error details)
    conda-env: /home/sriveroe/pypsa-eur/.snakemake/conda/2e95ad1e3640ec7dfb44488631b1e892_

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

I took a look at the xarray documentation and apparently xarray.open_rasterio was deprecated some time ago, so I had to downgrade the xarray package to the version indicated in the environment.fixed.yaml file, and the error disappeared.

Best regards,

Sebastian

[Fabian Neumann]

Hi Sebastian,

this is now fixed on the master branch through

https://github.com/PyPSA/pypsa-eur/pull/474

Best wishes,

Fabian

[Sebastián Rivero Equiza]

Great, thank you!

Best regards,

Sebastian