pypsa-eur: Error in build_renewable_profiles(technology=offwind-ac ) while solving the network

[swagat panda]

Hello Community,

I am getting the following error while solving a network using the default config file.

>>>

rule build_renewable_profiles:

input: networks/base.nc, data/bundle/corine/g250_clc06_V18_5.tif, resources/natura.tiff, data/bundle/GEBCO_2014_2D.nc, resources/country_shapes.geojson, resources/offshore_shapes.geojson, resources/regions_offshore.geojson, cutouts/europe-2013-era5.nc

output: resources/profile_offwind-ac.nc

log: logs/build_renewable_profile_offwind-ac.log 

jobid: 13 

benchmark: benchmarks/build_renewable_profiles_offwind-ac 

wildcards: technology=offwind-ac 

resources: mem=20000

Traceback (most recent call last):

 File "<string>", line 1, in <module>

 availability = cutout.availabilitymatrix(regions, excluder, **kwargs)

File "C:\LocalData\Swagat\pypsa-eur\envs\eurEnv1\lib\multiprocessing\spawn.py", line 116, in spawn_main

File "C:\LocalData\Swagat\pypsa-eur\envs\eurEnv1\lib\site-packages\atlite\gis.py", line 520, in compute_availabilitymatrix with mp.Pool(**kwargs) as pool:

exitcode = _main(fd, parent_sentinel) File "C:\LocalData\Swagat\pypsa-eur\envs\eurEnv1\lib\multiprocessing\context.py", line 119, in Pool

...

...

<<<

The complete process and error log and the config file are attached below.

Further Information:

1. This rule: "build_renewable_profiles" worked perfectly in the case where the wildcard technology == solar.
2. The following URL was used to download the data bundle:  https://zenodo.org/record/3517935/files/pypsa-eur-data-bundle.tar.xz
3. The following URL was used to download the cutout data:  

•   zenodo.org/record/4709858/files/ europe-2013-era5.nc
•   zenodo.org/record/4709858/files/ europe-2013-sarah.nc

       4. The following URL was used to download the nature raster

•    zenodo.org/record/4706686/files/natura.tiff        

I think the process should work to at least build the network based on the tutorial I found on the website using the default config file settings. So please help me out in figuring out what's wrong here.

Thanks in advance for your help😃.

Mit freundlichen Grüßen/Regards,

Swagat Panda

[swagat panda]

I'm guessing there is something wrong with the netCDF4 data ( europe-2013-era5.nc) provided by the link. Since whenever I try to run this code it throws an error, but works perfectly with the 'europe-2013-sarah.nc' data.

>>>import atlite

>>>cutout = atlite.Cutout(...\\...\\europe-2013-era5.nc)

KeyError: [<class 'netCDF4._netCDF4.Dataset'>......

....

....

...OSError: [Errno -101] NetCDF: HDF error:

[Johannes Hampp]

Dear Swagat Panda,

Did you try deleting and downloading the `cutouts/europe-2013-era5.nc`
again? The file may have been corrupted.

Else please open a bug report in our GitHub repository for pypsa-eur:

https://github.com/PyPSA/pypsa-eur/issues

and include the error log, your OS (presumably windows) and a list of
your installed packages (`conda list`).

The rule you are having trouble with underwent major overhaul recently
and passed all our internal checks. But we may have missed something.

Thanks & Best,
Johannes


Best regards,
Johannes Hampp (he/him)

Justus Liebig University Giessen (JLU)
Center for international Development and Environmental Research (ZEU)

mailto: johanne...@zeu.uni-giessen.de

Office 110
Senckenbergstr. 3
DE-35392 Giessen
https://uni-giessen.de/zeu

Am 03/06/2021 um 14:55 schrieb swagat panda:
> I'm guessing there is something wrong with the netCDF4 data (

> europe-2013-era5.nc <http://europe-2013-era5.nc/>) provided by the link.

> Since whenever I try to run this code it throws an error, but works

> perfectly with the 'europe-2013-sarah.nc <http://europe-2013-sarah.nc/>'

> data.
>>>>import atlite
>>>>cutout = atlite.Cutout(...\\...\\europe-2013-era5.nc

> <http://europe-2013-era5.nc/>)

> KeyError: [<class 'netCDF4._netCDF4.Dataset'>......
> ....
> ....
> ...OSError: [Errno -101] NetCDF: HDF error:
>
> On Wednesday, 2 June 2021 at 17:32:53 UTC+2 swagat panda wrote:
>
> Hello Community,
>
> I am getting the following error while solving a network using the
> default config file.
> >>>
> rule build_renewable_profiles:

> input: networks/base.nc <http://base.nc>,

> data/bundle/corine/g250_clc06_V18_5.tif, resources/natura.tiff,
> data/bundle/GEBCO_2014_2D.nc, resources/country_shapes.geojson,
> resources/offshore_shapes.geojson,
> resources/regions_offshore.geojson, cutouts/europe-2013-era5.nc

> <http://europe-2013-era5.nc>
> output: resources/profile_offwind-ac.nc <http://profile_offwind-ac.nc>

> log: logs/build_renewable_profile_offwind-ac.log 
> jobid: 13 
> benchmark: benchmarks/build_renewable_profiles_offwind-ac 
> wildcards: technology=offwind-ac 
> resources: mem=20000
>
> Traceback (most recent call last):
>  File "<string>", line 1, in <module>
>  availability = cutout.availabilitymatrix(regions, excluder, **kwargs)
> File
> "C:\LocalData\Swagat\pypsa-eur\envs\eurEnv1\lib\multiprocessing\spawn.py",
> line 116, in spawn_main
> File
> "C:\LocalData\Swagat\pypsa-eur\envs\eurEnv1\lib\site-packages\atlite\gis.py",
> line 520, in compute_availabilitymatrix with mp.Pool(**kwargs) as pool:
> exitcode = _main(fd, parent_sentinel) File
> "C:\LocalData\Swagat\pypsa-eur\envs\eurEnv1\lib\multiprocessing\context.py",
> line 119, in Pool
> ...
> ...
> <<<

> */The complete process and error log and the config file are
> attached below./*
> Further Information:
>
> 1. This rule: "build_renewable_profiles" worked perfectly in the

> case where the wildcard technology == solar.

> 2. The following URL was used to download the data
> bundle:  https://zenodo.org/record/3517935/files/pypsa-eur-data-bundle.tar.xz
> <https://zenodo.org/record/3517935/files/pypsa-eur-data-bundle.tar.xz>
> 3. The following URL was used to download the cutout data:  
>
> *   zenodo.org/record/4709858/files/
> <http://zenodo.org/record/4709858/files/> europe-2013-era5.nc
> <http://europe-2013-era5.nc>
> *   zenodo.org/record/4709858/files/
> <http://zenodo.org/record/4709858/files/> europe-2013-sarah.nc
> <http://europe-2013-sarah.nc>

>
>        4. The following URL was used to download the nature raster
>

> *    zenodo.org/record/4706686/files/natura.tiff
> <http://zenodo.org/record/4706686/files/natura.tiff>        

>
> I think the process should work to at least build the network based
> on the tutorial I found on the website using the default config file
> settings. So please help me out in figuring out what's wrong here.
>
> Thanks in advance for your help😃.
>
> Mit freundlichen Grüßen/Regards,
>
> Swagat Panda
>

> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pypsa/3ce5ea81-dfd2-4ca7-a74b-503f103357bcn%40googlegroups.com
> <https://groups.google.com/d/msgid/pypsa/3ce5ea81-dfd2-4ca7-a74b-503f103357bcn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[swagat panda]

Hello  Johannes,

Thanks for the reply. 

Yes, I tried it again, not only on my PC but also on another virtual machine. But unfortunately, the problem still persists.

Okay, I'll add a bug report with the error logs and the package list of my python virtual environment . In the meanwhile

please let me know if anything comes up regarding the problem.

---

Best Regards,

Swagat panda