[netCDF file] illegal data type for attribute b'network_opf_keep_files'

[Shuwei Zhang]

I tried to run the Pypsa-EUR (actually a previous version with data here: https://github.com/martavp/pypsa-eur-sec-30-path) with some revision on the definition of bus and links, in the Google Colab environment. 

But it is wrong when the NetCDF4 files were created (a boolean value problem, but why?)...

The layers of files are heavy, and how the setting of Network component was stored is an unknown for me.  

But grateful for you hints if you have.

-------

Traceback (most recent call last): File "/content/drive/model/.snakemake/scripts/tmplgalb1rw.prepare_network.py", line 1120, in <module> network.export_to_netcdf(snakemake.output.network_name) File "/usr/local/lib/python3.9/dist-packages/pypsa/io.py", line 666, in export_to_netcdf return exporter.ds File "/usr/local/lib/python3.9/dist-packages/pypsa/io.py", line 47, in __exit__ self.finish() File "/usr/local/lib/python3.9/dist-packages/pypsa/io.py", line 361, in finish self.ds.to_netcdf(self.path) File "/usr/local/lib/python3.9/dist-packages/xarray/core/dataset.py", line 1904, in to_netcdf return to_netcdf( # type: ignore # mypy cannot resolve the overloads:( File "/usr/local/lib/python3.9/dist-packages/xarray/backends/api.py", line 1231, in to_netcdf dump_to_store( File "/usr/local/lib/python3.9/dist-packages/xarray/backends/api.py", line 1278, in dump_to_store store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims) File "/usr/local/lib/python3.9/dist-packages/xarray/backends/common.py", line 268, in store self.set_attributes(attributes) File "/usr/local/lib/python3.9/dist-packages/xarray/backends/common.py", line 285, in set_attributes self.set_attribute(k, v) File "/usr/local/lib/python3.9/dist-packages/xarray/backends/netCDF4_.py", line 445, in set_attribute self.ds.setncattr(key, value) File "src/netCDF4/_netCDF4.pyx", line 3025, in netCDF4._netCDF4.Dataset.setncattr File "src/netCDF4/_netCDF4.pyx", line 1748, in netCDF4._netCDF4._set_att TypeError: illegal data type for attribute b'network_opf_keep_files', must be one of dict_keys(['S1', 'i1', 'u1', 'i2', 'u2', 'i4', 'u4', 'i8', 'u8', 'f4', 'f8']), got b1

============

I installed the !pip install atlite package, so I should have the netCDF4 files?

But I did NOT have the src/netCDF4 folder...

Thanks!

Shuwei

[Johannes Hampp]

Hi Shuwei,

If you are using an older version of the model, did you make sure to
also install the *old* dependencies, i.e. the versions of PyPSA and the
package dependency that were used to create the model?

That could cause the error.


Side notes:
The error is not related to atlite.
Also you do have the "src/netCDF4" folder as part of your Python
installation. It's not a folder in the same directory.


Best,
Johannes

Am 23/03/2023 um 12:51 schrieb Shuwei Zhang:
> I tried to run the Pypsa-EUR (actually a previous version with data
> here: https://github.com/martavp/pypsa-eur-sec-30-path) with some

> revision on the definition of bus and links, in the /_*Google Colab*_/
> environment.
>
> But it is wrong when the NetCDF4 files were created (*a boolean value
> problem, but why?*)...

> The layers of files are heavy, and how the setting of Network component
> was stored is an unknown for me.
>
> But grateful for you hints if you have.
>
> -------
> Traceback (most recent call last): File

> "/content/drive/model/.snakemake/scripts/tmplgalb1rw.prepare_network.py", line 1120, in <module> network.export_to_netcdf(snakemake.output.network_name) File "/usr/local/lib/python3.9/dist-packages/pypsa/io.py", line 666, in export_to_netcdf return exporter.ds File "/usr/local/lib/python3.9/dist-packages/pypsa/io.py", line 47, in __exit__ self.finish() File "/usr/local/lib/python3.9/dist-packages/pypsa/io.py", line 361, in finish self.ds.to_netcdf(self.path) File "/usr/local/lib/python3.9/dist-packages/xarray/core/dataset.py", line 1904, in to_netcdf return to_netcdf( # type: ignore # mypy cannot resolve the overloads:( File "/usr/local/lib/python3.9/dist-packages/xarray/backends/api.py", line 1231, in to_netcdf dump_to_store( File "/usr/local/lib/python3.9/dist-packages/xarray/backends/api.py", line 1278, in dump_to_store store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims) File "/usr/local/lib/python3.9/dist-packages/xarray/backends/common.py", line 268, in store self.set_attributes(attributes) File "/usr/local/lib/python3.9/dist-packages/xarray/backends/common.py", line 285, in set_attributes self.set_attribute(k, v) File "/usr/local/lib/python3.9/dist-packages/xarray/backends/netCDF4_.py", line 445, in set_attribute self.ds.setncattr(key, value) File "src/netCDF4/_netCDF4.pyx", line 3025, in netCDF4._netCDF4.Dataset.setncattr File "src/netCDF4/_netCDF4.pyx", line 1748, in netCDF4._netCDF4._set_att TypeError: *illegal data type for attribute b'network_opf_keep_files', must be one of dict_keys(['S1', 'i1', 'u1', 'i2', 'u2', 'i4', 'u4', 'i8', 'u8', 'f4', 'f8']), got b1*
> ============
>
> I installed the !pip install *_atlite_* package, so I should have the

> netCDF4 files?
> But I did NOT have the src/netCDF4 folder...
>
> Thanks!
> Shuwei
>

> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pypsa/9afb5f5a-e512-4c4e-8742-598976505bcbn%40googlegroups.com <https://groups.google.com/d/msgid/pypsa/9afb5f5a-e512-4c4e-8742-598976505bcbn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Shuwei Zhang]

Thank you, Johannes for the quick reply.

I have already sorted all the potential in-harmonisations before this with the latest version (I think!).

But unfortunately blocked in this file step to create the .nc file:).

Which part(s) of the settings related to the incompatible boolean values?

There exists many options but I think before the CDF4 file created.. all these options set for some value not TRUE/False..

BTW: the error is only existing for the year of 2050 actually..

--------------

rule prepare_networks_2050:

input:

options_name=config['results_dir'] + 'version-' + str(config['version']) + '/options/options-{flexibility}_{line_limits}_2050.yml',

population_name='data/population.csv',

solar_thermal_name='data/heating/solar_thermal-{angle}.h5'.format(angle=config['solar_thermal_angle']),

heat_demand_name='data/heating/daily_heat_demand.h5',

cop_name='data/heating/cop-2015_2.h5',

energy_totals_name='data/energy_totals.csv',

co2_totals_name='data/co2_totals.csv',

temp='data/heating/temp.h5',

transport_name='data/transport_data.csv',

cost_name='data/costs/costs_2050.csv',

output:

network_name=config['results_dir'] + 'version-' + str(config['version']) + '/prenetworks/prenetwork-green-{flexibility}_{line_limits}_2050.nc'

threads: 1

resources: mem_mb=20000

script: 'scripts/prepare_network.py'