sanitize_locations error

[Eduardo Díaz]

Hey there,

I am using the latest version of the master branch. 

When trying to solve an electricity only problem, I come across an error I previosuly did not encounter.

sanitize_locations function, in extra_components, calls network.buses.locations.

Where is this location attribute added to network.buses?

My networks do not have that attribute and I can not find the part of the workflow in which it is created.

Error log:

[Fri Feb 16 12:45:48 2024]
rule add_extra_components:
    input: resources/2024-test/networks/elec_s_2.nc, data/costs_2030_mod.csv
    output: resources/2024-test/networks/elec_s_2_ec.nc
    log: logs/2024-test/add_extra_components/elec_s_2.log
    jobid: 3
    benchmark: benchmarks/2024-test/add_extra_components/elec_s_2_ec
    reason: Missing output files: resources/2024-test/networks/elec_s_2_ec.nc; Input files updated by another job: resources/2024-test/networks/elec_s_2.nc
    wildcards: simpl=, clusters=2
    resources: tmpdir=C:\Users\SE91723\AppData\Local\Temp, mem_mb=4000, mem_mib=3815

INFO:pypsa.io:Imported network elec_s_2.nc has buses, carriers, generators, links, loads, storage_units
WARNING:add_electricity:tech_colors for carriers ['AC', 'DC', ''] not defined in config.
ERROR:root:Uncaught exception
Traceback (most recent call last):
  File "C:\Users\SE91723\Modelos\pypsa\pypsa-eur\.snakemake\scripts\tmpghiyhpxh.add_extra_components.py", line 269, in <module>
    sanitize_locations(n)
  File "C:\Users\SE91723\Modelos\pypsa\pypsa-eur\rules\..\scripts\add_electricity.py", line 185, in sanitize_locations
    n.buses["x"] = n.buses.x.where(n.buses.x != 0, n.buses.location.map(n.buses.x))
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\generic.py", line 6293, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'location'
[Fri Feb 16 12:45:57 2024]
Error in rule add_extra_components:
    jobid: 3
    input: resources/2024-test/networks/elec_s_2.nc, data/costs_2030_mod.csv
    output: resources/2024-test/networks/elec_s_2_ec.nc
    log: logs/2024-test/add_extra_components/elec_s_2.log (check log file(s) for error details)
    conda-env: C:\Users\SE91723\Modelos\pypsa\pypsa-eur\.snakemake\conda\3701581752c6e5590dda85748a41c931_

[Fabian Neumann]

Fixed now in 472a0998. Please re-run add_electricity rule.

[Eduardo Díaz]

Thanks, I tested this commit and it results in a consense error when aggregating the buses in cluster_network.py (I attached the error below).

I also see that on later commits these changes were reverted. I have pulled master branch but i get the same error as in my initial message.

Any thoughts on this? I am still trying to find where this location attribute is created during the electricity only workflow.

Best, 

Eduardo

------Error message-----

ERROR:root:Uncaught exception
Traceback (most recent call last):

  File "C:\Users\SE91723\Modelos\pypsa\pypsa-eur\.snakemake\scripts\tmpf_3g62se.cluster_network.py", line 646, in <module>
    clustering = clustering_for_n_clusters(
  File "C:\Users\SE91723\Modelos\pypsa\pypsa-eur\.snakemake\scripts\tmpf_3g62se.cluster_network.py", line 413, in clustering_for_n_clusters
    clustering = get_clustering_from_busmap(
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pypsa\clustering\spatial.py", line 536, in get_clustering_from_busmap
    buses = aggregatebuses(n, busmap, custom_strategies=bus_strategies)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pypsa\clustering\spatial.py", line 344, in aggregatebuses
    aggregated = n.buses.groupby(busmap).agg(strategies)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\groupby\generic.py", line 1432, in aggregate
    result = op.agg()
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\apply.py", line 190, in agg
    return self.agg_dict_like()
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\apply.py", line 423, in agg_dict_like
    return self.agg_or_apply_dict_like(op_name="agg")
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\apply.py", line 1608, in agg_or_apply_dict_like
    result_index, result_data = self.compute_dict_like(
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\apply.py", line 496, in compute_dict_like
    results = [
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\apply.py", line 497, in <listcomp>
    getattr(obj._gotitem(key, ndim=1), op_name)(how, **kwargs)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\groupby\generic.py", line 294, in aggregate
    return self._python_agg_general(func, *args, **kwargs)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\groupby\generic.py", line 327, in _python_agg_general
    result = self._grouper.agg_series(obj, f)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\groupby\ops.py", line 863, in agg_series
    result = self._aggregate_series_pure_python(obj, func)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\groupby\ops.py", line 884, in _aggregate_series_pure_python
    res = func(group)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pandas\core\groupby\generic.py", line 324, in <lambda>
    f = lambda x: func(x, *args, **kwargs)
  File "C:\Users\SE91723\Anaconda3\envs\pypsa-eur\lib\site-packages\pypsa\clustering\spatial.py", line 136, in consense
    assert (x == v).all() or x.isnull().all(), (
AssertionError: In Bus cluster location, the values of attribute location do not agree:
Bus
7928    7928
927      927
929      929
1004    1004
1005    1005
        ...
8655    8655
8508    8508
7814    7814
7825    7825
7106    7106
Name: location, Length: 748, dtype: object
[Mon Feb 19 10:05:37 2024]
Error in rule cluster_network:
    jobid: 4
    input: resources/2024-test/networks/elec_s.nc, resources/2024-test/regions_onshore_elec_s.geojson, resources/2024-test/regions_offshore_elec_s.geojson, resources/2024-test/busmap_elec_s.csv, data/costs_2030_mod.csv
    output: resources/2024-test/networks/elec_s_2.nc, resources/2024-test/regions_onshore_elec_s_2.geojson, resources/2024-test/regions_offshore_elec_s_2.geojson, resources/2024-test/busmap_elec_s_2.csv, resources/2024-test/linemap_elec_s_2.csv
    log: logs/2024-test/cluster_network/elec_s_2.log (check log file(s) for error details)

    conda-env: C:\Users\SE91723\Modelos\pypsa\pypsa-eur\.snakemake\conda\3701581752c6e5590dda85748a41c931_

[Fabian Hofmann]

Hey Eduardo,

can you confirm that you recreated all networks starting from the base network?

Best

Fabian Hofmann

--
You received this message because you are subscribed to the Google Groups "pypsa" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pypsa+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pypsa/e1b80794-fe8b-4da3-96a5-78b0d620e7cen%40googlegroups.com.

-- 
Fabian Hofmann 

Postdoctoral Researcher
Institute of Energy Technology
Technische Universität Berlin
http://fabianhofmann.org/

Group website: https://www.tu.berlin/ensys

[isabel asensio]

hi, 

I have just had the same problem. I have just cloned pypsa-eur, and the environment.

• I have the problem in rule add_extra_components, in the function sanitize_locations. 
• It calls to n.buses.locations, but apparently, it has no attribute 'location' (See below the error)

I can not find the part of the workflow in which this attribute is created.

can someone help me? 

thanks!!

Isabel
______ 

ERROR:root:Uncaught exception
Traceback (most recent call last):

  File "/trinity/home/asensis/pypsa-eur-nodal-nodalv10_13jun/.snakemake/scripts/tmpy4pqq3l2.add_extra_components.py", line 252, in <module>
    sanitize_locations(n)
  File "/trinity/home/asensis/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpxf_zzvov/file/trinity/home/asensis/pypsa-eur-v10_13jun/rules/../scripts/add_electricity.py", line 182, in sanitize_locations

    n.buses["x"] = n.buses.x.where(n.buses.x != 0, n.buses.location.map(n.buses.x))

                                                   ^^^^^^^^^^^^^^^^
  File "/trinity/home/asensis/.conda/envs/version10/lib/python3.11/site-packages/pandas/core/generic.py", line 6299, in __getattr__
    return object.__getattribute__(self, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

AttributeError: 'DataFrame' object has no attribute 'location'

[isabel asensio]

Hello all, 

can somebody help me solve the issue I have with the function "sanitize locations" , in the "rule add_extra_components"? 

I am on the latest version of pypsa, trying to solve an electricity only problem. 

I posted the error a week ago, and I am copying again my error, with bit more information. Thank you very much for the help

So: 

• I have the problem in rule add_extra_components, in the function sanitize_locations. 
• It calls to n.buses.locations, but apparently, it has no attribute 'location' (See below the error)

• I dont have any error before... and I can not find the part of the workflow in which this attribute (n.buses.location) is created.

Here below the code

Thank you very much for your help!

Isabel

______ 

ERROR:root:Uncaught exception
Traceback (most recent call last):
  File "/trinity/home/asensis/pypsa-eur-nodal-nodalv10_13jun/.snakemake/scripts/tmpy4pqq3l2.add_extra_components.py", line 252, in <module>
    sanitize_locations(n)
  File "/trinity/home/asensis/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpxf_zzvov/file/trinity/home/asensis/pypsa-eur-v10_13jun/rules/../scripts/add_electricity.py", line 182, in sanitize_locations
    n.buses["x"] = n.buses.x.where(n.buses.x != 0, n.buses.location.map(n.buses.x))
                                                   ^^^^^^^^^^^^^^^^
  File "/trinity/home/asensis/.conda/envs/version10/lib/python3.11/site-packages/pandas/core/generic.py", line 6299, in __getattr__
    return object.__getattribute__(self, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'DataFrame' object has no attribute 'location'

I am using the latest version of the master branch. 

When trying to solve an electricity only problem

has 

You received this message because you are subscribed to a topic in the Google Groups "pypsa" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pypsa/iNGgnejQyz4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pypsa+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pypsa/353755ce-ad86-4f51-a46d-9c191c64266fn%40googlegroups.com.

[Fabian Hofmann]

Hey Isabel, 

The issue was already fixed in the repo. Can you please pull the latest master. Then everything should work as expected. 

Best wishes, 

Fabian 

Am 19.06.2024 um 21:02 schrieb isabel asensio <isabel....@gmail.com>:



To view this discussion on the web, visit https://groups.google.com/d/msgid/pypsa/CAAADuSKXXOdKE5g%2B-i6yrux9-BWH_9AtM86XuEa0QxUoKg%3DKHg%40mail.gmail.com.