"xarray MergeError" in combination of timeslice, agent and region

[jamshid]

Dear MUSE team

While running a technology with 8 timeslice for 2 regions with 2 agents following error occured.
 
-- muse.objectives - INFO
Computing objective: capacity

Traceback (most recent call last):
  File "...\.venv\lib\site-packages\xarray\core\dataarray.py", line 3120, in _inplace_binary_op
    with self.coords._merge_inplace(other_coords):
  File "C:\Program Files\Python39\lib\contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "...\.venv\lib\site-packages\xarray\core\coordinates.py", line 191, in _merge_inplace
    variables, indexes = merge_coordinates_without_align(
  File "...\.venv\lib\site-packages\xarray\core\merge.py", line 345, in merge_coordinates_without_align
    return merge_collected(filtered, prioritized, combine_attrs=combine_attrs)
  File "...\.venv\lib\site-packages\xarray\core\merge.py", line 217, in merge_collected
    raise MergeError(
xarray.core.merge.MergeError: conflicting values for index 'timeslice' on objects to be combined:
first value: <xarray.core.indexes.PandasIndex object at 0x000002033697DA40>
second value: <xarray.core.indexes.PandasMultiIndex object at 0x0000020336963B80>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "...\test_01.py", line 32, in <module>
    rslts()
  File "...\test_01.py", line 28, in rslts
    model.run()
  File "...\.venv\lib\site-packages\muse\mca.py", line 346, in run
    _, new_market, self.sectors = self.find_equilibrium(new_market)
  File "...\.venv\lib\site-packages\muse\mca.py", line 209, in find_equilibrium
    return find_equilibrium(
  File "...\.venv\lib\site-packages\muse\mca.py", line 559, in find_equilibrium
    market, equilibrium_sectors = single_year_iteration(market, sectors)
  File "...\.venv\lib\site-packages\muse\mca.py", line 466, in single_year_iteration
    sector_market = sector.next(
  File "...\.venv\lib\site-packages\muse\sectors\sector.py", line 233, in next
    subsector.invest(
  File "...\.venv\lib\site-packages\muse\sectors\subsector.py", line 74, in invest
    lp_problem = self.aggregate_lp(
  File "...\.venv\lib\site-packages\muse\sectors\subsector.py", line 147, in aggregate_lp
    result = agent.next(
  File "...\.venv\lib\site-packages\muse\agents\agent.py", line 402, in next
    search = super().next(technologies, market, demand, time_period=time_period)
  File "...\.venv\lib\site-packages\muse\agents\agent.py", line 268, in next
    decision = self._compute_objective(demand, search_space, technologies, market)
  File "...\.venv\lib\site-packages\muse\agents\agent.py", line 280, in _compute_objective
    objectives = self.objectives(self, demand, search_space, technologies, market)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 142, in objectives
    result[name] = objective(agent, demand, search_space, *args, **kwargs)
  File "...\.venv\lib\site-packages\muse\registration.py", line 172, in decorated
    result = inner_decorated(*args, **kwargs)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 173, in decorated_objective
    result = function(agent, reduced_demand, search_space, *args, **kwargs)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 899, in equivalent_annual_cost
    npv = net_present_cost(agent, demand, search_space, technologies, market)
  File "...\.venv\lib\site-packages\muse\registration.py", line 172, in decorated
    result = inner_decorated(*args, **kwargs)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 173, in decorated_objective
    result = function(agent, reduced_demand, search_space, *args, **kwargs)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 860, in net_present_cost
    return -net_present_value(
  File "...\.venv\lib\site-packages\muse\registration.py", line 172, in decorated
    result = inner_decorated(*args, **kwargs)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 173, in decorated_objective
    result = function(agent, reduced_demand, search_space, *args, **kwargs)
  File "...\.venv\lib\site-packages\muse\objectives.py", line 836, in net_present_value
    results -= fuel_costs
  File "...\.venv\lib\site-packages\xarray\core\_typed_ops.py", line 290, in __isub__
    return self._inplace_binary_op(other, operator.isub)
  File "...\.venv\lib\site-packages\xarray\core\dataarray.py", line 3123, in _inplace_binary_op
    raise MergeError(
xarray.core.merge.MergeError: Automatic alignment is not supported for in-place operations.
Consider aligning the indices manually or using a not-in-place operation.
See https://github.com/pydata/xarray/issues/3910 for more explanations.
Process finished with exit code 1

The problem seems to be adding arrays with different order, what is the correct way to model "timeslices", "agents" and "regions" together?!

Thanks

[sgiarols]

Dear Jamshid

MUSE needs to read consistent timeslices in the TOML file, in the preset file, and in the technidatatimeslices.csv (if you are using it to change technology production by timeslice).

Have you changed the number of timeslices consistenly across these files?

Something more on the timeslice variation is present here https://muse-os.readthedocs.io/en/latest/user-guide/modify-timing-data.html

I hope this helps.

Best wishes

[jamshid]

Dear Sara,

Thanks for your kind reply.

Yes, both TOML and TechnodataTimeslices.csv files modified accordingly.
In the TOML file the sum of timeslices 8760 and plugin "output.py" added.
TechnodataTimeslices.csv has been modified based on the following tutorial:  https://muse-os.readthedocs.io/en/stable/user-guide/min-max-timeslice-constraints.html
Should I share files from "...\data\example\default" folder?

ps. I noticed that
• 8 timeslices, 2 regions, 1 agent
• 8 timeslices, 1 region, 2 agents
• 2 regions, 2 agents, without timeslice modifications
works great. The error occurs only in the case of
• N timeslices, 2 regions, 2 agents

I'm using "venv" environment (Python 3.9.16) with  "xarray" package version 2022.3.0

Regards,
Jamshid

[jamshid]

Dear Sara, 

Thank you very much for your support.
Pull #135 problem solved.

Regards,
Jamshid

[Alaa Qassabi]

Dear Jamshid,

I still get the same error message even though I'm running the latest version of muse.

Could you please explain how you overcome this agent-related error?

Most appreciated

Alaa

[jamshid]

Dear Alaa Qassabi, 

As mentioned in my last post, the developers already fixed this issue.
Please make sure you are using the latest version of Muse. Here is the github link

Regards,
Jamshid