Error when solving PyPSA-EUR with Xpress

[Peter Klein]

Hi Fabian

I am trying to solve the PyPSA-EUR tutorial model using Xpress. The model solves fine with CBC, but I am getting an error related to the warmstart in ilopf. Any suggestions for a quick fix?

Regards

Peter

Reading Problem pypsa-problem-ki6fpit6
Problem Statistics
      122767 (      0 spare) rows
       58823 (      0 spare) structural columns
      248508 (      0 spare) non-zero elements
Global Statistics
           0 entities        0 sets        0 set members
Traceback (most recent call last):
  File "/home/pklein/GitHub/pypsa-eur2/.snakemake/shadow/tmppqqh_5p7/.snakemake/scripts/tmp153y13_b.solve_network.py", line 370, in <module>
    n = solve_network(n, snakemake.config, opts, solver_dir=tmpdir,
  File "/home/pklein/GitHub/pypsa-eur2/.snakemake/shadow/tmppqqh_5p7/.snakemake/scripts/tmp153y13_b.solve_network.py", line 345, in solve_network
    ilopf(n, solver_name=solver_name, solver_options=solver_options,
  File "/home/pklein/anaconda3/envs/pypsa-eur-2/lib/python3.10/site-packages/pypsa/linopf.py", line 1606, in ilopf
    status, termination_condition = network_lopf(n, snapshots, **kwargs)
  File "/home/pklein/anaconda3/envs/pypsa-eur-2/lib/python3.10/site-packages/pypsa/linopf.py", line 1462, in network_lopf
    res = solve(
  File "/home/pklein/anaconda3/envs/pypsa-eur-2/lib/python3.10/site-packages/pypsa/linopt.py", line 1158, in run_and_read_xpress
    m.readbasis(warmstart)
TypeError: argument 1 must be str, not bool

[Fabian Neumann]

Hi Peter,

this seems to be indeed a bug with the Xpress warmstart implementation
in PyPSA:

https://github.com/PyPSA/PyPSA/blob/996952d73857a54a772782271e4a146a4e8f8288/pypsa/linopt.py#L1005

This is the line one would need to fix. If that is not possible, cann
just ignore this setting by replacing the line with:

logger.warning("Warmstart, not available in Xpress. Will be ignored.")

Would you mind flagging this as an issue on Github? A PR is of course
also welcome.

A quick fix without needing to change PyPSA is to set in config.yaml,
skip_iterations to True. This will skip the iterative impedance update
in response to line expansion.

Best wishes,

Fabian N

> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pypsa/0fbb82bb-bfa2-4ded-8a03-679924052b53n%40googlegroups.com
> <https://groups.google.com/d/msgid/pypsa/0fbb82bb-bfa2-4ded-8a03-679924052b53n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Dr.-Ing.
Fabian Neumann (he/him)
Postdoctoral Researcher

Department of Digital Transformation in Energy Systems
Institute of Energy Technology
Technische Universität Berlin

Group website: https://tub-ensys.github.io
Personal website: https://neumann.fyi
Schedule Meeting: https://calendly.com/fneum

Einsteinufer 25 (TA 8)
10587 Berlin

My working hours may not be your working hours. Do not feel pressure to
reply to this email outside your working hours.

[Peter Klein]

Thanks for the quick response and information. I am going on leave this week, but would be happy to consider a fix and pull request when I am back. 

A quick follow up question, if you don't mind? Is it impractical to do the ilopf without considering a warmstart? I find that the barrier method in Xpress, without crossover is very quick (say 10-30min for a large problem), but adding crossover can add 6h+. My understanding is that you need a simplex solution in order to get a basis function that can be used under warm start conditions? I am wondering if the added solver time to get the basis function in the first place is worth the effort, vs doing another barrier solve each iteration. 

[Fabian Neumann]

Hi Peter,

no, I think the warmstart is not necessary, and yes, we also disable
crossover per default for performance.

I would have to revisit how it is currently implemented in different
solvers. My feeling is that Gurobi, which we normally use, skips the
warmstart if no basis is available from the previous iteration rather
than failing.

Best wishes,

Fabian N

> <http://tmp153y13_b.solve_network.py>",

> > line 370, in <module>
> >     n = solve_network(n, snakemake.config, opts, solver_dir=tmpdir,
> >   File
> >
> "/home/pklein/GitHub/pypsa-eur2/.snakemake/shadow/tmppqqh_5p7/.snakemake/scripts/tmp153y13_b.solve_network.py

> <http://tmp153y13_b.solve_network.py>",

> <https://groups.google.com/d/msgid/pypsa/0fbb82bb-bfa2-4ded-8a03-679924052b53n%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/pypsa/0fbb82bb-bfa2-4ded-8a03-679924052b53n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> Dr.-Ing.
> Fabian Neumann (he/him)
> Postdoctoral Researcher
>
> Department of Digital Transformation in Energy Systems
> Institute of Energy Technology
> Technische Universität Berlin
>
> Group website: https://tub-ensys.github.io
> <https://tub-ensys.github.io>

> Personal website: https://neumann.fyi <https://neumann.fyi>

> Schedule Meeting: https://calendly.com/fneum
> <https://calendly.com/fneum>
>
> Einsteinufer 25 (TA 8)
> 10587 Berlin
>
> My working hours may not be your working hours. Do not feel pressure to
> reply to this email outside your working hours.
>

> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/pypsa/8bd3a412-44ce-40bf-b838-727cda1d1c46n%40googlegroups.com
> <https://groups.google.com/d/msgid/pypsa/8bd3a412-44ce-40bf-b838-727cda1d1c46n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Peter Klein]

I added this mods in bold to the run_and_read_xpress, which does just that. Happy to submit if you think useful.

    if warmstart:

        try:

            m.readbasis(n.basis_fn)

            logger.info("Basis loaded")

        except:

            logger.info("No model basis loaded")

            pass

    m.solve()

    if store_basis:

        n.basis_fn = solution_fn.replace(".sol", ".bss")

        try:

            m.writebasis(n.basis_fn)

        except:

            logger.info("No model basis stored")

            del n.basis_fn

[Fabian Neumann]

Hi Peter,

thanks! This looks good!

On 02.08.22 12:16, Peter Klein wrote:
> I added this mods in bold to the run_and_read_xpress, which does just
> that. Happy to submit if you think useful.
>     if warmstart:

> *        try:*
> *            m.readbasis(n.basis_fn)*
> *            logger.info("Basis loaded")*
> *        except:*
> *            logger.info("No model basis loaded")*
> *            pass*
>     m.solve()
>
>     if store_basis:
>         n.basis_fn = solution_fn.replace(".sol", *".bss")*

> > <http://tmp153y13_b.solve_network.py

> <http://tmp153y13_b.solve_network.py>>",
> > > line 370, in <module>
> > >     n = solve_network(n, snakemake.config, opts,
> solver_dir=tmpdir,
> > >   File
> > >
> >
> "/home/pklein/GitHub/pypsa-eur2/.snakemake/shadow/tmppqqh_5p7/.snakemake/scripts/tmp153y13_b.solve_network.py
> <http://tmp153y13_b.solve_network.py>

> > <http://tmp153y13_b.solve_network.py

> <https://groups.google.com/d/msgid/pypsa/8bd3a412-44ce-40bf-b838-727cda1d1c46n%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/pypsa/8bd3a412-44ce-40bf-b838-727cda1d1c46n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> Dr.-Ing.
> Fabian Neumann (he/him)
> Postdoctoral Researcher
>
> Department of Digital Transformation in Energy Systems
> Institute of Energy Technology
> Technische Universität Berlin
>
> Group website: https://tub-ensys.github.io

> <https://tub-ensys.github.io>
> Personal website: https://neumann.fyi <https://neumann.fyi>
> Schedule Meeting: https://calendly.com/fneum
> <https://calendly.com/fneum>
>
> Einsteinufer 25 (TA 8)
> 10587 Berlin
>
> My working hours may not be your working hours. Do not feel pressure to
> reply to this email outside your working hours.
>
> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/pypsa/8ccd2188-7c3d-44d5-b870-88bee577e966n%40googlegroups.com
> <https://groups.google.com/d/msgid/pypsa/8ccd2188-7c3d-44d5-b870-88bee577e966n%40googlegroups.com?utm_medium=email&utm_source=footer>.