Re: [openmod-initiative] HELM for real

94 views
Skip to first unread message

Robbie Morrison

unread,
Mar 10, 2020, 2:37:22 AM3/10/20
to openmod-i...@googlegroups.com

Hello Santiago

That all sounds really interesting.  I think it would be useful to start a openmod forum topic on this theme. I can help if you like.

Also to note there is a good article on wikipedia on HELM:

The wikipedia article also quotes US patents 7519506 and 7979239. I presume these patents do not cover aspects of the core method. Do you know anything about their scope and consequences.

Two of the early papers on HELM listed below.

I recall PyPSA was also looking at the method. Any news there from KIT?

with best wishes, Robbie

References

Trias, Antonio (July 2012). The holomorphic embedding load flow method. 2012 IEEE Power and Energy Society General Meeting. New Jersey, USA: IEEE. doi:10.1109/PESGM.2012.6344759.

Trias, Antonio (2015). “Fundamentals of the holomorphic embedding load-flow method”. arXiv. Abstract.


On 09/03/2020 19.58, Santiago Peñate wrote:

Dear all, 

It has been a while since I started reading and attempting to crack a working version of the Holomorphic Embeding Load Flow (HELM), being fascinated by it's almost magical properties.

In this search I found plenty of half-way papers, or papers with straight wrong formulations claiming otherwise. Last month, out of the blue a guy from Girona (Josep Fanals-Batllori) wrote me with his formulation and he wanted me to check it out. It worked! This was the first PQ+PV nodes formulation that I had seen working, providing almost perfectly convergent series. But it was not competitive in terms of speed or grid cases coverage (transformer taps, sparse matrix formulation, etc.)

After quite some work by Josep and myself, Today GridCal features a fully fledged holomorphic embedding power flow algorithm, with the "magical" convergence properties, that beat Newton-Raphson, and also beat the fast-decoupled algorithm in large grids. And that is a lot to say. However now, and unlike others before us, we provide hard evidence:


Enjoy.
--
You received this message because you are subscribed to the Google Groups "openmod initiative" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openmod-initiat...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openmod-initiative/4c52c178-82be-41a8-be0f-fbfe6f15dc6f%40googlegroups.com.

-- 
Robbie Morrison
Address: Schillerstrasse 85, 10627 Berlin, Germany
Phone: +49.30.612-87617

Robbie Morrison

unread,
Mar 10, 2020, 3:27:14 AM3/10/20
to openmod list

RESEND: for some reason the original send was not delivered after 00:50:00.  It could be because of some weird signature fields in the original posting?

Hello Santiago

That all sounds really interesting.  I think it would be useful to start a openmod forum topic on this theme. I can help if you like.

Also to note there is a good article on wikipedia on HELM:

The wikipedia article also quotes US patents 7519506 and 7979239. I presume these patents do not cover aspects of the core method. Do you know anything about their scope and consequences.

Two of the early papers on HELM listed below.

I recall PyPSA was also looking at the method. Any news there from KIT?

with best wishes, Robbie

References

Trias, Antonio (July 2012). The holomorphic embedding load flow method. 2012 IEEE Power and Energy Society General Meeting. New Jersey, USA: IEEE. doi:10.1109/PESGM.2012.6344759.

Trias, Antonio (2015). “Fundamentals of the holomorphic embedding load-flow method”. arXiv. Abstract.

-- 
Robbie Morrison
Address: Schillerstrasse 85, 10627 Berlin, Germany
Phone: +49.30.612-87617

Tom Brown

unread,
Mar 10, 2020, 5:42:15 AM3/10/20
to openmod-i...@googlegroups.com
Hi Robbie, all,

Yes, we wanted to implement it in PyPSA but were put off by exactly the
issue Josep and Santiago have solved, i.e. getting it to work with a
large number of PV buses, which break holomorphicity. (Finally, a use
for my math degree...)

Lay explanation: The method is for solving the power flow problem. The
standard Newton-Raphson method can run into numerical problems where it
fails to converge to a solution, despite a physical solution existing.
HELM is guaranteed not to have these types of convergence problems
thanks to the beautiful theory of complex functions. This is important
in a real-time control setting, where you don't want convergence problems.

Patent: I'm not a lawyer. My understanding was that the patents skirt
the issue that math should not be patentable in the US by patenting the
combination of algorithm and SCADA system. Either way, they're not valid
in Europe as I understand it. Please someone correct me if I'm wrong.

Performance: @Santiago: How well does the algorithm perform on networks
with 100 or 1000 buses?

Best wishes,

Tom





On 10/03/2020 07:37, Robbie Morrison wrote:
> Hello Santiago
>
> That all sounds really interesting.  I think it would be useful to start
> a openmod forum topic on this theme. I can help if you like.
>
> Also to note there is a good article on wikipedia on HELM:
>
> * https://en.wikipedia.org/wiki/Holomorphic_embedding_load_flow_method
>
> The wikipedia article also quotes US patents 7519506 and 7979239. I
> presume these patents do not cover aspects of the core method. Do you
> know anything about their scope and consequences.
>
> Two of the early papers on HELM listed below.
>
> I recall PyPSA was also looking at the method. Any news there from KIT?
>
> with best wishes, Robbie
>
> *References*
>
> Trias, Antonio (July 2012). /The holomorphic embedding load flow
> method/. 2012 IEEE Power and Energy Society General Meeting. New Jersey,
> USA: IEEE. doi:10.1109/PESGM.2012.6344759
> <https://dx.doi.org/10.1109/PESGM.2012.6344759>.
>
> Trias, Antonio (2015). “Fundamentals of the holomorphic embedding
> load-flow method” <https://arxiv.org/pdf/1509.02421>. /arXiv/. Abstract
> <https://arxiv.org/abs/1509.02421>.
>> <mailto:openmod-initiat...@googlegroups.com>.
>> <https://groups.google.com/d/msgid/openmod-initiative/4c52c178-82be-41a8-be0f-fbfe6f15dc6f%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> ​
>
> --
> Robbie Morrison
> Address: Schillerstrasse 85, 10627 Berlin, Germany
> Phone: +49.30.612-87617
>
> --
> You received this message because you are subscribed to the Google
> Groups "openmod initiative" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openmod-initiat...@googlegroups.com
> <mailto:openmod-initiat...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/openmod-initiative/e936d5de-8a31-a8bf-b59a-1316e1925c1f%40posteo.de
> <https://groups.google.com/d/msgid/openmod-initiative/e936d5de-8a31-a8bf-b59a-1316e1925c1f%40posteo.de?utm_medium=email&utm_source=footer>.

--
Karlsruhe Institute of Technology (KIT)
Institute for Automation and Applied Informatics (IAI)

Tom Brown (he/him)
Research Group Leader, Energy System Modelling

Phone: +49 721 608 25737
Fax: +49 721 608 22602
Group website: https://www.iai.kit.edu/english/ESM.php
Personal website: https://nworbmot.org/

Visitor Address:
Office 309
Campus North Building 445
Hermann-von-Helmholtz-Platz 1
76344 Eggenstein-Leopoldshafen

Santiago Peñate

unread,
Mar 10, 2020, 8:57:59 AM3/10/20
to openmod initiative
Hi guys, 


These area some results that I have just simulated on my very crappy laptop from work:

IEEE 118 Bus
Method Converged? Error Elapsed(s) Iterations
SolverType.HELM True 1.51E-11 0.0469 19
SolverType.HELM True 1.51E-11 0.0625193 19
SolverType.HELM True 1.51E-11 0.0533915 19
SolverType.HELM True 1.51E-11 0.0505242 19
SolverType.HELM True 1.51E-11 0.0523407 19
SolverType.NR True 3.93E-12 0.0156519 3
SolverType.NR True 3.93E-12 0.031256 3
SolverType.NR True 3.93E-12 0.0221415 3
SolverType.NR True 3.93E-12 0.0271504 3
SolverType.NR True 3.93E-12 0.0229998 3
Pegase 2869 Bus
Method Converged? Error Elapsed(s) Iterations
SolverType.HELM True 7.37E-11 2.48401 49
SolverType.HELM True 7.37E-11 2.44854 49
SolverType.HELM True 7.37E-11 2.63507 49
SolverType.HELM True 7.37E-11 2.49048 49
SolverType.HELM True 7.37E-11 2.4523 49
SolverType.NR True 3.52E-08 6.91317 5
SolverType.NR True 3.52E-08 6.88511 5
SolverType.NR True 3.52E-08 6.74982 5
SolverType.NR True 3.52E-08 6.8204 5
SolverType.NR True 3.52E-08 6.84771 5

You see that depending on the case it is either comparable or way faster than NR. My oppinion is that it will be difficult to tell since the NR speed will depend of the conditioning of the grid.
One of the great points of the HE method is that you can simulate 4 or 6 iterations and get an estimation of the result that goes in the right direction. This is very useful in RT systems or even system screening.

Regarding documentation you may find some here: https://github.com/u1946589/Holomorphic-Embedding-Josep
and in the documents of GridCal: https://gridcal.readthedocs.io/en/latest/theory/power_flow/holomorphic_embedding.html    (this one is WIP, and I intend to publish a comprehensive formulation)

Patent-wise, I believe we are ok since the formulation is sufficiently different, and we are not using this in a scada system.

BR,
Santiago


El martes, 10 de marzo de 2020, 7:37:22 (UTC+1), Robbie Morrison escribió:

Hello Santiago

That all sounds really interesting.  I think it would be useful to start a openmod forum topic on this theme. I can help if you like.

Also to note there is a good article on wikipedia on HELM:

The wikipedia article also quotes US patents 7519506 and 7979239. I presume these patents do not cover aspects of the core method. Do you know anything about their scope and consequences.

Two of the early papers on HELM listed below.

I recall PyPSA was also looking at the method. Any news there from KIT?

with best wishes, Robbie

References

Trias, Antonio (July 2012). The holomorphic embedding load flow method. 2012 IEEE Power and Energy Society General Meeting. New Jersey, USA: IEEE. doi:10.1109/PESGM.2012.6344759.

Trias, Antonio (2015). “Fundamentals of the holomorphic embedding load-flow method”. arXiv. Abstract.


On 09/03/2020 19.58, Santiago Peñate wrote:

Dear all, 

It has been a while since I started reading and attempting to crack a working version of the Holomorphic Embeding Load Flow (HELM), being fascinated by it's almost magical properties.

In this search I found plenty of half-way papers, or papers with straight wrong formulations claiming otherwise. Last month, out of the blue a guy from Girona (Josep Fanals-Batllori) wrote me with his formulation and he wanted me to check it out. It worked! This was the first PQ+PV nodes formulation that I had seen working, providing almost perfectly convergent series. But it was not competitive in terms of speed or grid cases coverage (transformer taps, sparse matrix formulation, etc.)

After quite some work by Josep and myself, Today GridCal features a fully fledged holomorphic embedding power flow algorithm, with the "magical" convergence properties, that beat Newton-Raphson, and also beat the fast-decoupled algorithm in large grids. And that is a lot to say. However now, and unlike others before us, we provide hard evidence:


Enjoy.
--
You received this message because you are subscribed to the Google Groups "openmod initiative" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openmod-initiative+unsub...@googlegroups.com.

Thurner, Leon, Dr.-Ing.

unread,
Mar 10, 2020, 9:58:51 AM3/10/20
to Santiago Peñate, openmod initiative, Alexander Scheidler, Schäfer, Florian

Hi Santiago,


congratulations on getting HELM up an running, great work!


The performance is of course very difficult to measure, because you are always measuring a specific implementation of an algorithm. For reference, the pegase2869 network that you measured at ~2.5s with HELM and ~7s with NR takes just 75ms to simulate with pandapower:



So in many cases, code optimization and removing bottlenecks is much more relevant than switching to a different algorithm. We also have a forward/backward sweep implemented in pandapower which is supposed to be faster than the NR in radial networks, but is actually slower because the implementation is not as heavily optimized as our NR.

Of course the real advantag of HELM is not performance, but convergence behaviour. Do you know of any examples where NR converges to the wrong solution or doesn't converge at all, but HELM finds the correct solution?

Regards
Leon



Von: openmod-i...@googlegroups.com <openmod-i...@googlegroups.com> im Auftrag von Santiago Peñate <santiago.p...@gmail.com>
Gesendet: Dienstag, 10. März 2020 13:57
An: openmod initiative
Betreff: Re: [openmod-initiative] HELM for real
 
To unsubscribe from this group and stop receiving emails from it, send an email to openmod-initiat...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openmod-initiative/c98e9f1a-a496-4e8d-ac06-6101575e1aaf%40googlegroups.com.

Santiago Peñate

unread,
Mar 10, 2020, 1:34:56 PM3/10/20
to openmod initiative
Hi All, 

For comparison, pandapower on my work's laptop was 0.28 seconds for the Pegase 2689 bus grid. That is 28 times faster.

From what I have seen you have come up with a ludicrously high tuned Jacobian formulation, and there is definitely a lot to learn about that.

In GridCal I have some algorithms and tweaks done to enhance convergence, namely:
- line search within Newton Raphson: If this one does not converge, probably there is something wrong (too high/low voltage settings, wrong impedances, too much imbalance, ...)
- Iwamoto Newton-Raphson: A somewhat optimal line search
- Levenberg Marquardt: A more robust convex solver

In GridCal's logic, if NR fails, Iwamoto goes next, then LM and finally a linear AC method that is always going to provide some solution (much like the linear DC would)

HELM is a very nice option because of the side effects; Once you have the voltage series there are a number of things that you can do almost automatically like check the grid distance to the collapse, asses the voltage collapse curve, and even check a node's installable power. For sure, a highly tuned HELM is an option and now with the formulation that is available anyone can put hours into making it blazing fast (faster convolutions, faster series computation, etc..)
It is better in many cases than the NR formulation that is featured in GridCal, but your NR implementation with numba code pulverises the solving times.

IMHO, pandapower's highly tuned Jacobian into a line search newton raphson is king right now.

Best regards,
Santiago

El martes, 10 de marzo de 2020, 7:37:22 (UTC+1), Robbie Morrison escribió:

Hello Santiago

That all sounds really interesting.  I think it would be useful to start a openmod forum topic on this theme. I can help if you like.

Also to note there is a good article on wikipedia on HELM:

The wikipedia article also quotes US patents 7519506 and 7979239. I presume these patents do not cover aspects of the core method. Do you know anything about their scope and consequences.

Two of the early papers on HELM listed below.

I recall PyPSA was also looking at the method. Any news there from KIT?

with best wishes, Robbie

References

Trias, Antonio (July 2012). The holomorphic embedding load flow method. 2012 IEEE Power and Energy Society General Meeting. New Jersey, USA: IEEE. doi:10.1109/PESGM.2012.6344759.

Trias, Antonio (2015). “Fundamentals of the holomorphic embedding load-flow method”. arXiv. Abstract.


On 09/03/2020 19.58, Santiago Peñate wrote:

Dear all, 

It has been a while since I started reading and attempting to crack a working version of the Holomorphic Embeding Load Flow (HELM), being fascinated by it's almost magical properties.

In this search I found plenty of half-way papers, or papers with straight wrong formulations claiming otherwise. Last month, out of the blue a guy from Girona (Josep Fanals-Batllori) wrote me with his formulation and he wanted me to check it out. It worked! This was the first PQ+PV nodes formulation that I had seen working, providing almost perfectly convergent series. But it was not competitive in terms of speed or grid cases coverage (transformer taps, sparse matrix formulation, etc.)

After quite some work by Josep and myself, Today GridCal features a fully fledged holomorphic embedding power flow algorithm, with the "magical" convergence properties, that beat Newton-Raphson, and also beat the fast-decoupled algorithm in large grids. And that is a lot to say. However now, and unlike others before us, we provide hard evidence:


Enjoy.
--
You received this message because you are subscribed to the Google Groups "openmod initiative" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openmod-initiative+unsub...@googlegroups.com.

Robbie Morrison

unread,
Mar 11, 2020, 2:16:57 AM3/11/20
to openmod-i...@googlegroups.com

Hi Tom

If someone needs to dig into the applicability of United States software patents in Europe, I suggest Haapanen (2017). Haapanen reviews software patents in Europe (p72–79) and software patents in the US (p79–86).  FOSS software licensing is covered too relative to legal theory.

Robbie

References

Haapanen, Anna (1 April 2017). Free and open source software licensing: and the mystery of licensor’s patents — PhD thesis. Helsinki, Finland: Faculty of Law, University of Helsinki.

On 10/03/2020 10.42, Tom Brown wrote:

Patent: I'm not a lawyer. My understanding was that the patents skirt
the issue that math should not be patentable in the US by patenting the
combination of algorithm and SCADA system. Either way, they're not valid
in Europe as I understand it. Please someone correct me if I'm wrong.
-- 
Robbie Morrison
Address: Schillerstrasse 85, 10627 Berlin, Germany
Phone: +49.30.612-87617

Thurner, Leon, Dr.-Ing.

unread,
Mar 11, 2020, 5:52:31 AM3/11/20
to Santiago Peñate, openmod initiative, Schäfer, Florian, Alexander Scheidler

Hi Santiago, all,


I was really curious about the HELM implementation, so I gave it a try and integrated into pandapower. Here is the short version:


  1. It works and gives almost the same results as the pandapower standard NR solver (<1e-4pu voltage difference on the first try)! Thanks and kudos for this implementation!
  2. For the ill-conditioned case11_iwamoto system thats included in pandapower, neither NR nor HELM find a valid solution. I would be interested in finding out why that is...


For more discussion, I suggest we move to the openmod forum to not spam everyone: https://forum.openmod-initiative.org/t/the-new-helm-powerflow-in-gridcal/1952


Cheers
Leon




Gesendet: Dienstag, 10. März 2020 18:34

An: openmod initiative
Betreff: Re: [openmod-initiative] HELM for real
To unsubscribe from this group and stop receiving emails from it, send an email to openmod-initiat...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openmod-initiative/952b4755-b39b-45a2-aac4-c91028a37fb7%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages