pypower 80 times slower than matpower

[kenneth...@gmail.com]

Hi guys,

I have the following problem.
When I do a runopf() in matpower and pypower with the same data it is 80 times slower in pypower.

I figured out the following code in opf_consfcn is the problem.

137 dg[:, iVaVmPgQg] = vstack([
138            ## P mismatch w.r.t Va, Vm, Pg, Qg
139            hstack([dSbus_dVa.real, dSbus_dVm.real, neg_Cg, blank]),
140            ## Q mismatch w.r.t Va, Vm, Pg, Qg
141            hstack([dSbus_dVa.imag, dSbus_dVm.imag, blank, neg_Cg])
142        ], "csr")

needs about 5 to 6 seconds.


168 dh[:, r_[iVa, iVm].T] = vstack([
169           hstack([df_dVa, df_dVm]), ## "from" flow limit
170           hstack([dt_dVa, dt_dVm]) ## "to" flow limit

needs about 4 seconds.

On this position in the code dg and dh are 11501x8674 lil_matrices.

Is there a workaround to speed up the code on this position?

Thanks

[Richard Lincoln]

I suspect the performance issue arises not from the stacking operations, but from the assignment to specific columns of dg and dh. Perhaps you could test this by separating the operations into two different statements and timing them. If the stacking operations are slow you could try converting the inputs to some other sparse matrix format. If that doesn't help you could try using CVXOPT:

http://cvxopt.org/userguide/matrices.html#cvxopt.sparse

I think the matrices can be converted like this:

>>> coo = csr.tocoo()
>>> sp = spmatrix(coo.data, coo.row.tolist(), coo.col.tolist())

It may be possible to optimize (or possibly ignore) the assignment operation by making some assumptions about order of the Va, Vm, Pg, Qg indexes.

This is an interesting bug and I'd like to see it fixed. I hope you make some progress.

Richard

--
You received this message because you are subscribed to the Google Groups "pypower" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pypower+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[kenneth...@gmail.com]

Hi Richard,

thanks for your response.

Your hint save me now 7.5 seconds per call.

This is my first change....

Code hier

#neg_Cg = sparse((-ones(ng), (gen[:, GEN_BUS], range(ng))), (nb, ng))
## construct Jacobian of equality constraints (power flow) and transpose it
# dg = lil_matrix((2 * nb, nxyz))
# blank = sparse((nb, ng))
neg_Cg = csc_matrix((-ones(ng), (gen[:, GEN_BUS], range(ng))), (nb, ng))
dg = csc_matrix((2 * nb, nxyz))
blank = csc_matrix((nb, ng))
cols = vstack([hstack([dSbus_dVa.real, dSbus_dVm.real, neg_Cg, blank]), hstack([dSbus_dVa.imag, dSbus_dVm.imag, blank, neg_Cg])], 'csc')

dg[:, iVaVmPgQg] = cols
#dg = dg.tolil()
# dg[:, iVaVmPgQg] = vstack([
#         ## P mismatch w.r.t Va, Vm, Pg, Qg
#         hstack([dSbus_dVa.real, dSbus_dVm.real, neg_Cg, blank]),
#         ## Q mismatch w.r.t Va, Vm, Pg, Qg
#         hstack([dSbus_dVa.imag, dSbus_dVm.imag, blank, neg_Cg])
#     ], "csr")

eingeben...

... and my second one:

dh = csc_matrix((2 * nl2, nxyz))
cols2 = vstack([

        hstack([df_dVa, df_dVm]),    ## "from" flow limit

        hstack([dt_dVa, dt_dVm])     ## "to" flow limit

    ], 'csr')
dh[:, r_[iVa, iVm].T] = cols2
#dh.tolil()
# dh = lil_matrix((2 * nl2, nxyz))
# dh[:, r_[iVa, iVm].T] = vstack([
#         hstack([df_dVa, df_dVm]),    ## "from" flow limit
#         hstack([dt_dVa, dt_dVm])     ## "to" flow limit
#     ], "csr")

I could reduce the needed time from 9.8 to 2.3 seconds.

Right now PYPOWER is just 17 times slower than MatPower.

I think it could be a bug in scipy.sparse.lil_matrix cause I get a warning when I use the csc-/csr-format.

C:\Python34\lib\site-packages\scipy\sparse\compressed.py:730: SparseEfficiencyWarning: Changing the sparsity structure of a csc_matrix is expensive. lil_matrix is more efficient.
  SparseEfficiencyWarning)

Thanks a lot.

Kenneth