Is there a way to reduce model writing time when passing the instance to the solver?

[Benjamín Maluenda]

Hello!

I read on some documents that Pyomo is significantly slower than other AMLs (such as AMPL) at writing big LPs so solvers can optimize them. It is stated there that this is caused by the way Pyomo maps model components.

For instance, I have a medium sized MILP model that takes 25 s to be solved by Gurobi (just the optimization). Meanwhile, the 'write' function of the PyomoModel object takes 45 s to be executed. That time is mostly spent on writing the LP (43,6 s). This optimization is being run with Gurobi 6.5.2 and Pyomo 4.2.10784 with all of the default solving options, on my local machine.

Is there a way around this issue? Do you recommend any good practices to reduce this writing time?

All the best,

Benjamin

[Carlos Suazo M.]

Benjamín,

You should give a try to Pyomo bindings that are significantly faster (on new version of Pyomo) for Gurobi than writing/reading LP's.

This can be simply done as follows:

opt = SolverFactory(solver,solver_io='python')

In our experience, Pyomo is not dramatically slower compared to AMPL, Mosel or GAMS.

Carlos Suazo

[Siirola, John D]

Carlos is absolutely correct: the direct python interfaces are generally significantly faster as they avoid generating and parsing files.  Alas, at this moment, they only exist for Gurobi and CPLEX.

 

Separately, Benjamin: would you mind sharing your model with us (off-list)?  I am collecting a library of larger models from the real world that we can use for performance profiling, benchmarking, and tuning.

 

Thanks!

 

John

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benjamín Maluenda]

Thanks Carlos!

Though, I tried out that option and the gurobi_direct plugin is working slower for me than the regular one. The small gains in speed during the optimization and post solve functions are offset by an increase in pre solving time. It seems the population of the gurobi instance is slower than writing and parsing an LP file. Some timings of a couple of tests I ran are as follows:

Small model:

With gurobi.py
-Print LP: 4.43 s
-Solvers.solve function: 6.54 s

With gurobi_direct.py
-Populate_gurobi_instance: 6.22 s
-Solvers.solve function: 7.15 s

Big model:

With gurobi.py
-Print LP: 77.90 s
-Solvers.solve function: 163.15 s

With gurobi_direct.py
-Populate_gurobi_instance: 112.32 s
-Solvers.solve function: 187.80 s

I found the following comment in the gurobi_direct plugin's docstring:

"Because of the direct connection with the Gurobi, no temporary files need be
 written or read.  That ostensibly makes this a faster plugin than the file-based
 Gurobi plugin.  However, you will likely not notice any speed up unless you are
 using the GLPK solver with PySP problems (due to the rapid re-solves)."

Does anybody have an idea on why would the direct plugin take longer to solve a model than the regular one? Maybe it's my machine's problem? I checked the changelog and there hasn't been any change on these plugins from version 4.2.10784 to the current one, so it shouldn't be an update problem.

Cheers,

Benjamín