Choosing a solver for performance

2,303 views
Skip to first unread message

Nicholas Gorman

unread,
Mar 14, 2018, 6:47:35 PM3/14/18
to pulp-or-discuss

Hi Folks,

I am using Pulp to solver some problems that include a mix of binary variables and linear variables. Typically these take between 2 and 5 s to solve and this has become the main bottle neck in my program. So far I have just used the default PULP_CBC_CMD solver, but wanted to explore using other solvers to speed up the program. I was wondering if someone had some tips on how to approach this or could point me in the direction of some resource that explained when to use which types of solvers and why. 

Kind Regards,
Nick

Clay Campaigne

unread,
Mar 14, 2018, 11:41:36 PM3/14/18
to pulp-or...@googlegroups.com
Hans Mittelmann
​'s list of benchmarks for MILP solution times is a standard place to look. CBC is generally the fastest free solver, but the proprietary ones (I'm thinking Gurobi and CPLEX) are generally 20-30x faster. SCIP semi-proprietary and its performance is halfway in between (geometrically).  You can use Gurobi for free if it's academic, SCIP for free if it's non-commercial (case-by-case by contacting the developers), and a SCIP license costs I think $1000 perpetual, for commercial purposes. Subject to negotiation depending on the application and pricing model of any resulting product. Gurobi has transparent pricing, and I think CPLEX requires contacting them and maybe negotiation.

​MILP benchmarks:​

I've used CBC, Gurobi, CPLEX, and SCIP, and my experience roughtly lines up with this, except in certaint MIP that more closely resempled combinatorial optimization, where SCIP was pretty close to CBC. 

best
Clay

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.



--
Clay Campaigne
Cell Phone: (773) 732-9406

Stuart Mitchell

unread,
Mar 14, 2018, 11:43:04 PM3/14/18
to pulp-or...@googlegroups.com
I think with those solve times you will be better off trying to avoid the disk writes, maybe a ramdisk?

Stu

Stuart Mitchell
PhD Engineering Science
Extraordinary Freelance Programmer and Optimisation Guru

Nicholas Gorman

unread,
Mar 15, 2018, 8:00:53 AM3/15/18
to pulp-or-discuss
Hi Stu,

I like the idea of the ramdisk and have done a bit of research, but can't seem to find a way to make Pulp write to a different file location that is not where it is installed. Would you expect this to be a bit complicated, like changing parts of Pulp, or is there a simple way I'm missing.

Cheers,
Nick


On Thursday, 15 March 2018 14:43:04 UTC+11, Stuart Mitchell wrote:
I think with those solve times you will be better off trying to avoid the disk writes, maybe a ramdisk?

Stu

Stuart Mitchell
PhD Engineering Science
Extraordinary Freelance Programmer and Optimisation Guru

On Thu, Mar 15, 2018 at 4:41 PM, Clay Campaigne <clay.ca...@gmail.com> wrote:
Hans Mittelmann
​'s list of benchmarks for MILP solution times is a standard place to look. CBC is generally the fastest free solver, but the proprietary ones (I'm thinking Gurobi and CPLEX) are generally 20-30x faster. SCIP semi-proprietary and its performance is halfway in between (geometrically).  You can use Gurobi for free if it's academic, SCIP for free if it's non-commercial (case-by-case by contacting the developers), and a SCIP license costs I think $1000 perpetual, for commercial purposes. Subject to negotiation depending on the application and pricing model of any resulting product. Gurobi has transparent pricing, and I think CPLEX requires contacting them and maybe negotiation.

​MILP benchmarks:​

I've used CBC, Gurobi, CPLEX, and SCIP, and my experience roughtly lines up with this, except in certaint MIP that more closely resempled combinatorial optimization, where SCIP was pretty close to CBC. 

best
Clay
On Wed, Mar 14, 2018 at 3:47 PM, Nicholas Gorman <n.gor...@gmail.com> wrote:

Hi Folks,

I am using Pulp to solver some problems that include a mix of binary variables and linear variables. Typically these take between 2 and 5 s to solve and this has become the main bottle neck in my program. So far I have just used the default PULP_CBC_CMD solver, but wanted to explore using other solvers to speed up the program. I was wondering if someone had some tips on how to approach this or could point me in the direction of some resource that explained when to use which types of solvers and why. 

Kind Regards,
Nick

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.
--
Clay Campaigne
Cell Phone: (773) 732-9406

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.

Stuart Mitchell

unread,
Mar 15, 2018, 4:33:12 PM3/15/18
to pulp-or...@googlegroups.com
use the `path` parameter to the solver


prob.solve(PULP_CBC_CMD(path=path_to_ram_disk))

regards 
Stu

Stuart Mitchell
PhD Engineering Science
Extraordinary Freelance Programmer and Optimisation Guru

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.

Nicholas Gorman

unread,
Mar 15, 2018, 5:42:21 PM3/15/18
to pulp-or-discuss

There seems to be line in the solvers code directly preventing this,

https://github.com/coin-or/pulp/blob/bac6d9d2214ba773d638d2de5149940cfd711359/src/pulp/solvers.py#L1558


I get the error "Use COIN_CMD if you want to set path". Would this mean I need to install COIN_CMD if I want to use the ramdisk?

Stuart Mitchell

unread,
Mar 15, 2018, 6:08:12 PM3/15/18
to pulp-or...@googlegroups.com
sorry that was the wrong parameter.

try setting keepFiles=1

then the files will be saved in the current directory

Stu

Stuart Mitchell
PhD Engineering Science
Extraordinary Freelance Programmer and Optimisation Guru

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.

Nicholas Gorman

unread,
Mar 15, 2018, 7:07:49 PM3/15/18
to pulp-or-discuss
This seems to work for the the .mps file which gets written to the ramdisk, but for some reason the .sol never gets written and an error is thrown when solve_CBC checks for it. 
https://github.com/coin-or/pulp/blob/bac6d9d2214ba773d638d2de5149940cfd711359/src/pulp/solvers.py#L1426





Note I just did keepFiles=1, and also changed the working directory to the ramdisk before creating the prob.

Stuart Mitchell

unread,
Mar 15, 2018, 7:10:23 PM3/15/18
to pulp-or...@googlegroups.com
does your lp name have spaces or other characters in it?

msg=1 

can help to see the solver log

Stu

Stuart Mitchell
PhD Engineering Science
Extraordinary Freelance Programmer and Optimisation Guru

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.

Nicholas Gorman

unread,
Apr 10, 2018, 6:01:41 AM4/10/18
to pulp-or-discuss
Hi Stuart,

You were dead right about the spaces in the name!

The ramdisk speeds the solve up from about  3 s to about 2 s, I did some profiling deeper into the solver and it looks like the cbc.wait() call is responsible for about 1 s of runtime. 


I understand this is the time taken for the subprocess to run the cbc executable ?  Do you think moving these lower level process to the ram disk would provide additional benefits or if this is even possible ?

Cheers,
Nick

Stuart Mitchell

unread,
Apr 12, 2018, 5:47:10 PM4/12/18
to pulp-or...@googlegroups.com
I guess you could try installing pulp into a virtualenv on the ram disk

Stu

Stuart Mitchell
PhD Engineering Science
Extraordinary Freelance Programmer and Optimisation Guru

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages