How to get CPLEX on PuLP

[Nicolò Golinucci]

Hi Stu,

I am using Python for LP modelling. I have used the default solver and I got a solution after several hours of programming. I need to speed up the code and I decided to use CPLEX.
I have installed CPLEX via Anaconda Prompt and I performed a run which was interrupted by an error such as "set too extended". So I thought that the version of CPLEX I have obtained was limited on number of variables (my problem needs 300k variables) and I downloaded an unlimted academic version but I cannot understand if pulp is able to use it. How do I let pulp use solvers I download?
Sorry if the question may seem trivial, but I am quite confused

Thank you in advantage for your time,

Nicolo

[Stuart Mitchell]

I haven't used the cplex in a while but first make sure that cplex is installed in python

do this by typing

>>>  import cplex

in the anaconda prompt

if you don't get an error then you can use the `CPLEX_PY` solver by doing the following

>>> prob.solve(CPLEX_PY())

Stu

Stuart Mitchell

PhD Engineering Science

Extraordinary Freelance Programmer and Optimisation Guru

www.stuartmitchell.com

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

[Nicolò Golinucci]

Thank you for answering so quickly! 

The fact is that my code uses 300k variables and therefore I cannot rely on the free to use version of CPLEX but I need to use the academic version. I have downloaded it and I have installed IBM ILOG Studio, but I cannot understand how to "let Anaconda know" that it has the power to use CPLEX at full power (more than 1k variables).
Alternatively, are there any other solvers that I can easier use which can performa an optimizaion with that large number of variables?
That is the error that I have:

Mail priva di virus. www.avg.com

[Stuart Mitchell]

I suggest you look at the cplex installation docs for this answer.

You will have already done it once for the promotional version, you just need to run the `setup.py` file for the academic version

Gurobi Academic version will also solve this problem size

Stu

Stuart Mitchell

PhD Engineering Science

Extraordinary Freelance Programmer and Optimisation Guru

www.stuartmitchell.com

On Tue, Apr 10, 2018 at 3:58 AM, Nicolò Golinucci <nicolo.g...@gmail.com> wrote:

Thank you for answering so quickly! 

The fact is that my code uses 300k variables and therefore I cannot rely on the free to use version of CPLEX but I need to use the academic version. I have downloaded it and I have installed IBM ILOG Studio, but I cannot understand how to "let Anaconda know" that it has the power to use CPLEX at full power (more than 1k variables).
Alternatively, are there any other solvers that I can easier use which can performa an optimizaion with that large number of variables?
That is the error that I have:

Mail priva di virus. www.avg.com

[Nicolò Golinucci]

Thank you! I have installed GUROBI and the problem has been solved. The thing I do not understand is the following:

My problem is devided into two formulation (primal and dual). The primal is comprehensive of 380k variables and similar magnitude of constraints; the dual has 8k variables and a similar number of constraints (almost 400k). Those two separated model are, right now, performed in series. When I run only the primal I get an optimal result; when I run the two model in series I got "Status: Not Solved" for both without any other error. I have tried to run the same model using a smaller data set and I very quickly get a solution.

Furthermore, the model need 2 hours and half for primal, and almost 3 for the dual. Is it possible to introduce multiprocessing in PuLP in order to make the computation quicker? It would be great for example to perform primal and dual formulation in parallel.

Best regards,

Nicolò

[Stuart Mitchell]

use msg=1 in the solver parameters to see what the solver is printing out.

gurobi is parallel by default so you will probably just slow everything down if you try and run primal and dual at the same time.

Stu

Stuart Mitchell

PhD Engineering Science

Extraordinary Freelance Programmer and Optimisation Guru

www.stuartmitchell.com

On Thu, Apr 12, 2018 at 8:31 PM, Nicolò Golinucci <nicolo.g...@gmail.com> wrote:

Thank you! I have installed GUROBI and the problem has been solved. The thing I do not understand is the following:

My problem is devided into two formulation (primal and dual). The primal is comprehensive of 380k variables and similar magnitude of constraints; the dual has 8k variables and a similar number of constraints (almost 400k). Those two separated model are, right now, performed in series. When I run only the primal I get an optimal result; when I run the two model in series I got "Status: Not Solved" for both without any other error. I have tried to run the same model using a smaller data set and I very quickly get a solution.

Furthermore, the model need 2 hours and half for primal, and almost 3 for the dual. Is it possible to introduce multiprocessing in PuLP in order to make the computation quicker? It would be great for example to perform primal and dual formulation in parallel.

Best regards,

Nicolò

[Siddharth Shetty]

Hello,

I am using the Osemosys pulp version for my thesis at KTH on a district heating module. Since the Osemosys pulp code uses a non-linear solver, I wish to use a LP solver such as cplex or glpk. Is there any information you can provide on using one of these solvers in the existing Osemosys pulp code?

Also, I tried using the information you have provided which is to use CPLEX_PY(), but that does not seem to work on the Osemosys pulp code. I assume it is because replacing model.solve(CPLEX_PY()) with prob.solve(CPLEX_PY()) does not work.

Thank you.

Best regards,

Siddharth

On Sunday, April 8, 2018 at 11:44:51 PM UTC+2, Stuart Mitchell wrote:

I haven't used the cplex in a while but first make sure that cplex is installed in python

do this by typing

>>>  import cplex

in the anaconda prompt

if you don't get an error then you can use the `CPLEX_PY` solver by doing the following

>>> prob.solve(CPLEX_PY())

Stu

Stuart Mitchell

PhD Engineering Science

Extraordinary Freelance Programmer and Optimisation Guru

www.stuartmitchell.com

On Mon, Apr 9, 2018 at 5:04 AM, Nicolò Golinucci <nicolo....@gmail.com> wrote:

Hi Stu,

I am using Python for LP modelling. I have used the default solver and I got a solution after several hours of programming. I need to speed up the code and I decided to use CPLEX.
I have installed CPLEX via Anaconda Prompt and I performed a run which was interrupted by an error such as "set too extended". So I thought that the version of CPLEX I have obtained was limited on number of variables (my problem needs 300k variables) and I downloaded an unlimted academic version but I cannot understand if pulp is able to use it. How do I let pulp use solvers I download?
Sorry if the question may seem trivial, but I am quite confused

Thank you in advantage for your time,

Nicolo

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.

[Sam Mathew]

Hi Siddharth,

If OSeMOSYS-PuLP is built on top of PuLP from Stuart, then it should ship with the default CBC solver which can very well solve LP or MILP problems. As the documentation of OSeMOSYS-PuLP says, you have a requirements.txt file that requires you to install PuLP 1.6.8 or greater. Check that out first.

Regards,

Sam

--
New posters to this group are moderated which can take up to 48 hours, so please be patient if your first post takes a while to turn up.
---

You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pulp-or-discuss/e5ff6378-3a9a-4521-bf2f-d0acb102e0c0%40googlegroups.com.

--

Consultant
Data Science and Engineering

IITM Research Park

[Siddharth Shetty]

Hello Sam,

Thank you for your response.

I do have PuLP 2.0 installed. I was recently informed that the CBC solver is MILP solver and not strictly a LP solver. For this thesis, is there anyway to make the CBC solver in solving the model in LP mode strictly? 

Thank you.

Best Regards,
Siddharth Shetty

To view this discussion on the web visit https://groups.google.com/d/msgid/pulp-or-discuss/CAEfK-1dM3igND523CRNsmcgDGu4VC8P1iU16mKC%3D%2BN8zsaZhuA%40mail.gmail.com.

[Sam Mathew]

Dear Siddharth,

LP is a subset of MILP, so CBC of course can solve LP problems as well. In fact, the LP solver CBC uses internally is called Clp. So, you can easily solve this problem with CBC.

I tried the following on a terminal (in GNU/Linux or Windows, with git installed):

git clone https://github.com/OSeMOSYS/OSeMOSYS_PuLP.git

cd OSeMOSYS_PuLP

python OSeMOSYS_PuLP.py

If the basic python code runs, then everything is installed correctly. Unless your project domain doesn't allow you, I would highly suggest you to read up a bit on LP and MILP on wiki before using something so advanced as OSeMOSYS_PuLP.

Regards,

Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/pulp-or-discuss/CAP1-sZDXmgX0JRX9o%2BeuLwEe5XV1RsdV422UaRnMT7B8dSd0eQ%40mail.gmail.com.