PulpSolverError cbc.exe

[Hyunju Jeong]

I built MILP to optimize a supply chain but I faced to the following error message.

PulpSolverError: Pulp: Error while trying to execute C:\Python27\ArcGIS10.3\lib\site-packages\pulp-1.6.0-py2.7.egg\pulp\solverdir\cbc\win\32\cbc.exe

I will be very appreciate if even a hint is given.

[Stuart Mitchell]

hmm are you able to run the file `cbc.exe` in the given location?

are you trying to run it from within GIS software?

Stu

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

--

Stuart Mitchell

PhD Engineering Science

Extraordinary Freelance Programmer and Optimisation Guru

www.stuartmitchell.com

[Hyunju Jeong]

Thank you for your response.

When I ran 'cbc.exe', I could see the DOS screen in the below image. At this moment, I run the script in PythonWin and will run it under ArcGIS later. But my data used in the script is from *.shp, which opens in ArcGIS, as you may check in the script.

[Hyunju Jeong]

The problem was solved!!!!!!!!!

My script has a binary variable and the only change that I made was calling the variable from y --> yb.

Even though I have no idea what the relation of the PulpError and the variable change, I could get the result. :)

[Stuart Mitchell]

strange, can't really tell you more without looking at your code.

Stu

--

You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

[Hyunju Jeong]

Dr. Mitchell,

My issue seems to happen according to variable ranges and equation factors used in constraints. I am running the script for three different ranges of one variable among five and, according to the ranges, different constants are used in prob +=. Amazingly, simulation was conducted for two ranges without any errors but it was not for the other. I wonder this observation can be related to the error of cbc.exe.

[Stuart Mitchell]

free free to send me the code (privately if you like) and I can have a look

Stu

On Sat, Jan 9, 2016 at 8:11 AM, Hyunju Jeong <criz...@gmail.com> wrote:

Dr. Mitchell,

My issue seems to happen according to variable ranges and equation factors used in constraints. I am running the script for three different ranges of one variable among five and, according to the ranges, different constants are used in prob +=. Amazingly, simulation was conducted for two ranges without any errors but it was not for the other. I wonder this observation can be related to the error of cbc.exe.

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

[Patrick Moore]

Hi Stu,

I am getting this same error.

I am using Pulp to solve geometry patches which are formulated as ILP problems in a user interactive modelling addon for the 3d modeling software Blender.  I have adapted this paper http://igl.ethz.ch/projects/patch-quad/pattern-based-quadrangulation.pdf

here a rough video of some initial testing https://www.youtube.com/watch?v=6t2KS604SWo

here is my branch for source code.  I can direct you to certain areas later.   https://github.com/CGCookie/retopoflow/tree/patricks_patches

Basically, each geometry patch boundary is permuted and tested against a set of predefined patterns.  The pattern defines the ILP constraints.  I'm attempting to do this in real time, but occasionally, I get the "Pulp Error while trying to execute"  as listed above.  For example, on a 6 sided patch, there are 12 permutations (rotation and mirroring of the sides) tested against 4 patterns for 48 individual ILP problems.  However they are very small in terms of #of variables and constraints compared to what others are posting.

I get the error when cbc.wait() != 0:

https://github.com/coin-or/pulp/blob/081d1e02355945b3bfd83ea68dcb380c238886c0/src/pulp/solvers.py#L1362

Most often.  cbc.wait() = 3, I have no idea what this error code means.

it seems that Popen.wait() can deadlock the subprocess. https://docs.python.org/2/library/subprocess.html  maybe this is what is happening?  It's just not ready to be asked to wait?

most of the ILP models take .03 seconds to solve

If i add a sleep(.05) before line 1362 in solvers.py, it successfully solves all 48 models without error. 

I can do some things on the interactive side of my software to schedule the ILP solvers to run in background, accumulating the set of feasible solutions while the program remains interactive, however I can't seem to figure out how to test if the cbc.exe is solver is busy so that I can give it more time.  I don't want to artificially add sleep timers because performance will be different on many systems.

Any tips!?

Best,

Patrick Moore

[Stuart Mitchell]

Hmm, I'll have to think about it are you running pulp with multithreading processing? 

Are you able to generate a minimal program that generates this error?

This will help me alot when I try and fix it.

Stu

[Patrick Moore]

1) I am not using multithreading. This app is cross platform (osx, windows and linux) so plain vanilla CBC is all we need.

2)I will provide code example, but its basically the same as demo code on the wiki. The trouble is the same code can complete successfully in some situations. It seems to be related to performance. I do have a lot of instances of the LpProblem, could that cause problems?

3)the crash is always on the same spot

[Patrick Moore]

Here is a code example

http://www.pasteall.org/63558

My tests have been on Windows 7, 64bit and on OSX

Windows Tests: 

Tested with CBC version 2.9.0 built 2/12/2015 and with version 2.8 built 9/2014.  Both fail in same manner.

CBC.exe has quit working  (this error pops up in it's own window, separate from python)

then in the python console "Pulp: Error while trying to execute"

If I print the value of cbc.wait() as mentioned above, sometimes it's 3, other times it is 255

OSX Tests:  The same script works fine on OSX

[Patrick Moore]

Actually, I have reproduced the same error on OSX-10.11.2 I tried running the script while the computer was working on something else (screen recording) and it crashed with the same error message.  Before, I was thinking this was a windows CBC.exe problem.  Now I am unsure if this is caused by my constraints, something in the way PuLp is communicating with CBC, or something within CBC.

Any steps you could recommend to help me isolate what the root cause is?

Best,

Patrick

[Stuart Mitchell]

try the msg=1 solver param

which should print out the output from cbc

also keepFiles=1 will keep the mps files around

Stu

Sorry I have got to this yet

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

[Patrick Moore]

No worries Stu,  everyone is busy!  However I think I am getting to the root of the problem.  Which is wild, because a few months ago, I started with "What is ILP?" in Google.

0) I used the LpDefaultSolver.msg and LpDefaultSolver.keepFiles params.  That didn't help me, except I could see what command string was being put into the CBC.exe

1)  I changed all my constraints to LpAffineExpressions to rule out any of that kind of problem because I initially thought this was performance based.

2) I have identified a set of criteria which crashes the first time, every time.  I think this rules out a performance issue, and that there are some fundamental issues with my problem, PuLp or CBC.exe

3.  I started messing with some parameters (blindly, with no knowledge of their implications for the validity/appropriateness of such changes).  Results as follows

   a)  turn off use_mps  =>> no crash

   b)  turn on in initial_solve  =>> no crash

   c)  turn on presolve  ==> Crash

4.  After that, I started reading/learning about those params.

    a)  mps is a file type, so this could be related to lp.writeMPS

    b)  initialSolve seems to only be appropriate if your problem is not MIP.  My problem is definitely Integer in nature, so I think I just got lucky with initialSolve.

    c)  presolve On and MPS OFF ==> no crash

All of this leads me to believe that there is a problem with lp.writeMPS with some very specific criteria

Here, is the minimal program that can reproduce a crash on my system every time

http://www.pasteall.org/63740/python

Here is the associated MPS file

https://www.dropbox.com/s/75t0zbxayachlvx/SixSidedPatch-pulp.mps?dl=0

Here is the associated Lp file

https://www.dropbox.com/s/mj0o0iofs5x1rho/SixSidedPatch-pulp.lp?dl=0

Here is the sol file, that I get when I don't use MPS

https://www.dropbox.com/s/xzjubxvqfgpmuiz/SixSidedPatch-pulp.sol?dl=0

I will confirm/refute the same problem on Mac ASAP and report here.

[Francesca Olivetta]

Sorry to bother you but I am facing a similar problem.

How didi you manage to do the following things:

   a)  turn off use_mps  =>> no crash

   b)  turn on in initial_solve  =>> no crash

?

Thank you in advance

Francesca

[Patrick Moore]

Hi Francesca,

It's been so long since I was deep in this project that I'm not sure!  What I do rememeber is that there are two file types that can be written out to pass to the solver.  And when I used one file type (mps I think) I got more crashes but when I switched the crashes stopped I was getting the results I needed and then kept on moving without spending any more time to dig in.  I'll see if I can dig up my code.

But, be encouraged, PuLP worked awesome for my project.

-P

--
New posters to this group are moderated which can take up to 48 hours, so please be patient if your first post takes a while to turn up.
---
You received this message because you are subscribed to a topic in the Google Groups "pulp-or-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pulp-or-discuss/x1bN9jQ8btQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pulp-or-discu...@googlegroups.com.

[Patrick Moore]

https://github.com/CGCookie/retopoflow/blob/patricks_patches/op_edgepatches/patch_math.py 

ok, here is the file that the magic happens, I need to read through it to find where the actual PuLP stuff is happening 

[Patrick Moore]

ok, yeas read the PatchSolver classes

-P