PuLP slow adding constraints?

[Jeffrey Milloy]

Hello,

I am trying to use PuLP for a large set of problems. One of my test problems has about 4000 constraints and 67 variables. At first, adding the constraints took 50 seconds, which is surprising. I realized that my constraints matrix is very sparse and adjusted the constraint expressions to only include nonzero coefficients; now adding the constraints takes about 4 seconds.

This still seems slow to me. Setting up the constraints is taking 90% of the runtime (solving the problem is quick). Using PyGLPK, for example, setting up the problem takes about 300 ms. I could generate the .lp file myself in no time.

Is this to be expected with PuLP, or am I doing something wrong that can be easily improved?

Any help would be greatly appreciated.

Jeff

def milp(f, A, b, N):

    m, n = A.shape

    prob = pulp.LpProblem("test", pulp.LpMinimize)

    #variables and bounds

    x = []

    for i in range(n):

        x.append(pulp.LpVariable('x%d' % i, 0, N, pulp.LpInteger))

    #equality constraint

    prob += pulp.lpSum(x[i] for i in range(n)) == N

    # ----- THIS IS THE SLOW LOOP ------ #

    

    #inequality constraints

    for r in range(m):

        prob += pulp.lpSum(A[r][i]*x[i] for i in range(n) if A[r][i]) <= b[r]

    

    # ---------------------------------- #

    # objective function

    prob += pulp.lpSum(f[i]/N * x[i] for i in range(n))

    

    # solve

    status = prob.solve()

    if status == 1:

        return [int(pulp.value(x[i])) for i in range(n)]

    else:

        return None

[Stuart Mitchell]

yeah setting up with pulp is slower then setting up the lp file yourself as there are more helper functions.

Pulp is not really designed to operate on the full 'A' matrix but is intended to help you construct it. 

In this case I would move pulp up the process and use it to construct the A matrix and take advantage of the sparsity there.

otherwise you can use the following construct.

for r in range(m):

        prob += pulp.LpAffineExpression((x[i], A[r][i]) for i in range(n) if A[r][i]) <= b[r]

   

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discu...@googlegroups.com.
To post to this group, send email to pulp-or...@googlegroups.com.
Visit this group at http://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

--

Stuart Mitchell

PhD Engineering Science

Extraordinary Freelance Programmer and Optimisation Guru

www.stuartmitchell.com

[Nathan Petty]

Running into the same problem...  Taking 3 minutes to add only 1500 constraints using:

 prob += lpSum([routeVars[f][t]] for f in list(set(allNodes.intersection(set(s)))) for t in list(allNodes.difference(set( allNodes.intersection(s)))) )>=1

What do you mean "move pulp up the process"?

Really desperate for help on this because I need speedy solves for real world application.

Thanks,

Nathan

[Stuart Mitchell]

I think it is your set operations taking all the time.

your code is looping over the set s.

I think you should try and optimise the building of that list.

http://www.stuartmitchell.com/journal/2012/2/3/my-top-n-tips-for-python-coding-in-optimisation-1.html

Otherwise without more code I don't think we can help, right now I think the code you provided is incorrect as I don't think the `[]` are in the right place.

Stu

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/pulp-or-discuss.

For more options, visit https://groups.google.com/d/optout.

[Nathan Petty]

Stu, thanks a bunch for your reply -  It does not appear that the set operations are actually what's taking so long as I did a test on just the set operations, spitting the results into a table and it took 0.2 seconds.  Then, using those lists, adding constraints to the problem (with no set ops) was still the long step.  See below for example:

#old way takes ~15 seconds

    for s in subtourList:

        prob += lpSum([routeVars[f][t]] for f in list(allNodes.intersection(set(s))) for t in list(allNodes.difference(set( allNodes.intersection(s)))) ) >=1

##-------------------------------------------------------------

#new way (in Two Steps) also takes ~15 seconds

#Step One - do set operations and put results into list (takes 0.2 seconds)

    intersectionList=[]

    differenceList=[]

    for s in subtourList:

        intersectionList.append(list(allNodes.intersection(set(s))))

        differenceList.append(list(allNodes.difference(set( allNodes.intersection(s)))))

#Step Two - add constraints to prob (takes 14 seconds)

    for s in range(len(intersectionList)):

        prob += lpSum([routeVars[f][t]] for f in intersectionList[s] for t in differenceList[s] ) >=1    

Any ideas for speeding up prob+=lpsum?

Visit this group at https://groups.google.com/group/pulp-or-discuss.

For more options, visit https://groups.google.com/d/optout.

[Nathan Petty]

Stu, thanks a bunch for your reply -  It does not appear that the set operations are actually what's taking so long as I did a test on just the set operations, spitting the results into a table and it took 0.2 seconds.  Then, using those lists, adding constraints to the problem (with no set ops) was still the long step.  See below for example:

#old way takes ~15 seconds

    for s in subtourList:

        prob += lpSum([routeVars[f][t]] for f in list(allNodes.intersection(set(s))) for t in list(allNodes.difference(set( allNodes.intersection(s)))) ) >=1

##-------------------------------------------------------------

#new way (in Two Steps) also takes ~15 seconds

#Step One - do set operations and put results into list (takes 0.2 seconds)

    intersectionList=[]

    differenceList=[]

    for s in subtourList:

        intersectionList.append(list(allNodes.intersection(set(s))))

        differenceList.append(list(allNodes.difference(set( allNodes.intersection(s)))))

#Step Two - add constraints to prob (takes 14 seconds)

    for s in range(len(intersectionList)):

        prob += lpSum([routeVars[f][t]] for f in intersectionList[s] for t in differenceList[s] ) >=1    

Any ideas for speeding up prob+=lpsum?

[Stuart Mitchell]

I am still not sure on the placement of your '[]'

to my mind the lpSum line should be

prob += lpSum([routeVars[f][t] for f in intersectionList[s] for t in differenceList[s]] ) >=1

or 

prob += lpSum(routeVars[f][t] for f in intersectionList[s] for t in differenceList[s] ) >=1

without having the rest of your code I can't even tell why it is working as it is.

Stu

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.

Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

[Nathan Petty]

Thanks Stu,  I tried your syntax - it works the same..including the slow run time.  Almost exact same run time, actually

--
You received this message because you are subscribed to a topic in the Google Groups "pulp-or-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.

[Nathan Petty]

Still looking for something, anything to speed up prob += lpSum in PULP.

As demonstrated with code in my previous post, building up the lists
using set operations is instantaneous while adding those lists of
constraints to the problem is exponentially slow. The 14 seconds in
my example was a small problem. For a medium problem this step takes
20 minutes. For a large problem, it could take upwards of an hour.
All that time simply to formulate the problem, let alone solve, is
starting on a very bad foot when it comes to practical implementations
for my client. Yes, the constraints are long and by long I mean most
of the variables from the problem are present in each of these
constraints, BUT on the bright side, there are very few of them - in
the realm of 500 to 5,000 in number. Obviously, the slowness of
prob+=lpsum is being driven by the number of columns/vars in the
constraint.

All that being said, should I look to move out of PULP and into
something like CyLP? or perhaps out of python altogether and into C?
I hope not, I feel like python can handle it?

>> You received this message because you are subscribed to a topic in the
>> Google Groups "pulp-or-discuss" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

[Stuart Mitchell]

Hi you can try just making the lpAffineExpression from scratch instead of using lpSum

like so

class LpAffineExpression(_DICT_TYPE):

    """

    A linear combination of :class:`LpVariables<LpVariable>`.

    Can be initialised with the following:

    #.   e = None: an empty Expression

    #.   e = dict: gives an expression with the values being the coefficients of the keys (order of terms is undetermined)

    #.   e = list or generator of 2-tuples: equivalent to dict.items()

    #.   e = LpElement: an expression of length 1 with the coefficient 1

    #.   e = other: the constant is initialised as e

    Examples:

       >>> f=LpAffineExpression(LpElement('x'))

       >>> f

       1*x + 0

       >>> x_name = ['x_0', 'x_1', 'x_2']

       >>> x = [LpVariable(x_name[i], lowBound = 0, upBound = 10) for i in range(3) ]

       >>> c = LpAffineExpression([ (x[0],1), (x[1],-3), (x[2],4)])

       >>> c

If that doesn't help if you can separately time the creation of the constraints and adding them to the problem.

Unfortunately for dense problems (most Lps are sparse) this line

https://github.com/coin-or/pulp/blob/master/src/pulp/pulp.py#L1312

can cause problems.

In this case you can probably just comment it out.

Stu

>>> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>>> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>>
>> --
>> Stuart Mitchell
>> PhD Engineering Science
>> Extraordinary Freelance Programmer and Optimisation Guru
>> www.stuartmitchell.com
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "pulp-or-discuss" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> pulp-or-discuss+unsubscribe@googlegroups.com.
>> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>

--

You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.

Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

[Nathan Petty]

No luck. prob+=lpsum being broken slow is a gamebreaker for PULP?
because its a critical component of any and all problems. Any plans
to fix?

>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "pulp-or-discuss" group.
>> >> To unsubscribe from this topic, visit
>> >>
>> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> pulp-or-discu...@googlegroups.com.
>> >> To post to this group, send email to pulp-or...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pulp-or-discu...@googlegroups.com.
>> To post to this group, send email to pulp-or...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Stuart Mitchell
> PhD Engineering Science
> Extraordinary Freelance Programmer and Optimisation Guru
> www.stuartmitchell.com
>
> --

> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Clay Campaigne]

This isn't very PuLPish, but if you're already spending the money on Gurobi, as another post of yours indicates, have you considered just trying the Gurobi Python interface? My memory is that the syntax/usage is extremely similar to PuLP. I don't know too much about interface performance, but it may be that the quicksum / multidict data structures in Gurobipy (and PySCIPOpt), or something, make this kind of thing more efficient. I've had experiences with Gurobipy speeding up problem construction by orders of magnitude over some Matlab interface, I forget if it was YALMIP or CVX. 

PuLP and Gurobipy are both lightweight interfaces that implement variables, constraints, and linear / affine expressions, and rely on your Python programming for everything else. Gurobipy also gives you more access to internals, callbacks, warm starts, etc., I think. If your codebase is not very large and complex, it would probably be pretty quick to convert it. 

But if maintaining portability across solvers is important to you, or if your current project requires a free solver, then my suggestion doesn't really apply. 

>> >>> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>> >>> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> >>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >>> For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Stuart Mitchell
>> >> PhD Engineering Science
>> >> Extraordinary Freelance Programmer and Optimisation Guru
>> >> www.stuartmitchell.com
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "pulp-or-discuss" group.
>> >> To unsubscribe from this topic, visit
>> >>
>> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> pulp-or-discuss+unsubscribe@googlegroups.com.
>> >> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Stuart Mitchell
> PhD Engineering Science
> Extraordinary Freelance Programmer and Optimisation Guru
> www.stuartmitchell.com
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> pulp-or-discuss+unsubscribe@googlegroups.com.
> To post to this group, send email to pulp-or-discuss@googlegroups.com.

> Visit this group at https://groups.google.com/group/pulp-or-discuss.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "pulp-or-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pulp-or-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pulp-or-discuss@googlegroups.com.

Visit this group at https://groups.google.com/group/pulp-or-discuss.
For more options, visit https://groups.google.com/d/optout.

--

Clay Campaigne

Cell Phone: (773) 732-9406

[Nathan Petty]

Thanks Clay for the timely response. Yes, I do need the ability to
solve with CBC or Gurobi. Would love to go GurobiPy if that was
possible.

>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "pulp-or-discuss" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >>
>> >> >>
>> >> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> pulp-or-discu...@googlegroups.com.
>> >> >> To post to this group, send email to
>> >> >> pulp-or...@googlegroups.com.
>> >> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> >
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "pulp-or-discuss" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> >> an
>> >> email to pulp-or-discu...@googlegroups.com.
>> >> To post to this group, send email to pulp-or...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> >
>> >
>> > --
>> > Stuart Mitchell
>> > PhD Engineering Science
>> > Extraordinary Freelance Programmer and Optimisation Guru
>> > www.stuartmitchell.com
>> >
>> > --

>> > You received this message because you are subscribed to a topic in the
>> > Google Groups "pulp-or-discuss" group.
>> > To unsubscribe from this topic, visit
>> >
>> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> > To unsubscribe from this group and all its topics, send an email to

>> > pulp-or-discu...@googlegroups.com.
>> > To post to this group, send email to pulp-or...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pulp-or-discu...@googlegroups.com.
>> To post to this group, send email to pulp-or...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --

> Clay Campaigne
> Cell Phone: (773) 732-9406
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Nathan Petty]

Looking for something from Stu or another dev that it's a known issue and makes the use of PULP unusable on anything but trivial problems.  I mean, if adding the constraints to the problem is taking longer than solve, we got a real problem.

>> >> >>> email to pulp-or-discuss+unsubscribe@googlegroups.com.

>> >> >>> To post to this group, send email to

>> >> >>> pulp-or-discuss@googlegroups.com.

>> >> >>> Visit this group at
>> >> >>> https://groups.google.com/group/pulp-or-discuss.
>> >> >>> For more options, visit https://groups.google.com/d/optout.
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Stuart Mitchell
>> >> >> PhD Engineering Science
>> >> >> Extraordinary Freelance Programmer and Optimisation Guru
>> >> >> www.stuartmitchell.com
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "pulp-or-discuss" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >>
>> >> >>
>> >> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> pulp-or-discuss+unsubscribe@googlegroups.com.

>> >> >> To post to this group, send email to

>> >> >> pulp-or-discuss@googlegroups.com.

>> >> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> >
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "pulp-or-discuss" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> >> an

>> >> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>> >> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> >
>> >
>> > --
>> > Stuart Mitchell
>> > PhD Engineering Science
>> > Extraordinary Freelance Programmer and Optimisation Guru
>> > www.stuartmitchell.com
>> >
>> > --
>> > You received this message because you are subscribed to a topic in the
>> > Google Groups "pulp-or-discuss" group.
>> > To unsubscribe from this topic, visit
>> >
>> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>> > To unsubscribe from this group and all its topics, send an email to

>> > pulp-or-discuss+unsubscribe@googlegroups.com.
>> > To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Clay Campaigne
> Cell Phone: (773) 732-9406
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> pulp-or-discuss+unsubscribe@googlegroups.com.
> To post to this group, send email to pulp-or-discuss@googlegroups.com.

[Stuart Mitchell]

Well most optimisation problems are very sparse so your dense matrix is something unusually.

gurobipy has an advantage because it will build its models directly in c whereas pulp has to build an in memory model then build the gurobi model from that.

Saying that the bit of code I identified above does seem like it could be hurting performance

https://github.com/coin-or/pulp/blob/master/src/pulp/pulp.py#L1312

Did you try commenting it out.

At the moment I'm super busy so don't have alot of cycles to devote to this.

I tend to try and do two releases of pulp a year.

Stu

[Nathan Petty]

Stu, Thanks for the timely response. I understand your busy and I'm
tremendously grateful for this tool and your willingness to help.

Commenting that line out of code, as suggested:
prob size, 90k vars, 3k constraints
time to add constraints normally: 120s
with line commented out: 117s

>>> >> >> >> You received this message because you are subscribed to a topic
>>> >> >> >> in
>>> >> >> >> the
>>> >> >> >> Google Groups "pulp-or-discuss" group.
>>> >> >> >> To unsubscribe from this topic, visit
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> >> >> >> To unsubscribe from this group and all its topics, send an email
>>> >> >> >> to

>>> >> > You received this message because you are subscribed to a topic in
>>> >> > the
>>> >> > Google Groups "pulp-or-discuss" group.
>>> >> > To unsubscribe from this topic, visit
>>> >> >
>>> >> >
>>> >> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> >> > To unsubscribe from this group and all its topics, send an email to

>>> >> > pulp-or-discu...@googlegroups.com.
>>> >> > To post to this group, send email to
>>> >> > pulp-or...@googlegroups.com.
>>> >> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> >> > For more options, visit https://groups.google.com/d/optout.
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "pulp-or-discuss" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an
>>> >> email to pulp-or-discu...@googlegroups.com.
>>> >> To post to this group, send email to pulp-or...@googlegroups.com.
>>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> >> For more options, visit https://groups.google.com/d/optout.
>>> >
>>> >
>>> >
>>> >
>>> > --

>>> > Clay Campaigne
>>> > Cell Phone: (773) 732-9406
>>> >
>>> > --
>>> > You received this message because you are subscribed to a topic in the
>>> > Google Groups "pulp-or-discuss" group.
>>> > To unsubscribe from this topic, visit
>>> >
>>> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> > To unsubscribe from this group and all its topics, send an email to

>>> > pulp-or-discu...@googlegroups.com.
>>> > To post to this group, send email to pulp-or...@googlegroups.com.
>>> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pulp-or-discu...@googlegroups.com.
>> To post to this group, send email to pulp-or...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Stuart Mitchell
> PhD Engineering Science
> Extraordinary Freelance Programmer and Optimisation Guru
> www.stuartmitchell.com
>
> --

> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Nathan Petty]

Stu, I commented out that line in source code.  How do I know if its actually taking effect because I got no performance gains.

The package is at C:\Program Files\Python35\Lib\site-packages\pulp

I went in there opened pulp.py and commented out the line and saved it - does that mean its going to use the version I edited during execution?

>>> >> >> >>> email to pulp-or-discuss+unsubscribe@googlegroups.com.

>>> >> >> >>> To post to this group, send email to

>>> >> >> >>> pulp-or-discuss@googlegroups.com.

>>> >> >> >>> Visit this group at
>>> >> >> >>> https://groups.google.com/group/pulp-or-discuss.
>>> >> >> >>> For more options, visit https://groups.google.com/d/optout.
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> Stuart Mitchell
>>> >> >> >> PhD Engineering Science
>>> >> >> >> Extraordinary Freelance Programmer and Optimisation Guru
>>> >> >> >> www.stuartmitchell.com
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> You received this message because you are subscribed to a topic
>>> >> >> >> in
>>> >> >> >> the
>>> >> >> >> Google Groups "pulp-or-discuss" group.
>>> >> >> >> To unsubscribe from this topic, visit
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> >> >> >> To unsubscribe from this group and all its topics, send an email
>>> >> >> >> to

>>> >> >> >> pulp-or-discuss+unsubscribe@googlegroups.com.

>>> >> >> >> To post to this group, send email to

>>> >> >> >> pulp-or-discuss@googlegroups.com.

>>> >> >> >> Visit this group at
>>> >> >> >> https://groups.google.com/group/pulp-or-discuss.
>>> >> >> >> For more options, visit https://groups.google.com/d/optout.
>>> >> >> >
>>> >> >> >
>>> >> >>
>>> >> >> --
>>> >> >> You received this message because you are subscribed to the Google
>>> >> >> Groups
>>> >> >> "pulp-or-discuss" group.
>>> >> >> To unsubscribe from this group and stop receiving emails from it,
>>> >> >> send
>>> >> >> an

>>> >> >> email to pulp-or-discuss+unsubscribe@googlegroups.com.

>>> >> >> To post to this group, send email to

>>> >> >> pulp-or-discuss@googlegroups.com.

>>> >> >> Visit this group at
>>> >> >> https://groups.google.com/group/pulp-or-discuss.
>>> >> >> For more options, visit https://groups.google.com/d/optout.
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Stuart Mitchell
>>> >> > PhD Engineering Science
>>> >> > Extraordinary Freelance Programmer and Optimisation Guru
>>> >> > www.stuartmitchell.com
>>> >> >
>>> >> > --
>>> >> > You received this message because you are subscribed to a topic in
>>> >> > the
>>> >> > Google Groups "pulp-or-discuss" group.
>>> >> > To unsubscribe from this topic, visit
>>> >> >
>>> >> >
>>> >> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> >> > To unsubscribe from this group and all its topics, send an email to

>>> >> > pulp-or-discuss+unsubscribe@googlegroups.com.

>>> >> > To post to this group, send email to

>>> >> > pulp-or-discuss@googlegroups.com.

>>> >> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> >> > For more options, visit https://groups.google.com/d/optout.
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "pulp-or-discuss" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an

>>> >> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>>> >> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> >> For more options, visit https://groups.google.com/d/optout.
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Clay Campaigne
>>> > Cell Phone: (773) 732-9406
>>> >
>>> > --
>>> > You received this message because you are subscribed to a topic in the
>>> > Google Groups "pulp-or-discuss" group.
>>> > To unsubscribe from this topic, visit
>>> >
>>> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> > To unsubscribe from this group and all its topics, send an email to

>>> > pulp-or-discuss+unsubscribe@googlegroups.com.
>>> > To post to this group, send email to pulp-or-discuss@googlegroups.com.

>>> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to pulp-or-discuss+unsubscribe@googlegroups.com.
>> To post to this group, send email to pulp-or-discuss@googlegroups.com.

>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Stuart Mitchell
> PhD Engineering Science
> Extraordinary Freelance Programmer and Optimisation Guru
> www.stuartmitchell.com
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> pulp-or-discuss+unsubscribe@googlegroups.com.
> To post to this group, send email to pulp-or-discuss@googlegroups.com.

[Nathan Petty]

Stu, I confirmed my method for editing pulp source on my machine is correct by inserting a test print statement.  It does indeed import the edited code, so I know that the portion you told me to comment out is indeed commented out.  Unfortunately I commenting that line out did not result in faster constraint loading.  Again as a reminder, my constraints are very long (have many columns/variables) but there are very few of them (just a few thousands).  Using various techniques I have gotten the actual solve time for large problems down significantly to the point where problem initialization (namely the step where i add these particular constraints) is taking longer than the solve itself.  Desperately looking for a way to speed this up.

[Stuart Mitchell]

Could you send me you source files or a small example (privately if you like) and I'll run a profile on it to see why it is slow. Pulp will always be slower than directly using the solver e.g gurobi as it has to build two models the pulp model and the gurobi one. But maybe there is some code in there that is slowing things down.

Stu

[Nathan Petty]

Stu, very much appreciate the offer.  Right now I am reformulating it in column-wise modeling as a necessity of implementing column generation, but I also hope  this method will end up being faster for loading the constraints.  I will update this thread with results.  My worst fear is that column-wise modeling would be somehow slower for loading constraints.  

This is the example I am following:

https://projects.coin-or.org/PuLP/browser/wiki/ColumnGeneration2.wiki

>>> >> >> >> You received this message because you are subscribed to a topic
>>> >> >> >> in
>>> >> >> >> the
>>> >> >> >> Google Groups "pulp-or-discuss" group.
>>> >> >> >> To unsubscribe from this topic, visit
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> >> >> >> To unsubscribe from this group and all its topics, send an email
>>> >> >> >> to

>>> >> > You received this message because you are subscribed to a topic in
>>> >> > the
>>> >> > Google Groups "pulp-or-discuss" group.
>>> >> > To unsubscribe from this topic, visit
>>> >> >
>>> >> >
>>> >> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> >> > To unsubscribe from this group and all its topics, send an email to

>>> >> > pulp-or-discu...@googlegroups.com.
>>> >> > To post to this group, send email to
>>> >> > pulp-or...@googlegroups.com.
>>> >> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> >> > For more options, visit https://groups.google.com/d/optout.
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "pulp-or-discuss" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an
>>> >> email to pulp-or-discu...@googlegroups.com.
>>> >> To post to this group, send email to pulp-or...@googlegroups.com.
>>> >> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> >> For more options, visit https://groups.google.com/d/optout.
>>> >
>>> >
>>> >
>>> >
>>> > --

>>> > Clay Campaigne
>>> > Cell Phone: (773) 732-9406
>>> >
>>> > --
>>> > You received this message because you are subscribed to a topic in the
>>> > Google Groups "pulp-or-discuss" group.
>>> > To unsubscribe from this topic, visit
>>> >
>>> > https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
>>> > To unsubscribe from this group and all its topics, send an email to

>>> > pulp-or-discu...@googlegroups.com.
>>> > To post to this group, send email to pulp-or...@googlegroups.com.
>>> > Visit this group at https://groups.google.com/group/pulp-or-discuss.
>>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pulp-or-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pulp-or-discu...@googlegroups.com.
>> To post to this group, send email to pulp-or...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/pulp-or-discuss.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Stuart Mitchell
> PhD Engineering Science
> Extraordinary Freelance Programmer and Optimisation Guru
> www.stuartmitchell.com
>
> --

> You received this message because you are subscribed to a topic in the
> Google Groups "pulp-or-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pulp-or-discuss/DgI25x_zvL8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> pulp-or-discu...@googlegroups.com.
> To post to this group, send email to pulp-or...@googlegroups.com.
> Visit this group at https://groups.google.com/group/pulp-or-discuss.
> For more options, visit https://groups.google.com/d/optout.