Optimization In Python Pdf

[Chapin Ratte]

Iam giving a small computer exercise that aims at teaching students the basics of a modelling language to model small optimization problems. So far I have been using the modelling language GAMS as this is used in many companies in industry.

To be totally honest, I have never been a big fan of GAMS mainly because I'd prefer to use a general-purpose programming language for optimization instead of a pure modelling language. Now I am thinking about using either Python or Julia for teaching.

The problem is that I really do not know if Julia or Python are used in the industry for optimization. So the question is not if those programming languages are generally used (of course, I know about the Python hype), it is about whether those languages are also used for operations research in industry. In fact on the webpage of Julia several case studies are listed, however I could not find many case studies from industry for Python.

What is your opinion towards this and what experience have you made? If I choose to use Python or Julia, can I tell students that they are (strongly) used in industry to motivate them? Which of those would you choose (or just proceed with GAMS)?

Despite being a great fan of Julia (and JuMP) I must admit that Python is most widely adopted in industry. I won't recommend PuLP however, which tends to be too slow. As alternatives, I would consider

My experience in Artelys, a firm specialized in optimization, is that most people are using Python nowadays, and prefer to stick to this language. We have some prototypes in Julia, but none of them have been industrialized. However, they do provide support for the Julia interface of the solver Knitro (but mostly used by academics so far).

I won't be so definitive as others about Julia, though. JuMP is really a game changer. For non-linear programming, the performance of JuMP AD backend is closed to those of AMPL (between 3x and 5x slower in my experience, which is way better than Pyomo). My bet is that the gap will close in the next years, with the current focus on AD in Julia. That's the reason why I prefer to use Julia for my teachings so far (having built-in linear algebra is gold to me). Also if you choose to use Julia, you could experiment cutting-edge packages developed by the JuliaOpt community. For instance, I do not know any equivalent of Dualization.jl (a package computing automatically the dual of an optimization problem) in other languages.

Those customers like to use Cvxpy or Mosek Fusion to interface the optimizer. You can see some Python notebooks at our Github tutorial page. This portfolio construction framework also provides a good example of what the financial industry are doing with Python and optimization

Staffjoy was an early user of Julia and JuMP for their start up providing workforce scheduling. They also release all of their internal software as open-source after they shut-down. See for example the autoscheduler based on JuMP.

I think that since Python is one of the most popular languages out there, mechanically it is used for optimization. However, I think it also depends on what you mean by optimization. Are you talking about software development, an industrial study, consulting, etc ? I think for software development, Python is often used with other low level languages such as C. For consulting or studies, Python is very appropriate in my experience.

I think Julia is promising, but too young to be compared to Python or any other language in fact. This being said, Atoptima solve their optimization problems with a branch and price framework implemented in Julia. I would not be surprised if in the coming years, Julia becomes more and more popular in the optimization community.

Yes, you should teach your students optimization using Python. For simple models, one simple open-source platform you could introduce is PuLP. It is solver-agnostic, and will work both with commercial solvers as well as open source (including COIN-OR stuff). For more complex stuff, you could teach the gurobipy interface.

Our, KLM, current optimizer products' codebases are all in python. The main reason for this is python is extremely powerful for fast prototyping. However, when it comes to the necessity of implementing more advanced techniques such as column generation and own branch-and-price algorithm, then python start lacking the performance you're looking for. In that case, python is again powerful since that part of your code can then well be implemented in c++ still within your codebase. Last but not least, the immense support with packages from the community makes it extremely handy.

Regarding Julia, I personally started experimenting and it seems quite nice. However, having no community support as much as python is an important fallback. Moreover, most commercial solvers do not have an official API for it. Maybe not a showstopper but certainly an issue to be thoroughly discussed.

We are Optimeering Aqua and our sister company Optimeering use Python and the (Fico) Xpress Python- API. We were alpha and beta users. For us this has been working well. We very early on used Fico's Mosel language, but found moving to a general programming language to have lots of advantages, with few disadvantages. I think there has been many debates on general purpose languages vs domain specific languages, so will not repeat that here.

Interesting that you ask - I've actually seen both julia and python used in industry. On the python side, I'd highly recommend cvxpy (for convex optimization). It was pretty easy to get started with, and it integrates well with other popular python numerical libraries. The stuff I've seen in julia was custom work, so I can't really comment on ease of use.

Python is well ahead of specific modelling languages. Many of solvers such as Gurobi, Cplex etc. have python interface. You can encounter small problems. For example for modelling problems, which package you will teach. You will have alternatives pyomo, pulp, python-mip or solver interface. I prefer pyomo which can be used with lots of commercial or free solvers. Also you can find heuristic and constraint programming packages for python such as google OR tools.

The main reason why Pyomo is being used in industry and JuMP is not is that JuMP is a version 0.21.6 package whereas Pyomo is v5.7.3. Naturally, most businesses are not going to use a package with version less than 1.

Also, according to the following discussion, first-class nonlinear optimization support is something for v2.0 (two or three years away): -dev/JuMP.jl/issues/2355This (and other related links) provide excellent and open discussion that in and of itself is a testament to JuMP's potential.

We have worked in OR for 20 years and have observed the huge rise of Python in the industry over the last 10 years. Many engineers and analysts have moved to Python, especially in scientific fields like data science and operations research.

Until now, the preferred languages of our clients were Java and C# because they were the main stacks used by IT services to build business applications that embed optimization engines. C++ is still used in some companies, particularly software editors, but it has become rare.

The minimize function provides a common interface to unconstrainedand constrained minimization algorithms for multivariate scalar functionsin scipy.optimize. To demonstrate the minimization function, consider theproblem of minimizing the Rosenbrock function of \(N\) variables:

Note that the Rosenbrock function and its derivatives are included inscipy.optimize. The implementations shown in the following sectionsprovide examples of how to define an objective function as well as itsjacobian and hessian functions. Objective functions in scipy.optimizeexpect a numpy array as their first parameter which is to be optimizedand must return a float value. The exact calling signature must bef(x, *args) where x represents a numpy array and argsa tuple of additional arguments supplied to the objective function.

The simplex algorithm is probably the simplest way to minimize a fairlywell-behaved function. It requires only function evaluations and is a goodchoice for simple minimization problems. However, because it does not useany gradient evaluations, it may take longer to find the minimum.

As an alternative to using the args parameter of minimize, simplywrap the objective function in a new function that accepts only x. Thisapproach is also useful when it is necessary to pass additional parameters tothe objective function as keyword arguments.

In order to converge more quickly to the solution, this routine usesthe gradient of the objective function. If the gradient is not givenby the user, then it is estimated using first-differences. TheBroyden-Fletcher-Goldfarb-Shanno (BFGS) method typically requiresfewer function calls than the simplex algorithm even when the gradientmust be estimated.

Here, expensive is called 12 times: six times in the objective function andsix times from the gradient. One way of reducing redundant calculations is tocreate a single function that returns both the objective function and thegradient.

When we call minimize, we specify jac==True to indicate that the providedfunction returns both the objective function and its gradient. Whileconvenient, not all scipy.optimize functions support this feature,and moreover, it is only for sharing calculations between the function and itsgradient, whereas in some problems we will want to share calculations with theHessian (second derivative of the objective function) and constraints. A moregeneral approach is to memoize the expensive parts of the calculation. Insimple situations, this can be accomplished with thefunctools.lru_cache wrapper.

The inverse of the Hessian is evaluated using the conjugate-gradientmethod. An example of employing this method to minimizing theRosenbrock function is given below. To take full advantage of theNewton-CG method, a function which computes the Hessian must beprovided. The Hessian matrix itself does not need to be constructed,only a vector which is the product of the Hessian with an arbitraryvector needs to be available to the minimization routine. As a result,the user can provide either a function to compute the Hessian matrix,or a function to compute the product of the Hessian with an arbitraryvector.

3a8082e126