# Optimizing math functions

1 view

### Esmail

May 23, 2009, 9:22:59 AM5/23/09
to pytho...@python.org
Hello all,

I would like to maximize or minimize a given math function over a
specific set of values, in Python preferably.

I was checking out Wolfram Alpha (http://www70.wolframalpha.com/)
and it can do simple optimization problems in math, such as

maximize 15*x - x**2 over 0 to 15

which finds the maximum value and input for x in the range 0 to 15.
Very cool. (This is of course a very simple example).

What it apparently can't do is for maximize (or minimize) functions
that contain two variables, x and y, or more. So for example a simple
example would be

maximize x**2 + y**2 over x:0 to 15, y:0-12 -- this does not work
-- the correct syntax too.

Is there some sort of simple Python module that would allow me to
evaluate this type of function?

In this particular instance I am interested in the minimum of

x * sin(4*x) + 1.1 * sin(2*y), where x,y in range 0-10

though in other problems the range may not be identical for x and y.

Thanks,

Esmail

ps: Does anyone know if Octave or some other free Linux (or Windows)
program might also do this in a simple way? My preference would
still be a Python solution that would be simple to use, ie plug in
the function, the ranges and have it pop out the solution :-)

### Steven D'Aprano

May 23, 2009, 9:55:43 AM5/23/09
to
On Sat, 23 May 2009 09:22:59 -0400, Esmail wrote:

> Hello all,
>
> I would like to maximize or minimize a given math function over a
> specific set of values, in Python preferably.

...

> What it apparently can't do is for maximize (or minimize) functions that
> contain two variables, x and y, or more.

Function minimization is a well-studied problem. You will find oodles of
references to doing function minimization if you google. In fact, if you

Minimizing functions of two variables is difficult, as a general rule.
Nevertheless, there are tools for doing so. Check out SciPy.

--
Steven

### Esmail

May 23, 2009, 10:06:09 AM5/23/09
to pytho...@python.org
Steven D'Aprano wrote:
> On Sat, 23 May 2009 09:22:59 -0400, Esmail wrote:
>
>> Hello all,
>>
>> I would like to maximize or minimize a given math function over a
>> specific set of values, in Python preferably.
> ...
>> What it apparently can't do is for maximize (or minimize) functions that
>> contain two variables, x and y, or more.
>
> Function minimization is a well-studied problem. You will find oodles of
> references to doing function minimization if you google. In fact, if you

Hi Steven,

Right you are (63,700!) ... I don't know what search string I used
before, but clearly not this rather obvious one .. duh.

> Minimizing functions of two variables is difficult, as a general rule.
> Nevertheless, there are tools for doing so. Check out SciPy.

I will - thanks. I have tools for 1D, I mostly am interested in 2D
optimization at this point, mostly as a way to verifying some results
I am calculating.

Hopefully SciPy will provide a nice simple interface to help me do
this.

Thanks,
Esmail

### Robert Kern

May 23, 2009, 5:21:46 PM5/23/09
to pytho...@python.org
On 2009-05-23 08:22, Esmail wrote:
> Hello all,
>
> I would like to maximize or minimize a given math function over a
> specific set of values, in Python preferably.
>
> I was checking out Wolfram Alpha (http://www70.wolframalpha.com/)
> and it can do simple optimization problems in math, such as
>
> maximize 15*x - x**2 over 0 to 15
>
> (http://www70.wolframalpha.com/input/?i=maximize+15*x+-+x**2+over+0+to+15)
>
> which finds the maximum value and input for x in the range 0 to 15.
> Very cool. (This is of course a very simple example).
>
> What it apparently can't do is for maximize (or minimize) functions
> that contain two variables, x and y, or more. So for example a simple
> example would be
>
> maximize x**2 + y**2 over x:0 to 15, y:0-12 -- this does not work
> -- though I'm unclear about
> -- the correct syntax too.
>
> Is there some sort of simple Python module that would allow me to
> evaluate this type of function?
>
> In this particular instance I am interested in the minimum of
>
> x * sin(4*x) + 1.1 * sin(2*y), where x,y in range 0-10
>
> though in other problems the range may not be identical for x and y.

We have several bounded optimization routines in scipy.

http://docs.scipy.org/doc/scipy/reference/optimize.html

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
an underlying truth."
-- Umberto Eco

### Charlie

May 24, 2009, 4:39:20 AM5/24/09
to pytho...@python.org
Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:

>
> On Sat, 23 May 2009 09:22:59 -0400, Esmail wrote:
>
> > Hello all,
> >
> > I would like to maximize or minimize a given math function over a
> > specific set of values, in Python preferably.
> ...
> > What it apparently can't do is for maximize (or minimize) functions that
> > contain two variables, x and y, or more.

You might also look at:
http://pyparasol.sourceforge.net/example_1.html

### Esmail

May 24, 2009, 8:30:13 AM5/24/09
to pytho...@python.org
Charlie wrote:
>
> You might also look at:
> http://pyparasol.sourceforge.net/example_1.html

Thanks for this lead, I had never heard of parasol before. Do you know
if this also works under Linux? The docs mention only the Windows platform,
but given that this is Python perhaps it is save to assume this would also
work under Linux?

Esmail

### Esmail

May 24, 2009, 8:42:59 AM5/24/09
to pytho...@python.org
Robert Kern wrote:
>
>
> We have several bounded optimization routines in scipy.
>
> http://docs.scipy.org/doc/scipy/reference/optimize.html

Hi Robert,

Thanks for the lead .. I briefly looked at the documentation, but
before I dig into this more deeply 2 quick questions:

1. Will is also allow me to maximize a function (I only saw minimum)?

2. Will it give me the global minima/maxima?

I only ask because I got all excited to see that R has an optimize
function but unfortunately it can not deal with multi-modal functions.
I am trying to come up with some complex test cases for a genetic
algorithm I'm working with, both for functions using f(x) and f(x,y) over a
variety of intervals so I would like to have a way to verify results for
more challenging functions.

Thanks a lot,

Esmail

### John Machin

May 24, 2009, 9:52:10 AM5/24/09
to
On May 24, 10:42 pm, Esmail <ebo...@hotmail.com> wrote:
> Robert Kern wrote:
>
> > We have several bounded optimization routines in scipy.
>
> >http://docs.scipy.org/doc/scipy/reference/optimize.html
>
> Hi Robert,
>
> Thanks for the lead .. I briefly looked at the documentation, but
> before I dig into this more deeply 2 quick questions:
>
> 1. Will is also allow me to maximize a function (I only saw minimum)?

To maximise f(x,y), minimise -f(x,y)

HTH,
John

### Esmail

May 24, 2009, 10:11:36 AM5/24/09
to pytho...@python.org
John Machin wrote:
>
>>
>> 1. Will is also allow me to maximize a function (I only saw minimum)?
>
> To maximise f(x,y), minimise -f(x,y)

Ooops .. yes of course!

Thanks,
Esmail

### Lawrence D'Oliveiro

May 25, 2009, 3:32:00 AM5/25/09
to
In message <0033dace\$0\$9725\$c3e...@news.astraweb.com>, Steven D'Aprano
wrote:

> Minimizing functions of two variables is difficult, as a general rule.
> Nevertheless, there are tools for doing so. Check out SciPy.

The name "Marquadt-Levenberg" comes to mind. As I recall, it involved
finding zeroes of the various partial derivatives.

### Charlie

May 25, 2009, 4:48:03 AM5/25/09
to pytho...@python.org

It might work under Linux, however, it was developed under Windows and, to my
knowledge, has never been tested on a Linux machine. Basic operation only
depends on installations of matplotlib, numpy, and scipy. Those packages are
all available on Linux.

If you try it, I'd like to know the outcome.

The parasol options to launch Microsoft Office apps Excel, Power Point, and
Word; or the ray tracing app POV-Ray, will very likely fail.

Charlie

### Esmail

May 25, 2009, 7:23:03 PM5/25/09
to pytho...@python.org
Charlie wrote:
>
> It might work under Linux, however, it was developed under Windows and, to my
> knowledge, has never been tested on a Linux machine. Basic operation only
> depends on installations of matplotlib, numpy, and scipy. Those packages are
> all available on Linux.
>
> If you try it, I'd like to know the outcome.

For sure, if I end up trying it I will post a message about my
results.

> The parasol options to launch Microsoft Office apps Excel, Power Point, and
> Word; or the ray tracing app POV-Ray, will very likely fail.

:-)

thanks again for the information about Parasol,

Esmail

### Robert Kern

May 26, 2009, 11:42:15 AM5/26/09
to pytho...@python.org
On 2009-05-24 07:42, Esmail wrote:
> Robert Kern wrote:
>>
>>
>> We have several bounded optimization routines in scipy.
>>
>> http://docs.scipy.org/doc/scipy/reference/optimize.html
>
> Hi Robert,
>
> Thanks for the lead .. I briefly looked at the documentation, but
> before I dig into this more deeply 2 quick questions:
>
> 1. Will is also allow me to maximize a function (I only saw minimum)?

> 2. Will it give me the global minima/maxima?

No, they are local optimizers.

> I only ask because I got all excited to see that R has an optimize
> function but unfortunately it can not deal with multi-modal functions.
> I am trying to come up with some complex test cases for a genetic
> algorithm I'm working with, both for functions using f(x) and f(x,y) over a
> variety of intervals so I would like to have a way to verify results for
> more challenging functions.

You may want to consider using artificial functions with known optima. Here is a
paper from the ACM Transactions on Mathematical Software doing such:

### Beni Cherniavsky

May 31, 2009, 5:41:14 AM5/31/09
to
On May 23, 4:22 pm, Esmail <ebo...@hotmail.com> wrote:
>
> Is there some sort of simple Python module that would allow me to
> evaluate this type of function?
>
> In this particular instance I am interested in the minimum of
>
>    x * sin(4*x) + 1.1 * sin(2*y), where x,y in range 0-10
>
> though in other problems the range may not be identical for x and y.

Take a look at http://openopt.org (python-scikits-openopt on debian/
ubuntu) - a lot of optimization engines under one interface, so you
can try different engines easily and without learning many interfaces.

>>> import openopt
>>> from numpy import *
# openopt passes parameters as single vector.
>>> def f((x,y)):
return x * sin(4*x) + 1.1 * sin(2*y)
# GLP finds global minimum - can be hard, but let's try.
# I'm not constraining the effort in hope it will converge quickly.
>>> opt = openopt.GLP(f, lb=array([0, 0]), ub=array([10, 10]))
# 'galileo' is one of the supported GLP solvers, included out of the
box.
>>> sol = opt.solve('galileo', plot=True)
-----------------------------------------------------
solver: galileo problem: unnamed
iter objFunVal
0 3.966e+00
10 -6.190e+00
20 -7.613e+00
30 -7.613e+00
35 -7.613e+00
istop: 11 (Non-Success Number > maxNonSuccess = 15)
Solver: Time Elapsed = 0.78 CPU Time Elapsed = 0.24
Plotting: Time Elapsed = 8.04 CPU Time Elapsed = 2.21
objFunValue: -7.6132332 (feasible, max constraint = 0)
### here you need to close the runtime-value graph to continue ###
>>> sol.ff
-7.6132331733254421
>>> sol.xf
array([ 7.3418726 , 5.44153445])

That is x=7.3418726, y=5.44153445 gives f=-7.6132331733254421.
Makes sense?

### Beni Cherniavsky

May 31, 2009, 5:51:18 AM5/31/09
to
On May 31, 12:41 pm, Beni Cherniavsky <beni.cherniav...@gmail.com>
wrote:
>
> Take a look athttp://openopt.org(python-scikits-openopt on debian/

> ubuntu) - a lot of optimization engines under one interface, so you
> can try different engines easily and without learning many interfaces.
>
Correction: don't use the debian package, it uses an old snapshot of
openopt.
Install from svn, it's very easy (checkout, setup.py).