22 views

Skip to first unread message

Apr 14, 2009, 9:03:58 AM4/14/09

to

Hello,

Is there a way to easily build an object that behaves exactly like a

float, but whose value can be changed? The goal is to maintain a list

[x, y,…] of these float-like objects, and to modify their value on the

fly (with something like x.value = 3.14) so that any expression like "x

+y" uses the new value.

I thought of two solutions, both of which I can't make to work:

1) Use a class that inherits from float. This takes care of the

"behave like float" part. But is it possible to change the value of

the float associated with an instance? That is, is it possible to

do: "x = MyFloat(1.23); x.change_value(3.14)" so that x's float value

becomes 3.14?

2) The other possibility I thought of was: use a class that defines a

'value' member (x.value). This takes care of the "value can be

changed" part. But is it possible/easy to make it fully behave like a

float (including when passed to functions like math.sin)?

Alternatively, I'd be happy with a way of handling numerical

uncertainties in Python calculations (such as in "calculate the value

and uncertainty of a*sin(b) knowing that a=3.0 +/- 0.1 and b=1.00 +/-

0.01").

Any idea would be much appreciated!

Apr 14, 2009, 11:14:18 AM4/14/09

to

It looks like what is needed here are a kind of "mutable float". Is

there a simple way of creating such a type? I don't mind changing the

value through x.value = 1.23 instead of x = 1.23... :)

there a simple way of creating such a type? I don't mind changing the

value through x.value = 1.23 instead of x = 1.23... :)

Apr 14, 2009, 11:31:54 AM4/14/09

to

I think you'll have to describe your use case a little better. I don't

see why you'd need a mutable float. As long as the reference x is

visible to the other parts of your code, when that code uses x, it'll

always get the right instance of a float object.

--David

Apr 14, 2009, 11:34:17 AM4/14/09

to Eric.L...@spectro.jussieu.fr, pytho...@python.org

On Tue, Apr 14, 2009 at 9:03 AM, <Eric.L...@spectro.jussieu.fr> wrote:

> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed? The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

>

It's not clear what your requirement is.> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed? The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

>

Why can't you just use floats?

For your list, why can't you use a list of floats?

Something is missing from your explanation.

Apr 14, 2009, 11:52:01 AM4/14/09

to Eric.L...@spectro.jussieu.fr, pytho...@python.org

Eric.L...@spectro.jussieu.fr wrote:

> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed? The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed? The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

Your approach doesn't follow the Python philosophy thus it's most likely

to fail. It is possible to implement a mutable float like object --

Python doesn't stop you from shooting yourself in the knee -- but please

don't harm yourself.

Python has an easy way to archive an equal goal:

data = [1.0, 2.0, -1.0, -7.0, 23.0]

for idx, value in enumerate(data):

if value < 0.0:

data[idx] = value**2

Important side note:

It's ok to modify elements of a list while iterating over the same list.

But you should not remove, append, insert or reorder the list while you

are iterating over it. It will lead to surprising effects.

Christian

Apr 14, 2009, 12:19:25 PM4/14/09

to pytho...@python.org

Christian Heimes wrote:

> Eric.L...@spectro.jussieu.fr wrote:

>> Hello,

>>

>> Is there a way to easily build an object that behaves exactly like a

>> float, but whose value can be changed? The goal is to maintain a list

>> [x, y,…] of these float-like objects, and to modify their value on the

>> fly (with something like x.value = 3.14) so that any expression like "x

>> +y" uses the new value.

>

>

The saying is actually "shooting yourself in the foot", but then that's

like what happens when you don't follow the Python philosophy! :-)

> Eric.L...@spectro.jussieu.fr wrote:

>> Hello,

>>

>> Is there a way to easily build an object that behaves exactly like a

>> float, but whose value can be changed? The goal is to maintain a list

>> [x, y,…] of these float-like objects, and to modify their value on the

>> fly (with something like x.value = 3.14) so that any expression like "x

>> +y" uses the new value.

>

>

> Your approach doesn't follow the Python philosophy thus it's most likely

> to fail. It is possible to implement a mutable float like object --

> Python doesn't stop you from shooting yourself in the knee -- but please

> don't harm yourself.

>

[snip]> to fail. It is possible to implement a mutable float like object --

> Python doesn't stop you from shooting yourself in the knee -- but please

> don't harm yourself.

>

The saying is actually "shooting yourself in the foot", but then that's

like what happens when you don't follow the Python philosophy! :-)

Apr 14, 2009, 12:26:16 PM4/14/09

to

Eric.L...@spectro.jussieu.fr wrote:

> Alternatively, I'd be happy with a way of handling numerical

> uncertainties in Python calculations (such as in "calculate the value

> and uncertainty of a*sin(b) knowing that a=3.0 +/- 0.1 and b=1.00 +/-

> 0.01").

Naive no warranties implementation:

from math import sin, sqrt

class Value(object):

def __init__(self, value, err):

self.value = value

self.err = err

def __str__(self):

return "%s +- %s" % (self.value, self.err)

def derive(f, values, i, eps=1e-5):

x1 = f(*values)

values = list(values)

values[i] += eps

x2 = f(*values)

return (x2-x1)/eps

def calc(f, *args):

values = [v.value for v in args]

errs = [v.err for v in args]

sigma = 0

for i, (v, e) in enumerate(zip(values, errs)):

x = derive(f, values, i)*e

sigma += x*x

return Value(f(*values), sqrt(sigma))

a = Value(3.0, 0.1)

b = Value(1.0, 0.01)

def f(x, y):

return x * sin(y)

print "a = %s" % a

print "b = %s" % b

print "c = %s" % calc(f, a, b)

Peter

Apr 14, 2009, 2:45:47 PM4/14/09

to pythonlist

Eric.L...@spectro.jussieu.fr wrote:

> It looks like what is needed here are a kind of "mutable float". Is

> there a simple way of creating such a type? I don't mind changing the

> value through x.value =.23 instead of x = 1.23... :)> It looks like what is needed here are a kind of "mutable float". Is

> there a simple way of creating such a type? I don't mind changing the

>

> On Apr 14, 3:03 pm, Eric.Le.Bi...@spectro.jussieu.fr wrote:

>

>> Hello,

>>

>> Is there a way to easily build an object that behaves exactly like a

>> float, but whose value can be changed? The goal is to maintain a list

>> [x, y,…] of these float-like objects, and to modify their value on the

>> +y" uses the new value.

>>

>> I thought of two solutions, both of which I can't make to work:

>>

>> 1) Use a class that inherits from float. This takes care of the

>> "behave like float" part. But is it possible to change the value of

>> the float associated with an instance? That is, is it possible to

>> becomes 3.14?

>>

>> 2) The other possibility I thought of was: use a class that defines a

>> 'value' member (x.value). This takes care of the "value can be

>> changed" part. But is it possible/easy to make it fully behave like a

>> float (including when passed to functions like math.sin)?

>>

>> Alternatively, I'd be happy with a way of handling numerical

>> uncertainties in Python calculations (such as in "calculate the value

>> 0.01").

>>

>> Any idea would be much appreciated!

>>

The answer to your original question is no. If the value can be changed, then it doesn't behave like a float. And that's not just a pedantic answer, it's a serious consideration.

You have to decide what characteristics of a float you need to mimic, and which ones you don't care about, and which ones you want to change. Only after having a pretty good handle on those answers can you pick a "best" implementation.

Let's call this new type a nfloat, and let's assume you have a function that returns one. That might be a constructor, but it may not, so we're keeping our options open.

myval = newfunction(42.0)

What do you want to happen when you execute b = myval ? Presumably

you want them to be "equal" but in what sense? Suppose you then change

one of them with your suggested attribute/method.

myval.value = newfunction("aaa")

is b the same as it was (like a float would be), or is b also changed?

Do you need lots of functions to work on one of these nfloats, or could

you use them as follows:

sin( b.value )

Instead of using .value to change the underlying float, how about if you

use [0] ? Just use a list of size 1.

Apr 14, 2009, 3:36:02 PM4/14/09

to

Thank you all for your input. It is not yet obvious how to achieve

the goal/need that I had in mind in the original post. Basically, I

would need to be able to calculate the derive() function of Peter, but

without knowing what arguments are passed to the function f under

study. Here is why:

the goal/need that I had in mind in the original post. Basically, I

would need to be able to calculate the derive() function of Peter, but

without knowing what arguments are passed to the function f under

study. Here is why:

I'll give more details, as David S. and David R. were asking for. The

code could look like this:

import crystals

my_crystal = crystals.Crystal("Quartz 111")

which would set some attributes of my_crystal as "floats with

uncertainty" FloatWithUncert, which behave exactly like floats in

calculations (they return the central value of the confidence

interval: x returns the value, as for floats, while, x.uncert returns

the uncertainty). Now, I'd like to perform a calculation of some

physical quantity associated to the crystal:

print my_crystal.lattice_spacing(temperature = 273.15)

setup = Experiment(my_crystal, my_mirror); print setup.bragg_angle

() # An Experiment object also defines and uses FloatWithUncert

objects

Everything is fine up to now (I have a FloatWithUncert class which

inherits from float).

Now, I would like to get the uncertainty on the result, even though we

have no idea of what quantities are used in lattice_spacing() for the

calculation (it can be attribute that are floats, attributes that are

FloatWithUncert, module globals defined as FloatWithUncert, etc.).

The idea that prompted my initial post was as follows: perform the

same calculation of lattice_spacing() many times, but each time change

on of existing FloatWithUncert numbers (this is akin to the derive()

function of Peter) and deduce the uncertainty on lattice_spacing() (as

with the calc() function of Peter). So I thought that the

FloatWithUncert class could keep a list createdNumbers of all created

FloatWithUncert numbers, be instructed to change the "float" value of

the n-th float to "central value + uncertainty", and the calculation

would then be performed again, but with a single updated number (as in

calc() above):

FloatWithUncert.shift_number(n = 3) # The 3rd FloatWithUncert ever

created will return "central value + uncertainty"; others return the

central value

print my_crystal.lattice_spacing(temperature = 273.15) # This

should give a new result

The original post was essentially asking: is it possible to write

shift_number() in Python? i.e., we have objects 'x' of type

FloatWithUncert, which return a single float value when used in

expressions such as 'x+1', which can be tracked in a list

FloatWithUncert.createdNumbers (created by FloatWithUncert), and

modified later (FloatWithUncert.createdNumbers[3].value = ...). In

other words, as I was saying in my second post, a kind of mutable

float would effectively be needed.

Now, to respond to David S., David R. and Christian, it's not possible

to use a simple list of floats [x, y,...] because this would not help

making the result of "x+y" change when you change one of the floats

_through the list_ (again, I have no other information on what

variables, globals, instance attributes, etc. are used in the

calculation whose uncertainty is being calculated). I would not like

to write my numerous mathematical expressions as list_of_floats

[0]+list_of_floats[1]*sin(...), etc. (This would be illegible, and

would not be robust.)

Peter's solution is nice when you call functions with explicit

arguments. But my class methods perform calculations through instance

attributes, globals, etc., and I don't want to modify all my

calculation code in order to implement error propagation.

I hope that the problem is clearer, now. :)

A couple of ideas I had:

1) Define a FloatWithUncert object, but get instance values as x(), as

in "x()+y()". The code is relatively legible. 'x' is mutable. But

formulas don't look so good, and you can't drop a float replacement

for 'x', as floats are not callable.

2) Write all expressions that could contain FloatWithUncert objects

with a 'float()' wrapper ("float(x)+float(y)"), after defining the

FloatWithUncert.__float__() method. FloatWithUncert would be

mutable. The code is a little bit heavy, but it is explicit. 'x'

could be a pure float.

Sorry for this long post. Any thought/idea/remark would be most

welcome!

Apr 14, 2009, 3:48:19 PM4/14/09

to

Thanks Dave for your thoughtful remarks, which you sent right when I

was writing a response to the previous posts.

was writing a response to the previous posts.

I was wondering about a kind "mutable float"; so you're right, it's

not fully a float, because it's mutable. I'd like to have an object

that behaves like a float in numerical calculations. I understand

that mutability would require to handle such objects with care, as in

your example, but Python programmers are used to this with any mutable

object. All my calculations use "constants with an uncertainty" (or

regular floats).

There are many such calculations in my code, and I'd like it to be

very clean. The first idea I mentioned (using "x()") is essentially

what you propose with using "x[0]" in calculations.

So, it looks like it's not possible to have float-like objects

(calculation/formula-wise) that are mutable?! and it also looks like

the price to pay for the mutability is to have to write "heavier"

versions of any formula that uses these "special floats" (that carry

an uncertainty): "x[0]+y[0]*sin(...)", "x()+y()", "float(x)+float

(y)",... which, by the way, essentially prevents any float-like

object from being used as a float in formulas that you don't write

yourself.

This does not look good. Python is failing me!!! :/ I heard that

this would be easy to do in Ruby (not confirmed, though)...

More ideas and thoughts would still be most welcome!

On Apr 14, 8:45 pm, Dave Angel <da...@ieee.org> wrote:

Apr 14, 2009, 9:41:15 PM4/14/09

to

On Tue, 14 Apr 2009 14:45:47 -0400, Dave Angel wrote:

> The answer to your original question is no. If the value can be

> changed, then it doesn't behave like a float. And that's not just a

> pedantic answer, it's a serious consideration.

Oh nonsense. Many programming languages have mutable floats.

--

Steven

Apr 14, 2009, 11:35:14 PM4/14/09

to

On Tue, 14 Apr 2009 06:03:58 -0700, Eric.Le.Bigot wrote:

> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed?

Yes, have a look at the source code for UserString.MutableString for some

ideas.

> The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

Why is that the goal? If you want to change the value of x, just change

the value of x.

--

Steven

Apr 15, 2009, 12:17:40 AM4/15/09

to

As you're aware (but Eric may not be), Python doesn't have “change the

value of x”. The closest would be to re-bind the name ‘x’ to a different

value, which would not be what Eric is asking for.

If I understand correctly, Eric wants something with the following

behaviour:

>>> foo = [floatref(3.14), floatref(1.41)]

>>> bar = foo[0]

>>> baz = foo[1]

>>> foo

[3.14, 1.41]

>>> (bar, baz)

(3.14, 1.41)

>>> foo[1].changevalue(1.62)

>>> foo

[3.14, 1.62]

>>> (bar, baz)

(3.14, 1.62)

and is asking how to get such a ‘floatref’.

--

\ “If you go parachuting, and your parachute doesn't open, and |

`\ you friends are all watching you fall, I think a funny gag |

_o__) would be to pretend you were swimming.” —Jack Handey |

Ben Finney

Apr 15, 2009, 1:13:51 AM4/15/09

to

On Tue, 14 Apr 2009 12:36:02 -0700, Eric.Le.Bigot wrote:

> I'll give more details, as David S. and David R. were asking for. The

> code could look like this:

>

> import crystals

> my_crystal = crystals.Crystal("Quartz 111")

>

> which would set some attributes of my_crystal as "floats with

> uncertainty" FloatWithUncert, which behave exactly like floats in

> calculations (they return the central value of the confidence interval:

> x returns the value, as for floats, while, x.uncert returns the

> uncertainty).

So far so good -- this is just straight-forward interval arithmetic.

Nothing about this requires mutability.

> Now, I'd like to perform a calculation of some physical

> quantity associated to the crystal:

>

> print my_crystal.lattice_spacing(temperature = 273.15) setup =

> Experiment(my_crystal, my_mirror); print setup.bragg_angle

> () # An Experiment object also defines and uses FloatWithUncert objects

>

> Everything is fine up to now (I have a FloatWithUncert class which

> inherits from float).

Your programming model is not clear. Does the Experiment class represent

the things you do to perform the experimental procedure? Or does it

represent the result(s) you get after you perform the experimental

procedure?

> Now, I would like to get the uncertainty on the result,

What's "the result"? Is setup "the result"? Or setup.bragg_angle? Or

something else?

> even though we

> have no idea of what quantities are used in lattice_spacing() for the

> calculation (it can be attribute that are floats, attributes that are

> FloatWithUncert, module globals defined as FloatWithUncert, etc.).

I'm going to assume that you have a valid way of combining floats with

FloatWithUncert that doesn't invalidate your error estimates.

Still there's nothing here that requires mutability.

> The

> idea that prompted my initial post was as follows: perform the same

> calculation of lattice_spacing() many times, but each time change on of

> existing FloatWithUncert numbers (this is akin to the derive() function

> of Peter) and deduce the uncertainty on lattice_spacing()

I think you are asking for this behaviour:

>>> my_crystal.magic = 1 # hidden input as an attribute (say)

>>> x = my_crystal.lattice_spacing(temperature=290)

>>> print x, x.uncert

(1.234, 0.001)

>>> my_crystal.magic = 2 # change an attribute

>>> print x, x.uncert # and magic happens

(4.567, 0.003)

If so, that is bad and evil. You might think it is a good idea now, but

trust me, it will lead to tears.

Do this instead:

>>> my_crystal.magic = 1 # hidden input as an attribute (say)

>>> x = my_crystal.lattice_spacing(temperature=290)

>>> print x, x.uncert

(1.234, 0.001)

>>> my_crystal.magic = 2 # change an attribute

>>> y = my_crystal.lattice_spacing(temperature=290)

>>> print x, x.uncert # stability is a good thing

(1.234, 0.001)

>>> print y, t.uncert

(4.567, 0.003)

You might think mutable values changing by magic is a good thing now, but

trust me, it isn't. Let me put it this way... if you were doing a real

experiment, and you had a note pad and *wrote down* the lattice spacing

after an experiment, and then a week later somebody came into the lab and

changed some component in the experimental setup, and the numbers in your

notepad changed, that would be a disaster. What you're suggesting (or at

least what I think you're suggesting) is the same thing, only not quite

as extreme.

--

Steven

Apr 15, 2009, 3:51:52 AM4/15/09

to

Eric.L...@spectro.jussieu.fr wrote:

> Now, I would like to get the uncertainty on the result, even though we

> have no idea of what quantities are used in lattice_spacing() for the

> calculation (it can be attribute that are floats, attributes that are

> FloatWithUncert, module globals defined as FloatWithUncert, etc.).

I think this is your main problem. What you describe is a source of hard to

find bugs and a maintenance nightmare.

I also don't understand what you want to do with the results. Consider a

simple example

a = 1 +- 0.1

b = 1 +- 0.1

Now let's calculate a + b. With my approach you get

a + b = 2 +- 0.14

i. e. the result falls in the interval 1.86...2.14 with the same likelihood

that a and b are within 0.9...1.1. You don't get 2 +- 0.2 because the

errors sometimes cancel out.

With your approach you get (some permutation of)

a + b = [1.9, 1.9, 2.0, 2.0, 2.1, 2.1]

What is that supposed to mean? Let's assume you have one additional variable

c = 42 +- 7 # whatever

Your result will become

a + b = [1.9, 1.9, 2.0, 2.0, 2.0, 2.0, 2.0, 2.1, 2.1]

Hmm...

Here's the code I used to get that result:

class F(object):

all = []

def __init__(self, value, err):

self.value = value

self.err = err

self.shift = 0

self.all.append(self)

def __float__(self):

return self.value + self.err * self.shift

def __add__(self, other):

return float(self) + float(other)

__radd__ = __add__

def __mul__(self, other):

return float(self) * float(other)

__rmul__ = __mul__

def __str__(self):

return "F(%s)" % float(self)

a = F(1.0, 0.1)

b = F(1.0, 0.1)

unused = F(42.0, 7.0)

results = []

for f in F.all:

for f.shift in [-1, 0, 1]:

print "a = %s, b = %s, a+b = %s" % (a, b, a + b)

results.append(a+b)

f.shift = 0

print "[%s]" % ", ".join("%.1f" % r for r in sorted(results))

If you add enough arithmetic operations it might even "work" with your

codebase.

Peter

Apr 15, 2009, 4:05:33 AM4/15/09

to

Ben F., you're right on the money! You expressed exactly what I'm

looking for. Why should I want this? because the place in the code

where (foo, baz) is calculated has _no idea_ of what foo and baz are,

of where they were defined, etc.; on the other hand, the floatref

class can keep track of the values that were constructed, and I'm

looking for a way of modifying them from its perspective

(floatref.change_value(index, new_value)).

looking for. Why should I want this? because the place in the code

where (foo, baz) is calculated has _no idea_ of what foo and baz are,

of where they were defined, etc.; on the other hand, the floatref

class can keep track of the values that were constructed, and I'm

looking for a way of modifying them from its perspective

(floatref.change_value(index, new_value)).

Steven, t

To Steven: Thank you for your suggestion about checking

UserString.MutableString.

I do understand that mutable objects can be surprising, but, again,

Python programmers constantly use such objects (lists, dicts,....): I

don't think that manipulating a mutable float is more of a problem

than using a list, for a Python programmer.

As for your idea of "straight-forward interval arithmetic", it's a

good one, but I'd have to redefine lots of functions from the math

module, to use them explicitly in my code, etc.: this is heavy; I was

looking for a light-weight alternative, where all calculations are

expressed in Python as if all numbers were regular floats, and where

you don't have to redefine any mathematical operation. In other

words, I'm looking for a robust way of implementing the derive()

function of Peter without knowing anything about which function uses

which "numbers with an uncertainty": the equivalent of Peter's derive

() would simply successively change all "numbers with uncertainty"

defined so far and see how the result of a given calculation varies--

the variables that are not used in the calculated expression don't

change the result, for instance. I understand that this is

computationally inefficient (too many variables might be changed), but

I don't need speed, only robustness in the propagation of errors, and

a simple and very legible calculation code, and ideally with only a

few lines of Python.

Anyway, Steven gave me hope that some kind of mutable float is

possible in Python, and Ben perfectly expressed my need. Again,

alternatively, any module that handles in a convenient manner

calculations with error propagation would be great, but I have not

found any single such module!

Apr 15, 2009, 4:44:07 AM4/15/09

to pytho...@python.org

Eric.L...@spectro.jussieu.fr wrote:

> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed? The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

> Hello,

>

> Is there a way to easily build an object that behaves exactly like a

> float, but whose value can be changed? The goal is to maintain a list

> [x, y,…] of these float-like objects, and to modify their value on the

> fly (with something like x.value = 3.14) so that any expression like "x

> +y" uses the new value.

Hi Eric,

Numpy's array object can do something like what you want:

In [27]: x=array(0.0)

In [28]: print x, sin(x)

0.0 0.0

In [29]: x.itemset(pi/2)

In [30]: print x, sin(x)

1.57079632679 1.0

Not sure if this a recommended way of using array or not, but it seems

to work. The problem is that any calculation you do with such an object

will result in a float, and not another numpy array (although inplace

operations work):

In [40]: print (x*2).__class__

<type 'numpy.float64'>

In [41]: x *= 2

In [42]: print x.__class__

<type 'numpy.ndarray'>

So it's not a perfect solution, but it might be OK for what you need.

Dan

Apr 15, 2009, 4:53:03 AM4/15/09

to

To Peter: What I had in mind was to implement your calc() function;

you could do something similar with your loop in the previous post by

replacing "for f.shift..." by "f.shift = 1"; this would give you 2

values, which you can combine with your unused variable in order to

obtain the same value as with your calc() function.

you could do something similar with your loop in the previous post by

replacing "for f.shift..." by "f.shift = 1"; this would give you 2

values, which you can combine with your unused variable in order to

obtain the same value as with your calc() function.

To Peter and Steven: I now realize that the interval arithmetic that

you propose is not trivial, if one wants optimal confidence intervals:

correlations between uncertainties are very hard to handle. The most

robust method is to implement Peter's calc() method. But again, when

calculations are not done through explicit argument passing, as in

Peter's example, mutable floats are useful!

Apr 15, 2009, 4:59:21 AM4/15/09

to

On Wed, 15 Apr 2009 01:05:33 -0700, Eric.Le.Bigot wrote:

> Ben F., you're right on the money! You expressed exactly what I'm

> looking for. Why should I want this? because the place in the code

> where (foo, baz) is calculated has _no idea_ of what foo and baz are, of

> where they were defined, etc.;

This makes no sense. The piece of code which calculates (foo, baz) knows

EXACTLY what foo and baz are, because it has just calculated them. It has

them *right there*. If it wants to know what they are, it can just look

at them:

def calculate():

# lots of calculations

result = (foo, baz)

# what is foo?

foo

# what is baz?

baz

# what are their types?

type(foo)

type(baz)

As for where they were defined... why do you care where they were

defined? What's important is *what they are*, and you know what they are,

because you have them, right there.

> on the other hand, the floatref class can

> keep track of the values that were constructed,

What advantage does that give you? I can think of big disadvantages: it

is complicated and inefficient and will lead to bugs in your code.

[...]

> To Steven: Thank you for your suggestion about checking

> UserString.MutableString.

> I do understand that mutable objects can be surprising, but, again,

> Python programmers constantly use such objects (lists, dicts,....): I

> don't think that manipulating a mutable float is more of a problem than

> using a list, for a Python programmer.

Lists and floats tend to be used in completely different ways.

I don't have (much) problem with the idea of having mutable floats per

se, although I think they are unnecessary. But even given mutable floats,

the use you seem to want to put them will lead you into trouble.

> As for your idea of

> "straight-forward interval arithmetic", it's a good one, but I'd have to

> redefine lots of functions from the math module, to use them explicitly

> in my code, etc.:

But you need to do that anyway, because the math module doesn't calculate

the error estimates that you need. Given some FloatWithError x, if you

call math.sin(x) the math.sin() function makes no effort to calculate an

error term.

> this is heavy; I was looking for a light-weight

> alternative, where all calculations are expressed in Python as if all

> numbers were regular floats, and where you don't have to redefine any

> mathematical operation. In other words, I'm looking for a robust way of

> implementing the derive() function of Peter without knowing anything

> about which function uses which "numbers with an uncertainty": the

> equivalent of Peter's derive () would simply successively change all

> "numbers with uncertainty" defined so far and see how the result of a

> given calculation varies-- the variables that are not used in the

> calculated expression don't change the result, for instance. I

> understand that this is computationally inefficient (too many variables

> might be changed), but I don't need speed, only robustness in the

> propagation of errors, and a simple and very legible calculation code,

> and ideally with only a few lines of Python.

Nothing you have described gives me any confidence in the robustness of

your solution. In fact the opposite: given the description of what you

are planning, I'd have ZERO confidence in ANY calculation you did,

because I'd wonder how all the other calculations being performed were

effecting the calculations you've already done.

As far as I can tell, you are conflating four different issues as if they

were one:

(1) how to calculate and propagate errors and do interval arithmetic;

(2) how to use the functions in the math module with arguments which

aren't floats;

(3) how to make a mutable float type;

(4) how to have the result of a calculation magically change when you

change the arguments, without re-doing the calculation (at least not

explicitly re-doing the calculation).

(1) - (3) are reasonable; (3) is I think pointless but not actively

harmful; but (4) is just creating a world of pain and almost-impossible

to track down bugs.

> Anyway, Steven gave me hope that some kind of mutable float is possible

> in Python, and Ben perfectly expressed my need. Again, alternatively,

> any module that handles in a convenient manner calculations with error

> propagation would be great, but I have not found any single such module!

http://pyinterval.googlecode.com/svn/trunk/html/index.html

http://docs.sympy.org/modules/mpmath/intervals.html

You should also read this:

http://conference.scipy.org/proceedings/SciPy2008/paper_3/

--

Steven

Apr 15, 2009, 5:21:24 AM4/15/09

to

Dan, wow! This looks extremely promising!

I initially tried to create a Float_ref class that inherits from

numpy.array, so that objects of the new class behave like numpy.array

in calculations, but also contain an "uncertainty" atribute, but this

is apparently not allowed ("cannot create 'builtin_function_or_method'

instances").

So I thought I'd directly add an attribute to a numpy.array: "x =

numpy.array(3.14); x.uncertainty = 0.01", but this is not allowed

either.

Thus, it is not obvious for me to extend the 1x1 numpy.array object so

that it also contains an "uncertainty" attribute.

However, I think your suggestion can largely solve the original

problem, with a small price to pay: numbers with uncertainties can be

defined in a module that keeps declared values in a list of

numpy.arrays, and the associated uncertainties in another list: after

"registering" many "mutable float" objects, the modules would contain

the following lists:

>>> mutable_floats = [ numpy.array(3.14), numpy.array(1.41),...]

>>> uncertainties = [ 0.01, 0.01 ,...]

The floatref class could instead be a simple function that returns the

last numpy.array created, after adding an element to both the

mutable_float and uncertainties lists. What we lose here is that it's

hard to get the uncertainty associated to a given variable (this is

obviously possible if no other variable has the same value, though).

Dan, you gave me hope that precise error propagation calculations can

be done by repeated calls to unmodified calculation code designed for

floats! Thanks!

On Apr 15, 10:44 am, Dan Goodman <dg.gm...@thesamovar.net> wrote:

Apr 15, 2009, 5:31:05 AM4/15/09

to pythonlist

>>>

>>> On Apr 14, 3:03 pm, Eric.Le.Bi...@spectro.jussieu.fr wrote:

>>>

>>>> Hello,

>>>>

>>>> Is there a way to easily build an object that behaves exactly like a

>>>> float, but whose value can be changed? The goal is to maintain a list

>>>> [x, y,…] of these float-like objects, and to modify their value on the

>>>> +y" uses the new value.

>>>>

>>>> I thought of two solutions, both of which I can't make to work:

>>>>

>>>> 1) Use a class that inherits from float. This takes care of the

>>>> "behave like float" part. But is it possible to change the value of

>>>> the float associated with an instance? That is, is it possible to

>>>> becomes 3.14?

>>>>

>>>> 2) The other possibility I thought of was: use a class that defines a

>>>> 'value' member (x.value). This takes care of the "value can be

>>>> changed" part. But is it possible/easy to make it fully behave like a

>>>> float (including when passed to functions like math.sin)?

>>>>

>>>> Alternatively, I'd be happy with a way of handling numerical

>>>> uncertainties in Python calculations (such as in "calculate the value

>>>> 0.01").

>>>>

>>>> Any idea would be much appreciated!

>>>>

>> The answer to your original question is no. If the value can be changed, then it doesn't behave like a float. And that's not just a pedantic answer, it's a serious consideration.

>>

>> You have to decide what characteristics of a float you need to mimic, and which ones you don't care about, and which ones you want to change. Only after having a pretty good handle on those answers can you pick a "best" implementation.

>>

>> Let's call this new type a nfloat, and let's assume you have a function that returns one. That might be a constructor, but it may not, so we're keeping our options open.

>>

>> What do you want to happen when you execute b =yval ? Presumably

>> you want them to be "equal" but in what sense? Suppose you then change

>> one of them with your suggested attribute/method.

>>

>> is b the same as it was (like a float would be), or is b also changed?

>>

>> Do you need lots of functions to work on one of these nfloats, or could

>> you use them as follows:

>> sin( b.value )

>>

>> Instead of using .value to change the underlying float, how about if you

>> use [0] ? Just use a list of size 1.

>>

>

>

>

after. Seems to me you're going to have a specific list of values, which

you want to be able to individually tweak each time you do the

calculations. And you want to have a name for each of those values, so

that the formulas look pretty straightforward.

I'm not convinced this is the best approach for uncertainty

calculations, but if those numbers do form a fairly easily specified

list (and one that's not dynamic), what you probably want is an object

that's a reference to a list item, while the list item is itself a

float. And you want the object to support many of the usual floating

point operations. So why not define a single static list (could be a

class object), and define a class whose instances contain an index into

that list. The class could have methods __add__() and __mul__() etc. And

it could have one more method for storing to the list.

class IndirectFloat(object):

liss = []

def __init__(self, value=0.0):

self.index = len(self.liss)

self.liss.append(value)

def modify(self, newvalue):

self.liss[self.index]= newvalue

def __add__(self, val): #handle IndirectFloat + float

return self.liss[self.index] + float(val)

__radd__ = __add__ #handle float + IndirectFloat

def __float__(self): #just in case someone wants to explicitly convert

return self.liss[self.index]

x = IndirectFloat(12.0)

y = IndirectFloat(20.0)

print x + 13.0

print x + y

print 3.0 + x

x.modify(15.0)

print x + 13.0

Notice that if you ever do things like z = y, you'll be using the *same*

object, and modifying one will also modify the other.

Now, in addition to __add__ and __radd__, you want __mul__ and __rmul__,

and many others, presumably. Notice that for the non-commutative

methods, you can't just use the same method for both, the way we do for

add and multiply.

I still think an error-propagation class would be better.

Something like:

class ErrorPropFloat(object):

def __init__(self, value=0.0, err=0.0):

self.value = value

self.err = err

def __add__(self, val):

res = self.value + float(val)

if type(val) == type(0.0):

err = self.err

else:

err = self.err + val.err

return ErrorPropFloat(res, err)

__radd__ = __add__

def __float__(self): #just in case someone wants to explicitly convert

return self.value

def __repr__(self):

return "ErrorProp(%g, %g)" % (self.value, self.err)

#return "ErrorProp(" + self.value + "," + self.err + ")"

x = ErrorPropFloat(12.0, 0.1)

y = ErrorPropFloat(20.0, 0.2)

print x, y

print x + 13.0

print x + y

print 3.0 + x

Apr 15, 2009, 5:38:57 AM4/15/09

to

>>>>> Eric.L...@spectro.jussieu.fr (ELB) wrote:

[snip]

>ELB> A couple of ideas I had:

>ELB> 1) Define a FloatWithUncert object, but get instance values as x(), as

>ELB> in "x()+y()". The code is relatively legible. 'x' is mutable. But

>ELB> formulas don't look so good, and you can't drop a float replacement

>ELB> for 'x', as floats are not callable.

>ELB> 2) Write all expressions that could contain FloatWithUncert objects

>ELB> with a 'float()' wrapper ("float(x)+float(y)"), after defining the

>ELB> FloatWithUncert.__float__() method. FloatWithUncert would be

>ELB> mutable. The code is a little bit heavy, but it is explicit. 'x'

>ELB> could be a pure float.

Would this be what you want?

class Float(object):

def __init__(self, value, uncert=0.0):

self.value = value

self.uncert = uncert

def __float__(self):

return self.value

def __str__(self):

return str(self.value)

def __add__(self, other):

return self.value + other

__radd__ = __add__

# etc for all other arithmetic and logical operators (but this can be

# automated, see the thread "Automatically generating arithmetic

# operations for a subclass")

>>> x = Float(3.14, 0.01)

>>> print x

3.14

>>> from math import sin

>>> sin(x)

0.0015926529164868282

>>> x.value = -3.14

>>> print x

-3.14

>>> sin(x)

-0.0015926529164868282

>>> x + 1

-2.1400000000000001

>>> y = Float(7.0, 0.3)

>>> print y

7.0

>>> x + y

3.8599999999999999

>>>

If you write your own functions they might have to coerce your Float

objects to real floats ocassionally by using float(param) instead of

param.

--

Piet van Oostrum <pi...@cs.uu.nl>

URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]

Private email: pi...@vanoostrum.org

Apr 15, 2009, 5:48:57 AM4/15/09

to

Steven, I'd appreciate if you could refrain from criticizing so

bluntly so many points. I'd be great if you trusted me more for

knowing what I'm talking about; I've been a programmer for 25 years,

now, and I pretty well know what my own code looks like! I appreciate

your input, but please soften your style!

bluntly so many points. I'd be great if you trusted me more for

knowing what I'm talking about; I've been a programmer for 25 years,

now, and I pretty well know what my own code looks like! I appreciate

your input, but please soften your style!

For instance, for your information, to respond only to your first

point (below), I'd like to be more precise: I don't want a function

that calculates a result (such as f() in Peter's code above) to

explicitly contain code for handling uncertainties. This is what I

had in mind when I said, perhaps inadequately, that "the place in the

code where (foo, baz) is calculated has no idea (...) of where they

were defined, etc.". Thus, either f() manipulates numbers without

knowing that it is actually performing error calculations when doing

x*y, or some code calls f() and does the job of calculating the

uncertainty. The first method does not work, for many reasons: (1)

correlated errors are really hard to take into account, (2) the 2

modules you list are interesting, but none of them gives the standard

deviation of an arbitrary mathematical expression.

Steven, there is obviously a strong misunderstanding problem here.

For instance, I do not "conflate four different issues as if they were

one". There are, however, various issues raised in this stimulating

thread, and many paths are worth exploring.

It looks to me like Dan has paved the way to a good solution to the

initial problem: calculate the uncertainty on any calculation that was

written for code that works with floats, without modifying this code;

the method consists essentially in performing Peter's calc()

calculation, but changing the values of the quantities used through

references (as Ben described). But, again, I don't yet see how to

unambiguously keep track of the uncertainty associated to a numerical

value. I'll try this, to see if the devil does lie in the details,

here. :)

The other, approximate solution, would be to coerce with float() any

parameter that can carry an uncertainty, in calculations (numbers with

uncertainty would have a __float__() method that returns a result that

can be changed after instantiation); but this would modify (and make

less legible) existing code.

On Apr 15, 10:59 am, Steven D'Aprano

<ste...@REMOVE.THIS.cybersource.com.au> wrote:

> On Wed, 15 Apr 2009 01:05:33 -0700, Eric.Le.Bigot wrote:

> > Ben F., you're right on the money! You expressed exactly what I'm

> > looking for. Why should I want this? because the place in the code

> > where (foo, baz) is calculated has _no idea_ of what foo and baz are, of

> > where they were defined, etc.;

>

> This makes no sense. The piece of code which calculates (foo, baz) knows

> EXACTLY what foo and baz are, because it has just calculated them. It has

> them *right there*. If it wants to know what they are, it can just look

> at them:

>

> def calculate():

> # lots of calculations

> result = (foo, baz)

> # what is foo?

> foo

> # what is baz?

> baz

> # what are their types?

> type(foo)

> type(baz)

>

> As for where they were defined... why do you care where they were

> defined? What's important is *what they are*, and you know what they are,

> because you have them, right there.

Apr 15, 2009, 5:59:39 AM4/15/09

to pythonlist

likely alter the OP's code's behavior. It doesn't matter if it's

possible, it matters whether the existing code's behavior might change,

and of course if a future maintainer might have trouble making sense of it.

BTW, just what languages have mutable floats? I don't think I've come

across any since Fortran (on a CDC 6400), and I don't think that was

deliberate. In that implementation it was possible to change the value

of a literal 2.0 to something else. I remember writing a trivial

program that printed the value of 2.0 + 2.0 as 5.0. It was about 1971,

I believe. I know I haven't used Fortran since 1973.

If you're going to use the fact that many languages pass arguments by

reference, then you should realize that a reference to a float is a

different kind of variable than a float. And although that language

might use the terminology of mutable, it wouldn't mean the same thing

that mutable does in Python.

Apr 15, 2009, 6:14:11 AM4/15/09

to

To Dave A. and Piet: I appreciate your taking the time to make

suggestions. I understand that there is a hitch in the approach that

you describe, which I would like to insist on: how do you handle

functions that use math.sin(), for instance? numpy does this kind of

magic, but I'm not sure it's wise to spend time reproducing it. I'd

to not have to modify any calculation code, so that it is legible and

general (in particular, so that it can work with Python floats). And

again, interval arithmetic fails to produce real standard deviations,

because of correlations between uncertainties.

suggestions. I understand that there is a hitch in the approach that

you describe, which I would like to insist on: how do you handle

functions that use math.sin(), for instance? numpy does this kind of

magic, but I'm not sure it's wise to spend time reproducing it. I'd

to not have to modify any calculation code, so that it is legible and

general (in particular, so that it can work with Python floats). And

again, interval arithmetic fails to produce real standard deviations,

because of correlations between uncertainties.

Hence my idea to have mutable floats, that would be changed in some

_external_ error calculation routine (i.e. a routine which is

completely independent from the calculation f()), in a way equivalent

to Peter's derive() function--except that the values used in f() are

only accessible through a list of objects, as in Ben's post. I'll

implement this with 1x1 numpy.array objects, even though this will

mean that numbers will not have an unambiguous uncertainty attribute,

as I mentioned in a previous post. I'll post here the result of my

investigations.

Apr 15, 2009, 7:00:19 AM4/15/09

to

Eric.L...@spectro.jussieu.fr writes:

> As for your idea of "straight-forward interval arithmetic", it's a

> good one, but I'd have to redefine lots of functions from the math

> module, to use them explicitly in my code, etc.: this is heavy; I was

> looking for a light-weight alternative, where all calculations are

> expressed in Python as if all numbers were regular floats, and where

> you don't have to redefine any mathematical operation. In other

> words, I'm looking for a robust way of implementing the derive()

> function of Peter without knowing anything about which function uses

> which "numbers with an uncertainty": the equivalent of Peter's derive

> () would simply successively change all "numbers with uncertainty"

> defined so far and see how the result of a given calculation varies--

> the variables that are not used in the calculated expression don't

> change the result, for instance. I understand that this is

> computationally inefficient (too many variables might be changed), but

> I don't need speed, only robustness in the propagation of errors, and

> a simple and very legible calculation code, and ideally with only a

> few lines of Python.

I still don't understand why you need mutable floats.

Here is a suggestion, based on Petter Otten's Value class and his derive

and calc functions. I've modified Value slightly so that it implements

unary - and binary +, *, -, /, **. The 'for name in dir(math)' loop at

the end wraps each function in the math module in valueified (defined

below) so that they accept floats or Values.

---------------- uncert.py ---------------

def valueified(f):

"""

Change a function that accepts floats to a function that accepts

any of float or Value.

"""

def decorated(*args):

ret = calc(f, map(Value, args))

return ret if ret.err else ret.value

return decorated

class Value(object):

def __init__(self, value, err=0):

if isinstance(value, Value):

value, err = value.value, value.err

self.value = float(value)

self.err = float(err)

def __repr__(self):

return "%r +- %r" % (self.value, self.err)

__neg__ = valueified(float.__neg__)

for op in 'add', 'sub', 'mul', 'div', 'pow':

for r in '', 'r':

exec """__%s__ = valueified(float.__%s__)""" % (r+op, r+op)

del op, r

def derive(f, values, i, eps=1e-5):

x1 = f(*values)

values = list(values)

values[i] += eps

x2 = f(*values)

return (x2-x1)/eps

def calc(f, args):

values = [v.value for v in args]

errs = [v.err for v in args]

sigma = 0

for i, (v, e) in enumerate(zip(values, errs)):

x = derive(f, values, i)*e

sigma += x*x

return Value(f(*values), sigma**0.5)

builtinf = type(sum)

import math

for name in dir(math):

obj = getattr(math, name)

if isinstance(obj, builtinf):

setattr(math, name, valueified(obj))

---------- end of uncert.py ---------------

Example:

marigold:junk arno$ python -i uncert.py

>>> a = Value(2, 0.1)

>>> b = Value(7, 2)

>>> a+b

9.0 +- 2.0024984393742682

>>> a*b

14.0 +- 4.0607881007017825

>>> from math import *

>>> sin(a)

0.90929742682568171 +- 0.041615138303141563

>>> sin(2)

0.90929742682568171

>>> 2*a

4.0 +- 0.20000000000131024

>>> 2**b

128.0 +- 177.44629319622615

>>> a/b

0.2857142857142857 +- 0.082873111674175479

>>> sqrt(a**2+b**2)

7.2801098892805181 +- 1.9232454006952127

>>> hypot(a, b)

7.2801098892805181 +- 1.9232442191491188

>>> #etc

Isn't this what you need?

--

Arnaud

Apr 15, 2009, 7:13:29 AM4/15/09

to

It looks like Dan found what is in effect a mutable float

(numpy.array).

(numpy.array).

Now, with respect to the initial problem of having mutable floats that

also contain an uncertainty attribute, I'd like to note that

numpy.ndarray can be subclassed: it now looks possible to create a

mutable float class that also contains an uncertainty attribute!

So, I'll see how/whether this can be implemented without pain... I'll

leave a message here when I've got news!

Thanks again everybody for helping me out!

On Apr 15, 10:44 am, Dan Goodman <dg.gm...@thesamovar.net> wrote:

Apr 15, 2009, 7:41:37 AM4/15/09

to

>>>>> Eric.L...@spectro.jussieu.fr (ELB) wrote:

>ELB> To Dave A. and Piet: I appreciate your taking the time to make

>ELB> suggestions. I understand that there is a hitch in the approach that

>ELB> you describe, which I would like to insist on: how do you handle

>ELB> functions that use math.sin(), for instance? numpy does this kind of

>ELB> magic, but I'm not sure it's wise to spend time reproducing it. I'd

>ELB> to not have to modify any calculation code, so that it is legible and

>ELB> general (in particular, so that it can work with Python floats). And

>ELB> again, interval arithmetic fails to produce real standard deviations,

>ELB> because of correlations between uncertainties.

Mine does allow you to use math.sin(x) as you can see. Dave's also I

think. His solution is similar to mine but he puts the list stuff in it

whereas I came up with only the float.

In my solution I think you can mostly use normal formula's with the

Float objects, although there may be some functions that give problems

for example if they have explicit tests like type(x) == float. Why don't

you give it a try?

Apr 15, 2009, 9:01:11 AM4/15/09

to

Arnaud, your code is very interesting!

On Apr 15, 1:00 pm, Arnaud Delobelle <arno...@googlemail.com> wrote:

> I still don't understand why you need mutable floats.

Here is why: the code that your proposed (or any code that does

straightforward error propagation, for that matter) does not generally

calculate uncertainties correctly:

>>> a=Value(1, 0.1)

>>> a-a

0.0 +- 0.14142135623823598

The correct result is obviously 0.0 +- 0.0. This is the effect of

what I referred to in previous posts as "correlated errors".

Anyway, I learned some interesting stuff about what you can do in

Python, thanks to your code! :)

Apr 15, 2009, 9:26:48 AM4/15/09