175 views

Skip to first unread message

Jul 4, 2011, 6:47:53 AM7/4/11

to

Let Mathematica (6.x, 7.x) compute quite a simple product

In[1]:= N[N[2.0, 20] * N[Sqrt[2] , 20], 20]

Out[1]= 2.82843

This is a bug.

Why?

Now we analyze it in details:

1. N[2.0,20] should give 2 with accuracy/precision/whatsever about 20

decimal digits, i.e. 2.00000000000000000000

2. Sqrt[2] should give... Sqrt[2]

3. N[Sqrt[2]] should give 1.4142135623730950488 (this is copy-paste from

Mathematica output to N[Sqrt[2]] )

4. The product 2.00000000000000000000 * 1.4142135623730950488 is

2.8284271247461900976 (again copy-paste)

5. BUT THE RESULT OF N[2.0, 20] * N[Sqrt[2] , 20] "truncated to 20 digits"

is Out[1]= 2.82843

Where are missing digits?!

slawek

Jul 4, 2011, 7:14:46 AM7/4/11

to

The answer to this puzzle is that the N[2.0,20] is 2.0, not

2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

in you have already limited yourself to machine precision, and N[2.0,20]

is then just machine accuracy.

2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

in you have already limited yourself to machine precision, and N[2.0,20]

is then just machine accuracy.

Kevin

Jul 4, 2011, 7:18:14 AM7/4/11

to

BTW, you can skip the outer N[]:

N[2,20]N[Sqrt[2],20]

produces:

2.8284271247461900976

Kevin

On 7/4/2011 6:47 AM, slawek wrote:

Jul 5, 2011, 5:17:49 AM7/5/11

to

Uďż˝ytkownik "Kevin J. McCann" <k...@KevinMcCann.com> napisaďż˝ w wiadomoďż˝ci grup

dyskusyjnych:ius7b6$30t$1...@smc.vnet.net...

> The answer to this puzzle is that the N[2.0,20] is 2.0, not

> 2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

> in you have already limited yourself to machine precision, and N[2.0,20]

> is then just machine accuracy.

It is still a-bug-and-a-feature.

And this bug make Mathematica nearly useless in numerical computations. "MS

Windows Calculator" is much more reliable!

The number of written digits IS NEITHER the precision NOR the accuracy.

Mathematica treat 2.0 as a 2.0+-0.1, but it is not the proper way to handle

numbers.

I know, that it is common mistake to treat 2.0 as "not an integer number"

and/or "exact" number, but 2.0 is an integer number AND also it is a

rational number AND also a real number AND also a complex number. And 2.0 is

simply 1+1+ 0/10 . Therefore, as you see, there is no "roudning", "limited

precision", "error" or "uncertinaity". It is only a matter of a notation of

decimal fractions. And decimal fractions are exact. Any "tolerance" is not

indicated in any way by this notation. Thus it is a bug. Nasty, big, fat bug

in the core of Mathematica.

Even from "CS view" 2.0 is translated to IEEE representation with 56-bits of

the mantisa. Nobody declare float x = 2.0000000000 to iniject the float

point two into a code.

slawek

Jul 5, 2011, 5:15:47 AM7/5/11

to

In brief: yes to both.

On 7/4/2011 4:18 AM, Kevin J. McCann wrote:

...

>>

>> 1. N[2.0,20] should give 2 with accuracy/precision/whatsever about 20

>> decimal digits, i.e. 2.00000000000000000000

>

WRI will defend this as a feature.

You thought that the semantics of N[] were the same as SetPrecision

e.g.

N[SetPrecision[2.0, 20]*N[Sqrt[2], 20], 20]

works as you expected.

So from your perspective, and from the perspective of anyone else who

thinks along the same lines, it is a bug. I would prefer to call it

a design error.

2.0 (indeed, any floating point number) is a perfectly respectable way

of denoting a number that can be expressed in higher precision by adding

binary zeros. WRI doesn't agree.

RJF

Jul 6, 2011, 5:34:31 AM7/6/11

to

U=BFytkownik "Richard Fateman" <fat...@cs.berkeley.edu> napisa=B3 w wiadomo=B6ci

grup dyskusyjnych:iuuko3$eqi$1...@smc.vnet.net...

> 2.0 (indeed, any floating point number) is a perfectly respectable way

> of denoting a number that can be expressed in higher precision by adding

> binary zeros. WRI doesn't agree.

Mathematica is very simple to use. Of course, it is the entire programming

language, advanced concepts, etc. However it seems that in order to

calculate 2 Sqrt[2] is need no special knowledge nor special care.

Nevertheless, it appears that although the language itself does not enforce

this, calculations require very formal definitions of all variables.

slawek

Jul 6, 2011, 5:35:02 AM7/6/11

to

Well, I certainly disagree with your assessment. The rules regarding

exact and machine precision numbers are straightforward and stated in

the documentation. But if you feel that the MS Windows Calculator is a

better option, by all means use that. It is free and easy to use.

exact and machine precision numbers are straightforward and stated in

the documentation. But if you feel that the MS Windows Calculator is a

better option, by all means use that. It is free and easy to use.

2.0 is not an integer nor a rational, it is machine precision. On the

other hand 2 is an integer and exact.

As far as whether or not 2.0 should be considered as 2.0+-0.1 or as

2.000..., 2.0 is a machine precision number, which means

2.000000000000000 or as close as you can get in about 16 places. Telling

Mathematica that you want 20 places is somewhat like asking Mathematica to give more

accuracy *after* the calculation is finished. At any rate my friend

David Park reminded me that you can specify your 2.0 to 20 places with

the following:

2.0`20

Precision[%]

20.

Kevin

On 7/5/2011 5:17 AM, slawek wrote:

> U�ytkownik "Kevin J. McCann"<k...@KevinMcCann.com> napisa� w wiadomo�ci grup

Jul 6, 2011, 5:39:07 AM7/6/11

to

On Tue, 05 Jul 2011 10:17:49 +0100, slawek <sla...@host.pl> wrote:

> U=BFytkownik "Kevin J. McCann" <k...@KevinMcCann.com> napisa=B3 w wiadomo=B6ci

> grup dyskusyjnych:ius7b6$30t$1...@smc.vnet.net...

>> The answer to this puzzle is that the N[2.0,20] is 2.0, not

>> 2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

>> in you have already limited yourself to machine precision, and N[2.0,20]

>> is then just machine accuracy.

>

> It is still a-bug-and-a-feature.

> And this bug make Mathematica nearly useless in numerical computations..

> "MS Windows Calculator" is much more reliable!

>

> The number of written digits IS NEITHER the precision NOR the accuracy..

> Mathematica treat 2.0 as a 2.0+-0.1, but it is not the proper way to

> handle numbers.

>

In fact:

In[1] :=

Interval[2.0] // InputForm

Out[1]//InputForm =

Interval[{1.9999999999999998, 2.0000000000000004}]

While:

In[2] :=

Interval[2.0``1] // InputForm

Out[2]//InputForm =

Interval[{1.875`1.2730012720637347, 2.125`1.327358934386329}]

Precision (which, as defined by Mathematica, means relative uncertainty)

and accuracy (absolute uncertainty) are expressed as annotations after the

number. In the special case of a number with a decimal point and no

annotation, the number is taken to be a machine precision real. I agree

that these definitions and the notational convention chosen by Mathematica

are strange. However, there is nothing "improper" about it as a choice of

formalism--at least, this is no worse a design choice than for Mathematica

to have standardized on decimal notation for input and presentation of

numerical values rather than binary as it uses internally.

The purpose of this approach is as a crude but often adequate

approximation to interval arithmetic, whereby these (approximations of)

errors are carried through arithmetic operations using first-order

algebraic methods. When functions (such as N) that pay attention to

Precision and Accuracy (by Mathematica's definitions) see them decreasing,

they increase the working precision so as to avoid numerical instability

being expressed in the final result. This is by no means intended to be

rigorous; it is merely a heuristic, but one that comes at little cost and

works in many cases. Of course, if a user's own code treats this

approximation as somehow sacrosanct and ignores the precision adjustments

necessary during the calculation while taking the final answer as correct,

it is more likely that the approximation will have fallen apart somewhere

down the line.

If you don't like significance arithmetic, you have (at least) two other

options at hand: either work in fixed precision ($MinPrecision

$MaxPrecision = prec) or use interval arithmetic. These have their own

drawbacks, of course (most notably that Mathematica tacitly assumes all

intervals are uncorrelated), but your hand isn't forced either way and you

may even use all three methods simultaneously if you wish. Alternatively

you may program your own, more accurate algebraic or Monte Carlo error

propagation methods if you prefer.

Jul 6, 2011, 5:41:10 AM7/6/11

to

"Mathematica treat 2.0 as a 2.0+-0.1"

No, it treats it as 2 with machine precision -- almost 16 digits at my

machine -- not two digits, as you're describing.

Bobby

On Tue, 05 Jul 2011 04:12:05 -0500, slawek <sla...@host.pl> wrote:

>

> U=BFytkownik "Kevin J. McCann" <k...@KevinMcCann.com> napisa=B3 w wiadomo=B6ci

> grup

> dyskusyjnych:ius7b6$30t$1...@smc.vnet.net...

>> The answer to this puzzle is that the N[2.0,20] is 2.0, not

>> 2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

>> in you have already limited yourself to machine precision, and N[2.0,20]

>> is then just machine accuracy.

>

> It is still a-bug-and-a-feature.

> And this bug make Mathematica nearly useless in numerical computations.

> "MS

> Windows Calculator" is much more reliable!

>

>

> The number of written digits IS NEITHER the precision NOR the accuracy.

> Mathematica treat 2.0 as a 2.0+-0.1, but it is not the proper way to

> handle

> numbers.

>

> I know, that it is common mistake to treat 2.0 as "not an integer number"

> and/or "exact" number, but 2.0 is an integer number AND also it is a

> rational number AND also a real number AND also a complex number. And

> 2.0 is

> simply 1+1+ 0/10 . Therefore, as you see, there is no "roudning",

> "limited

> precision", "error" or "uncertinaity". It is only a matter of a notation

> of

> decimal fractions. And decimal fractions are exact. Any "tolerance" is

> not

> indicated in any way by this notation. Thus it is a bug. Nasty, big, fat

> bug

> in the core of Mathematica.

>

> Even from "CS view" 2.0 is translated to IEEE representation with

> 56-bits of

> the mantisa. Nobody declare float x == 2.0000000000 to iniject the float

> point two into a code.

>

> slawek

>

>

Jul 7, 2011, 7:33:49 AM7/7/11

to

>

This is an excellent summary of Mathematica's approach to arithmetic on

numbers. Unfortunately many people come to use Mathematica with their

own notions of numbers, accuracy, precision, and equality. These words

are redefined in a non-standard way in Mathematica, leading to

unfortunate situations sometimes. "unexplainable" behavior. confusion.

Or worse, erroneous results silently delivered and accepted as true by a

user, who "knows" about precision, accuracy, floating-point arithmetic, etc.

WRI argues that this is a winning proposition. Perhaps Wolfram still

believes that someday all the world will use Mathematica for all

programming purposes and everyone will accept his definition of terms

like Precision and Accuracy, and that (see separate thread on how to

write a mathematical paper) it will all be natural and consistent.

(or that people who want to hold to the standard usage will be forced to

use something like SetGlobalPrecision[prec_]:=

$MaxPrecision=MinPrecision=prec.

I believe this is routinely used by people who find Mathematica's

purportedly "user-friendly" amateurish error control to be hazardous.

)

.........

'When I use a word,' Humpty Dumpty said, in rather a scornful tone, 'it

means just what I choose it to mean =97 neither more nor less.'

'The question is,' said Alice, 'whether you can make words mean so many

different things.'

'The question is,' said Humpty Dumpty, 'which is to be master =97 that's

all.'

Jul 7, 2011, 7:34:50 AM7/7/11

to

On Jul 5, 4:17 am, "slawek" <sla...@host.pl> wrote:

> U ytkownik "Kevin J. McCann" <k...@KevinMcCann.com> napisa w wiadomo ci grup

> dyskusyjnych:ius7b6$30...@smc.vnet.net...

>

> > The answer to this puzzle is that the N[2.0,20] is 2.0, not

> > 2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

> > in you have already limited yourself to machine precision, and N[2.0,20]

> > is then just machine accuracy.

>

> It is still a-bug-and-a-feature.

> And this bug make Mathematica nearly useless in numerical computations. "MS

> Windows Calculator" is much more reliable!

> U ytkownik "Kevin J. McCann" <k...@KevinMcCann.com> napisa w wiadomo ci grup

> dyskusyjnych:ius7b6$30...@smc.vnet.net...

>

> > The answer to this puzzle is that the N[2.0,20] is 2.0, not

> > 2.00000000... Try N[2,20] and all is well. I think that when you put 2.0

> > in you have already limited yourself to machine precision, and N[2.0,20]

> > is then just machine accuracy.

>

> It is still a-bug-and-a-feature.

> And this bug make Mathematica nearly useless in numerical computations. "MS

> Windows Calculator" is much more reliable!

I have used Mathematica for numerical computation for nearly 20 years

and will say that on most days I do not find it to be useless. I am

not familiar with MS Windows Calculator so cannot comment on any

comparison between the two.

> The number of written digits IS NEITHER the precision NOR the accuracy.

> Mathematica treat 2.0 as a 2.0+-0.1, but it is not the proper way to handle

> numbers.

As written this is not correct. In Mathematica if you input 2.0 then

it is treated as a machine double, which on all platforms today has

around 16 decimal places. In any case it is 2. +-$MachineEpsilon.

The formatting of machine floats attempts (I believe always

successfully) to show the decimal approximating to the underlying

binary value (that is, within half an error-in-the-last place) that

has the least number of digits. So for example 2.1 will be printed as

2.1, even though the underlying binary representation is not

identically equal to 21/10.

> I know, that it is common mistake to treat 2.0 as "not an integer number"

> and/or "exact" number, but 2.0 is an integer number AND also it is a

> rational number AND also a real number AND also a complex number.

This is a matter to be determined by the language semantics (for any

language, not just Mathematica). A "design decision", if you will (and

as another response states). You may not like the design, and you may

have reasons for not liking it, or for preferring other programs. That

does not make this design intrinsically wrong.

> And 2.0 is

> simply 1+1+ 0/10 . Therefore, as you see, there is no "roudning", "limited

> precision", "error" or "uncertinaity". It is only a matter of a notation of

> decimal fractions. And decimal fractions are exact.

This is only true if the underlying representation is decimal. In some

programs that might be the case. In Mathematica underlying

representations are binary based. So a decimal such as 2.1 will not be

2 + 1/10 because that cannot be supported by finite binary

representation.

A different tack would be to translate, at parse time say, decimal

input such as 2.1 to exact input such as 21/10. I will spare anyone

who has read this far from all the reasons why that would be an

incredibly bad idea. Suffice it to say that if one wants to work with

exactly 21/10 then one should use 21/10, and not an approximate input

variant thereof.

> Any "tolerance" is not

> indicated in any way by this notation. Thus it is a bug. Nasty, big, fat bug

> in the core of Mathematica.

>

> Even from "CS view" 2.0 is translated to IEEE representation with 56-bits of

> the mantisa. Nobody declare float x = 2.0000000000 to iniject the float

> point two into a code.

>

> slawek

Daniel Lichtblau

Wolfram Research

Jul 7, 2011, 7:35:52 AM7/7/11

to

I don't think this holds up for all floats in the sense you seem to

indicate. Take 2.1 for example. We can (and will) use SetPrecision to

pad with binary zeroes, just as you use it above on 2.0.

In[606]:= N[SetPrecision[2.1, 20]*N[Sqrt[2], 20], 20]

Out[606]= 2.9698484809834997281

But this is not an accurate representation of 21/10 * Sqrt[2] to 20

places.

In[607]:= N[21/10*Sqrt[2], 20]

Out[607]= 2.9698484809834996025

They are off by arould a machine epsilon. No surprise here, I think.

In[608]:= % - %%

Out[608]= -1.256*10^-16

I also think that this sort of numeric discrepancy is inevitable* in

any program that uses decimal input and binary representation.

Daniel Lichtblau

Wolfram Research

*Perhaps more inevitable than these threads. Now that's pretty

inevitable.

Jul 7, 2011, 7:40:29 AM7/7/11

to

U=BFytkownik "Kevin J. McCann" <Kevin....@umbc.edu> napisa=B3 w wiadomo=B6ci

grup dyskusyjnych:iv1a86$sji$1...@smc.vnet.net...

> 2.0 is not an integer nor a rational, it is machine precision. On the

> other hand 2 is an integer and exact.

The decimal fraction 2.0 is in mathematics (the science) exactly the same as

the sum 2 + 0/10

The convention that 2.0 is less accurate than 2.00 is applied ONLY in

Mathematica (the computer program).

Jul 8, 2011, 4:51:53 AM7/8/11

to

On Jul 7, 5:40 am, "slawek" <sla...@host.pl> wrote:

> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

> Mathematica (the computer program).

Not true. This is a long-standing convention in experimental science.

Jul 8, 2011, 4:53:25 AM7/8/11

to

On 7/7/2011 4:35 AM, Daniel Lichtblau wrote:

> On Jul 5, 4:15 am, Richard Fateman<fate...@cs.berkeley.edu> wrote:

>> In brief: yes to both.

>>

>> On 7/4/2011 4:18 AM, Kevin J. McCann wrote:

>> ...

>>

>>

>>

>>>> 1. N[2.0,20] should give 2 with accuracy/precision/whatsever about 20

>>>> decimal digits, i.e. 2.00000000000000000000

>>

>> WRI will defend this as a feature.

>>

>> You thought that the semantics of N[] were the same as SetPrecision

>> e.g.

>>

>> N[SetPrecision[2.0, 20]*N[Sqrt[2], 20], 20]

>>

>> works as you expected.

>>

>> So from your perspective, and from the perspective of anyone else who

>> thinks along the same lines, it is a bug. I would prefer to call it

>> a design error.

>>

>> 2.0 (indeed, any floating point number) is a perfectly respectable way

>> of denoting a number that can be expressed in higher precision by adding

>> binary zeros. WRI doesn't agree.

>>

>> RJF

>

> I don't think this holds up for all floats in the sense you seem to

> indicate. Take 2.1 for example.

> On Jul 5, 4:15 am, Richard Fateman<fate...@cs.berkeley.edu> wrote:

>> In brief: yes to both.

>>

>> On 7/4/2011 4:18 AM, Kevin J. McCann wrote:

>> ...

>>

>>

>>

>>>> 1. N[2.0,20] should give 2 with accuracy/precision/whatsever about 20

>>>> decimal digits, i.e. 2.00000000000000000000

>>

>> WRI will defend this as a feature.

>>

>> You thought that the semantics of N[] were the same as SetPrecision

>> e.g.

>>

>> N[SetPrecision[2.0, 20]*N[Sqrt[2], 20], 20]

>>

>> works as you expected.

>>

>> So from your perspective, and from the perspective of anyone else who

>> thinks along the same lines, it is a bug. I would prefer to call it

>> a design error.

>>

>> 2.0 (indeed, any floating point number) is a perfectly respectable way

>> of denoting a number that can be expressed in higher precision by adding

>> binary zeros. WRI doesn't agree.

>>

>> RJF

>

> I don't think this holds up for all floats in the sense you seem to

> indicate. Take 2.1 for example.

My contention is that 2.1, written that way, is a perfectly respectable

way of denoting a number that can be expressed in higher precision by

adding binary zeros. 2.1 is a double-float in most systems today.

What is that number? In decimal, to higher precision, it looks like

2.10000000000000008881784197001....

A program that decodes a float into its component fraction, exponent,

and sign gives this information for the fraction:

4728779608739021 or in binary

10000110011001100110011001100110011001100110011001101

That is the number to which we add binary zeros.

Let us call this number P.

>We can (and will) use SetPrecision to

> pad with binary zeroes, just as you use it above on 2.0.

>

> In[606]:= N[SetPrecision[2.1, 20]*N[Sqrt[2], 20], 20]

>

> Out[606]= 2.9698484809834997281

>

> But this is not an accurate representation of 21/10 * Sqrt[2] to 20

> places.

Just because you are doing arithmetic on numbers represented to 20

digits does not mean that the result of the arithmetic will be right to

20 digits, however, in the single operation of multiplication, you

should not lose any digits. What's going on here?

You simply computed two different quantities. Let S= sqrt(2) to 20

decimal places. In line 606 you multiplied P*S and rounded it to 20

digits [ actually you did something to the binary reps, but it doesn't

matter here].

In line 607, below, you multiplied 21/10 by S. Since P is not equal

to 21/10, why should the results be the same? Oh, Mathematica thinks

that P ==21/10, but that is merely the terribly broken notion of

equality that Mathematica uses. We used to call that "close enough for

government work". But it is terribly broken because == fails to

satisfy the axioms of "equivalence relations" such as a==b and b==c

implies a==c.

>

> In[607]:= N[21/10*Sqrt[2], 20]

> Out[607]= 2.9698484809834996025

>

> They are off by arould a machine epsilon. No surprise here, I think.

>

> In[608]:= % - %%

> Out[608]= -1.256*10^-16

>

> I also think that this sort of numeric discrepancy is inevitable* in

> any program that uses decimal input and binary representation.

Absent any interior calculations, I would tend to agree. But there is

something else going on. I think that Mathematica has embedded in it

several design decisions that elevate the (typical)

one-half-unit-in-last-place error in binary-to-decimal and

decimal-to-binary conversion that happens in input or output, into a

global slush bucket that infects and propagates into all interior

computations with "software floats".

>

> Daniel Lichtblau

> Wolfram Research

>

> *Perhaps more inevitable than these threads. Now that's pretty

> inevitable.

>

maybe so.

Jul 8, 2011, 5:04:08 AM7/8/11

to

Apart from the curious definitions given to Precision and Accuracy (one

imagines ApproximateRelativeError and ApproximateAbsoluteError were

considered too verbose), I do not think Mathematica's way of doing things

is particularly arbitrary or confusing in the broader context of

multiprecision arithmetic. Mathematically, finite-precision numbers

represent distributions over finite fields, and they therefore possess

quantized upper and lower bounds, as well as quantized expectation values.

Strictly, then, any two such distributions cannot be said to be equal if

they represent numbers of different precision: they are then distributions

over two entirely different fields, irrespective of whether their

expectation values may be equal.

However, this definition is not very useful numerically and we are usually

satisfied in practice that two finite-precision numbers equal if the

expectation values are equal within quantization error. Note that the

question of true equality for numbers of different precisions, i.e. the

means of the distributions being equal, is impossible to resolve in

general given that the means (which represent the exact values) are not

available to us. Heuristically, the mean should be close, in a relative

sense, to the expectation value, hence the tolerance employed by Equal;

the exact magnitude of this tolerance may perhaps be a matter for debate

but either way it is set using Internal`$EqualTolerance, which takes a

machine real value indicating the number of decimal digits' tolerance that

should be applied, i.e. Log[2]/Log[10] times the number of least

significant bits one wishes to ignore. This setting has been discussed in

this forum at least once in the past: see e.g.

<http://forums.wolfram.com/mathgroup/archive/2009/Dec/msg00013.html>.

Note that if one wishes to be more rigorous when determining equality,

SameQ operates in a similar manner to Equal for numeric comparands, except

that its tolerance is 1 (binary) ulp. This is also adjustable, via

Internal`$SameQTolerance.

In regard to erroneous results: undoubtedly it is a possibility. However,

one would expect that an approximate first order method for dealing with

error propagation should at least be better in the majority of cases than

a zeroth-order method such as working in fixed precision. As stated

previously, if one desires more accurate approximations then one is in any

case free to implement them, although given the above it should be clear

that all that is generally possible within the domain of finite-precision

numbers is a reasonable approximation unless other information is

available from which to make stronger deductions. I will also note that

none of the example "problems" in this present topic are anything directly

to do with significance arithmetic; they instead represent a combination

of confusion due to Mathematica's (admittedly confusing) choice of

notation, combined with an apparent misunderstanding of concepts related

to multiprecision arithmetic in general.

Jul 8, 2011, 5:06:40 AM7/8/11

to

On 7/7/11 at 7:31 AM, sla...@host.pl (slawek) wrote:

>U=BFytkownik "Kevin J. McCann" <Kevin....@umbc.edu> napisa=B3 w

>wiadomo=B6ci grup dyskusyjnych:iv1a86$sji$1...@smc.vnet.net...

>>2.0 is not an integer nor a rational, it is machine precision. On

>>the other hand 2 is an integer and exact.

>The decimal fraction 2.0 is in mathematics (the science) exactly the

>same as the sum 2 + 0/10

True enough.

>The convention that 2.0 is less accurate than 2.00 is applied ONLY

>in Mathematica (the computer program).

Not true.

First

In[10]:= Precision[2.0] === Precision[2.00]

Out[10]= True

In[11]:= Accuracy[2.0] === Accuracy[2.00]

Out[11]= True

But it is true in Mathematica 2 + 0/10 is different than 2.0.

But the fact these are different is not at all unique to

Mathematica. Any application that understands floating point

values and integer values that is compliant with IEEE standards

will report 2 + 0/10 is different than 2.0

Jul 8, 2011, 5:09:44 AM7/8/11

to

This happens in EVERY language that converts decimal (real) input to

binary storage.

binary storage.

Binary storage is not mathematically required, but the result is MUCH

faster code. Perhaps, at some point, we'll abandon this trade-off when

computers are so fast that the gain in speed isn't worth the confusion.

But that day isn't here, yet.

(Well. It is for ME, but not for a lot of other, very impatient people. If

they can get a result in a second with decimal arithmetic, they'd rather

have it in a microsecond with binary arithmetic, no matter WHAT the cost

in confusion factors.)

Bobby

On Thu, 07 Jul 2011 06:31:49 -0500, slawek <sla...@host.pl> wrote:

>

> U=BFytkownik "Kevin J. McCann" <Kevin....@umbc.edu> napisa=B3 w

> wiadomo=B6ci

> grup dyskusyjnych:iv1a86$sji$1...@smc.vnet.net...

>> 2.0 is not an integer nor a rational, it is machine precision. On the

>> other hand 2 is an integer and exact.

>

> The decimal fraction 2.0 is in mathematics (the science) exactly the

> same as

> the sum 2 + 0/10

>

> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

> Mathematica (the computer program).

>

>

>

>

Jul 9, 2011, 7:34:48 AM7/9/11

to

On 7/8/2011 2:04 AM, Oleksandr Rasputinov wrote:

.. snip...

>

> Apart from the curious definitions given to Precision and Accuracy (one

> imagines ApproximateRelativeError and ApproximateAbsoluteError were

> considered too verbose), I do not think Mathematica's way of doing things

> is particularly arbitrary or confusing in the broader context of

> multiprecision arithmetic. Mathematically, finite-precision numbers

> represent distributions over finite fields, and they therefore possess

> quantized upper and lower bounds, as well as quantized expectation values.

This is one view, but one that is not especially useful computationally.

A much more useful view is that a finite-precision number is simply a

single value. That's all. The number 3 has no fuzz around it. The

double-float number 0.1d0 has no fuzz around it either. It is not

exactly 1/10, but

7205759403792794 X 2 ^(-56).

exactly. It is a finite precision number because the fraction part, here

7205759403792794, is limited to a fixed number of bits. There is, I

repeat, nothing that makes it a distribution.

Another way of writing it is

3602879701896397/36028797018963968

The advantages of this view:

(1) arithmetic is well-defined and executable on a computer

(2) should you choose to implement some kind of distribution arithmetic

-- intervals, Gaussian bell curves, significance arithmetic,

fuzzy-set-theoretic whatever, you can do so knowing that the underlying

implementation of arithmetic is supportive of any model that can be

reduced to ordinary mathematics.

> Strictly, then, any two such distributions cannot be said to be equal if

> they represent numbers of different precision: they are then distributions

> over two entirely different fields, irrespective of whether their

> expectation values may be equal.

This is something you are free to implement. I personally object to an

arithmetic system and a notion of equality that does not support the

fundamental properties of equivalence relations. I much prefer that if

a==b, then a-b==0. This of course fails, in Mathematica.

>

> However, this definition is not very useful numerically and we are usually

> satisfied in practice that two finite-precision numbers equal if the

> expectation values are equal within quantization error.

No, I disagree. It used to be that programmers were taught that one

should never (or almost never) compare floating point numbers for

equality. You can easily ask if two numbers are relatively or

absolutely close. That is not the same as being equal.

Note that the

> question of true equality for numbers of different precisions, i.e. the

> means of the distributions being equal, is impossible to resolve in

> general given that the means (which represent the exact values) are not

> available to us.

The fault here is that you view numbers as distributions. I have no

problem telling if a number 0.1d0 is equal to

3602879701896397/36028797018963968 .

It is. That is its value. It is equal to all other objects with the

same exact value. That's what equal means.

Is it equal to 0.099999999? No, but it is relatively close by some measure.

Heuristically, the mean should be close, in a relative

> sense, to the expectation value, hence the tolerance employed by Equal;

You may choose to believe this, but it is mathematical nonsense.

> the exact magnitude of this tolerance may perhaps be a matter for debate

> but either way it is set using Internal`$EqualTolerance, which takes a

> machine real value indicating the number of decimal digits' tolerance that

> should be applied, i.e. Log[2]/Log[10] times the number of least

> significant bits one wishes to ignore. This setting has been discussed in

> this forum at least once in the past: see e.g.

> <http://forums.wolfram.com/mathgroup/archive/2009/Dec/msg00013.html>.

I suspect there are more insightful discussions in the archives.

>

> Note that if one wishes to be more rigorous when determining equality,

> SameQ operates in a similar manner to Equal for numeric comparands, except

> that its tolerance is 1 (binary) ulp. This is also adjustable, via

> Internal`$SameQTolerance.

It is nice that one can try to scrape out all the garbage of the

arithmetic by setting internal flags, but the way it is set up in

Mathematica, the ordinary user with default system settings is exposed

to a really defective arithmetic system.

>

> In regard to erroneous results: undoubtedly it is a possibility. However,

> one would expect that an approximate first order method for dealing with

> error propagation should at least be better in the majority of cases than

> a zeroth-order method such as working in fixed precision.

You might think that, but the choice is not so clear.

If you set $MinPrecision=$MaxPrecision

then

(i = 1.100000000000000000; Do[(i = 2*i - i; Print[i]), {4}])

gives Overflow[] 4 times.

In the default setting, $MinPRecision is 0.

then

(i = 1.100000000000000000; Do[(i = 2*i - i; Print[i]), {50}]

sets i to decreasingly precise values, ending in 0., 0., 0X10^1, ... 0X10^5

(try it.. it is only one line!)

What is the naive user to do?

If you think this example is a good result, imagine what would happen to

a naive user (most are) who has a more elaborate program (most are)

which internally produces an answer " 0. " which is entirely bogus?

As stated

> previously, if one desires more accurate approximations then one is in any

> case free to implement them, although given the above it should be clear

> that all that is generally possible within the domain of finite-precision

> numbers is a reasonable approximation unless other information is

> available from which to make stronger deductions.

Quite the opposite. All numbers in a computer calculation should be

considered exact unless other information is available from which to

make WEAKER deductions. Then the kind of deductions (based, perhaps, on

knowledge of physical measurements or uncertainties) can be incorporated

in the calculation.

I will also note that

> none of the example "problems" in this present topic are anything directly

> to do with significance arithmetic; they instead represent a combination

> of confusion due to Mathematica's (admittedly confusing) choice of

> notation,

I think that is right.

combined with an apparent misunderstanding of concepts related

> to multiprecision arithmetic in general.

I think that the concepts of multiprecision arithmetic as implemented

in Mathematica are different from other implementations in general.

Bringing such outside knowledge to bear on an utterance in Mathematica

is hazardous.

Jul 9, 2011, 7:37:23 AM7/9/11

to

On Fri, 08 Jul 2011 09:51:53 +0100, Noqsi <noqsiae...@gmail.com> wrote:

> On Jul 7, 5:40 am, "slawek" <sla...@host.pl> wrote:

>

>> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

>> Mathematica (the computer program).

>

> Not true. This is a long-standing convention in experimental science.

>

Unfortunately so, given that it is severely erroneous: see e.g.

<http://www.av8n.com/physics/uncertainty.htm>. However, Mathematica's

approximation of how these uncertainties propagate is first-order, not

zeroth-order. This does not make it completely reliable, of course, but

certainly it is not almost always wrong as is the significant digits

convention. Within the bounds of its own applicability, Mathematica's

approximation is reasonable, although it would still be a mistake to apply

it to experimental uncertainty analysis given the much broader scope of

the latter.

Jul 10, 2011, 5:08:00 AM7/10/11

to

Is using "`" an acceptable work-around?

a = 2.`30; b = Sqrt[2]; c = a b;

check[x_] := {x, Precision[x]}

check[a] {2.00000000000000000000000000000, 30.}

check[b] {Sqrt[2], \[Infinity]}

check[c] {2.82842712474619009760337744842, 30.}

-FranD

Slawek wrote:

> Let Mathematica (6.x, 7.x) compute quite a simple

> product

>

> In[1]:= N[N[2.0, 20] * N[Sqrt[2] , 20], 20]

>

> Out[1]= 2.82843

>

> This is a bug.

>

> Why?

>

> Now we analyze it in details:

>

> 1. N[2.0,20] should give 2 with

> accuracy/precision/whatsever about 20

> decimal digits, i.e. 2.00000000000000000000

>

Jul 12, 2011, 7:01:05 AM7/12/11

to

Użytkownik "DrMajorBob" <btr...@austin.rr.com> napisał w wiadomości grup

dyskusyjnych:iv6hgo$sch$1...@smc.vnet.net...

> Binary storage is not mathematically required, but the result is MUCH

> faster code. Perhaps, at some point, we'll abandon this trade-off when

> computers are so fast that the gain in speed isn't worth the confusion.

> But that day isn't here, yet.

BCD - Binary Coded Decimals.

Jul 12, 2011, 7:02:07 AM7/12/11

to

Użytkownik "Oleksandr Rasputinov" <oleksandr_...@hmamail.com> napisał

w wiadomości grup dyskusyjnych:iv6h68$s97$1...@smc.vnet.net...

> considered too verbose), I do not think Mathematica's way of doing things

> is particularly arbitrary or confusing in the broader context of

If 1.4 is not the same as 14/10, then Mathematica should evaluate 1.4 ==

14/10 as False.

Jul 12, 2011, 7:01:36 AM7/12/11

to

Użytkownik "Noqsi" <noqsiae...@gmail.com> napisał w wiadomości grup

dyskusyjnych:iv6gf9$ruo$1...@smc.vnet.net...

There is no such convention. There is always possible that 2.00 +- 0.53 .

Nobody should believe that 2.00 is more exact than 2.0 or even 2 . (If so,

then 2 Pi have got 10% std. dev. ;)

Jul 13, 2011, 3:11:53 AM7/13/11

to

On Jul 9, 4:37 am, "Oleksandr Rasputinov"

<oleksandr_rasputi...@hmamail.com> wrote:

> On Fri, 08 Jul 2011 09:51:53 +0100, Noqsi <noqsiaerosp...@gmail.com> wrote:

> > On Jul 7, 5:40 am, "slawek" <sla...@host.pl> wrote:

>

> >> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

> >> Mathematica (the computer program).

>

> > Not true. This is a long-standing convention in experimental science.

>

> Unfortunately so, given that it is severely erroneous:

<oleksandr_rasputi...@hmamail.com> wrote:

> On Fri, 08 Jul 2011 09:51:53 +0100, Noqsi <noqsiaerosp...@gmail.com> wrote:

> > On Jul 7, 5:40 am, "slawek" <sla...@host.pl> wrote:

>

> >> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

> >> Mathematica (the computer program).

>

> > Not true. This is a long-standing convention in experimental science.

>

> Unfortunately so, given that it is severely erroneous:

Such an unqualified statement is unjustified. Whether this produces

troublesome error or not depends on what you're doing.

> see e.g.

> <http://www.av8n.com/physics/uncertainty.htm>.

I have some sympathy with this point of view. But I recall a rather

vehement dispute between two colleagues in a tame case where recording

only significant digits was fine. That dispute poisoned human

relationships unnecessarily. The real folly is to be ideological about

this, instead of understanding what you're doing *from the perspective

of the problem domain*. One size does not fit all.

> However, Mathematica's

> approximation of how these uncertainties propagate is first-order, not

> zeroth-order. This does not make it completely reliable, of course, but

> certainly it is not almost always wrong as is the significant digits

> convention. Within the bounds of its own applicability, Mathematica's

> approximation is reasonable, although it would still be a mistake to apply

> it to experimental uncertainty analysis given the much broader scope of

> the latter.

Experimental uncertainty analysis almost always involves

approximation. Avoiding shortcuts in the statistical formulation can

lead to extremely large calculations even when the data set is very

small. See, for example, Loredo and Lamb 1989, bibcode 1989NYASA.

571..601L. This example still required numerical approximation. In

this case, I think the enormous human and computational effort was

justified, but the appropriateness of any particular approach depends

on the problem to be solved.

Jul 13, 2011, 3:13:56 AM7/13/11

to

On 7/12/11 at 6:59 AM, sla...@host.pl (slawek) wrote:

>U=C2=BFytkownik "Oleksandr Rasputinov" <oleksandr_...@hmamail.com>

>napisa=C2=B3 w wiadomo=C2=B6ci grup dyskusyjnych:iv6h68$s97$1...@smc.vnet.net...

>>considered too verbose), I do not think Mathematica's way of doing

>>things is particularly arbitrary or confusing in the broader

>>context of

>If 1.4 is not the same as 14/10, then Mathematica should evaluate

>1.4 == 14/10 as False.

The documentation for Equal (==) specifically states:

Approximate numbers with machine precision or higher are

considered equal if they differ in at most their last seven

binary digits (roughly their last two decimal digits).

Since the exact value 14/10 differs from less than the last

seven bits of the binary representation of 1.4 14/10 == 1.4

returns true.

Note, by default whenever both machine precision values and

exact values are in the same expression, Mathematica evaluates

the expression as if everything was machine precision. And in

general this is a good thing.

The other choices would seem to be either leave 1.4 == 14/10

unevaluated or to return False. Both seem undesirable as they

would likely cause far more problem than returning True as

Mathematica presently does. Either of the other choices would

certainly make expressions containing both exact and approximate

values much more problematic to evaluate.

Ultimately, since the developers are unlikely to change such a

fundamental aspect of Mathematica, the only sensible thing is to

understand how Mathematica does things if you want to

effectively use Mathematica. The alternative would be to find a

system that operates more to your liking.

Jul 13, 2011, 3:17:00 AM7/13/11

to

On 7/12/2011 4:01 AM, slawek wrote:

> U=BFytkownik "DrMajorBob"<btr...@austin.rr.com> napisa=B3 w wiadomo=B6=

ci grup

> dyskusyjnych:iv6hgo$sch$1...@smc.vnet.net...

>> Binary storage is not mathematically required, but the result is MUCH

>> faster code. Perhaps, at some point, we'll abandon this trade-off when

>> computers are so fast that the gain in speed isn't worth the confusion=

.

>> But that day isn't here, yet.

>

> BCD - Binary Coded Decimals.

>

>

> U=BFytkownik "DrMajorBob"<btr...@austin.rr.com> napisa=B3 w wiadomo=B6=

ci grup

> dyskusyjnych:iv6hgo$sch$1...@smc.vnet.net...

>> Binary storage is not mathematically required, but the result is MUCH

>> faster code. Perhaps, at some point, we'll abandon this trade-off when

.

>> But that day isn't here, yet.

>

> BCD - Binary Coded Decimals.

>

>

IEEE 754-2008 is an update to the floating-point standard that not only

includes binary, but decimal arithmetic (and radix-independent). This

is presumably implemented in hardware, or intended to be implemented in

hardware.

At least one computer algebra system does software arbitrary precision

floats in what looks like decimal. Though I think it is base 1000 coded

in 1024 bits or some such thing.

For more reading on IEEE use google or just check out

http://en.wikipedia.org/wiki/IEEE_754-2008

Jul 13, 2011, 3:12:24 AM7/13/11

to

On Jul 12, 4:01 am, "slawek" <sla...@host.pl> wrote:

> U=BFytkownik "Noqsi" <noqsiaerosp...@gmail.com> napisa=B3 w wiadomo=B6ci grup

> dyskusyjnych:iv6gf9$ru...@smc.vnet.net...

>

> > On Jul 7, 5:40 am, "slawek" <sla...@host.pl> wrote:

>

> >> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

> >> Mathematica (the computer program).

>

> > Not true. This is a long-standing convention in experimental science.

>

> U=BFytkownik "Noqsi" <noqsiaerosp...@gmail.com> napisa=B3 w wiadomo=B6ci grup

> dyskusyjnych:iv6gf9$ru...@smc.vnet.net...

>

> > On Jul 7, 5:40 am, "slawek" <sla...@host.pl> wrote:

>

> >> The convention that 2.0 is less accurate than 2.00 is applied ONLY in

> >> Mathematica (the computer program).

>

> > Not true. This is a long-standing convention in experimental science.

>

> There is no such convention.

You mean you're not familiar with it. But it exists.

> There is always possible that 2.00 +- 0.53 .

That's a different convention. It also exists.

> Nobody should believe that 2.00 is more exact than 2.0 or even 2 . (If so,

> then 2 Pi have got 10% std. dev. ;)

There are different traditions here. Those who use the "2.00" to

indicate that further digits are unknown do generally not find "2 Pi"

confusing. The clash of conventions in engineering, though, is more

troublesome, where the units change by factors of 1000 and fractions

are often avoided. So, does "200 mm" mean "0.2 m" or "0.200 m"? Still,

this rarely causes serious confusion.

Jul 14, 2011, 5:22:10 AM7/14/11

to

On 7/13/2011 12:13 AM, Bill Rowe wrote:

> On 7/12/11 at 6:59 AM, sla...@host.pl (slawek) wrote:

>

>> U=C2=BFytkownik "Oleksandr Rasputinov"<oleksandr_...@hmamail.com>

>> napisa=C2=B3 w wiadomo=C2=B6ci grup dyskusyjnych:iv6h68$s97$1...@smc.vnet.net...

>>> considered too verbose), I do not think Mathematica's way of doing

>>> things is particularly arbitrary or confusing in the broader

>>> context of

>

>> If 1.4 is not the same as 14/10, then Mathematica should evaluate

>> 1.4 == 14/10 as False.

>

> The documentation for Equal (==) specifically states:

>

> Approximate numbers with machine precision or higher are

> considered equal if they differ in at most their last seven

> binary digits (roughly their last two decimal digits).

> On 7/12/11 at 6:59 AM, sla...@host.pl (slawek) wrote:

>

>> U=C2=BFytkownik "Oleksandr Rasputinov"<oleksandr_...@hmamail.com>

>> napisa=C2=B3 w wiadomo=C2=B6ci grup dyskusyjnych:iv6h68$s97$1...@smc.vnet.net...

>>> considered too verbose), I do not think Mathematica's way of doing

>>> things is particularly arbitrary or confusing in the broader

>>> context of

>

>> If 1.4 is not the same as 14/10, then Mathematica should evaluate

>> 1.4 == 14/10 as False.

>

> The documentation for Equal (==) specifically states:

>

> Approximate numbers with machine precision or higher are

> considered equal if they differ in at most their last seven

> binary digits (roughly their last two decimal digits).

You mean "considered Equal[] by Mathematica.

>

> Since the exact value 14/10 differs from less than the last

> seven bits of the binary representation of 1.4 14/10 == 1.4

> returns true.

>

> Note, by default whenever both machine precision values and

> exact values are in the same expression, Mathematica evaluates

> the expression as if everything was machine precision. And in

> general this is a good thing.

>

> The other choices would seem to be either leave 1.4 == 14/10

> unevaluated or to return False.

Clearly Rasputinov thinks that if they are not equal they should not be

Equal. Thus the answer is False.

Both seem undesirable as they

> would likely cause far more problem than returning True as

> Mathematica presently does.

Actually, you are assuming users are routinely comparing floats and

exact numbers for equality which they should not be doing anyway.

Programmers in FORTRAN are told to not test for equality involving floats.

Either of the other choices would

> certainly make expressions containing both exact and approximate

> values much more problematic to evaluate.

Or less, depending on what you expect for numbers.

>

> Ultimately, since the developers are unlikely to change such a

> fundamental aspect of Mathematica, the only sensible thing is to

> understand how Mathematica does things if you want to

> effectively use Mathematica. The alternative would be to find a

> system that operates more to your liking.

It might be fun to test to see if any of your code broke if you did this:

Unprotect[Equal]

Equal[a_Real,b_]:= Equal[Rationalize[SetAccuracy[a,Infinity]],b]

Equal[a_,b_Real]:= Equal[a,Rationalize[SetAccuracy[b,Infinity]]]

For example,

0.1d0 is exactly p= 3602879701896397/36028797018963968

so my new and improved Equal thinks that

0.1d0 and 1/10 are NOT equal, (indeed, they differ by 1/180143985094819840)

but

0.1d0 and p ARE equal.

So the question is: would any of YOUR code break if you used this patch

on Equal? Really?

RJF

>

>

Jul 14, 2011, 5:23:11 AM7/14/11

to

On 7/13/2011 12:11 AM, Noqsi wrote:

..

..

>> see e.g.

>> <http://www.av8n.com/physics/uncertainty.htm>.

>

>

..

Learning mathematics from a physicist is hazardous.

Learning computer science from a physicist is hazardous too.

Numbers in a computer are different from experimental measurements.

nevertheless, I like this article. It says, among other things,

The technique of propagating the uncertainty from step to step

throughout the calculation is a very bad technique. It might sometimes

work for super-simple “textbook” problems but it is unlikely to work for

real-world problems.

Jul 14, 2011, 9:18:56 PM7/14/11

to

Dear group!

The number of posts has already become too large to read every of them, so

I appologize if I repeat something which is already said but I think some

people would like to see Equal to behave like SameQ.

Some now might think it is too much to write

In[1]:= 1.4===14/10

Out[1]= False

instead of:

In[2]:= 1.4==14/10

Out[2]= True

so one way would be to overwrite Equal itself:

In[2]:= Unprotect[Equal]

Out[2]= {Equal}

In[3]:= Equal[a_,b__]:=SameQ[a,b]

In[4]:= Protect[Equal]

Out[4]= {Equal}

In[5]:= 1.4==14/10

Out[5]= False

My question is: how dangerous is it to overwrite such a fundamental

function as Equal?

Best,

Christoph

Jul 14, 2011, 9:19:27 PM7/14/11

to

On 14 Jul 2011, at 11:21, Richard Fateman wrote:

> On 7/13/2011 12:11 AM, Noqsi wrote:

> ..

>

>>> see e.g.

>>> <http://www.av8n.com/physics/uncertainty.htm>.

>>

>>

> ..

> Learning mathematics from a physicist is hazardous.

> Learning computer science from a physicist is hazardous too.

> Numbers in a computer are different from experimental measurements.

>

>

> nevertheless, I like this article. It says, among other things,

>

> The technique of propagating the uncertainty from step to step

> throughout the calculation is a very bad technique. It might sometimes

> work for super-simple textbook problems but it is unlikely to work for

> real-world problems.

>

Well, here is a quote from a very well known book on numerical analysis by a mathematician (Henrici, "Elements of Numerical Analysis").

> It is plain that, on a given machine and for a given problem, the local

> rounding errors are not, in fact, random variables. If the same problem

> is run on the same machine a number of times, there will result always the

> same local rounding errors, and therefore also the same accumulated

> error. We may, however, adopt a stochastic model of the propagation of

> rounding error, where the local errors are treated as if they were random

> variables. This stochastic model has been applied in the literature to a

> number of different numerical problems and has produced results that are

> in complete agreement with experimentally observed results in several

> important cases.

The book then describes the statistical method of error propagation of which Mathematica's approach can be regarded as a first order approximation (as pointed out by Oleksandr Rasputinov, who should not be confused with the OP of this thread so:

> Clearly Rasputinov thinks that if they are not equal they should not be

> Equal. Thus the answer is False.

is *clearly* False. In fact Oleksander expressed something closer to the opposite view.)

And if one quote is not enough, here is another, from another text on numerical analysis. (Conte, de Boor, ""Elementary Numerical Analysis).

It describes 4 approaches to error analysis, interval arithmetic, significant-digit arithmetic, the "statistical approach" and backward error analysis. Here is what it says about the second and the third one:

> A third approach is significant-digit arithmetic. As pointed out earlier, whenever two nearly equal machine numbers are subtracted, there is a danger that some significant digits will be lost. In significant-digit arithmetic an attempt is made to keep track of digits so lost. In one version

> only the significant digits in any number are retained, all others being discarded. At the end of a computation we will thus be assured that all digits retained are significant. The main objection to this method is that some information is lost whenever digits are discarded, and that the results obtained are likely to be much too conservative. Experimentation with this technique is still going on, although the experience to date is not too promising.

>

> A fourth method which gives considerable promise of providing an adequate mathematical theory of round-off-error propagation is based on a statistical approach. It begins with the assumption that round-off errors are independent. This assumption is, of course, not valid, because if the same problem is run on the same machine several times, the answers will always be the same. We can, however, adopt a stochastic model of the propagation of round-off errors in which the local errors are treated as if they were random variables. Thus we can assume that the local round-off errors are either uniformly or normally distributed between their extreme values. Using statistical methods, we can then obtain the standard devia- tion, the variance of distribution, and estimates of the accumulated round- off error. The statistical approach is considered in some detail by Ham- ming [1] and Henrici [2]. The method does involve substantial analysis and additional computer time, but in the experiments conducted to date it has obtained error estimates which are in remarkable agreement with experimentally available evidence.

The fundamental paper of Mathematica's error propagation is "Precise numerical computation" by Mark Sofroniou and Giulia Spaletta in The Journal of Logic and Algebraic Programming 64 (2005) 113=96134. This paper describes Mathematica's "significance arithmetic" as a first order approximation to Interval Arithmetic. It makes no mention of distributions. Oleksandr Rasputionov, in an earlier post here, interpreted "significance arithmetic" as a first order approximation to the fourth method above. I have not considered this very carefully, but it seems pretty clear that he is right, and that the two "first order" approximations are in fact isomorphic. The first order approach is, of course, justified on grounds of performance. It is perfectly "rigorous" in the same sense as any "first order" approach is (i.e. taking a linear approximation to the Taylor series of a non-linear function). It works fine under certain conditions and will produce nonsense when these conditions do not hold.

The fact that significance arithmetic is "useful" needs no justification other than the fact that it is used successfully by NSolve and Reduce in achieving validated symbolic results by numerical methods which are vastly faster than purely symbolic ones.

It is also useful, for users such as myself, who sometimes need fast first order error anlysis.

I have lots of posts by Richard on this topic (or perhaps it was the same post lots of time, it's so hard to tell), but I have never understood what his main point is. It seems to me that is because he himself has not yet decided this, although he has been posting on this topic for over 20 years (I think).

Sometimes he seems to be disparaging significance arithmetic itself. When Daniel points out how effective it is in his implementation of numerical Groebner basis, or in Adam Strzebonski's work on Reduce, he either ignores this altogether or claims that Groebner bases, etc. are themselves not "useful".

On other occasions he takes on the role of the defender of the interest of the "naive user" (presumably like the OP, who however would be better described as "intentionally naive") who is going to be confused by the "quirky" nature of significance arithmetic (at low precision). In doing so he conveniently ignores that fact of the existence of thousands of "naive users" who never become confused (sometimes because they always work with machine precision numbers and only use significance arrhythmic unknowingly, e.g. when using Reduce). And moreover, for those who do find a need for some sort of error analysis he offers no alternative, except perhaps to learn backward error analysis. Except, of course, that should they do so they would no longer be "naive" and thus outside of Richard's area of concern. And in any case, anyone who needs and understands backward error analysis can use it now, and can't imagine that even Richard would claim that reducing users' options is a good thing.

Finally, perhaps all that Richard is so upset about is simply Mathematica's habit of defining numbers as "fuzz balls" or "distributions". In other words, if Mathematica used a more usual "definition" of number, and used significance arithmetic for "error" propagation, that would be done by applying some separate function or turning on an option, than everything would be fine? If that is all, than it seems to me that Richard has for years been making mountains of molehills.

Andrzej Kozlowski

Jul 14, 2011, 9:23:04 PM7/14/11

to

On Jul 14, 2:23 am, Richard Fateman <fate...@cs.berkeley.edu> wrote:

> On 7/13/2011 12:11 AM, Noqsi wrote:

> ..

>

> >> see e.g.

> >> <http://www.av8n.com/physics/uncertainty.htm>.

>

> ..

> Learning mathematics from a physicist is hazardous.

> On 7/13/2011 12:11 AM, Noqsi wrote:

> ..

>

> >> see e.g.

> >> <http://www.av8n.com/physics/uncertainty.htm>.

>

> ..

> Learning mathematics from a physicist is hazardous.

In some ways. But if you want to relate mathematics to reality, you

might do well to consider the viewpoints of physicists.

> Learning computer science from a physicist is hazardous too.

Learning computer science from computer scientists is in some ways

worse, unless your interest is pointless bit pushing. Matthew 23:24 is

very relevant (some things never change).

In extracting useful results from computers in the real world

ignorance of the application domain is more debilitating than

ignorance of computer science. One can muddle through the latter, but

not the former.

Consider the howler you yourself committed a little while ago,

discussing the role of computers in the Apollo moon landings:

> you've got to wonder how the Russians, probably WITHOUT much in the way

> of computers, put up an artificial satellite.

This illustrates a profound ignorance of the problem. If you don't

care what orbit you go into, the computations are readily performed by

slide rule ahead of time. But Apollo had to perform a series of

precise maneuvers using imprecise rockets, with repeated re-

computation of the trajectories. For the early missions, they didn't

even know the lunar gravity very well, so actual orbits diverged

rapidly from predictions even when the initial conditions were known.

Apollo's indirect "lunar orbital rendezvous" approach was thus a

triumph of computation. Even the Saturn V was not big enough to

support the less computationally intensive direct approach.

Perhaps the "programmer" I learned the most from in a long career was

a physicist whose code was littered with silly (from a CS point of

view) constructions like:

TA=(COS(EA)-EC)/(1.-EC*COS(EA))

IF(ABS(TA).GT.1.) TA=SIGN(.99999,TA)

TA=ACOS(TA)

What was so great about his code? It's that every program he wrote was

an illuminating exercise in extracting important knowledge from

measurable information. The sloppy technique didn't matter so much. He

put rigor into the place it really counted: serving the needs of his

research. There were several much better technical programmers in that

research group, but they were not as good at conceptualizing how to

actually *use* the computer, rather than simply programming it.

> Numbers in a computer are different from experimental measurements.

But the experimental measurements relate much better to reality.

>

> nevertheless, I like this article. It says, among other things,

>

> The technique of propagating the uncertainty from step to=

step

> throughout the calculation is a very bad technique. It might sometimes

> work for super-simple =93textbook=94 problems but it is unlikely to work =

for

> real-world problems.

Except that error propagation techniques are used successfully in many

fields. Your cell phone works because engineers found a good balance

of power consumption and radio sensitivity from error propagation

methods, rather than the impractical method of tracking each electron

through the circuits. Getting back to orbits, one extremely useful

application of error propagation is to use it "backwards" to determine

which observations would best improve knowledge of an orbit.

There is no universal method for tracking uncertainty that is accurate

and practical. Your own ideologically favored method, interval

arithmetic, yields unrealistically large estimates of error in many

cases, and that can be a very bad thing. Or it can be useful to have

an upper bound. What's good depends on what the *application* needs,

not some ivory tower ideology.

I am *really* tired of your smug, patronizing attitude. You're a blind

man attempting to explain a rainbow. Why not, instead of whining all

the time that Mathematica doesn't conform to your profoundly narrow

notions of what computation is, spend some time with it actually

computing something of relevance to the real world? If you were

actually interested in applications, you would *rejoice* in the fact

that there are a variety of approaches available. But instead, you

obviously see Mathematica as a threat to your narrow ideology.

Jul 14, 2011, 9:25:07 PM7/14/11

to

On Jul 14, 2011, at 11:21 AM, Richard Fateman <fat...@cs.berkeley.edu> wrote:

> Learning mathematics from a physicist is hazardous.

> Learning computer science from a physicist is hazardous too.

> Numbers in a computer are different from experimental measurements.

I believe that I won't be the only one who objects to this hyperbole and limited world view. I've learned poor quality math from poor quality instructors---but this is not "hazardous.". I've learned what I believe to be high quality math from excellent physics and mathematical physics instructors.

Every experimental measurement of a continuously varying quantity will have some imprecision. Entering that number into a computer may or may not add to the imprecision. Keeping track of the sources of imprecision is what good experimentalist does.

Of course, an experiment is different from a number; but the experimentally d

etermined number and its sources of imprecision in a computer is just another number on the computer. There is no difference in meaning unless the precision of the experiment exceeds that of the measurement.

WCC

Jul 14, 2011, 9:26:08 PM7/14/11

to

On 7/14/2011 7:12 AM, W. Craig Carter wrote:

> On Jul 14, 2011, at 11:21 AM, Richard Fateman<fat...@cs.berkeley.edu> wrote:

>> Learning mathematics from a physicist is hazardous.

>> Learning computer science from a physicist is hazardous too.

>> Numbers in a computer are different from experimental measurements.

> I believe that I won't be the only one who objects to this hyperbole and limited world view. I've learned poor quality math from poor quality instructors---but this is not "hazardous.". I've learned what I believe to be high quality math from excellent physics and mathematical physics instructors.

Generally hazardous. You may however be lucky. I too have had some > On Jul 14, 2011, at 11:21 AM, Richard Fateman<fat...@cs.berkeley.edu> wrote:

>> Learning mathematics from a physicist is hazardous.

>> Learning computer science from a physicist is hazardous too.

>> Numbers in a computer are different from experimental measurements.

> I believe that I won't be the only one who objects to this hyperbole and limited world view. I've learned poor quality math from poor quality instructors---but this is not "hazardous.". I've learned what I believe to be high quality math from excellent physics and mathematical physics instructors.

excellent physics instructors, but it is generally hazardous, judging

from papers I've read, and programs I've looked at written by

physicists. By the way, I was a physics major as an undergraduate.

> Every experimental measurement of a continuously varying quantity will have some imprecision. Entering that number into a computer may or may not add to the imprecision.

Assume we are talking about floating-point representation. Next assume

that the number is scaled so that it is within range of some

(floating-point) representation. The external number can then be stored

in the computer with at most one rounding error, good to half a unit in

the last place (ULP). The precision of this number is related to the

floating-point data format, and varies depending upon

single/double/extended/ software-arbitrary-precision etc choices.

If you, additionally, wish to represent the uncertainty [[NOT

PRECISION]] of the number in the computer, you can do so by storing

more data in the computer.

> Keeping track of the sources of imprecision is what good experimentalist does.

computational result is often a goal of a numerical computational

scientist. Estimating the computed error would depend on the uncertainty

of the input and the computational steps.

> Of course, an experiment is different from a number; but the experimentally determined number and its sources of imprecision in a computer is just another number on the computer.

Yes, though I would again not use "imprecision" but "uncertainty". I

would also distinguish between the uncertainty of the input and the

potential for subsequent computational errors.

> There is no difference in meaning unless the precision of the experiment exceeds that of the measurement.

measurements, and computing an average, to get an extra

decimal digit??? I don't see how that is related to numerical

computation (as on a computer).

Fundamentally, taking the rules of thumb that you learn in an

undergraduate physics lab about uncertainties in reading a meter-stick

or a thermometer, etc, sort of work if you are only doing a few

arithmetic operations on them and you know personally that the numbers

are not correlated. If you put a few thousand numbers in a computer and

do a few billion arithmetic operations on them, the same rules don't

work so well.

RJF

Jul 14, 2011, 9:26:38 PM7/14/11

to

On 7/14/2011 6:22 AM, Christoph Lhotka wrote:

> Dear group!

>

....

> Dear group!

>

....

> My question is: how dangerous is it to overwrite such a fundamental

> function as Equal?

>

> Best,

>

> Christoph

>

An excellent question. Anyone who runs Mathematica in serious

applications could try it

and report back.

Unfortunately, SameQ doesn't operate the way I suspect you think it does.

Consider a = 1.100000000000000000000000000; b=1.1

SameQ[a,b] is True.

Yet a and b are different. You can see this if you do

SetPrecision[a,100] and SetPrecision[b,100].

1.10000000000000000000000000000000000000117549435082228750796873653722\

2245677818665556772087521508752

1.10000000000000008881784197001252323389053344726562500000000000000000\

0000000000000000000000000000000

RJF

Jul 15, 2011, 4:12:25 AM7/15/11

to

You are clearly confusing significant digit arithmetic, which is not

what Mathematica uses, with significance arithmetic, which is a first

order approximation to interval arithmetic or distribution based

approach. Obviously you don't read the posts you reply to and confuse

both the posters and the contents of what they post. Here is a quote

from Oleksandr Rasputionov that makes this completely clear:

what Mathematica uses, with significance arithmetic, which is a first

order approximation to interval arithmetic or distribution based

approach. Obviously you don't read the posts you reply to and confuse

both the posters and the contents of what they post. Here is a quote

from Oleksandr Rasputionov that makes this completely clear:

Unfortunately so, given that it is severely erroneous: see e.g.

<http://www.av8n.com/physics/uncertainty.htm>. However, Mathematica's

approximation of how these uncertainties propagate is first-order, not

zeroth-order. This does not make it completely reliable, of course, but

certainly it is not almost always wrong as is the significant digits

convention. Within the bounds of its own applicability, Mathematica's

approximation is reasonable, although it would still be a mistake to apply

it to experimental uncertainty analysis given the much broader scope of

the latter.

Note the "first-order not zeroth-order". Also, do take a look at Sofroniou and Spaletta and then you may perhaps understand what "order" means and how "significance arithmetic" differs from the "significant digits" convention. Good grief, did you ever learn about the Taylor series? It must have been a long time ago, I take.

Andrzej Kozlowski

On 14 Jul 2011, at 19:55, Richard Fateman wrote:

> On 7/14/2011 6:27 AM, Andrzej Kozlowski wrote:

>> On 14 Jul 2011, at 11:21, Richard Fateman wrote:

>>

>>> On 7/13/2011 12:11 AM, Noqsi wrote:

>>> ..

>>>

>>>>> see e.g.

>>>>> <http://www.av8n.com/physics/uncertainty.htm>.

>>>>

>>> ..

>>> Learning mathematics from a physicist is hazardous.

>>> Learning computer science from a physicist is hazardous too.

>>> Numbers in a computer are different from experimental measurements.

>>>

>>>

>>> nevertheless, I like this article. It says, among other things,

>>>

>>> The technique of propagating the uncertainty from step to step

>>> throughout the calculation is a very bad technique. It might sometimes

>>> work for super-simple textbook problems but it is unlikely to work for

>>> real-world problems.

>>>

>> Well, here is a quote from a very well known book on numerical

analysis by a mathematician (Henrici, "Elements of Numerical Analysis").

>>

>>

>>> It is plain that, on a given machine and for a given problem, the local

>>> rounding errors are not, in fact, random variables. If the same problem

>>> is run on the same machine a number of times, there will result always the

>>> same local rounding errors, and therefore also the same accumulated

>>> error. We may, however, adopt a stochastic model of the propagation of

>>> rounding error, where the local errors are treated as if they were random

>>> variables. This stochastic model has been applied in the literature to a

>>> number of different numerical problems and has produced results that are

>>> in complete agreement with experimentally observed results in several

>>> important cases.

>>

>> The book then describes the statistical method of error propagation of which Mathematica's approach can be regarded as a first order approximation (as pointed out by Oleksandr Rasputinov, who should not be confused with the OP of this thread so:

>

> So are we to conclude that Henrici recommends this as a general numerical computational method?

>

> I don't see that here. I think what he is saying is that if you do some mathematics (see below), then

> you will get results consistent with what you will get if you actually run the experiment on the computer.

> This is not surprising. It is a result that says that "theoretical" numerical analysis agrees with

> "computer experiments" in arithmetic. It doesn't say Henrici recommends running a computation this way.

>

> When Henrici says "adopt a stochastic model ...." he doesn't mean to write a program. He means to

> think about each operation like this.. (I show for multiplication of numbers P and Q with errors a b resp.)

>

> P*(1+a) times Q*(1+b) P*Q *(1+a)*(1+b)* (1+c) where c is a new "error" bounded by roundoff, e.g. half unit in last place.

>

> For each operation in the calculation, make up another error letter... a,b,c,d,e,f,g...

> assume they are uncorrelated.

>

> The fact that this theoretical approach and numerically running "several important cases" is a statement

> about correlation of roundoff in these cases, not a statement of advisability of whatever for a model of

> how to write a computer system.

>

> By the way, I think that Henrici was an extremely fine theoretical numerical analyst, and a fine writer too.

>

>

>

>>> Clearly Rasputinov thinks that if they are not equal they should not be

>>> Equal. Thus the answer is False.

>> is *clearly* False. In fact Oleksander expressed something closer to the opposite view.)

> This thread is too long. I don't know at this point if you are agreeing that it is false or contradicting that "is False" is false.

>> And if one quote is not enough, here is another, from another text on numerical analysis. (Conte, de Boor, ""Elementary Numerical Analysis).

>> It describes 4 approaches to error analysis, interval arithmetic, significant-digit arithmetic, the "statistical approach" and backward error analysis. Here is what it says about the second and the third one:

> Huh, if we are talking about the second and third one, why does he say third and fourth?

> Are you using 0-based indexing and deBoor is using 1-based indexing????

>

>

>>> A third approach is significant-digit arithmetic. As pointed out earlier, whenever two nearly equal machine numbers are subtracted, there is a danger that some significant digits will be lost. In significant-digit arithmetic an attempt is made to keep track of digits so lost. In one version

>>> only the significant digits in any number are retained, all others being discarded. At the end of a computation we will thus be assured that all digits retained are significant. The main objection to this method is that some information is lost whenever digits are discarded, and that the results obtained are likely to be much too conservative. Experimentation with this technique is still going on, although the experience to date is not too promising.

>>>

> OK, so deBoor (who is retired and therefore not likely to revise the "experience to date") says this method "is not too promising".

> This sounds to me like he is not endorsing what Mathematica does.

>

>>> A fourth method which gives considerable promise of providing an adequate mathematical theory of round-off-error propagation is based on a statistical approach. It begins with the assumption that round-off errors are independent. This assumption is, of course, not valid, because if the same problem is run on the same machine several times, the answers will always be the same. We can, however, adopt a stochastic model of the propagation of round-off errors in which the local errors are treated as if they were random variables. Thus we can assume that the local round-off errors are either uniformly or normally distributed between their extreme values. Using statistical methods, we can then obtain the standard devia- tion, the variance of distribution, and estimates of the accumulated round- off error. The statistical approach is considered in some detail by Ham- ming [1] and Henrici [2]. The method does involve substantial analysis and additional computer time, but in the experiments conducted to date it has obtained error estimates which are in remarkable agreement with experimentally available evidence.

> deBoor is essentially quoting Henrici, and this statistical approach is to say that all those error terms I mentioned above, a,b,c,d,e,f,...

> can be chosen from some distribution (the way I've written it, a ,....,z .... would essentially be chosen from {-u,u} where u = 2^(-W) where the fraction part of the floating-point number is W bits. ) and you can compute the final expression as ANSWER+<somehorrendousfunctionof>(a,b,c,....).

>

> What deBoor says is that this (theoretical numerical analysis) "method" promises to provide

> a theory of round-off error propagation. He is not saying this is a practical method for scientific computing. When he uses the work "method"

> he means a mathematical method for analyzing roundoff.

>

> In any case, Mathematica does not do this. I would further argue that Mathematica makes it hard to carry out the experiments that might be done to demonstrate that this theory applied in any particular sample computation.

>>

>> The fundamental paper of Mathematica's error propagation is "Precise numerical computation" by Mark Sofroniou and Giulia Spaletta in The Journal of Logic and Algebraic Programming 64 (2005) 113=96134. This paper describes Mathematica's "significance arithmetic" as a first order approximation to Interval Arithmetic. It makes no mention of distributions.

> I thought I read this paper in some Mathematica documentation or conference proceedings.

>> Oleksandr Rasputionov, in an earlier post here, interpreted "significance arithmetic" as a first order approximation to the fourth method above.

> Huh? First of all, the original poster was slawek. Rasputinov seems to think that Mathematica numbers are like Intervals (basically a good intuition until you think about equality.) and refers to them as distributions. This is not deBoor's 4th "method" of theoretically analyzing round-off.

> In fact it is deBoor's 1st method, interval arithmetic. This has the advantage of being maybe 4 to 8 times slower than regular arithmetic, and also has a huge literature (see "Reliable Computation") describing variations, advantages, disadvantages, etc.

>

>> I have not considered this very carefully, but it seems pretty clear that he is right, and that the two "first order" approximations are in fact isomorphic.

> Uh, this is unclear, unless you mean that Mathematica's number system is essentially interval arithmetic, but with a confusing front end.

>> The first order approach is, of course, justified on grounds of performance. It is perfectly "rigorous" in the same sense as any "first order" approach is (i.e. taking a linear approximation to the Taylor series of a non-linear function). It works fine under certain conditions and will produce nonsense when these conditions do not hold.

>>

>> The fact that significance arithmetic is "useful" needs no justification other than the fact that it is used successfully by NSolve and Reduce in achieving validated symbolic results by numerical methods which are vastly faster than purely symbolic ones.

> Really?

>

> 1. This does not mean that other methods, e.g. validated methods for accurately evaluating polynomials (etc) NOT based on significance arithmetic would not be faster and better. DanL claims it is used and useful there, so that's nice. BUT..

>

> 2. This does not mean that this arithmetic should be used by default by user arithmetic.

>> It is also useful, for users such as myself, who sometimes need fast first order error anlysis.

> OK, If you find it useful yourself, fine. I daresay you are not a typical user.

>

>> I have lots of posts by Richard on this topic (or perhaps it was the same post lots of time, it's so hard to tell), but I have never understood what his main point is.

>

> Main points: Mathematica's number system is non-standard, peculiar, hard to explain, capable of returning mysterious non-answers without indications of error to innocent users, a bad foundation for building higher-level functionality.

>

> I have other criticisms of Mathematica, but I think that the sentence above is enough for you to process today.

>

>> It seems to me that is because he himself has not yet decided this, although he has been posting on this topic for over 20 years (I think).

>

>> Sometimes he seems to be disparaging significance arithmetic itself.

> I think deBoor did that.

>

>> When Daniel points out how effective it is in his implementation of numerical Groebner basis, or in Adam Strzebonski's work on Reduce, he either ignores this altogether or claims that Groebner bases, etc. are themselves not "useful".

> I think Reduce is a very nice program when it works. If I am not mistaken, all work on numerical Groebner basis should be reducible to the evaluation of polynomials, for which there are faster and more accurate methods available not using significance arithmetic. On the other hand, I might be mischaracterizing DanL work, since I admit to not having studied it.

>

>

>> On other occasions he takes on the role of the defender of the interest of the "naive user" (presumably like the OP, who however would be better described as "intentionally naive") who is going to be confused by the "quirky" nature of significance arithmetic (at low precision).

> No, I don't think that slawek was a "troll". I think he was genuinely confused, as might anyone be who has some prior computer arithmetic exposure and for the first time encounters a system with exact rational arithmetic.

>> In doing so he conveniently ignores that fact of the existence of thousands of "naive users" who never become confused (sometimes because they always work with machine precision numbers and only use significance arrhythmic unknowingly, e.g. when using Reduce).

> Most naive users don't tread near the dangerous spots. Some naive users never notice that their answers are nonsense.

> Every so often we get a naive user who DOES notice, and he/she sends email to this newsgroup.

>> And moreover, for those who do find a need for some sort of error analysis he offers no alternative, except perhaps to learn backward error analysis.

> Actually, there's a whole bunch of libraries of methods with error estimates for common computations, where backward error analysis or some other method was used to provide extra information.

>

>> Except, of course, that should they do so they would no longer be "naive" and thus outside of Richard's area of concern.

> They would probably not be writing in Mathematica.

>> And in any case, anyone who needs and understands backward error analysis can use it now, and can't imagine that even Richard would claim that reducing users' options is a good thing.

> I have no problem with Mathematica providing as an option, some other arithmetic. It is WRI that has made it rather hard for the user to

> figure out how to "use the arithmetic of the rest of the world".

>

>> Finally, perhaps all that Richard is so upset about is simply Mathematica's habit of defining numbers as "fuzz balls" or "distributions".

> I'm not sure that "upset" is the right term. After all, I don't have to use Mathematica for numerical computation. And I rarely do.

>> In other words, if Mathematica used a more usual "definition" of number, and used significance arithmetic for "error" propagation, that would be done by applying some separate function or turning on an option, than everything would be fine?

> Actually, that's pretty close to correct.

>> If that is all, than it seems to me that Richard has for years been making mountains of molehills.

> So why are you (and WRI) so opposed to this notion? Note that it would also have to affect other parts of the system including of course,

> Equal and friends.

>

>

>>

>> Andrzej Kozlowski

>>

>>

>>

>>

>

>

Jul 15, 2011, 4:09:20 AM7/15/11

to

Richard, thanks for pointing this out.

>From what you say, it seems that the documentation for SameQ is incorrect.

>From what you say, it seems that the documentation for SameQ is incorrect.

On Thu, Jul 14, 2011 at 6:21 PM, Richard Fateman

<fat...@eecs.berkeley.edu>wrote:

Jul 15, 2011, 4:10:53 AM7/15/11

to

On 7/14/2011 6:23 PM, Noqsi wrote:

> On Jul 14, 2:23 am, Richard Fateman<fate...@cs.berkeley.edu> wrote:

>> On 7/13/2011 12:11 AM, Noqsi wrote:

>> ..

>>

>>>> see e.g.

>>>> <http://www.av8n.com/physics/uncertainty.htm>.

>>

>> ..

>> Learning mathematics from a physicist is hazardous.

>

> In some ways. But if you want to relate mathematics to reality, you

> might do well to consider the viewpoints of physicists.

>

>> Learning computer science from a physicist is hazardous too.

>

> Learning computer science from computer scientists is in some ways

> worse, unless your interest is pointless bit pushing. Matthew 23:24 is

> very relevant (some things never change).

> On Jul 14, 2:23 am, Richard Fateman<fate...@cs.berkeley.edu> wrote:

>> On 7/13/2011 12:11 AM, Noqsi wrote:

>> ..

>>

>>>> see e.g.

>>>> <http://www.av8n.com/physics/uncertainty.htm>.

>>

>> ..

>> Learning mathematics from a physicist is hazardous.

>

> In some ways. But if you want to relate mathematics to reality, you

> might do well to consider the viewpoints of physicists.

>

>> Learning computer science from a physicist is hazardous too.

>

> Learning computer science from computer scientists is in some ways

> worse, unless your interest is pointless bit pushing. Matthew 23:24 is

> very relevant (some things never change).

I suppose it depends on what you mean by computer science.

>

> In extracting useful results from computers in the real world

> ignorance of the application domain is more debilitating than

> ignorance of computer science. One can muddle through the latter, but

> not the former.

I think you may be confusing "computer science" with "applications of

computers to whatever [e.g. physics, engineering, business]" which is

usually taught in the department of 'whatever'.

>

> Consider the howler you yourself committed a little while ago,

> discussing the role of computers in the Apollo moon landings:

>

>> you've got to wonder how the Russians, probably WITHOUT much in the way

>> of computers, put up an artificial satellite.

>

> This illustrates a profound ignorance of the problem. If you don't

> care what orbit you go into, the computations are readily performed by

> slide rule ahead of time. But Apollo had to perform a series of

> precise maneuvers using imprecise rockets, with repeated re-

> computation of the trajectories. For the early missions, they didn't

> even know the lunar gravity very well, so actual orbits diverged

> rapidly from predictions even when the initial conditions were known.

> Apollo's indirect "lunar orbital rendezvous" approach was thus a

> triumph of computation. Even the Saturn V was not big enough to

> support the less computationally intensive direct approach.

Do you know for a fact that the Russians didn't care what orbit the

first manned satellite had?

>

> Perhaps the "programmer" I learned the most from in a long career was

> a physicist whose code was littered with silly (from a CS point of

> view) constructions like:

>

> TA=(COS(EA)-EC)/(1.-EC*COS(EA))

> IF(ABS(TA).GT.1.) TA=SIGN(.99999,TA)

> TA=ACOS(TA)

>

> What was so great about his code? It's that every program he wrote was

> an illuminating exercise in extracting important knowledge from

> measurable information. The sloppy technique didn't matter so much. He

> put rigor into the place it really counted: serving the needs of his

> research. There were several much better technical programmers in that

> research group, but they were not as good at conceptualizing how to

> actually *use* the computer, rather than simply programming it.

I think you are confusing application knowledge with computer science.

It is fairly clear that computer scientists cannot be held responsible

for the content of all programs.

>

>> Numbers in a computer are different from experimental measurements.

>

> But the experimental measurements relate much better to reality.

The computation deals with representations in the computer. Mapping

those representations to the external world is a separate matter that

deals with sensors, actuators (robots, displays, computer-controlled

instruments, sound boards, digital cameras, scanners, microphones...)

These are usually parts of some other engineering discipline.

>

>>

>> nevertheless, I like this article. It says, among other things,

>>

>> The technique of propagating the uncertainty from step to=

> step

>> throughout the calculation is a very bad technique. It might sometimes

>> work for super-simple =93textbook=94 problems but it is unlikely to work =

> for

>> real-world problems.

>

> Except that error propagation techniques are used successfully in many

> fields.

A simple problem perhaps.

Your cell phone works because engineers found a good balance

> of power consumption and radio sensitivity from error propagation

> methods, rather than the impractical method of tracking each electron

> through the circuits.

Of course there are many methods that can be programmed. You are

assuming that analysis of signals is done by some kind of particle

tracking? I assume that programs are designed by persons familiar with

differential equations and electromagnetic radiation, as well as more

seat-of-the-pants stuff like antenna design and sun spots.

Also I assume

that cell phones use signal strength and feedback, and do not need great

accuracy. Though maybe GPS stuff is tricky if you have few

triangulation points. Using 10 points, maybe not so tricky. Not

something I've cared to look at.

Anyway, after a few billion computations, significance arithmetic tends

to lose.

Getting back to orbits, one extremely useful

> application of error propagation is to use it "backwards" to determine

> which observations would best improve knowledge of an orbit.

>

> There is no universal method for tracking uncertainty that is accurate

> and practical.

Ah, so you are saying that Mathematica is not accurate and practical??

Your own ideologically favored method, interval

> arithmetic, yields unrealistically large estimates of error in many

> cases, and that can be a very bad thing. Or it can be useful to have

> an upper bound. What's good depends on what the *application* needs,

> not some ivory tower ideology.

It's not my favorite. I point it out as a method that has been widely

studied. It does not provide estimates of error. It provides bounds on

error, and those bounds may be very pessimistic.

>

> I am *really* tired of your smug, patronizing attitude. You're a blind

> man attempting to explain a rainbow. Why not, instead of whining all

> the time that Mathematica doesn't conform to your profoundly narrow

> notions of what computation is, spend some time with it actually

> computing something of relevance to the real world?

My concern is that someone attempting to compute something of relevance

will fall into a real-world hole.

If you were

> actually interested in applications, you would *rejoice* in the fact

> that there are a variety of approaches available. But instead, you

> obviously see Mathematica as a threat to your narrow ideology.

I am primarily interested in building systems appropriate for a range of

applications. (That's more computer science). From that perspective I

think that Mathematica falls short. I don't see that as a threat, but I

am inclined to object to statements that claim (in my view incorrectly)

that Mathematica (arithmetically speaking) is the best, or even the only

way to do floating-point calculations. In some ways the Mathematica

system is just fine, if you use its library routines for special

functions for arbitrary precision, and you are within the appropriate

ranges where they actually deliver what is promised. (I have mixed

experiences near singular points...)

But I think I've stated my perspective pretty clearly, even if

patronizingly.

RJF

>

>

Jul 15, 2011, 5:07:40 AM7/15/11

to

yes, indeed!

On 15/07/2011 09:34, Andrzej Kozlowski wrote:

> Well, here is one example of what would happen:

>

> In[3]:= Unprotect[Equal]

>

> Out[3]= {Equal}

>

> In[5]:= Equal[a_, b__] := SameQ[a, b]

>

> In[6]:= Protect[Equal]

>

> Out[6]= {Equal}

>

> In[7]:= Solve[3 x == 1, x]

>

> Out[7]= {}

>

> Doesn't look like a great idea to me.

>

> Andrzej Kozlowski

>

>

>

>

> On 15 Jul 2011, at 03:18, Christoph Lhotka wrote:

>

>> Dear group!

>>

>> The number of posts has already become too large to read every of them, so

>> I appologize if I repeat something which is already said but I think some

>> people would like to see Equal to behave like SameQ.

>>

>> Some now might think it is too much to write

>>

>> In[1]:= 1.4===14/10

>> Out[1]= False

>>

>> instead of:

>>

>> In[2]:= 1.4==14/10

>> Out[2]= True

>>

>> so one way would be to overwrite Equal itself:

>>

>> In[2]:= Unprotect[Equal]

>> Out[2]= {Equal}

>>

>> In[3]:= Equal[a_,b__]:=SameQ[a,b]

>>

>> In[4]:= Protect[Equal]

>> Out[4]= {Equal}

>>

>> In[5]:= 1.4==14/10

>> Out[5]= False

>>

>> My question is: how dangerous is it to overwrite such a fundamental

>> function as Equal?

>>

>> Best,

>>

>> Christoph

>>

>>

>>> Clearly Rasputinov thinks that if they are not equal they should not be

>>> Equal. Thus the answer is False.

>>>

Jul 15, 2011, 5:07:09 AM7/15/11

to

Jul 16, 2011, 5:43:56 AM7/16/11

to

On 7/14/11 at 5:20 AM, fat...@cs.berkeley.edu (Richard Fateman)

wrote:

wrote:

>On 7/13/2011 12:13 AM, Bill Rowe wrote:

>>On 7/12/11 at 6:59 AM, sla...@host.pl (slawek) wrote:

>>The documentation for Equal (==) specifically states:

>>Approximate numbers with machine precision or higher are considered

>>equal if they differ in at most their last seven binary digits

>>(roughly their last two decimal digits).

>You mean "considered Equal[] by Mathematica.

No. I meant exactly what I wrote. The excerpt was copy/paste

from the documentation for Equal. Now if you want to interpret

the documentation fine. But this is what is in the

documentation. And if your comment is to the effect something

other than Mathematica might treat 1.4 as unequal to 14/10, so

what? The existence of other ways to treating this has no impact

on what is in the Mathematica documentation or how Mathematica works.

>>Ultimately, since the developers are unlikely to change such a

>>fundamental aspect of Mathematica, the only sensible thing is to

>>understand how Mathematica does things if you want to effectively

>>use Mathematica. The alternative would be to find a system that

>>operates more to your liking.

>It might be fun to test to see if any of your code broke if you did

>this:

>Unprotect[Equal] Equal[a_Real,b_]:=

>Equal[Rationalize[SetAccuracy[a,Infinity]],b] Equal[a_,b_Real]:=

>Equal[a,Rationalize[SetAccuracy[b,Infinity]]]

Fun?? Modifying built-in Mathematica commands is risky. To much

of Mathematica uses other Mathematica functions. I could do then

this spend lots of time testing and getting results that lead me

to believe there is no impact only to get burned much later and

likely have a very difficult time determining why something

didn't work as expected. This isn't my idea of fun. I have far

better things to do with my time.

>For example, 0.1d0 is exactly p= 3602879701896397/36028797018963968

>so my new and improved Equal thinks that 0.1d0 and 1/10 are NOT

>equal, (indeed, they differ by 1/180143985094819840)

>but

>0.1d0 and p ARE equal.

>So the question is: would any of YOUR code break if you used this

>patch on Equal? Really?

I don't know the answer. And since I assume you don't have

access to Mathematica's source code, you cannot know the answer

either. I have no intention of attempting to determine the

answer for the reasons stated above. So, a discussion of whether

this patch would or would not break some code is pointless to me.

Jul 16, 2011, 5:45:28 AM7/16/11

to

On 15 Jul 2011, at 16:49, Richard Fateman wrote:

> On 7/14/2011 11:55 PM, Andrzej Kozlowski wrote:

>

> Gee, Andrzej, all I can think of is the childhood playground chant (I don't know where you might have been a child, so this may not

> bring back memories...)

>

> "I'm rubber, you're glue; everything You say sticks to YOU!"

Yes, I can also think of a few playground chants that would apply nicely, but unfortunately you would not understand them. So let's have something English instead, like "Sticks and stones...".

Now, I really have no time or patience to go over all your misrepresentations in detail. But let me point out (or remind you perhaps) that the already mentioned paper of Sofroniou and Spaletta makes the following remark:

> The choice of significance arithmetic as the default in Mathematica has not been universally favorable (see for example [9]) although some of these criticisms relate to early deficiencies in the implementation.

Reference [9] is, of course:

[9] R.J. Fateman, A review of Mathematica, J. Symbolic Comput. 13 (1992) 545=96579.

In fact the "early" version was much more like "significant digits convention", while the current version is quite a lot more sophisticated, as described in the article. Somehow you don't seem to have noticed it since that is the "early version" that you always seem to be describing. (Significant digits is something that used to be taught in primary school, if I recall correctly).

As for difference between theoretical analysis of the propagation of round-off error and practical scientific computations, I am well aware of it, but was it not you who gleefully quoted the statement:

The technique of propagating the uncertainty from step to step

throughout the calculation is a very bad technique. It might sometimes

work for super-simple textbook problems but it is unlikely to work for

real-world problems.

So if agree with this (as you seem to), are you implying that the entire chapter 16 of Henrici's book is concerned with something that only applies to "super-simple text book problems"?

The point is, of course, that you never make it clearly what exactly you are talking about probably because you find it convenient for rhetorical purposes (90% of these discussions are really just rhetorics).

I should add that I myself long ago and more then once wrote on this forum that I do not think Mathematica's "approximate numbers" are a good model for empirical errors, but they are very good tools for solving certain purely mathematical problems. Often it can be accomplished in 100% provably correct way (barring bugs of course).

As for things like numerical Groebner basis etc, you keep arguing that it probably can be done in some other way (without significance arithmetic) and in a certain, rather trivial, sense you undoubtedly right. But the fact is, as Daniel has pointed out, that, in practice, nobody seems to have done it in another way, and not for want of trying. So again here we have the distinction between "theory" and "practical application" but except that this time the shoe is on the other foot.

Andrzej

>> You are clearly confusing significant digit arithmetic, which is not what Mathematica uses, with significance arithmetic, which is a first order approximation to interval arithmetic or distribution based approach.

>

> I view significant digit arithmetic, if by digit you allow binary digit or "bit", to be a crude version of interval arithmetic where the endpoints of the interval are not numbers of (essentially) arbitrary precision, but where the endpoints must be represented by the central point plus or minus some power of two. It is an approximation to interval arithmetic. I think it is a reasonable idea to restrict the =

number of digits carried in the value of the interval endpoints (making =

them "cruder" than you might really know) in the interests of efficiency =

in time and space. Thus using properly rounded machine floats is =

appropriate, at least when the range is suitable and the difference =

between upper and lower bounds is large. Using just one bit makes the =

width of the interval grow faster than using more bits...

>

> Since you do not understand the context of the paragraphs you have =

quoted, nor do you seem to acknowledge the difference between =

theoretical analysis of the propagation of round-off error in =

well-chosen standard examples, and somehow running interval arithmetic =

all the time, it is hard to know how to educate you. How comforting is =

your proud quote, below? Mathematica's arithmetic is "not ... =

completely reliable". Rasputinov says,

> further, "it is not almost always wrong". I'm not sure why he says =

the significant digit convention is almost always wrong unless he means =

almost always too pessimistic. I'm also not aware of why Rasputinov's =

opinion should sway the discussion especially (sorry Oleksandr).

>

>

> My point remains:

> WRI could have a much more direct notion of number, and equality, that =

would make it easy to implement my choice of arithmetic or theirs or =

yours.

> They didn't. The default is not completely reliable. People write to =

this newsgroup, periodically, saying Huh? what's going on? I found a =

bug!

>

> As for the experimental and computational notions of (im)precision, I =

think there is an issue of using the same words with different meanings. =

Similar but unfortunately different. Precision of a floating point =

number F is simply the number of bits in the fraction. If you impute =

some uncertainty to F, you can store that in another piece of data D in =

the computer. If you want to assert that F and D together represent a =

distribution of a certain kind you can also compute with that.

>

> RJF

>

>> Obviously you don't read the posts you reply to and confuse both the =

posters and the contents of what they post. Here is a quote from =

Oleksandr Rasputionov that makes this completely clear:

>>

>>

>> Unfortunately so, given that it is severely erroneous: see e.g.

>> <http://www.av8n.com/physics/uncertainty.htm>. However, Mathematica's

>> approximation of how these uncertainties propagate is first-order, =

not

>> zeroth-order. This does not make it completely reliable, of course, =

but

>> certainly it is not almost always wrong as is the significant digits

>> convention. Within the bounds of its own applicability, Mathematica's

>> approximation is reasonable, although it would still be a mistake to =

apply

>> it to experimental uncertainty analysis given the much broader scope =

of

>> the latter.

>>

>> Note the "first-order not zeroth-order". Also, do take a look at =

Sofroniou and Spaletta and then you may perhaps understand what "order" =

means and how "significance arithmetic" differs from the "significant =

digits" convention. Good grief, did you ever learn about the Taylor =

series? It must have been a long time ago, I take.

>>

>>

>> Andrzej Kozlowski

>>

>>

>> On 14 Jul 2011, at 19:55, Richard Fateman wrote:

>>

>>> On 7/14/2011 6:27 AM, Andrzej Kozlowski wrote:

>>>> On 14 Jul 2011, at 11:21, Richard Fateman wrote:

>>>>

>>>>> On 7/13/2011 12:11 AM, Noqsi wrote:

>>>>> ..

>>>>>

>>>>>>> see e.g.

>>>>>>> <http://www.av8n.com/physics/uncertainty.htm>.

>>>>> ..

>>>>> Learning mathematics from a physicist is hazardous.

>>>>> Learning computer science from a physicist is hazardous too.

>>>>> Numbers in a computer are different from experimental =

measurements.

>>>>>

>>>>>

>>>>> nevertheless, I like this article. It says, among other things,

>>>>>

>>>>> The technique of propagating the uncertainty from step to step

>>>>> throughout the calculation is a very bad technique. It might =

sometimes

>>>>> work for super-simple =84textbook=89 problems but it is unlikely =

to work for

>>>>> real-world problems.

>>>>>

>>>> Well, here is a quote from a very well known book on numerical =

analysis by a mathematician (Henrici, "Elements of Numerical Analysis").

>>>>

>>>>

>>>>> It is plain that, on a given machine and for a given problem, the =

local

>>>>> rounding errors are not, in fact, random variables. If the same =

problem

>>>>> is run on the same machine a number of times, there will result =

always the

>>>>> same local rounding errors, and therefore also the same =

accumulated

>>>>> error. We may, however, adopt a stochastic model of the =

propagation of

>>>>> rounding error, where the local errors are treated as if they were =

random

>>>>> variables. This stochastic model has been applied in the =

literature to a

>>>>> number of different numerical problems and has produced results =

that are

>>>>> in complete agreement with experimentally observed results in =

several

>>>>> important cases.

>>>> The book then describes the statistical method of error propagation =

of which Mathematica's approach can be regarded as a first order =

approximation (as pointed out by Oleksandr Rasputinov, who should not be =

confused with the OP of this thread so:

>>> So are we to conclude that Henrici recommends this as a general =

numerical computational method?

>>>

>>> I don't see that here. I think what he is saying is that if you do =

some mathematics (see below), then

>>> you will get results consistent with what you will get if you =

actually run the experiment on the computer.

>>> This is not surprising. It is a result that says that "theoretical" =

numerical analysis agrees with

>>> "computer experiments" in arithmetic. It doesn't say Henrici =

recommends running a computation this way.

>>>

>>> When Henrici says "adopt a stochastic model ...." he doesn't mean =

to write a program. He means to

>>> think about each operation like this.. (I show for multiplication =

of numbers P and Q with errors a b resp.)

>>>

>>> P*(1+a) times Q*(1+b) = P*Q *(1+a)*(1+b)* (1+c) where c is a =

new "error" bounded by roundoff, e.g. half unit in last place.

>>>

>>> For each operation in the calculation, make up another error =

letter... a,b,c,d,e,f,g...

>>> assume they are uncorrelated.

>>>

>>> The fact that this theoretical approach and numerically running =

"several important cases" is a statement

>>> about correlation of roundoff in these cases, not a statement of =

advisability of whatever for a model of

>>> how to write a computer system.

>>>

>>> By the way, I think that Henrici was an extremely fine theoretical =

numerical analyst, and a fine writer too.

>>>

>>>

>>>

>>>>> Clearly Rasputinov thinks that if they are not equal they should =

not be

>>>>> Equal. Thus the answer is False.

>>>> is *clearly* False. In fact Oleksander expressed something closer =

to the opposite view.)

>>> This thread is too long. I don't know at this point if you are =

agreeing that it is false or contradicting that "is False" is false.

>>>> And if one quote is not enough, here is another, from another text =

on numerical analysis. (Conte, de Boor, ""Elementary Numerical =

Analysis).

>>>> It describes 4 approaches to error analysis, interval arithmetic, =

significant-digit arithmetic, the "statistical approach" and backward =

error analysis. Here is what it says about the second and the third one:

>>> Huh, if we are talking about the second and third one, why does he =

say third and fourth?

>>> Are you using 0-based indexing and deBoor is using 1-based =

indexing????

>>>

>>>

>>>>> A third approach is significant-digit arithmetic. As pointed out =

earlier, whenever two nearly equal machine numbers are subtracted, there =

is a danger that some significant digits will be lost. In =

significant-digit arithmetic an attempt is made to keep track of digits =

so lost. In one version

>>>>> only the significant digits in any number are retained, all others =

being discarded. At the end of a computation we will thus be assured =

that all digits retained are significant. The main objection to this =

method is that some information is lost whenever digits are discarded, =

and that the results obtained are likely to be much too conservative. =

Experimentation with this technique is still going on, although the =

experience to date is not too promising.

>>>>>

>>> OK, so deBoor (who is retired and therefore not likely to revise the =

"experience to date") says this method "is not too promising".

>>> This sounds to me like he is not endorsing what Mathematica does.

>>>

>>>>> A fourth method which gives considerable promise of providing an =

adequate mathematical theory of round-off-error propagation is based on =

a statistical approach. It begins with the assumption that round-off =

errors are independent. This assumption is, of course, not valid, =

because if the same problem is run on the same machine several times, =

the answers will always be the same. We can, however, adopt a stochastic =

model of the propagation of round-off errors in which the local errors =

are treated as if they were random variables. Thus we can assume that =

the local round-off errors are either uniformly or normally distributed =

between their extreme values. Using statistical methods, we can then =

obtain the standard devia- tion, the variance of distribution, and =

estimates of the accumulated round- off error. The statistical approach =

is considered in some detail by Ham- ming [1] and Henrici [2]. The =

method does involve substantial analysis and additional computer time, =

but in the experiments conducted to date it has obtained error estimates =

which are in remarkable agreement with experimentally available =

evidence.

>>> deBoor is essentially quoting Henrici, and this statistical approach =

is to say that all those error terms I mentioned above, a,b,c,d,e,f,...

>>> can be chosen from some distribution (the way I've written it, a =

,....,z .... would essentially be chosen from {-u,u} where u = =

2^(-W) where the fraction part of the floating-point number is W bits. =

) and you can compute the final expression as =

ANSWER+<somehorrendousfunctionof>(a,b,c,....).

>>>

>>> What deBoor says is that this (theoretical numerical analysis) =

"method" promises to provide

>>> a theory of round-off error propagation. He is not saying this is =

a practical method for scientific computing. When he uses the work =

"method"

>>> he means a mathematical method for analyzing roundoff.

>>>

>>> In any case, Mathematica does not do this. I would further argue =

that Mathematica makes it hard to carry out the experiments that might =

be done to demonstrate that this theory applied in any particular =

sample computation.

>>>> The fundamental paper of Mathematica's error propagation is =

"Precise numerical computation" by Mark Sofroniou and Giulia Spaletta =

in The Journal of Logic and Algebraic Programming 64 (2005) 113=96134. =

This paper describes Mathematica's "significance arithmetic" as a first =

order approximation to Interval Arithmetic. It makes no mention of =

distributions.

>>> I thought I read this paper in some Mathematica documentation or =

conference proceedings.

>>>> Oleksandr Rasputionov, in an earlier post here, interpreted =

"significance arithmetic" as a first order approximation to the fourth =

method above.

>>> Huh? First of all, the original poster was slawek. Rasputinov seems =

to think that Mathematica numbers are like Intervals (basically a good =

intuition until you think about equality.) and refers to them as =

distributions. This is not deBoor's 4th "method" of theoretically =

analyzing round-off.

>>> In fact it is deBoor's 1st method, interval arithmetic. This has =

the advantage of being maybe 4 to 8 times slower than regular =

arithmetic, and also has a huge literature (see "Reliable Computation") =

describing variations, advantages, disadvantages, etc.

>>>

>>>> I have not considered this very carefully, but it seems pretty =

clear that he is right, and that the two "first order" approximations =

are in fact isomorphic.

>>> Uh, this is unclear, unless you mean that Mathematica's number =

system is essentially interval arithmetic, but with a confusing front =

end.

>>>> The first order approach is, of course, justified on grounds of =

performance. It is perfectly "rigorous" in the same sense as any "first =

order" approach is (i.e. taking a linear approximation to the Taylor =

series of a non-linear function). It works fine under certain conditions =

and will produce nonsense when these conditions do not hold.

>>>>

>>>> The fact that significance arithmetic is "useful" needs no =

justification other than the fact that it is used successfully by NSolve =

and Reduce in achieving validated symbolic results by numerical methods =

which are vastly faster than purely symbolic ones.

>>> Really?

>>>

>>> 1. This does not mean that other methods, e.g. validated methods for =

accurately evaluating polynomials (etc) NOT based on significance =

arithmetic would not be faster and better. DanL claims it is used and =

useful there, so that's nice. BUT..

>>>

>>> 2. This does not mean that this arithmetic should be used by default =

by user arithmetic.

>>>> It is also useful, for users such as myself, who sometimes need =

fast first order error anlysis.

>>> OK, If you find it useful yourself, fine. I daresay you are not a =

typical user.

>>>

>>>> I have lots of posts by Richard on this topic (or perhaps it was =

the same post lots of time, it's so hard to tell), but I have never =

understood what his main point is.

>>> Main points: Mathematica's number system is non-standard, peculiar, =

hard to explain, capable of returning mysterious non-answers without =

indications of error to innocent users, a bad foundation for building =

higher-level functionality.

>>>

>>> I have other criticisms of Mathematica, but I think that the =

sentence above is enough for you to process today.

>>>

>>>> It seems to me that is because he himself has not yet decided =

this, although he has been posting on this topic for over 20 years (I =

think).

>>>> Sometimes he seems to be disparaging significance arithmetic =

itself.

>>> I think deBoor did that.

>>>

>>>> When Daniel points out how effective it is in his implementation =

of numerical Groebner basis, or in Adam Strzebonski's work on Reduce, he =

either ignores this altogether or claims that Groebner bases, etc. are =

themselves not "useful".

>>> I think Reduce is a very nice program when it works. If I am not =

mistaken, all work on numerical Groebner basis should be reducible to =

the evaluation of polynomials, for which there are faster and more =

accurate methods available not using significance arithmetic. On the =

other hand, I might be mischaracterizing DanL work, since I admit to not =

having studied it.

>>>

>>>

>>>> On other occasions he takes on the role of the defender of the =

interest of the "naive user" (presumably like the OP, who however would =

be better described as "intentionally naive") who is going to be =

confused by the "quirky" nature of significance arithmetic (at low =

precision).

>>> No, I don't think that slawek was a "troll". I think he was =

genuinely confused, as might anyone be who has some prior computer =

arithmetic exposure and for the first time encounters a system with =

exact rational arithmetic.

>>>> In doing so he conveniently ignores that fact of the existence of =

thousands of "naive users" who never become confused (sometimes because =

they always work with machine precision numbers and only use =

significance arrhythmic unknowingly, e.g. when using Reduce).

>>> Most naive users don't tread near the dangerous spots. Some naive =

users never notice that their answers are nonsense.

>>> Every so often we get a naive user who DOES notice, and he/she sends =

email to this newsgroup.

>>>> And moreover, for those who do find a need for some sort of error =

analysis he offers no alternative, except perhaps to learn backward =

error analysis.

>>> Actually, there's a whole bunch of libraries of methods with error =

estimates for common computations, where backward error analysis or some =

other method was used to provide extra information.

>>>

>>>> Except, of course, that should they do so they would no longer be =

"naive" and thus outside of Richard's area of concern.

>>> They would probably not be writing in Mathematica.

>>>> And in any case, anyone who needs and understands backward error =

analysis can use it now, and can't imagine that even Richard would claim =

that reducing users' options is a good thing.

>>> I have no problem with Mathematica providing as an option, some =

other arithmetic. It is WRI that has made it rather hard for the user =

to

>>> figure out how to "use the arithmetic of the rest of the world".

>>>

>>>> Finally, perhaps all that Richard is so upset about is simply =

Mathematica's habit of defining numbers as "fuzz balls" or =

"distributions".

>>> I'm not sure that "upset" is the right term. After all, I don't =

have to use Mathematica for numerical computation. And I rarely do.

>>>> In other words, if Mathematica used a more usual "definition" of =

number, and used significance arithmetic for "error" propagation, that =

would be done by applying some separate function or turning on an =

option, than everything would be fine?

>>> Actually, that's pretty close to correct.

>>>> If that is all, than it seems to me that Richard has for years been =

making mountains of molehills.

>>> So why are you (and WRI) so opposed to this notion? Note that it =

Jul 16, 2011, 5:46:30 AM7/16/11

to

On 7/15/2011 11:54 AM, Andrzej Kozlowski wrote:

>

> On 15 Jul 2011, at 16:49, Richard Fateman wrote:

>

>> On 7/14/2011 11:55 PM, Andrzej Kozlowski wrote:

>>

>> Gee, Andrzej, all I can think of is the childhood playground chant

(I don't know where you might have been a child, so this may not

>> bring back memories...)

>>

>> "I'm rubber, you're glue; everything You say sticks to YOU!"

>

> Yes, I can also think of a few playground chants that would apply nicely,

but unfortunately you would not understand them. So let's have something English instead, like "Sticks and stones...".

>

> Now, I really have no time or patience to go over all your misrepresentations in detail. But let me point out (or remind you perhaps) that the already mentioned paper of Sofroniou and Spaletta makes the following remark:

>

>> The choice of significance arithmetic as the default in Mathematica has not been universally favorable (see for example [9]) although some of these criticisms relate to early deficiencies in the implementation.

>

> Reference [9] is, of course:

>

> [9] R.J. Fateman, A review of Mathematica, J. Symbolic Comput. 13 (1992) 545=E2=80=93579.>

> On 15 Jul 2011, at 16:49, Richard Fateman wrote:

>

>> On 7/14/2011 11:55 PM, Andrzej Kozlowski wrote:

>>

>> Gee, Andrzej, all I can think of is the childhood playground chant

(I don't know where you might have been a child, so this may not

>> bring back memories...)

>>

>> "I'm rubber, you're glue; everything You say sticks to YOU!"

>

> Yes, I can also think of a few playground chants that would apply nicely,

but unfortunately you would not understand them. So let's have something English instead, like "Sticks and stones...".

>

> Now, I really have no time or patience to go over all your misrepresentations in detail. But let me point out (or remind you perhaps) that the already mentioned paper of Sofroniou and Spaletta makes the following remark:

>

>> The choice of significance arithmetic as the default in Mathematica has not been universally favorable (see for example [9]) although some of these criticisms relate to early deficiencies in the implementation.

>

> Reference [9] is, of course:

>

>

> In fact the "early" version was much more like "significant digits convention", while the current version is quite a lot more sophisticated, as described in the article. Somehow you don't seem to have noticed it since that is the "early version" that you always seem to be describing. (Significant digits is something that used to be taught in primary school, if I recall correctly).

(x = 1.00000000000000000000; While[x != 0, x = 2*x - x; Print[x]])

terminates when Equal[x,0] is True.

And[x == 1, x == 0, x == -1]

returns True.

says it all.

> As for difference between theoretical analysis of the propagation of round-off error and practical scientific computations, I am well aware of it, but was it not you who gleefully quoted the statement:

>

> The technique of propagating the uncertainty from step to step

> throughout the calculation is a very bad technique. It might sometimes

> work for super-simple textbook problems but it is unlikely to work for

> real-world problems.

While you claim to understand the difference, you now show you don't.

The technique of replacing x*y by (x*(1+a)) * (y*(1+b))-->

z*(1+c) for

suitable a,b,c, for theoretical analysis of propagation of roundoff

as a function of a, b, is different from doing the actual

calculations with fuzz-balls.

> So if agree with this (as you seem to), are you implying that the entire chapter 16 of Henrici's book is concerned with something that only applies to "super-simple text book problems"?

I don't have a copy of his book handy. It is true that the

theoretical analysis becomes pretty hairy, e.g. for programs with

branches.

> The point is, of course, that you never make it clearly what exactly you are talking about probably because you find it convenient for rhetorical purposes (90% of these discussions are really just rhetorics).

>

> I should add that I myself long ago and more then once wrote on this forum that I do not think Mathematica's "approximate numbers" are a good model for empirical errors, but they are very good tools for solving certain purely mathematical problems. Often it can be accomplished in 100% provably correct way (barring bugs of course).

>

> As for things like numerical Groebner basis etc, you keep arguing that it probably can be done in some other way (without significance arithmetic) and in a certain, rather trivial, sense you undoubtedly right. But the fact is, as Daniel has pointed out, that, in practice, nobody seems to have done it in another way, and not for want of trying. So again here we have the distinction between "theory" and "practical application" but except that this time the shoe is on the other foot.

It seems to me that numerical computation of GB has a certain history.

E.g.

http://www.risc.jku.at/publications/download/risc_273/Nr.7_paper-revised.pdf

(see references)

or

http://www.maths.lth.se/vision/publdb/reports/pdf/byrod-josephson-etal-iccv-07.pdf

and several PhD dissertations.

Given that most people messing with floating-point GB find themselves

facing stability problems,

the introduction of arbitrary precision numbers seems like a way out

of some difficulties. Most

people seem to concentrate on other ways though, because double-floats

are so much faster

than software floats.

It is nice that Dan has done some software-float implementation and

put it in Mathematica. I am not aware of

others who have tried this tactic and failed. Even you seem to agree

that Mathematica's particular kind of

bigfloat arithmetic is not essential to this. So I don't see your point.

<big snip>

RJF

Jul 16, 2011, 5:47:31 AM7/16/11

to

On 07/08/2011 03:50 AM, Richard Fateman wrote:

> On 7/7/2011 4:35 AM, Daniel Lichtblau wrote:

>> On Jul 5, 4:15 am, Richard Fateman<fate...@cs.berkeley.edu> wrote:

>>> In brief: yes to both.

>>>

>>> On 7/4/2011 4:18 AM, Kevin J. McCann wrote:

>>> ...

>>>

>>>

>>>

>>>>> 1. N[2.0,20] should give 2 with accuracy/precision/whatsever about 20

>>>>> decimal digits, i.e. 2.00000000000000000000

>>>

>>> WRI will defend this as a feature.

>>>

>>> You thought that the semantics of N[] were the same as SetPrecision

>>> e.g.

>>>

>>> N[SetPrecision[2.0, 20]*N[Sqrt[2], 20], 20]

>>>

>>> works as you expected.

>>>

>>> So from your perspective, and from the perspective of anyone else who

>>> thinks along the same lines, it is a bug. I would prefer to call it

>>> a design error.

>>>

>>> 2.0 (indeed, any floating point number) is a perfectly respectable way

>>> of denoting a number that can be expressed in higher precision by adding

>>> binary zeros. WRI doesn't agree.

>>>

>>> RJF

>>

>> I don't think this holds up for all floats in the sense you seem to

>> indicate. Take 2.1 for example.

>

> My contention is that 2.1, written that way, is a perfectly respectable

> way of denoting a number that can be expressed in higher precision by

> adding binary zeros. 2.1 is a double-float in most systems today.

> What is that number? In decimal, to higher precision, it looks like

> 2.10000000000000008881784197001....

>

> A program that decodes a float into its component fraction, exponent,

> and sign gives this information for the fraction:

> 4728779608739021 or in binary

> 10000110011001100110011001100110011001100110011001101

>

> That is the number to which we add binary zeros.

>

> Let us call this number P.

> On 7/7/2011 4:35 AM, Daniel Lichtblau wrote:

>> On Jul 5, 4:15 am, Richard Fateman<fate...@cs.berkeley.edu> wrote:

>>> In brief: yes to both.

>>>

>>> On 7/4/2011 4:18 AM, Kevin J. McCann wrote:

>>> ...

>>>

>>>

>>>

>>>>> 1. N[2.0,20] should give 2 with accuracy/precision/whatsever about 20

>>>>> decimal digits, i.e. 2.00000000000000000000

>>>

>>>

>>> You thought that the semantics of N[] were the same as SetPrecision

>>> e.g.

>>>

>>> N[SetPrecision[2.0, 20]*N[Sqrt[2], 20], 20]

>>>

>>> works as you expected.

>>>

>>> So from your perspective, and from the perspective of anyone else who

>>> thinks along the same lines, it is a bug. I would prefer to call it

>>> a design error.

>>>

>>> 2.0 (indeed, any floating point number) is a perfectly respectable way

>>> of denoting a number that can be expressed in higher precision by adding

>>> binary zeros. WRI doesn't agree.

>>>

>>> RJF

>>

>> I don't think this holds up for all floats in the sense you seem to

>> indicate. Take 2.1 for example.

>

> My contention is that 2.1, written that way, is a perfectly respectable

> way of denoting a number that can be expressed in higher precision by

> adding binary zeros. 2.1 is a double-float in most systems today.

> What is that number? In decimal, to higher precision, it looks like

> 2.10000000000000008881784197001....

>

> A program that decodes a float into its component fraction, exponent,

> and sign gives this information for the fraction:

> 4728779608739021 or in binary

> 10000110011001100110011001100110011001100110011001101

>

> That is the number to which we add binary zeros.

>

> Let us call this number P.

This is what SetPrecision will do. N will not do that.

If I recall correctly (unlikely, by now) the original point of

contention was that N[] did not also do this. That is by design. N[]

will not invent new digits, binary or otherwise. For that one requires

the more arcane SetPrecision.

>> We can (and will) use SetPrecision to

>> pad with binary zeroes, just as you use it above on 2.0.

>>

>> In[606]:= N[SetPrecision[2.1, 20]*N[Sqrt[2], 20], 20]

>>

>> Out[606]= 2.9698484809834997281

>>

>> But this is not an accurate representation of 21/10 * Sqrt[2] to 20

>> places.

>

>

> Just because you are doing arithmetic on numbers represented to 20

> digits does not mean that the result of the arithmetic will be right to

> 20 digits, however, in the single operation of multiplication, you

> should not lose any digits. What's going on here?

>

> You simply computed two different quantities. Let S= sqrt(2) to 20

> decimal places. In line 606 you multiplied P*S and rounded it to 20

> digits [ actually you did something to the binary reps, but it doesn't

> matter here].

I was (and remain) fairly sure that your notion, applied to 2.1 rather

than 2.0, would be to do exactly this. I never claimed it was a good idea...

> In line 607, below, you multiplied 21/10 by S. Since P is not equal

> to 21/10, why should the results be the same? Oh, Mathematica thinks

> that P ==21/10, but that is merely the terribly broken notion of

> equality that Mathematica uses. We used to call that "close enough for

> government work". But it is terribly broken because == fails to

> satisfy the axioms of "equivalence relations" such as a==b and b==c

> implies a==c.

It satisfies trichotomy. That is to say, it plays nice with inequalities.

In any implementation of equality and inequality predicates that

involves approximate numbers, something from exact arithmetic must be

jettisoned. The choice we took might not be the one you would have

preferred. But it has its justifications.

I think we've discussed this before. (Recursively, I probably said this

the last time around.)

>> In[607]:= N[21/10*Sqrt[2], 20]

>> Out[607]= 2.9698484809834996025

>>

>> They are off by arould a machine epsilon. No surprise here, I think.

>>

>> In[608]:= % - %%

>> Out[608]= -1.256*10^-16

>>

>> I also think that this sort of numeric discrepancy is inevitable* in

>> any program that uses decimal input and binary representation.

>

> Absent any interior calculations, I would tend to agree. But there is

> something else going on. I think that Mathematica has embedded in it

> several design decisions that elevate the (typical)

> one-half-unit-in-last-place error in binary-to-decimal and

> decimal-to-binary conversion that happens in input or output, into a

> global slush bucket that infects and propagates into all interior

> computations with "software floats".

For input, I don't see that this slush bucket is any different than the

underpinnings of other approximate floating point arithmetic. For output

purposes it seems irrelevant. Maybe I'm misunderstanding the claim you

are making.

Daniel Lichtblau

Wolfram Research

Jul 16, 2011, 5:42:54 AM7/16/11

to

(in a different thread, where replacing Equal by SameQ was proposed)

On 7/15/2011 2:07 AM, Andrzej Kozlowski wrote:

> Well, here is one example of what would happen:

>

> In[3]:= Unprotect[Equal]

>

> Out[3]= {Equal}

>

> In[5]:= Equal[a_, b__] := SameQ[a, b]

>

> In[6]:= Protect[Equal]

>

> Out[6]= {Equal}

>

> In[7]:= Solve[3 x == 1, x]

>

> Out[7]= {}

>

> Doesn't look like a great idea to me.

>

> Andrzej Kozlowski

>

That's because 3*x==1 is immediately changed to False, and

Solve[False, x] is indeed not a great idea.

Equal[a_?NumberQ, b_?NumberQ] := SameQ[a, b];

works better. That Solve example works just fine.

Indeed, if one does this..

Equal[a_?NumberQ, b_?NumberQ] := (Print[test[a, b]]; SameQ[a, b]);

one can detect when, if ever, the Solve program calls Equal on two

numbers. It will print test[...] on such occasions.

When does Solve call Equal on 2 numbers? I poked around a little and

found -- never. I tried Integrate, Factor, Do, Product, Plot.. --never.

note that f[x_] := If [x == 0, 1, f[x - 1]*x]; f[5] computes 5! and

calls Equal 6 times... but Factorial[5] does not.

Perhaps the internal support routines do not call Equal?

or calls to it are compiled away and not in reach of redefinition by the

user? Or do they call SameQ? (no, they don't. I tried it).

Cheers

RJF

Jul 16, 2011, 5:40:49 AM7/16/11

to

I should have added that actually Mathematica's significance arithmetic

is considerably more sophisticated than the description as a "first

order" approximation suggests. This is also explained by Sofroniou and

Spaletta:

is considerably more sophisticated than the description as a "first

order" approximation suggests. This is also explained by Sofroniou and

Spaletta:

Numerical algorithms for computing elementary functions can be written

in terms of addition and multiplication at some level. However, relying

on the error propagation rules for these operations would often give

very pessimistic error bounds in significance arithmetic. Much tighter

bounds can be obtained by directly imposing error estimates based on

properties of functions during their numerical computation.

In other words, Mathematica uses special estimates for a variety of

functions, based on properties of these functions and thus obtains bounds

are considerably tighter than a simple minded approach would produce.

Andrzej Kozlowski

On 15 Jul 2011, at 10:09, Andrzej Kozlowski wrote:

> You are clearly confusing significant digit arithmetic, which is not

> what Mathematica uses, with significance arithmetic, which is a first

> order approximation to interval arithmetic or distribution based

> approach. Obviously you don't read the posts you reply to and confuse

> both the posters and the contents of what they post. Here is a quote

> from Oleksandr Rasputionov that makes this completely clear:

>

>

> Unfortunately so, given that it is severely erroneous: see e.g.

> <http://www.av8n.com/physics/uncertainty.htm>. However, Mathematica's

> approximation of how these uncertainties propagate is first-order, not

> zeroth-order. This does not make it completely reliable, of course, but

> certainly it is not almost always wrong as is the significant digits

> convention. Within the bounds of its own applicability, Mathematica's

> approximation is reasonable, although it would still be a mistake to apply

> it to experimental uncertainty analysis given the much broader scope of

> the latter.

>

> Note the "first-order not zeroth-order". Also, do take a look at Sofroniou and Spaletta and then you may perhaps understand what "order" means and how "significance arithmetic" differs from the "significant digits" convention. Good grief, did you ever learn about the Taylor series? It must have been a long time ago, I take.

>

>

> Andrzej Kozlowski

>

>

> On 14 Jul 2011, at 19:55, Richard Fateman wrote:

>

>> On 7/14/2011 6:27 AM, Andrzej Kozlowski wrote:

>>> On 14 Jul 2011, at 11:21, Richard Fateman wrote:

>>>

>>>> On 7/13/2011 12:11 AM, Noqsi wrote:

>>>> ..

>>>>

>>>>>> see e.g.

>>>>>> <http://www.av8n.com/physics/uncertainty.htm>.

>>>>>

>>>> ..

>>>> Learning mathematics from a physicist is hazardous.

>>>> Learning computer science from a physicist is hazardous too.

>>>> Numbers in a computer are different from experimental measurements.

>>>>

>>>>

>>>> nevertheless, I like this article. It says, among other things,

>>>>

>>>> The technique of propagating the uncertainty from step to step

>>>> throughout the calculation is a very bad technique. It might sometimes

>>>> work for super-simple textbook problems but it is unlikely to work for

>>>> real-world problems.

>>>>

>>> Well, here is a quote from a very well known book on numerical

> analysis by a mathematician (Henrici, "Elements of Numerical Analysis").

>>>

>>>

>>>> It is plain that, on a given machine and for a given problem, the local

>>>> rounding errors are not, in fact, random variables. If the same problem

>>>> is run on the same machine a number of times, there will result always the

>>>> same local rounding errors, and therefore also the same accumulated

>>>> error. We may, however, adopt a stochastic model of the propagation of

>>>> rounding error, where the local errors are treated as if they were random

>>>> variables. This stochastic model has been applied in the literature to a

>>>> number of different numerical problems and has produced results that are

>>>> in complete agreement with experimentally observed results in several

>>>> important cases.

>>>

>>> The book then describes the statistical method of error propagation of which Mathematica's approach can be regarded as a first order approximation (as pointed out by Oleksandr Rasputinov, who should not be confused with the OP of this thread so:

>>

>> So are we to conclude that Henrici recommends this as a general numerical computational method?

>>

>> I don't see that here. I think what he is saying is that if you do some mathematics (see below), then

>> you will get results consistent with what you will get if you actually run the experiment on the computer.

>> This is not surprising. It is a result that says that "theoretical" numerical analysis agrees with

>> "computer experiments" in arithmetic. It doesn't say Henrici recommends running a computation this way.

>>

>> When Henrici says "adopt a stochastic model ...." he doesn't mean to write a program. He means to

>> think about each operation like this.. (I show for multiplication of numbers P and Q with errors a b resp.)

>>

>> P*(1+a) times Q*(1+b) P*Q *(1+a)*(1+b)* (1+c) where c is a new "error" bounded by roundoff, e.g. half unit in last place.

>>

>> For each operation in the calculation, make up another error letter... a,b,c,d,e,f,g...

>> assume they are uncorrelated.

>>

>> The fact that this theoretical approach and numerically running "several important cases" is a statement

>> about correlation of roundoff in these cases, not a statement of advisability of whatever for a model of

>> how to write a computer system.

>>

>> By the way, I think that Henrici was an extremely fine theoretical numerical analyst, and a fine writer too.

>>

>>

>>

>>>> Clearly Rasputinov thinks that if they are not equal they should not be

>>>> Equal. Thus the answer is False.

>>> is *clearly* False. In fact Oleksander expressed something closer to the opposite view.)

>> This thread is too long. I don't know at this point if you are agreeing that it is false or contradicting that "is False" is false.

>>> And if one quote is not enough, here is another, from another text on numerical analysis. (Conte, de Boor, ""Elementary Numerical Analysis).

>>> It describes 4 approaches to error analysis, interval arithmetic, significant-digit arithmetic, the "statistical approach" and backward error analysis. Here is what it says about the second and the third one:

>> Huh, if we are talking about the second and third one, why does he say third and fourth?

>> Are you using 0-based indexing and deBoor is using 1-based indexing????

>>

>>

>>>> A third approach is significant-digit arithmetic. As pointed out earlier, whenever two nearly equal machine numbers are subtracted, there is a danger that some significant digits will be lost. In significant-digit arithmetic an attempt is made to keep track of digits so lost. In one version

>>>> only the significant digits in any number are retained, all others being discarded. At the end of a computation we will thus be assured that all digits retained are significant. The main objection to this method is that some information is lost whenever digits are discarded, and that the results obtained are likely to be much too conservative. Experimentation with this technique is still going on, although the experience to date is not too promising.

>>>>

>> OK, so deBoor (who is retired and therefore not likely to revise the "experience to date") says this method "is not too promising".

>> This sounds to me like he is not endorsing what Mathematica does.

>>

>>>> A fourth method which gives considerable promise of providing an adequate mathematical theory of round-off-error propagation is based on a statistical approach. It begins with the assumption that round-off errors are independent. This assumption is, of course, not valid, because if the same problem is run on the same machine several times, the answers will always be the same. We can, however, adopt a stochastic model of the propagation of round-off errors in which the local errors are treated as if they were random variables. Thus we can assume that the local round-off errors are either uniformly or normally distributed between their extreme values. Using statistical methods, we can then obtain the standard devia- tion, the variance of distribution, and estimates of the accumulated round- off error. The statistical approach is considered in some detail by Ham- ming [1] and Henrici [2]. The method does involve substantial analysis and additional computer time, but in the ex!

> periments conducted to date it has obtained error estimates which are in remarkable agreement with experimentally available evidence.

>> deBoor is essentially quoting Henrici, and this statistical approach is to say that all those error terms I mentioned above, a,b,c,d,e,f,...

>> can be chosen from some distribution (the way I've written it, a ,....,z .... would essentially be chosen from {-u,u} where u == 2^(-W) where the fraction part of the floating-point number is W bits. ) and you can compute the final expression as ANSWER+<somehorrendousfunctionof>(a,b,c,....).

>>

>> What deBoor says is that this (theoretical numerical analysis) "method" promises to provide

>> a theory of round-off error propagation. He is not saying this is a practical method for scientific computing. When he uses the work "method"

>> he means a mathematical method for analyzing roundoff.

>>

>> In any case, Mathematica does not do this. I would further argue that Mathematica makes it hard to carry out the experiments that might be done to demonstrate that this theory applied in any particular sample computation.

>>>

>>> The fundamental paper of Mathematica's error propagation is "Precise numerical computation" by Mark Sofroniou and Giulia Spaletta in The Journal of Logic and Algebraic Programming 64 (2005) 1139=6134. This paper describes Mathematica's "significance arithmetic" as a first order approximation to Interval Arithmetic. It makes no mention of distributions.

>> I thought I read this paper in some Mathematica documentation or conference proceedings.

>>> Oleksandr Rasputionov, in an earlier post here, interpreted "significance arithmetic" as a first order approximation to the fourth method above.

>> Huh? First of all, the original poster was slawek. Rasputinov seems to think that Mathematica numbers are like Intervals (basically a good intuition until you think about equality.) and refers to them as distributions. This is not deBoor's 4th "method" of theoretically analyzing round-off.

>> In fact it is deBoor's 1st method, interval arithmetic. This has the advantage of being maybe 4 to 8 times slower than regular arithmetic, and also has a huge literature (see "Reliable Computation") describing variations, advantages, disadvantages, etc.

>>

>>> I have not considered this very carefully, but it seems pretty clear that he is right, and that the two "first order" approximations are in fact isomorphic.

>> Uh, this is unclear, unless you mean that Mathematica's number system is essentially interval arithmetic, but with a confusing front end.

>>> The first order approach is, of course, justified on grounds of performance. It is perfectly "rigorous" in the same sense as any "first order" approach is (i.e. taking a linear approximation to the Taylor series of a non-linear function). It works fine under certain conditions and will produce nonsense when these conditions do not hold.

>>>

>>> The fact that significance arithmetic is "useful" needs no justification other than the fact that it is used successfully by NSolve and Reduce in achieving validated symbolic results by numerical methods which are vastly faster than purely symbolic ones.

>> Really?

>>

>> 1. This does not mean that other methods, e.g. validated methods for accurately evaluating polynomials (etc) NOT based on significance arithmetic would not be faster and better. DanL claims it is used and useful there, so that's nice. BUT..

>>

>> 2. This does not mean that this arithmetic should be used by default by user arithmetic.

>>> It is also useful, for users such as myself, who sometimes need fast first order error anlysis.

>> OK, If you find it useful yourself, fine. I daresay you are not a typical user.

>>

>>> I have lots of posts by Richard on this topic (or perhaps it was the same post lots of time, it's so hard to tell), but I have never understood what his main point is.

>>

>> Main points: Mathematica's number system is non-standard, peculiar, hard to explain, capable of returning mysterious non-answers without indications of error to innocent users, a bad foundation for building higher-level functionality.

>>

>> I have other criticisms of Mathematica, but I think that the sentence above is enough for you to process today.

>>

>>> It seems to me that is because he himself has not yet decided this, although he has been posting on this topic for over 20 years (I think).

>>

>>> Sometimes he seems to be disparaging significance arithmetic itself.

>> I think deBoor did that.

>>

>>> When Daniel points out how effective it is in his implementation of numerical Groebner basis, or in Adam Strzebonski's work on Reduce, he either ignores this altogether or claims that Groebner bases, etc. are themselves not "useful".

>> I think Reduce is a very nice program when it works. If I am not mistaken, all work on numerical Groebner basis should be reducible to the evaluation of polynomials, for which there are faster and more accurate methods available not using significance arithmetic. On the other hand, I might be mischaracterizing DanL work, since I admit to not having studied it.

>>

>>

>>> On other occasions he takes on the role of the defender of the interest of the "naive user" (presumably like the OP, who however would be better described as "intentionally naive") who is going to be confused by the "quirky" nature of significance arithmetic (at low =

>> I have no problem with Mathematica providing as an option, some other =

arithmetic. It is WRI that has made it rather hard for the user to

>> figure out how to "use the arithmetic of the rest of the world".

>>

>>> Finally, perhaps all that Richard is so upset about is simply =

Mathematica's habit of defining numbers as "fuzz balls" or =

"distributions".

>> I'm not sure that "upset" is the right term. After all, I don't have =

Jul 16, 2011, 5:41:52 AM7/16/11

to

On Jul 15, 1:10 am, Richard Fateman <fate...@cs.berkeley.edu> wrote:

> > In extracting useful results from computers in the real world

> > ignorance of the application domain is more debilitating than

> > ignorance of computer science. One can muddle through the latter, but

> > not the former.

>

> I think you may be confusing "computer science" with "applications of

> computers to whatever [e.g. physics, engineering, business]" which is

> usually taught in the department of 'whatever'.

So, in your view, the substance is all in the departments of

'whatever'? Because without applications, what's left? Computers are

not natural objects: they are artificial, so without applications the

whole field is just a made-up story. A true science needs the

discipline of relating to *something* in the real world. You're

evading this discipline. Your view makes computer science

indistinguishable from religious scholasticism.

> > Consider the howler you yourself committed a little while ago,

> > discussing the role of computers in the Apollo moon landings:

>

> >> you've got to wonder how the Russians, probably WITHOUT much in the wa=

y

> >> of computers, put up an artificial satellite.

>

> > This illustrates a profound ignorance of the problem. If you don't

> > care what orbit you go into, the computations are readily performed by

> > slide rule ahead of time. But Apollo had to perform a series of

> > precise maneuvers using imprecise rockets, with repeated re-

> > computation of the trajectories. For the early missions, they didn't

> > even know the lunar gravity very well, so actual orbits diverged

> > rapidly from predictions even when the initial conditions were known.

> > Apollo's indirect "lunar orbital rendezvous" approach was thus a

> > triumph of computation. Even the Saturn V was not big enough to

> > support the less computationally intensive direct approach.

>

> Do you know for a fact that the Russians didn't care what orbit the

> first manned satellite had?

I know the consequences of getting it wrong. Basically, they knew if

they got the perigee high enough to go around once, nothing really bad

could result from the other orbital parameters. The rocket wasn't

powerful enough to do something silly like put Gagarin into an escape

trajectory. The limitations of the rocket combined with very basic

orbital mechanics guaranteed that about an hour and a half after

launch, the spacecraft would return to a point near overhead to where

the launch site had been. Consider the rotation of the Earth, and then

all they had to do was fire the retrorocket in roughly the right

direction at roughly the right time and Gagarin was guaranteed to come

down about 23 degrees west of where he was launched. The Soviet Union

was a huge place: they didn't need to do this accurately at all.

Satisfying a single inequality is enormously easier than rendezvous in

orbit around a body with poorly known gravity, where you must satisfy

six equations to high precision.

> I think you are confusing application knowledge with computer science.

Without applications, computer science is vacuous.

> Of course there are many methods that can be programmed. You are

> assuming that analysis of signals is done by some kind of particle

> tracking? I assume that programs are designed by persons familiar with

> differential equations and electromagnetic radiation, as well as more

> seat-of-the-pants stuff like antenna design and sun spots.

>

> Also I assume

> that cell phones use signal strength and feedback, and do not need great

> accuracy.

How do you determine how large to make the transistors when

manufacturing the cell phone? This requires calculation. How would you

do that calculation?

> Though maybe GPS stuff is tricky if you have few

> triangulation points. Using 10 points, maybe not so tricky. Not

> something I've cared to look at.

If von Neumann was alive, he'd understand the issues completely before

you'd even finished describing the problem. He was a *real* computer

scientist.

> > There is no universal method for tracking uncertainty that is accurate

> > and practical.

>

> Ah, so you are saying that Mathematica is not accurate and practical??

Not universally. However, given a specific problem, it is often the

tool of choice.

> I am primarily interested in building systems appropriate for a range of

> applications. (That's more computer science).

Without knowledge of applications, you have no foundation to stand on

here. And that's Wolfram's advantage: he and his people *do*

understand a very wide range of applications.

> From that perspective I

> think that Mathematica falls short.

Since many of us find Mathematica a very effective tool in real

applications, this judgement is obviously based on nothing but

ideology. You have repeatedly demonstrated your complete lack of any

useful perspective here, and your unwillingness to do the necessary

studying to gain that perspective. Instead, you carefully define

"computer science" in a way that excuses you from studying anything

you don't wish to study.

> I don't see that as a threat, but I

> am inclined to object to statements that claim (in my view incorrectly)

> that Mathematica (arithmetically speaking) is the best, or even the only

> way to do floating-point calculations.

There is no best. It depends on what you're doing. Mathematica is very

effective over a wide range of applications. It is not the right tool

for every application. But you need the application knowledge to

understand this.

Jul 16, 2011, 5:42:23 AM7/16/11