That would violate the spec.
Russ
> so overflows are an outcome that are desirable.
>
> http://en.wikipedia.org/wiki/Ariane_5_Flight_501
>
> <http://en.wikipedia.org/wiki/Ariane_5_Flight_501>;-)
>
> I can see the headlines now...
>
> "golang responsible for Ariane 6 explosion'
You are misrepresenting the spec by confusing different types of
overflow and by not noting that Go does not permit exceptions on
overflow or on floating point to integer type conversion. The specific
problem that happened there can not happen in Go. It is also less
likely to happen by accident in Go, as all type conversions must be
stated explicitly.
Ian
Well, the choices I see for handling signed integer overflow are:
1) Explicitly wrap. This is what Go does, and Java too.
2) Explicitly panic. This is what Ada does. In C# you can use checked
to do an integer computation with trapping. In Fortran some compilers
have optional overflow checks along with ON INTEGER OVERFLOW. gcc has a
-ftrapv option to do this, but it doesn't work reliably.
3) Automatically switch to bignums. This is what Python and various
LISP variants do. It's not usually a good choice for a compiled
language.
4) Explicitly do not define what happens. This is what C and C++ do.
It permits additional compiler optimizations at the cost of confusing
behaviour when overflow does actually occur.
For Go I think we can rule out choice 3, and we have decided to rule out
choice 4 because it is confusing for programmers. The question then
becomes whether to implement choice 1 or 2. Choice 1 gives better
runtime performance on most processors. It also matches the model of
computer arithmetic, which is always and unavoidably different from real
arithmetic. What are the advantages of choice 2?
Ian
You know ASAP that the Horrible Thing has happened and that
Something Must Be Done.
Otherwise some consequent Horrible Thing happens elsewhere and
elsewhen and headscratching happens.
Unrealistic Suggestion:
x, ok [:]= L <someInfixOp> R
x gets the usual result of L sIO R, ok gets true if it overflows, false
if not.
Probably completely horrible (since nested expressions become
a dreadful assembly-level collection of quads). On the other hand,
makes multi-precision arithmetic easier ...
Chris
--
Chris "allusive" Dollin
What if we had separate types that explicitly panicked? That would
provide programmers with the option to choose which they wanted. If
we thought of the question in this way, we could ask which int we
would wish to use for which purposes, which I think might be a better
question.
For indexing purposes (i.e. the argument i to a[i], or the size n in
make([]int, n)) I'd prefer a panicking int, since otherwise I could
end up with an out-of-bounds access that ends up being in-bounds due
to overflow. Sure, it's a rare case, but it'd be nice to panic
especially in the rare case where I have a rare bug. One can hope
that by using ints rather than uints we'll end up panicking because a
negative value is provided when a positive one is needed, but it seems
nicer to have an assurance that we crash (and sooner) rather than
continue with bogus results.
The wrapping integers seem more useful for random number generators,
bit fields, and things like that, which are explicitly written to use
ordinary binary arithmetic.
So I guess the question would be what the cost of panicking on
overflow would be. I imagine it's pretty high, and that I'd rather
stick with the current option 1. But if it could be done reasonably
cheaply, then it seems like I'd like for int at least to default to
panicking. One appealing option would be for int and uint to be the
only integral types that panic. If you want wrapping, you'd then have
to explicitly specify where you want the wrapping to occur. Although
that raises the question of int64 and file positions, where you
probably don't want wrapping, but you also need something bigger than
an int. If we had this distinction, I'd be tempted to add a "long"
type, with the understanding that an int can address any allocated
array or slice, and a long can address any file.
--
David Roundy
I prefer a well-defined behavior over a panic any day. I do enough
panicking by myself.
--
.o. | Sterling (Chip) Camden | http://camdensoftware.com
..o | ster...@camdensoftware.com | http://chipsquips.com
ooo | 2048R/D6DBAF91 | http://chipstips.com
What's not well-defined about a (Go) panic?
I mean, that the operation has a well-defined result instead of a panic.
IMO a panic should be reserved for cases in which no other result is
remotely sane. You might argue that overflow is insane, but it's already
been shown that overflow has some use cases.
That's nice if it's possible, but not all well-defined results are useful.
> IMO a panic should be reserved for cases in which no other result is
> remotely sane. You might argue that overflow is insane, but it's already
> been shown that overflow has some use cases.
Some. Sadly there are cases where the wrap-around isn't useful, eg
when you're counting. MAXwossname + 1 being 0 (or MINwossname)
may be well-defined but it's not helpful. In that case, I'd rather have
an inmyface panic than a well-defined fake.
On 03/23/2011 05:25 PM, David Roundy wrote:
> On Wed, Mar 23, 2011 at 8:42 AM, Ian Lance Taylor<ia...@google.com> wrote:
>> Well, the choices I see for handling signed integer overflow are:
>>
>> 1) Explicitly wrap. This is what Go does, and Java too.
>>
>> 2) Explicitly panic. This is what Ada does. In C# you can use checked
>> to do an integer computation with trapping. In Fortran some compilers
>> have optional overflow checks along with ON INTEGER OVERFLOW. gcc has a
>> -ftrapv option to do this, but it doesn't work reliably.
>
> What if we had separate types that explicitly panicked? That would
> provide programmers with the option to choose which they wanted. If
> we thought of the question in this way, we could ask which int we
> would wish to use for which purposes, which I think might be a better
> question.
>
var a int
var b pint // a panicking int
c := a + b
Is c now an int or a pint?
This sounds very confusing to me.
> var a int
> var b pint // a panicking int
>
> c := a + b
>
> Is c now an int or a pint?
Same as for Go's other different ints -- that would be a compile-time
error. Convert to the type your heart yearns for.
> If we limit a language's functionality to what a given programmer
> considers "useful" then we end up with Java.
Maybe, but no-one was suggesting we do that.
> May I quote you?
Sure. What was so quotable?
Array overflow is always an error. There is no way for the spec to
give a reasonable defined behaviour in such a case.
Integer overflow is not always an error. Sometimes it's desirable behaviour
and the language can give a reasonable and well defined result in
the case of overflow.
It's then the job of the programmer to write code that fits within the
spec of the language.
The language can't save you if you don't validate your input data.
- jessta
--
=====================
http://jessta.id.au
> Array overflow is always an error. There is no way for the spec to
> give a reasonable defined behaviour in such a case.
Sure there is; just like overflow, do wrap-around.
> Integer overflow is not always an error. Sometimes it's desirable behaviour
> and the language can give a reasonable and well defined result in
> the case of overflow.
> It's then the job of the programmer to write code that fits within the
> spec of the language.
> The language can't save you if you don't validate your input data.
But it could offer more help for the cases where one cares about overflow
and would like to know if it has happened.
That's much more drastic. In the case of integer overflow, it's merely
defining the result of an operation, given a specific input. For array
bounds overflow, it would mean interpreting the index as a different value
than the one specified.
>
> > Integer overflow is not always an error. Sometimes it's desirable behaviour
> > and the language can give a reasonable and well defined result in
> > the case of overflow.
> > It's then the job of the programmer to write code that fits within the
> > spec of the language.
> > The language can't save you if you don't validate your input data.
>
> But it could offer more help for the cases where one cares about overflow
> and would like to know if it has happened.
>
Offering more help is nice, but not at the expense of those who don't
want it. It's better to build such helpful constructs as add-ons (e.g.,
an 'if') where they are desired. I don't object to someone creating an
'int that panics on overflow' type if they want one, but let's not layer
gratuitous nannyisms onto the native int, please.
That's pretty much the same thing. In both cases, we have a value
(the array index; the result of some arithmetic operation) which won't
fit (the index is outside the bounds of the array; the value won't fit
into the number of bits available). For overflow, we discard the bits
that don't fit -- we reduce by the modulus. Doing the same thing with
an array index doesn't seem to be "much more drastic". In both cases,
the value that gets used isn't the value that was available.
>> But it could offer more help for the cases where one cares about overflow
>> and would like to know if it has happened.
>>
> Offering more help is nice, but not at the expense of those who don't
> want it.
Was someone doing that?
> It's better to build such helpful constructs as add-ons (e.g.,
> an 'if') where they are desired.
It's not clear that it's "better". It's a choice that needs evaluation.
What's my choice in current Go for eg "add these two numbers and
tell me if the result overflows"? Is it a reasonable choice? What
alternatives would one want, what would be the language &
usage cost? What happens if it's not a single addition we're
talking about but an entire expression?
> I don't object to someone creating an
> 'int that panics on overflow' type if they want one, but let's not layer
> gratuitous nannyisms onto the native int, please.
Let's not, no. But how does one tell whether something is a
"gratuitous nannyism" or a useful feature that's being bad-mouthed?
When is dealing with overflow by modulus reduction perfectly
fine and when is that asking for trouble, and how could those two
different contexts be handled in code?
I'm deeply uncomfortable about languages in which the normal,
encouraged machine arithmetic will wrap around silently and
give completely wrong answers with no warning or decoration.
Relying on the programmer to handle this themselves, everywhere,
seems to me a touch on the optimistic side.
Note that I am NOT saying that everyone should be nannied. But
it would be nice if we had the Mary Poppins option as well as
legal streaking.
As I said, I'm not opposed to an option that would catch these cases, as
long as it's an *option* and doesn't preclude the current behavior.
> What's my choice in current Go for eg "add these two numbers and
> tell me if the result overflows"?
Something like this is about the best I can come up with.
func Add(a, b int) int {
switch {
case a > 0 && b > 0:
if math.MaxInt32 - a < b {
panic("overflow")
}
case a < 0 && b < 0:
if math.MinInt32 - a > b {
panic("overflow")
}
}
return a + b
}
This particular implementation assumes that int is 32 bits. The math
package doesn't currently have MaxInt/MinInt, although perhaps it
should.
This is a case where operator methods might be interesting. One could
define operator methods for a type such that all operations checked for
overflow.
Ian
Try this:
func Xor(cond1, cond2 bool) bool {
return (cond1 || cond2) && !(cond1 && cond2);
}
func Add(a, b int) int {
r := a + b
if Xor((r < a), (b < 0)) {
panic("overflow")
}
return r
}
Because the overflow is defined in the spec, we can rely on it.
> Try this:
>
> func Xor(cond1, cond2 bool) bool {
> return (cond1 || cond2) && !(cond1 && cond2);
> }
>
> func Add(a, b int) int {
> r := a + b
> if Xor((r < a), (b < 0)) {
> panic("overflow")
> }
> return r
> }
>
> Because the overflow is defined in the spec, we can rely on it.
That's a ... complicated way of expressing Xor, and in any case we
don't need the function, since it's easier to write:
if (r < a) != (b < 0) { ... BOOM ... }
It's still significantly heavier than my strawman
r, ok := a + b
if !ok { ... BOOM ... }
Touché. Now I know why Go doesn't need a logical Xor.
>
> It's still significantly heavier than my strawman
>
> r, ok := a + b
> if !ok { ... BOOM ... }
>
I'd be OK with that syntax, as long as the overflow still occurred. I
imagine the implementation would get a bit tricky, though.
I don't know about the x86-style machines, but if I recall correctly
it's essentially free on the ARM; do an ADDS and then the
appropriate conditional instruction looking at the V flag (or the
C flag for unsigned. I think.)
I was referring to the syntactic complications of combined operations.
r, ok := a + b * c + d
To which operation does ok refer? Does it mean that none/(any) of the
operations overflowed? Or would the syntax only be supported for single
operations?
> I was referring to the syntactic complications of combined operations.
Ah! OK.
> r, ok := a + b * c + d
>
> To which operation does ok refer? Does it mean that none/(any) of the
> operations overflowed? Or would the syntax only be supported for single
> operations?
These are good questions, cunningly avoided for the present by my
earlier use of the term "strawman" ...
... I think it would have to be "at least one overflow happened". And
another question would be "what about arguments to functions" -- if
one's going for a wide-scope overflow test, would one want to deal with
the entire argument list in one lump?
I haven't looked at the Go compiler sources, but I'd venture to say that
returning a status for an operation embedded in an argument would not be
possible. That is, in:
r, ok := somefunc(a + b)
ok could never refer to an overflow of a + b, because it's returned by
somefunc().
Perhaps likewise, in combined operations, it might only refer to the last one.
Because:
r, ok := a + b * c + d
is equivalent to:
r, ok := (a + (b * c)) + d
I think that ok could only refer to the operation of adding d to the
intermediate result. But I await correction.
It /could/, if we determined that it /should/. The expression would have to
be evaluated in a remember-overflow mode, with a hidden variable being
set true if there was an overflow. (I hesitate to suggest that somefunc could
be called in such a way as to expose any overflows that happened during
/its/ execution ...)
But I think that would be too complicated (not to mention potentially
inefficient).
> Perhaps likewise, in combined operations, it might only refer to the last one.
> Because:
>
> r, ok := a + b * c + d
>
> is equivalent to:
>
> r, ok := (a + (b * c)) + d
>
> I think that ok could only refer to the operation of adding d to the
> intermediate result.
I think that if that were the case, we'd be better off restricting the
expression
to varorconst op varorconst, rather than allowing the full set but only checking
the outermost op. Having `r, ok := a + b * c + d` check all the overflows seems
like a good thing to me.
If we were prepared to have r be no-particular-value if an overflow
happened, then a following `if !ok` can be jumped to as soon as the
overflow is detected.
This can't be an unconsidered problem in language design. And I
bet the Go team has opinions to hand. Any pointers, anyone?
I dislike that last idea. Consider:
r, ok := a * b + somefunc()
if !ok ...
You're saying that if "a * b" overflows, don't call somefunc(). I'd also
still like to have r contain the usual result.
what would r, _ = a +b * c + d mean then? I'd expect give me r and I
don't care if it overflows not r may be nonsense.
>> If we were prepared to have r be no-particular-value if an overflow
>> happened, then a following `if !ok` can be jumped to as soon as the
>> overflow is detected.
>
> I dislike that last idea. Consider:
>
> r, ok := a * b + somefunc()
> if !ok ...
>
> You're saying that if "a * b" overflows, don't call somefunc().
I'm saying that /if/ we were prepared to have r be no-particular-value
/then/ we could exploit quick exit. I'm so prepared, and you're not.
(Also I think I'd not permit function calls in the scope of an
overflow-detection expression anyway.)
> I'd also still like to have r contain the usual result.
At this point we need more data to make choices with and I
don't know where to get it off-hand.
>> If we were prepared to have r be no-particular-value if an overflow
>> happened, then a following `if !ok` can be jumped to as soon as the
>> overflow is detected.
>
> what would r, _ = a +b * c + d mean then?
It would mean "r becomes some value or other", but you
don't know if its the overflowed value or some junk that
got allowed by the if-overflowed-could-be-anything clause.
> I'd expect give me r and I
> don't care if it overflows not r may be nonsense.
Something happend to your syntax but I think you meant
the same thing.
> I think the idea would also need wider acceptance than a couple of geeks
> on the mailing list.
Oh, surely. This is just exploring the territory and hoping for existing maps.
At any moment a Voice might say NEVER GOING TO HAPPEN and we have
to pick another topic.
What about a 0-sized array? How will it wrap around?
That reminds me too much of Visual Basic, "On Error Goto label"[1]
specifically.
[1] http://msdn.microsoft.com/en-us/library/5hsw66as(v=vs.80).aspx
This could introduce panics in to code that expects and handles
overflows as defined in the spec but doesn't expect a panic.
Optional parts of a spec are a bad idea. Different developers with
different expectations of what code should do.
Myself I think I'd prefer more local control over than the entire
body of a function. (Yes, I know one could bundle the code to
check in a function literal and call it, but part of the point of
the straw men is to make checking sufficiently syntactically
cheap that (a) it gets used and (b) one can see what's going on.)
What, global to the compilation of en entire package? (That's
what I'd expect a "compilation option" to mean -- a compiler
-switch). That would definitely be too coarse.
Oh, good catch. I should have thought of that. Hmm. Can't currently
justify the obvious ad-hoc answer.
(One-element arrays are amusing in this context too.)
That's not at all functional (as in FP). It creates side effects for
other statements, causing their operation to differ based on context.
And, as Ibrahim pointed out, it's too similar to "On Error", the source
of numerous bugs in VB (and DIBOL).
I rather like that idea, though perhaps it needs a different name in
that case -- such as "detect_overflow" or something similar. Naming an
ostensible function "overflow" tells me that it's going to overflow,
unconditionally.
We've already talked about that. You can't do it unconditionally,
since the spec explicitly allows the current wraparound behaviour
and code has been written against it. You need new arithmetic types
and/or overflow detection scoping constructs.
(Also, the program "not continuing" might be a Very Bad Indeed thing.)
I guess one way to do this today without changing the compiler would
be to write an AST walker which would transform the source. It would
need to replace most integer operations with a call to a function like
the one mentioned earlier in the thread. Probably not too hard given
that all of the hard work has already been done in the go/parser
package, and gofmt command. The generated code would be slow, but
perhaps ok if only for testing.
Here's a c implementation of some of the checks:
http://tinyurl.com/yg4os3m
The same technique could also be used to prototype the various
proposed language additions.
> I'd prefer a compiler option that enables an overflow check thatI guess one way to do this today without changing the compiler would
> panics upon detection.
be to write an AST walker which would transform the source. It would
need to replace most integer operations with a call to a function like
the one mentioned earlier in the thread. Probably not too hard given
that all of the hard work has already been done in the go/parser
package, and gofmt command. The generated code would be slow, but
perhaps ok if only for testing.