Error Handling Question

[Daniel Lepage]

Hi all,

I'm not sure if this is the right forum to ask this question - please let me know if there's somewhere better!

I'm wondering why most of the Go 2 error handling proposals I've seen are intent on not letting a function continue after detecting an error - this seems weird to me, since we already have panic() as a way for a function to signal an error it doesn't expect the caller to recover from. If a function returns an error instead of panicking, doesn't that mean the author of the function believes that it is both possible and reasonable for a caller to recover from this error and keep going?

I looked through a bunch of different error proposals among the PRs and linked from various summary pages, but I can't find if anyone has proposed something simple like

var x0 float

try {

   x0 = check DoSomeMath(check FetchSomething(), check ComputeSomething())

} handle err {

   log.Info("Unable to estimate initial approximation, defaulting to 1...")

   x0 = 1

}

// code continues and does things with x

This makes it easy to see what error handling will happen at any point within the function, keeps the control flow linear (so that, unlike defer()-based recovery, you don't have to skip ahead in the function to get context before the handler makes sense - the context comes first, followed by the handling code), and allows code to recover from errors without aborting an entire function.

As a bonus it'll be easier for new Go programmers to learn because it's structured the same way as try/catch or try/except blocks in numerous other languages.

This seems like an obvious enough suggestion that someone must have made it already, but I haven't been able to find that among all the other error handling proposals and discussions.

Has this in fact been proposed already, and if so where can I find discussion on why it was rejected?

Thanks,

Dan

[Ian Lance Taylor]

I can't recall any error handling proposals that are quite like that,
though you may want to look at the meta-issue
https://go.dev/issue/40432.

Most error handling proposals are trying to reduce the boilerplate of
error checking. If I rewrite your example with Go as it exists today
I come up with this:

var x0 float
x0, err = check DoSomeMath(check FetchSomething(), check ComputeSomething())
if err != nil {

log.Info("Unable to estimate initial approximation, defaulting to 1...")
x0 = 1
}
// code continues and does things with x

That isn't noticeably longer than what you wrote. So your suggestion
doesn't appear to reduce boilerplate much.

Specifically, most error handling proposals are focused on reducing
the boilerplate in

if err != nil {
return 0, fmt.Errorf("Oh noes: %v", err)
}

That isn't the kind of case that you are describing. If a function is
going to keep going, then it will almost certainly need some
function-specific code to recover from the error. That recovery code
is unlikely to be boilerplate. We could add new syntax to avoid "if
err != nil", but it's not like "if err != nil" is all that long, so
the new syntax seems unlikely to help much.

Ian

[Daniel Lepage]

Sorry, I should have been clearer - what I am proposing is both try/handle blocks and a `check` expression that triggers the handler. The line `check DoSomeMath(check FetchSomething(), check ComputeSomething())` cannot (as far as I know) be written in existing Go; in modern Go my example would have to look something like:

var x0 float

var err error
a, err := FetchSomething()

if err != nil {
   log.Info("Unable to estimate initial approximation, defaulting to 1...")
   x0 = 1

} else {
   b, err := ComputeSomething()

   if err != nil {
      log.Info("Unable to estimate initial approximation, defaulting to 1...")
      x0 = 1

   } else {

      x0, err = DoSomeMath(a, b)

      if err != nil {
         log.Info("Unable to estimate initial approximation, defaulting to 1...")
         x0 = 1
      }
   }
}
// code continues and does things with x

 

But I also don't want to get bogged down by this specific example - the above could be somewhat streamlined with an intermediate ComputeX0 function, for example, but that's beside the more general point I'm trying to make, which is just that there definitely are cases where a function wants to continue after handling one or more errors, and I am perplexed as to why every proposal I've seen seems designed to prevent this.

 

Specifically, most error handling proposals are focused on reducing
the boilerplate in

if err != nil {
    return 0, fmt.Errorf("Oh noes: %v", err)
}

TBH I'm less concerned with the boilerplate in general (though I wouldn't mind reducing it), and more concerned about the specific problem of using function return values within an expression - I think there are many cases where it's useful to be able to write code of the form `fn1(fn2())`, but if fn2 could produce an error then you have to fall back to

tmp, err := fn2()

if err != nil {

    return fmt.Errorf(...)

}

fn1(tmp)

which means you're now using temporary variables and obscuring what's actually happening by sticking a bunch of error handling in between the various calls.

The premise that being able to chain functions like this is useful is, I think, validated by the existence of various testing code that passes a testing.TB around so that the helpers can call t.Fatal instead of returning errors. As a real-world example, consider the open-source Ondatra project (https://github.com/openconfig/ondatra), which has numerous functions that use the "pass in a TB and Fatal on errors" pattern; you can see this API being used in e.g. this test from the OpenConfig featureprofiles project, which contains the snippet:

func configureDUT(t *testing.T, dut *ondatra.DUTDevice) {
    dc := dut.Config()
    i1 := dutSrc.NewInterface(dut.Port(t, "port1").Name())
    dc.Interface(i1.GetName()).Replace(t, i1)
    i2 := dutDst.NewInterface(dut.Port(t, "port2").Name())
    dc.Interface(i2.GetName()).Replace(t, i2)
}

The Port method and the Replace method above both can fail; if they do, they call t.Fatal instead of returning an error. If they returned errors instead, the above code would be significantly longer and harder to read, something like:

func configureDUT(dut *ondatra.DUTDevice) error {
   dc := dut.Config()
   port1, err := dut.Port("port1")
   if err != nil {
      return err
   }
   i1 := dutSrc.NewInterface(port1.Name())
   if _, err := dc.Interface(i1.GetName()).Replace(i1); err != nil {
      return err
   }
   port2, err := dut.Port("port2")
   if err != nil {
      return err
   }
   i2 := dutDst.NewInterface(port2.Name())
   if _, err := dc.Interface(i2.GetName()).Replace(i2); err != nil {
      return err
   }
   return nil
}

And this is just a single function from a much longer test, which is a single test out of dozens within the project, and that's just within this particular project which is definitely not the only project using Ondatra.

So it makes sense that the Ondatra authors opted to pass a TB everywhere, but it also means A) that none of the tooling they built can ever be used for anything BUT tests, B) that any test using these helpers MUST abort completely if an error happens, and C) it's not obvious just looking at the test code that every call to Replace or Port is actually an assertion helper that will kill your test if something's wrong. I would argue that they shouldn't have had to make this tradeoff - it should be possible in Go to achieve the streamlined style without assertion helpers.

The point of try/check/handle would be that you could rewrite the above as:

func configureDUT(dut *ondatra.DUTDevice) error {
  try {
    dc := dut.Config()
    i1 := dutSrc.NewInterface((check dut.Port("port1")).Name())
    check dc.Interface(i1.GetName()).Replace(i1)
    i2 := dutDst.NewInterface((check dut.Port(t, "port2")).Name())
    check dc.Interface(i2.GetName()).Replace(t, i2)
  } handle err {
    return fmt.Errorf("configuring DUT: %w", err)
  }
}

The actual logic is just as streamlined as it is right now, but it achieves this without any hidden assertions or dependency on testing.TB, and now it allows the caller to decide whether the error is fatal for the test or not.

The biggest differences between this and e.g. the original Go 2 draft proposal are 1) it can be used to consolidate error boilerplate regardless of whether or not the boilerplate returns (and thus terminates the functions), and 2) it limits "check" to within explicit "try" blocks, such that every function must still have explicit error handling for any errors that arise in it.

Note that point 2 could be changed - if "check" expressions were allowed outside of try/handle blocks, essentially by giving every function an implicit "try { <func body> } handle err { return ..., err }", then this would eliminate more boilerplate but at the cost of allowing functions to have entirely implicit error handling, so I think whether or not to do that should be a separate discussion beyond the scope of this proposal. This proposal is solely about allowing the error handling of multiple calls to be consolidated into a single handler block, reducing duplicate boilerplate and allowing natural use of return values even when functions also return errors.

It sounds like this hasn't been proposed before, so I can go ahead and write up a formal proposal for this.

Thanks,

Dan

[Ian Lance Taylor]

On Thu, Oct 20, 2022 at 9:52 PM Daniel Lepage <dple...@google.com> wrote:
>
> Sorry, I should have been clearer - what I am proposing is both try/handle blocks and a `check` expression that triggers the handler.

Sorry, I did miss the "check". That aspect of this idea seems similar
to the try proposal (https://go.dev/issue/32437). In that issue there
was considerable resistance to having flow of control change as part
of an expression. No other part of Go works that way: flow of control
change is always a keyword, or a call to panic which can only occur in
statement context. After going through the try proposal it's unlikely
that we would adopt that kind of change.

Ian

[Daniel Lepage]

> That aspect of this idea seems similar to the try proposal (https://go.dev/issue/32437).

Yes, I read over that proposal and the comments; it seemed like the biggest objections were:

1. handle() is very similar to defer/recover but not quite the same, which could be confusing (e.g. that a defer() inside a block is just a normal defer, while a handle() inside a block does not apply outside of that block)

2. It's confusing to take the previous if err != nil {} blocks that were obviously in response to the preceding line and have the actual handling code be earlier and far away, and

3. It would encourage people to just pass errors up without modification, because it's easier to just add a bunch of try's to your code and use the default handler.

My proposal doesn't have these issues - the handler goes with the try block so there's no separate function being deferred and no ambiguity about what code a given handler applies to, the handle block always comes right after the block being try'd, and there is no "default handler" - you can't use check to avoid having any error handling, only to consolidate several error handlers that would be identical.

> In that issue there was considerable resistance to having flow of control change as part of an expression. No other part of Go works that way: flow of control change is always a keyword, or a call to panic which can only occur in statement context.

This is not true - testing.TB.Fatal, log.Fatal, etc. affect control flow, as do numerous expressions like nil dereferences or zero divisions, and of course any function call that triggers any of those (and in extreme cases like running out of memory practically ANY expression could trigger a control flow change).

The link I provided earlier to the Ondatra package demonstrates that real-world Go programmers already ARE using expressions for control flow, by (in this example) calling functions that take a testing.TB and call t.Fatal() if anything goes wrong; the authors clearly decided that this was more readable than using idiomatic error handling.

I see panic/TB.Fatal as being somewhat analogous to Java's unchecked exceptions: they change control flow without warning and you can't be sure what functions trigger them. Errors-as-return-values is analogous to checked exceptions: you explicitly mark things you know could go wrong and the compiler will yell at you if you don't address those possibilities. Java's single biggest mistake (IMO) was making unchecked exceptions easier to use than checked exceptions - as a result many developers just didn't do any error handling at all. Go has mostly avoided this because checking errors is usually easier to write and read than using any of the aforementioned unchecked methods, but the t.Fatal examples make it clear that there still are cases where unchecked errors make code easier to read and write. I'm proposing to fix that, not by making unchecked errors harder to use or by turning checked errors into unchecked ones, but rather by streamlining the worst cases in proper error checking, primarily the all-too-common case where "if ComputeSomething(t, FetchThingOne(t), FetchThingTwo(t)) > 1" is so, so much easier to read and write than the equivalent properly checked version (which requires many more lines and several extra variables).

Dan

[Robert Engels]

Unchecked exceptions in Java denote programming errors or critical unrecoverable errors. 

If these are being trapped/handled it is indicative of a system with design errors. 

(That being said, some programming errors can be ignored/handled at the request level - indicating that the chance any particular request triggers the bug is low). 

On Oct 21, 2022, at 2:30 PM, 'Daniel Lepage' via golang-nuts <golan...@googlegroups.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAAViQtjfE2FdauJ888bu1HC%2BZPPCiff5Bz5_N%3DeM2%3DVtx2fFsg%40mail.gmail.com.

[Andrew Harris]

Comparing 'panic' and 't.Fatal', they are similar in that there's no notion of recovering and making further progress, they abruptly terminate the process. Still, a wonky thing to say is that a 't.Fatal' call is completely legitimate as a form of "checked"-style error handling. It will log what went badly rather and prevents further tests from running. This is a reasonable application-level behavior for testing (for non-fatal intent, 't.Error' is available).

[Daniel Lepage]

On Fri, Oct 21, 2022 at 5:16 PM Robert Engels <ren...@ix.netcom.com> wrote:

Unchecked exceptions in Java denote programming errors or critical unrecoverable errors. 

Yes, that's exactly what I'm saying. This is what panics are used for in Go; if you have an error that is recoverable you ought to be returning an error, not panicking.

 

If these are being trapped/handled it is indicative of a system with design errors. 

(That being said, some programming errors can be ignored/handled at the request level - indicating that the chance any particular request triggers the bug is low). 

I don't think handling panics is necessarily indicative of design errors, but I do agree that it's not something that should be done lightly. A big part of why Go has largely succeeded in getting people to actually use checked errors is that recovering from a panic is just complex enough that you wouldn't attempt it without a really good reason (compared to Java, where "recovering" from an unchecked exception by ignoring it or tossing in an empty catch {} block is trivial to write, so people do it all the time, unfortunately).

But t.Fatal is another panic-like operator, in that it reroutes control flow without any error handling, and I've seen projects using it as such, writing `f(t, g(t), h(t))` where each function calls t.Fatal() if anything goes wrong, instead of having each function return a value and an error. That's what the Ondatra example I linked was demonstrating - real code in the wild using t.Fatal as a way to have "unchecked exceptions".

Dan

[Daniel Lepage]

On Fri, Oct 21, 2022 at 5:57 PM Andrew Harris <harr...@spu.edu> wrote:

Comparing 'panic' and 't.Fatal', they are similar in that there's no notion of recovering and making further progress, they abruptly terminate the process. Still, a wonky thing to say is that a 't.Fatal' call is completely legitimate as a form of "checked"-style error handling. It will log what went badly rather and prevents further tests from running. This is a reasonable application-level behavior for testing (for non-fatal intent, 't.Error' is available).

To be clear, I'm not saying that tests shouldn't use `t.Fatal` to stop the test when a fatal error occurs. I'm just saying that test *helpers* shouldn't be using it to get around having to return errors. The fact that existing projects ARE using it this way suggests not only that tests would benefit from a more streamlined form of error handling, but also that there are probably many non-test APIs written in Go that would choose to allow more concise patterns if Go supported it.

Dan

[Ian Lance Taylor]

On Fri, Oct 21, 2022 at 12:30 PM Daniel Lepage <dple...@google.com> wrote:
>
> > In that issue there was considerable resistance to having flow of control change as part of an expression. No other part of Go works that way: flow of control change is always a keyword, or a call to panic which can only occur in statement context.
>
> This is not true - testing.TB.Fatal, log.Fatal, etc. affect control flow, as do numerous expressions like nil dereferences or zero divisions, and of course any function call that triggers any of those (and in extreme cases like running out of memory practically ANY expression could trigger a control flow change).

I understand what you are saying. Yes, any function can call panic.
Yes, Go also has run-time panics such as division by zero. Yes, there
is code that explicitly uses panic and recover to simplify error
handling (you don't have to refer to external packages, there is code
like that in the standard library, such as encoding/gob).

However, I think my broader point still holds despite those facts.


I also want to be clear: I'm happy to talk about these issues. But
there is basically zero chance that the Go language will add a
try/handle syntactic construct. It's possible that the language will
make changes regarding error handling, if we ever come to any sort of
consensus as to what can and should be done. But it's not going to
look like try/handle.

Ian

[Daniel Lepage]

I understand what you are saying.  Yes, any function can call panic.
Yes, Go also has run-time panics such as division by zero.  Yes, there
is code that explicitly uses panic and recover to simplify error
handling (you don't have to refer to external packages, there is code
like that in the standard library, such as encoding/gob).

However, I think my broader point still holds despite those facts.

The part I don't understand is that Go straight-up encourages having a panic instead of handling exceptions in certain specific cases. Suppose I have a map m from string ids to some Record interface, and some of those records satisfy a ColorRecord interface. Then I can write a function that handles exceptions properly like:

func isGreen(key string) (bool, error) {
    record, ok := m[key]
    if !ok || record == nil{
        return false, fmt.Errorf("no record with key %q", key)
    }
    if !record.IsActive() {
        return false, nil
    }
    cRecord, ok := record.(ColorRecord)
    if !ok {
        return false, fmt.Errorf("record with key %q is not a ColorRecord", key)
    }
    return cRecord.Color() == colors.GREEN
}

Or, I can streamline this considerably by ignoring the second return arguments from type assertions and map lookups, which means if anything goes wrong it'll panic:

func isGreen(key string) bool {
    r := m[key].(ColorRecord)
    return r.IsActive() && r.Color() == colors.GREEN
}

And I agree that the above function is much easier to read and much faster to write than the first version! But now I'm raising unchecked exceptions instead of handling errors properly.

The ability to ignore the second return argument is obviously not a standard feature of expressions in go - it's a special case that's hard-coded into the language for a few specific expressions, and as I understand it exists specifically to avoid having to check errors in these expressions. It seems to me that the existence of this language feature proves that there genuinely are cases where most people agree that streamlining exception handling is not just worthwhile, but in fact so useful that we're ok adding special cases to the language itself AND sacrificing compiler-enforced error handling entirely to achieve it.

How does this align with the philosophy that expressions shouldn't ever affect control flow? Or do opponents of control-flow expressions also believe that one should never fetch from a map or do a type assertion without checking ok, even if you know it should be impossible for it to fail?

Dan

[Brian Candler]

> And I agree that the above function is much easier to read and much faster to write than the first version! But now I'm raising unchecked exceptions instead of handling errors properly.

However you're not "raising an unchecked exception"; you're panicking, which is something different. Go does not encourage this at all.  It would be fine to write it that way if calling isGreen on a missing or non-color key should never happen - i.e. you know that the programmer seriously screwed up if it ever got this far.  But it would almost never be appropriate to attempt to recover() from such a situation.

panic/recover has two main uses:

1. something very serious went wrong. Either it was a programmer error or an internal runtime error; either way, data structures are likely in an inconsistent state and it's better to crash than to attempt to carry on.

2. some very limited specific use cases which require being able to unwind a deep call chain.

Regarding your sample code: it's OK but I can think of alternative APIs, depending on how isGreen is intended to be used.

1. The user is allowed to call isGreen on something which may or may not exist, or may or may not be a color, and they just want to know if it's a green thing or not a green thing.  In that case, return only a bool.

func isGreen(key string) bool {
    record := m[key]   // aside: no need for 2-arg form, missing map key gives zero value
    if record == nil {
        return false
    }
    if !record.IsActive() {
        return false

    }
    cRecord, ok := record.(ColorRecord)
    if !ok {
        return false
    }

    return cRecord.Color() == colors.GREEN
}

It's a little verbose, but the intent is absolutely clear, and IMO much more obvious that your brief version.  The brief version does raise the question "what happens if I call isGreen in situation X, Y or Z"?  The answer is "you must never call isGreen in those situations".

2. The user calls isGreen but also wants to know if it was a color thing or not.  Then you could return (bool green, bool ok), in the same way to the two-argument forms of map lookup and type assertions.

3. The user wants to know exactly *why* it wasn't a color thing.  Then you can make it similar to your original code, but return an error which is from a set of constants, rather than fmt.Errorf - which is only useful for printing and not much more.  That's closest to properly "checked" exceptions.

[Volker Dobler]

On Saturday, 22 October 2022 at 22:25:16 UTC+2 dple...@google.com wrote:

> that the above function is much easier to read and much faster to write

True but in my opinion not relevant because:

1. Time to write a function is almost negligible compare to its
time needed for maintenance, rewrites, updates and changes.

2. The function is easier to _read_ but not necessarily easier
to _understand_ due to more implicite control flow.

I do understand your frustration. I'm annoyed myself at how

much error code I have to write, but honestly: My code is

"better" with all this explicit error handling and in the long

run I'm thankful Go forces me to do it at least halfway right

from the beginning.

V.

[Daniel Lepage]

On Sun, Oct 23, 2022 at 1:08 AM Brian Candler <b.ca...@pobox.com> wrote:

> And I agree that the above function is much easier to read and much faster to write than the first version! But now I'm raising unchecked exceptions instead of handling errors properly.

However you're not "raising an unchecked exception"; you're panicking, which is something different. Go does not encourage this at all.  It would be fine to write it that way if calling isGreen on a missing or non-color key should never happen - i.e. you know that the programmer seriously screwed up if it ever got this far.  But it would almost never be appropriate to attempt to recover() from such a situation.

When I say "unchecked exception" I mean in the Java sense of "an error condition that the compiler will not force you to address". The behavior of an unchecked exception is that it travels up the call stack, aborting each function until it either reaches the end of the stack and terminates the program, or reaches a call frame that indicates it will try to recover from it. This is exactly the behavior of panic - the only difference between `panic(e)` in Go and `throw e` in Java is that recovery in Java is written as "catch(Exception)" while in Go it's written as "defer func() { r := recover(); ... }()".

Also exactly like in Go, most Java style guides A) discourage you from throwing unchecked exceptions, and B) agree that it is almost never appropriate to attempt to catch and recover from one.

The only reason Go doesn't have severe problems with panic/recover being used everywhere for normal control flow while Java does is because in Java there's no immediate cost to just declaring your exception type to be a runtime exception, and it saves you some typing and shuts up some compiler errors, so people take that shortcut all the time, whereas in Go it's (correctly, in my opinion) a separate mechanism from checked errors (i.e. ones where it's a compiler error (usually) to forget to handle them).

In practice, though, Go code *does* have a lot of unchecked exception usage - every time someone ignores the 'ok' on a type assertion, calls panic() or t.Fatal() from a helper function instead of returning an error, or writes a MustFoo() version of some function that panics instead of returning an error, that's a point where they *could* be using checked exceptions (i.e. return values), and have chosen not to because the code is easier to write.

Regarding your sample code: it's OK but I can think of alternative APIs, depending on how isGreen is intended to be used.

I said earlier that I didn't want to get bogged down on specific examples, and this is what I meant - I know that there absolutely are better ways to implement the specific example I gave, I was just trying to demonstrate that Go not only already allows "expressions that change control flow", but in fact has explicit special cases in the language in order to make it possible. The "isGreen" example was just to demonstrate this - I know that this particular example could be reimplemented to sidestep the problem, but I've seen code where this wasn't true (I didn't include any here because the examples I can think of are difficult to understand without reading the whole of a library first, and I obviously don't expect anyone to do that just so that I can make an example).

Reflecting on this more, I think there are several distinct points that I'm arguing here. I'll try to make them explicit as I think them through:

1. There are places where streamlining error handling is acceptable and encouraged.

I don't mean that it's *always* good, or even "usually acceptable", just that cases do exist where it's desirable.

The evidence for this is strong:

 * As I mentioned earlier, Go has special case syntax for certain operations like type assertions explicitly to streamline error handling. If streamlining errors was never a good idea, then Go would require you to write f, ok := x.(Foo) even if you know for a fact that x cannot be anything but a Foo, the way it currently requires you to write b, err := Bar(x) even if you know for a fact that x is a valid argument for Bar that won't cause an error.

 * In many cases, including in the standard libraries, there are functions that return errors and then accompanying functions with names like MustFoo() that just call Foo() and panic if there's an error. This is also error streamlining, and like with type assertions the streamlining comes at the cost of using panics. Checking sourcegraph.com suggests that hundreds of Go projects already choose to make this tradeoff: https://sourcegraph.com/search?q=context:global+lang:go++/func%5Cs%28%5C%28%5B%5E%29%5D%2B%5C%29%5Cs%29%3FMust/&patternType=standard (yes, some fraction of these are exclusively for module-level initialization, but at least half of the handful I randomly sampled were being called in other functions).

* The Ondatra library I linked earlier, as well as other test helper libraries I've seen, use t.Fatal in the same way - they judged that "if got, want := ondatra.DUT(t, "name").Config().System().Hostname().Get(t), "localhost"; got != want { t.Errorf(...) }", which Fatals if there is no "name" entry in the DUT table, if the attempt to fetch the hostname fails, or if the hostname is unset, was still preferable to making people write out the fatal checks themselves every single time, even though it means that if you don't actually want this to terminate your test you have to workaround it with panics and recovers.

2. Streamlining doesn't have to mean using unchecked exceptions like panic() or t.Fatal().

All of the dozens of error handling proposals I've read so far are fundamentally aimed at letting you use the return value of a function that might return an error in contexts other than just assignment, and between them there are lots of different examples of errors that are more streamlined but still must be handled. My proposal is an example of one way to do this, but it is not the only way.

3. Streamlining shouldn't only apply to error handling that terminates the function.

Unlike panics, errors are values, and should be treated as such, which means that the calling function should be able to decide what to do, and this should include continuing. Frequently it won't - a lot of error handling is just return fmt.Errorf("more context %w", err) - but any proposal that assumes that it *always* won't is, IMO, confusing errors with panics. This is the question that first started this thread - I didn't understand why all the existing error proposals explicitly required that a function terminate when it encounters an error, and AFAICT the answer is "because people are used to thinking of errors more like panics than like return values".

4. The best way to have people write high-quality code is to make it easier to write high-quality code.

Nobody should ever have to look at their code and think "My API would be easier to use in normal circumstances if I just panicked in abnormal ones..." and then have to decide whether it's more important that their callers have a streamlined API or a principled one.

This is not hypothetical - I talked to some of the engineers behind the Ondatra project that I linked above, and they went through exactly this, where they wanted to return errors but their users all felt it made the tests too verbose, and that's why they use t.Fatal() everywhere.

I didn't start this thread with the intent of creating a new error handling proposal - I just wanted to understand why all the existing proposals don't care about point 3. But the more I think about this, the more I'm surprised it hasn't already been proposed.

Is part of the problem that the discussion around the try/check/handle/etc. proposals got so involved that nobody wants to even consider anything that looks too similar to those? Would it be more palatable if I proposed it with names that made it clearer that this is about the consolidation of error handling rather than an attempt to replace it entirely?

onErrors {

    if must Foo(must Bar(), must Baz()) > 1 {

      ...

   }

} on err {

   ...

}

Thanks,

Dan

[Harri L]

Hi,

On Friday, October 21, 2022 at 12:14:52 AM UTC+3 dple...@google.com wrote:

var x0 float

try {

   x0 = check DoSomeMath(check FetchSomething(), check ComputeSomething())

} handle err {

   log.Info("Unable to estimate initial approximation, defaulting to 1...")

   x0 = 1

}

// code continues and does things with x

 

I wanted to show what the previous block looks like with the help of the current Go and some help from the err2-package (I'm the author). The Go playground.

     calcSomething := func() (x0 int) {

        defer err2.Catch(func(err error) {
            log.Printf("Unable to estimate initial approximation: %v, defaulting to 1...", err)
            x0 = 1
        })
        return try.To1(DoSomeMath(try.To1(FetchSomething()), try.To1(ComputeSomething())))
    }
    x0 := calcSomething()

Of course, that's not exactly what you are proposing:

This makes it easy to see what error handling will happen at any point within the function, keeps the control flow linear (so that, unlike defer()-based recovery, you don't have to skip ahead in the function to get context before the handler makes sense - the context comes first, followed by the handling code), and allows code to recover from errors without aborting an entire function.

 

I also wanted to say that the `defer` -based error handling isn't so bad when you are used to the defer keyword already in the Go. 

Best Regards,

-Harri

[Robert Engels]

Dan,

If it walks like a duck…

You make a great case why Go should have checked exceptions. If anything the Go error handling model tries to be checked exceptions - every error should be explicitly handled - without any of the conveniences of exceptions - common blocks, stack traces and call stack unwinding.  

On Oct 24, 2022, at 4:52 AM, Harri L <har...@gmail.com> wrote:

Hi,

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1d1e051d-4485-4fe0-a9c8-16afdc5d80e6n%40googlegroups.com.

[Andrew Harris]

> In many cases, including in the standard libraries, there are functions that return errors and then accompanying functions with names like MustFoo() that just call Foo() and panic if there's an error.

At least inside the standard lib, the uses of MustXxx document why they behave the way they do. I think the language could be written something like "this package considers any error in constructing a template to be a severe error that necessitates panic", or "... to be a runtime exception" without changing the reasoning about why that behavior has been chosen.

[Axel Wagner]

ISTM that a lot of your arguments boil down to throwing two disparate things into the same pot:

1. Detecting and handling failure modes outside the current process, which have to be expected and should be dealt with gracefully by a correct program.

2. Limitations of Go's type system, which result in bugs, which are detected at runtime, which imply that the program is fundamentally incorrect and can't be relied on.

They are fundamentally different things. Languages with exceptions tend to use the same (or similar) mechanisms for both, but Go is not one of that.

In Go, the first is an `error`, the second a `panic`. In Java, the first is an Exception and the second a RuntimeException.

Talking about "checked" vs. "unchecked exception" for the difference between `return err` and `panic` is IMO misunderstanding the nomenclature. The difference between those two is "no exception" vs. "exception". The difference between "checked and unchecked exceptions" is between returning `error` and returning `*os.PathError`, for example. I would say Go uses "checked" anything, if the compiler would force a function to declare which error types it can return and then force the caller to type-switch on those types.

On Mon, Oct 24, 2022 at 6:31 AM 'Daniel Lepage' via golang-nuts <golan...@googlegroups.com> wrote:

In practice, though, Go code *does* have a lot of unchecked exception usage - every time someone ignores the 'ok' on a type assertion, calls panic() or t.Fatal() from a helper function instead of returning an error, or writes a MustFoo() version of some function that panics instead of returning an error, that's a point where they *could* be using checked exceptions (i.e. return values), and have chosen not to because the code is easier to write.

I am one of the most adamant supporters of gratuitous usage of `panic` I know of and I almost never do it to make code easier to write. It is always because 1. I know that a correct program should fulfill some invariant, 2. the Go type system can't express that, so 3. a panic is the next best thing.

That's even true for the template `Must` functions. In an ideal world, the compiler would already verify that the template strings are well-formed (similar to `go vet` does for `Printf`). But it can't, so we have to delay that type-check to runtime.

Panicing in those situations is the right thing to do, because that code shouldn't have run in the first place. Pretending that incorrect code can police itself by returning an error or somesuch is, in my opinion, delusional. It just leads to an exponentially exploding space of incorrect states the program could be in and the programmer must consider. And to *even buggier and less stable* code, because it's impossible to reason about it.

Writing v.(T) isn't "lazy error handling". It's "I'm restricting the state-space of the program to those executions in which v has dynamic value T - I wish the compiler had done it for me".

 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAAViQtiXBHsVg4r-aTWR1mQWDX_7eRfhtPOKZ_YaNiP2ai7iSw%40mail.gmail.com.

[Ian Lance Taylor]

On Sun, Oct 23, 2022 at 9:31 PM 'Daniel Lepage' via golang-nuts
<golan...@googlegroups.com> wrote:

...

> 3. Streamlining shouldn't only apply to error handling that terminates the function.
>
> Unlike panics, errors are values, and should be treated as such, which means that the calling function should be able to decide what to do, and this should include continuing. Frequently it won't - a lot of error handling is just return fmt.Errorf("more context %w", err) - but any proposal that assumes that it *always* won't is, IMO, confusing errors with panics. This is the question that first started this thread - I didn't understand why all the existing error proposals explicitly required that a function terminate when it encounters an error, and AFAICT the answer is "because people are used to thinking of errors more like panics than like return values".

For what it's worth, I see this differently. The existing language is
not going to go away, and it's pretty good at handling the cases where
an error occurs and the function does not return. Those cases are by
their nature all distinct. They are not boilerplate. The way we
write them today is fine: easy to read and not too hard to write.
When people writing Go complain about error handling, what they are
complaining about is the repetitive boilerplate, particularly "if err
!= nil { return nil, err }". If we make any substantive changes to
the language or standard library for better error handling, that is
what we should address. If we can address other cases, fine, but as
they already work OK they should not be the focus of any substantive
change.

> Is part of the problem that the discussion around the try/check/handle/etc. proposals got so involved that nobody wants to even consider anything that looks too similar to those? Would it be more palatable if I proposed it with names that made it clearer that this is about the consolidation of error handling rather than an attempt to replace it entirely?
>
> onErrors {
> if must Foo(must Bar(), must Baz()) > 1 {
> ...
> }
> } on err {
> ...
> }

While error handling is important, the non-error code path is more
important. Somebody reading a function should be able to easily and
naturally focus on the non-error code. That works moderately well
today, as the style is "if err != nil { ... }" where the "..." is
indented out of the normal flow. It's fairly easy for the reader to
skip over the error handling code in order to focus on the non-error
code. A syntactic construct such as you've written above buries the
lede: what you see first is the error path, but in many cases you
actually want to focus on the non-error path.

Ian

[Robert Engels]

But that highlights the value of exceptions - the non error path is very clean. For example when writing a file - it often doesn’t matter the reason it failed within the write function - could be an invalid path, illegal file name , out of disk space. If the code is properly decomposed that function can’t handle it - so it throws - and hopefully a higher level function is able to cope (by handling the specific exception) - maybe asking the user for a different file name or a different destination device.

And the writing function can easily cleanup any temp state due to the stack unwinding and AutoClosable, etc.

> On Oct 24, 2022, at 6:18 PM, Ian Lance Taylor <ia...@golang.org> wrote:
>
> On Sun, Oct 23, 2022 at 9:31 PM 'Daniel Lepage' via golang-nuts

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXHaQk9TfuAz-TUFCsz_-0kDKa_14f3gYER2ufHbhM73Q%40mail.gmail.com.

[Ian Lance Taylor]

On Mon, Oct 24, 2022 at 5:57 PM Robert Engels <ren...@ix.netcom.com> wrote:
>
> But that highlights the value of exceptions - the non error path is very clean. For example when writing a file - it often doesn’t matter the reason it failed within the write function - could be an invalid path, illegal file name , out of disk space. If the code is properly decomposed that function can’t handle it - so it throws - and hopefully a higher level function is able to cope (by handling the specific exception) - maybe asking the user for a different file name or a different destination device.
>
> And the writing function can easily cleanup any temp state due to the stack unwinding and AutoClosable, etc.

I did not mean to imply that that was the only consideration for error handling.

Go style is to avoid exceptions for other reasons
(https://go.dev/doc/faq#exceptions).

Ian

[Robert Engels]

I’ve read that many times and I don’t believe it holds much water. Even the example cited about handling the inability to open a file - the function can’t handle this because it does not know the intent which leads to the

If err != nil {
return err
}

boilerplate. This is exactly what checked exceptions are designed to address.

Sure, it you don’t properly decompose your functions the Go error handling makes this safer, but properly decompose the functions and checked exceptions make things far far easier.

> On Oct 24, 2022, at 10:28 PM, Ian Lance Taylor <ia...@golang.org> wrote:

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUBs8UWdHMji-gLK0Z_Lt1mZ59tFaK9YT3UzEQ%2BPYKU8A%40mail.gmail.com.

[Ian Lance Taylor]

On Mon, Oct 24, 2022 at 8:49 PM Robert Engels <ren...@ix.netcom.com> wrote:
>
> I’ve read that many times and I don’t believe it holds much water. Even the example cited about handling the inability to open a file - the function can’t handle this because it does not know the intent which leads to the
>
> If err != nil {
> return err
> }
>
> boilerplate. This is exactly what checked exceptions are designed to address.
>
> Sure, it you don’t properly decompose your functions the Go error handling makes this safer, but properly decompose the functions and checked exceptions make things far far easier.

Thanks, but this has been discussed at great length in the past. We
don't need to rehash yet again.

[robert engels]

Totally understandable, but then I think the Go team should also drop any proposals related to “improved error handling” - because you are going to arrive back where you started - maybe with the a slightly different syntax and that hardly seems worth the effort. Great engineering is built by standing on the shoulders of giants. I haven’t seen any arguments that refute their findings.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUDCMegGnDHJRxqwBnqt3jaEtCoXJJ9LYFP0kGCbj0Yvw%40mail.gmail.com.

[Robert Engels]

This is an interesting read for a language that doesn’t have exceptions (Swift) but interops with a language that does (obj-c). https://docs.swift.org/swift-book/LanguageGuide/ErrorHandling.html

Seems like a promising path for Go. 

On Oct 24, 2022, at 11:04 PM, robert engels <ren...@ix.netcom.com> wrote:

Totally understandable, but then I think the Go team should also drop any proposals related to “improved error handling” - because you are going to arrive back where you started - maybe with the a slightly different syntax and that hardly seems worth the effort. Great engineering is built by standing on the shoulders of giants. I haven’t seen any arguments that refute their findings.

[Ian Lance Taylor]

On Mon, Oct 24, 2022, 9:03 PM robert engels <ren...@ix.netcom.com> wrote:

Totally understandable, but then I think the Go team should also drop any proposals related to “improved error handling” - because you are going to arrive back where you started - maybe with the a slightly different syntax and that hardly seems worth the effort. Great engineering is built by standing on the shoulders of giants. I haven’t seen any arguments that refute their findings.

I'm not sure that the problem that you see is a problem that we are interested in solving.  Go is not Java, nor should it be.  We already have Java, and it's a fine language.

In any case it's been years since the Go team has made any proposals related to error handing.

[Robert Engels]

My apologies. I thought there was still an active Go2 “error proposal”. Time flies…

On Oct 25, 2022, at 1:14 AM, Ian Lance Taylor <ia...@golang.org> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVwoujwfrfMyXPMGGs0D89Cw8FURT7GokaJJXxtfqUc%3Dw%40mail.gmail.com.

[Daniel Lepage]

On Mon, Oct 24, 2022 at 7:18 PM Ian Lance Taylor <ia...@golang.org> wrote:

On Sun, Oct 23, 2022 at 9:31 PM 'Daniel Lepage' via golang-nuts
<golan...@googlegroups.com> wrote:

...


> 3. Streamlining shouldn't only apply to error handling that terminates the function.
>
> Unlike panics, errors are values, and should be treated as such, which means that the calling function should be able to decide what to do, and this should include continuing. Frequently it won't - a lot of error handling is just return fmt.Errorf("more context %w", err) - but any proposal that assumes that it *always* won't is, IMO, confusing errors with panics. This is the question that first started this thread - I didn't understand why all the existing error proposals explicitly required that a function terminate when it encounters an error, and AFAICT the answer is "because people are used to thinking of errors more like panics than like return values".

For what it's worth, I see this differently.  The existing language is
not going to go away, and it's pretty good at handling the cases where
an error occurs and the function does not return.  Those cases are by
their nature all distinct.  They are not boilerplate.  The way we
write them today is fine: easy to read and not too hard to write.
When people writing Go complain about error handling, what they are
complaining about is the repetitive boilerplate, particularly "if err
!= nil { return nil, err }".  If we make any substantive changes to
the language or standard library for better error handling, that is
what we should address.  If we can address other cases, fine, but as
they already work OK they should not be the focus of any substantive
change.

Ok, that makes sense, and I think answers my initial question - all recent error-handling proposals have been termination-based not because anyone is specifically against streamlining continuation cases, just because it's also not important to anyone.

> Is part of the problem that the discussion around the try/check/handle/etc. proposals got so involved that nobody wants to even consider anything that looks too similar to those? Would it be more palatable if I proposed it with names that made it clearer that this is about the consolidation of error handling rather than an attempt to replace it entirely?
>
> onErrors {
>     if must Foo(must Bar(), must Baz()) > 1 {
>       ...
>    }
> } on err {
>    ...
> }

While error handling is important, the non-error code path is more
important.  Somebody reading a function should be able to easily and
naturally focus on the non-error code.  That works moderately well
today, as the style is "if err != nil { ... }" where the "..." is
indented out of the normal flow.  It's fairly easy for the reader to
skip over the error handling code in order to focus on the non-error
code.  A syntactic construct such as you've written above buries the
lede: what you see first is the error path, but in many cases you
actually want to focus on the non-error path.

Sorry, "onErrors" was a poor choice of word to replace 'try'; I wasn't proposing putting the handling before the non-error code, just wondering if using different names for try/check/handle would help. Using my original proposal but with must/on instead of check/handle, code would look like this:

try {
    if foo, limit := must Foo(), must Limit(); foo > limit {
        if foo = must DiminishFoo(foo); foo > limit {
            return fmt.Errorf("foo value %d exceeds limit %d, even after diminishment", foo, limit)
      }
  }
} on err {
  return fmt.Errorf("enforcing foo limit: %w", err)
}

You can read the normal flow pretty easily here - you compute a value and a limit, and if the value exceeds the limit you try to reduce it, and if it still exceeds the limit you return an error. Foo(), Limit(), and DiminishFoo() all return (value, error) pairs, but we don't consider this part of the "normal" control flow, so the code that handles if any of them fail is separated into its own block slightly further down.

In contrast, the modern Go version buries the "normal" flow in a pile of error handling, and IMO is a lot harder to follow:

foo, err := Foo()
if err != nil {
    return fmt.Errorf("enforcing foo limit: %w", err)
}
limit, err := Limit()
if err != nil {
    return fmt.Errorf("enforcing foo limit: %w", err)
}
if foo > limit {
    foo, err = DiminishFoo(foo)
    if err != nil {
        return fmt.Errorf("enforcing foo limit: %v", err)
    }
    if foo > limit {
        return fmt.Errorf("foo value %d exceeds limit %d, even after diminishment", foo, limit)
    }
}

This is fundamentally what I'm trying to propose - I'm not trying to address the existence of boilerplate, just trying to make it easier to move identical error handling blocks to the end so that the normal flow is more visible.

I have no idea if "must" is better than "check" - it's maybe clearer that it's going to jump to error handling if it fails, but it also might be confusing given that existing funcs like MustCompile use panics instead of errors.

Thanks,

Dan

 

[Bakul Shah]

On Oct 25, 2022, at 2:28 PM, 'Daniel Lepage' via golang-nuts <golan...@googlegroups.com> wrote:
>
> In contrast, the modern Go version buries the "normal" flow in a pile of error handling, and IMO is a lot harder to follow:
>
> foo, err := Foo()
> if err != nil {
> return fmt.Errorf("enforcing foo limit: %w", err)
> }
> limit, err := Limit()
> if err != nil {
> return fmt.Errorf("enforcing foo limit: %w", err)
> }
> if foo > limit {
> foo, err = DiminishFoo(foo)
> if err != nil {
> return fmt.Errorf("enforcing foo limit: %v", err)
> }
> if foo > limit {
> return fmt.Errorf("foo value %d exceeds limit %d, even after diminishment", foo, limit)
> }
> }

Some languages have optional types so for example you can do

foo := Foo() or { return fmt.Error("enforcing foo limit: %w", err) }
limit := Limit() or { return fmt.Errorf("enforcing foo limit: %w", err) }
if foo > limit {
...
}

If you just want to punt the error to the caller, you do

foo := Foo()?
limit := Limit()?

etc. Now this is just syntactic sugar and semantically almost
identical to what Go does. Except that err becomes a builtin.
Making it a builtin can potentially be useful as the compiler
can embed information such as which line or expression generated
the error (or passed it on) for debugging.

A function returning such a value may be declared as

fn Foo() ?typeFoo {
var foo typeFoo
...
return foo
...
return err...
}

I find this easier to read as the error handling code doesn't
overwhelm the non-error logic and the compiler can check if a
Foo() caller doesn't handle the error case and print a message.
And I find this much less odious than try ... catch ... +
compilers can't easily check if exceptions are indeed handled
without adding more boilerplate code.

Though I don't think Go authors like this style.

[Ian Lance Taylor]

On Tue, Oct 25, 2022 at 2:28 PM Daniel Lepage <dple...@google.com> wrote:
>
> On Mon, Oct 24, 2022 at 7:18 PM Ian Lance Taylor <ia...@golang.org> wrote:
>>
>> While error handling is important, the non-error code path is more
>> important. Somebody reading a function should be able to easily and
>> naturally focus on the non-error code. That works moderately well
>> today, as the style is "if err != nil { ... }" where the "..." is
>> indented out of the normal flow. It's fairly easy for the reader to
>> skip over the error handling code in order to focus on the non-error
>> code. A syntactic construct such as you've written above buries the
>> lede: what you see first is the error path, but in many cases you
>> actually want to focus on the non-error path.
>
>
> Sorry, "onErrors" was a poor choice of word to replace 'try'; I wasn't proposing putting the handling before the non-error code, just wondering if using different names for try/check/handle would help. Using my original proposal but with must/on instead of check/handle, code would look like this:
>
> try {
> if foo, limit := must Foo(), must Limit(); foo > limit {
> if foo = must DiminishFoo(foo); foo > limit {
> return fmt.Errorf("foo value %d exceeds limit %d, even after diminishment", foo, limit)
> }
> }
> } on err {
> return fmt.Errorf("enforcing foo limit: %w", err)
> }

To be clear, what I'm grumbling about here is not "must". It's "try"
and "on err'.

> In contrast, the modern Go version buries the "normal" flow in a pile of error handling, and IMO is a lot harder to follow:
>
> foo, err := Foo()
> if err != nil {
> return fmt.Errorf("enforcing foo limit: %w", err)
> }
> limit, err := Limit()
> if err != nil {
> return fmt.Errorf("enforcing foo limit: %w", err)
> }
> if foo > limit {
> foo, err = DiminishFoo(foo)
> if err != nil {
> return fmt.Errorf("enforcing foo limit: %v", err)
> }
> if foo > limit {
> return fmt.Errorf("foo value %d exceeds limit %d, even after diminishment", foo, limit)
> }
> }

Obviously I've gotten pretty used to reading Go over the years, but I
find this code easy to follow. Every time I see "if err != nil" I
skip straight down to the right curly brace. I don't have to look at
the error handling code at all. Which is also true for your
example--except that I have to start with "try" and "on err" and
distinguish those. But perhaps that would become its own kind of
easily skipped boilerplate, I don't know.

Ian