io.Reader can return (n, err), both non-zero

[Nigel Tao]

The ubiquitous io.Reader interface's Read method returns a (int,
error). Surprisingly (for programmers coming from other languages with
a built-in or idiomatic Result<T, E> type), it's possible for both
return values to be non-zero. There's (often overlooked??) commentary
in https://pkg.go.dev/io#Reader that says:

> Callers should always process the n > 0 bytes returned before considering the error err.

So that (again, especially for programmers used to saying result? or
"try result" or similar Result<T, E> aware mechanisms in other
languages), the following Go code is incorrect:

n, err := r.Read(buffer, etc)
if err != nil {
return err
}
doStuffWith(buffer[:n])

---

Do any of the early Gophers remember the motivations for designing
Read's semantics in that way? At the libc or syscall level (and my
reading of FD.Read in internal/poll/fd_unix.go), I don't think
read(fd, etc) can return both non-zero.

Is it just that, if you've got a buffer of some sort (including the
compress/* packages), it can be more efficient (1 Read call instead of
2) to return (positiveN, io.EOF)?

[Ian Lance Taylor]

Some earlier discussions:

https://groups.google.com/g/golang-nuts/c/b78eRMOS0FA/m/jq8lAQhenaoJ
https://groups.google.com/g/golang-nuts/c/WGYJx3ICCW4/m/1L0WcN8eqmkJ

See in particular this message from Russ:
https://groups.google.com/g/golang-nuts/c/b78eRMOS0FA/m/sbbgr9MUo9QJ

Ian

[Nigel Tao]

On Wed, Mar 20, 2024 at 2:13 PM Ian Lance Taylor <ia...@golang.org> wrote:
> Some earlier discussions:
>
> https://groups.google.com/g/golang-nuts/c/b78eRMOS0FA/m/jq8lAQhenaoJ
> https://groups.google.com/g/golang-nuts/c/WGYJx3ICCW4/m/1L0WcN8eqmkJ
>
> See in particular this message from Russ:
> https://groups.google.com/g/golang-nuts/c/b78eRMOS0FA/m/sbbgr9MUo9QJ

Russ said:

> Once in a while we think about changing the definition
> to require n==0 when err==EOF but it doesn't help the
> more general case, a partial read that returns an error
> saying why more data wasn't returned (disk error, etc).

OK, partial reads are a thing, but one possible design (in hindsight,
not now) was to push the complexity in tracking that into Read
callees, not Read callers. Instead of a callee returning (positiveN,
nonNilErr), have it save nonNilErr to a struct field and return (0,
nonNilErr) on the next Read call.

I'd expect fewer programmers writing callees than callers, and they'd
generally be more experienced Go programmers. Anecdotally, when I was
doing a lot of Go code reviews some years ago, I'd often have to point
out the "Callers should always process the n > 0 bytes returned before
considering the error err" comment, often to the programmer's
surprise.

While it's relatively easy, *if you already know*, to do this:

n, err := r.Read(buffer)
doStuffWith(buffer[:n])

if err != nil {
return err
}

instead of this:

n, err := r.Read(buffer)

if err != nil {
return err
}
doStuffWith(buffer[:n])

it's not really obvious if you don't know that you don't know. And
sometimes doStuffWith is more than just "something += n". If
doStuffWith can itself lead to further errors (e.g. you're parsing the
bytes you've just read), you also have to be careful not to clobber
the err variable.

Anyway, it's far too late to change it. I was just wondering if there
was some motivating reason that I'd otherwise missed.

[Harri L]

+1

I’d love to hear more about the motivation behind the io.EOF as an error. And why select a strategy to prefer no discriminated unions for the error results? Why transport non-error information in error values (io.EOF, 'http.ErrServerClosed)?

“Normal” vs. “error” [control flow] is a fundamental semantic distinction, and probably the most important distinction in any programming language - Herb Shutter

The io.Reader has caused many problems during its existence. Even so, the os packages official documentation and the File.Read example starts with a potential bug and incorrect io.Reader usage. Should the if clause be:

if err != nil && err != io.EOF { ...

The issue #21852 gives an excellent overall picture of how deep unintuitive io.EOF usage can affect Go’s code and test base. It includes an interesting comment that maybe Go 2 will fix the io.Reader. Now it’s official that there will never be Go 2, maybe io.Reader2?

​

[Christian Stewart]

When sending Read() via an RPC call or traversing locks, it is significantly faster to return EOF in the same call as returning the rest of the available data, than to call Read a second time to get the EOF.

It's not that hard to handle the EOF semantics. Yes, you have to know what io.EOF means. But it's equally as hard as knowing to check if the file ended with any other syntax.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3f26b58e-b451-4fdd-b0b8-17638a30ce4en%40googlegroups.com.

[Diego Joss]

Hi

On Thu, 21 Mar 2024 at 19:23, 'Christian Stewart' via golang-nuts <golan...@googlegroups.com> wrote:

When sending Read() via an RPC call or traversing locks, it is significantly faster to return EOF in the same call as returning the rest of the available data, than to call Read a second time to get the EOF.

Just for sake of discussion/argumentation, it's still possible for the callee implementation to cache the error status which is returned in the next Read call. Thus a single RPC (or lock) call is performed.

--

Diego Joss

[Tamás Gulácsi]

I don't really get it.

The current documentation says that bespite err!=nil, some data may be read (n!=0).

If one obeys this rule, then it will consume the data, then handle the error.

If we change then io.Reader's documentation (and all std lib implementations)...

Ok, now I may be understand: If an io.Reader implementation obeys this stricter rule, then it would become wrong with the new (lax) rule...

What about some documentation and an "func io.Read(io.Reader, p) (int, error)" wrapper function that caches the error and only returns err!=nil iff n!=0 ?

But this maybe just complicate things?

[Diego Joss]

Sorry I might've been to terse in my previous email.

What I meant is that the single RPC (or lock) call is not a valid justification for the non-exclusive return values of io.Reader. 

Let's consider io.Read specification that must return err != nil iff n == 0, then it is possible to implement it without extra RPC calls by caching the error value.

-Diego