io.Reader's Read with n > 0, os.EOF

[rh]

The documentation on io.Reader's read notes:

// At the end of the input stream, Read returns 0, os.EOF.

// Read may return a non-zero number of bytes with a non-nil err.

// In particular, a Read that exhausts the input may return n > 0, os.EOF.

However, returning n > 0, os.EOF doesn't seem to fit with what some of the io functions expect.

For example, MultiReader's Read notes:

// Well-behaved Readers should never

// return non-zero bytes read with an

// EOF.  But if so, we clean it.

Who is correct?

[Jessta]

io.multiReader.Read() looks kind of wrong to me.
It will return n < cap(p) with err == nil. Which seems strange to me.
I wouldn't expect to get less than what I'd asked for unless an error
occurred or EOF.

- jessta

--
=====================
http://jessta.id.au

[bflm]

The io.Reader may return n > 0 and os.EOF at once, the MultiReader
deals with that situation correctly. I fail to see where the problem
is.

[chris dollin]

The problem is in the comment that says "Well-behaved Readers should
never return non-zero bytes read with an EOF". Is that true or not? If
code routinely copes with this, is it really an issue?

Chris

--
Chris "allusive" Dollin

[Jessta]

On Sat, Mar 26, 2011 at 10:14 AM, bflm <befeleme...@gmail.com> wrote:
> The io.Reader may return n > 0 and os.EOF at once, the MultiReader
> deals with that situation correctly. I fail to see where the problem
> is.

But the multiReader is also an io.Reader().
But doesn't follow the same convention as the io.Reader()

[bflm]

On Mar 26, 11:26 am, chris dollin <ehog.he...@googlemail.com> wrote:
> The problem is in the comment that says "Well-behaved Readers should
> never return non-zero bytes read with an EOF". Is that true or not? If
> code routinely copes with this, is it really an issue?

If one doesn't accidentally cut of the "But if so, we clean it." part
of the cited comment then i fail to see where the problem is supposed
to be.

[chris dollin]

The problem is not that MultiReader cleans up. The problem is that
it's not clear whether it's true or not that "Well-behaved Readers should
never return non-zero bytes read with an EOF" and what one should
do in consequence.

* if I am writing a Reader, am I allowed to return n > 0 bytes with EOF?

* if I am using a Reader, should I allow for n > 0 with EOF?

* if implementors "clean up" anyway, what point is there in calling
some behaviour "well behaved", since we don't actually care?

One would just like to know what the contract is without needing
a well-behavedness weasel.

[yy]

2011/3/26 chris dollin <ehog....@googlemail.com>:

>
> * if I am writing a Reader, am I allowed to return n > 0 bytes with EOF?

Allowed? Yes. Encouraged? No.

> * if I am using a Reader, should I allow for n > 0 with EOF?

It is allowed, so yes, you should. In many situations, it won't matter
if you don't, but you should if you want to use arbitrary Readers.

> * if implementors "clean up" anyway, what point is there in calling
>  some behaviour "well behaved", since we don't actually care?

We cannot care. Reader is an interface, and you cannot impose limits
on how it is implemented. You can give some guidelines on how you
expect it to be used, but you cannot assume it will be used like that.

--
- yiyus || JGL .

[Jessta]

On Sat, Mar 26, 2011 at 11:06 AM, yy <yiyu...@gmail.com> wrote:
> 2011/3/26 chris dollin <ehog....@googlemail.com>:
>>
>> * if I am writing a Reader, am I allowed to return n > 0 bytes with EOF?
>
> Allowed? Yes. Encouraged? No.

Since the docs for io.Reader seem to say
this is expected behaviour, how can this not be encouraged?

[chris dollin]

On 26 March 2011 11:06, yy <yiyu...@gmail.com> wrote:
> 2011/3/26 chris dollin <ehog....@googlemail.com>:
>>
>> * if I am writing a Reader, am I allowed to return n > 0 bytes with EOF?
>
> Allowed? Yes. Encouraged? No.

And the reason for the discouragement is ... what?

>> * if implementors "clean up" anyway, what point is there in calling
>>  some behaviour "well behaved", since we don't actually care?
>
> We cannot care. Reader is an interface, and you cannot impose limits
> on how it is implemented.

So the notion of "well behaved" doesn't buy us anything.

I'm missing something.

> You can give some guidelines on how you
> expect it to be used, but you cannot assume it will be used like that.

Mmmm.

There's more to an interface than just method declarations; you
just can't always write that more using only the programming language.

[bflm]

On Mar 26, 11:59 am, chris dollin <ehog.he...@googlemail.com> wrote:
> The problem is not that MultiReader cleans up. The problem is that
> it's not clear whether it's true or not that  "Well-behaved Readers should
> never return non-zero bytes read with an EOF" and what one should
> do in consequence.
>
> * if I am writing a Reader, am I allowed to return n > 0 bytes with EOF?
>
> * if I am using a Reader, should I allow for n > 0 with EOF?
>
> * if implementors "clean up" anyway, what point is there in calling
>   some behaviour "well behaved", since we don't actually care?
>
> One would just like to know what the contract is without needing
> a well-behavedness weasel.

The io. Reader docs says that "Read that exhausts the input may return
n > 0, os.EOF." and that answers your first two questions.

The comment in MultiReader documents the correct handling of the
situation where n > 0 && err == os.EOF. Why not call an io.Reader
which gives the last data chunk with n > 0 and err == nil (with the
following Read() returning 0, os.EOF) "well-behaved"? Should "usual"
or "all of our own" be better? Maybe. Additionally that comment is not
part of the API definition, so I still fail to see where the problem
is seen as long as the MultiReader code works correctly in the way it
is publicly documented.

[chris dollin]

On 26 March 2011 11:18, bflm <befeleme...@gmail.com> wrote:
> On Mar 26, 11:59 am, chris dollin <ehog.he...@googlemail.com> wrote:
>> The problem is not that MultiReader cleans up. The problem is that
>> it's not clear whether it's true or not that  "Well-behaved Readers should
>> never return non-zero bytes read with an EOF" and what one should
>> do in consequence.
>>
>> * if I am writing a Reader, am I allowed to return n > 0 bytes with EOF?
>>
>> * if I am using a Reader, should I allow for n > 0 with EOF?
>>
>> * if implementors "clean up" anyway, what point is there in calling
>>   some behaviour "well behaved", since we don't actually care?
>>
>> One would just like to know what the contract is without needing
>> a well-behavedness weasel.
>
> The io. Reader docs says that "Read that exhausts the input may return
> n > 0, os.EOF." and that answers your first two questions.

Good.

> The comment in MultiReader documents the correct handling of the
> situation where n > 0 && err == os.EOF. Why not call an io.Reader
> which gives the last data chunk with n > 0 and err == nil (with the
> following Read() returning 0, os.EOF) "well-behaved"?

Because it doesn't have any useful consequence.

> Should "usual" or "all of our own" be better?

Just don't say anything about it at all. The case has to be handled,
so handle it.

> Maybe. Additionally that comment is not
> part of the API definition, so I still fail to see where the problem
> is seen as long as the MultiReader code works correctly in the way it
> is publicly documented.

The code (well, the comment in the code) is actively misleading,
even though it works correctly. It says "this shouldn't happen", despite
the fact that "this" is explicitly allowed for by the io.Reader documentation
you quote above.

Something more like, "this rarely happens but we must be careful to handle
it correctly" would, I believe, be clearer. It doesn't suggest obligations that
are in fact not imposed by the interface being used and which the code
has in any case to assume my not be satisfied.

[Russ Cox]

io.Reader is correct.

Once in a while we think about changing the definition
to require n==0 when err==EOF but it doesn't help the
more general case, a partial read that returns an error
saying why more data wasn't returned (disk error, etc).
You should handle that case correctly (usually += n before
looking at err), and if you do, handling EOF falls out naturally.

Russ

[jimmy frasche]

You could always do something like this: (thoroughly untested)

type EZReader struct {
r io.Reader
err os.Error
}

func NewEZReader(rin io.Reader) io.Reader {
return &EZReader{r: rin}
}

func (r *EZReader) Read(p []byte) (int, os.Error) {
if r.err != nil {
return r.err
}
n, err := r.Read(p)
if err != nil {
r.err = err
}
return n, nil
}