compress/gzip and io.UnexpectedEOF

[Константин Иванов]

Hello.

I was fight with the compress/gzip package and the snappy. The problem is the packages returns non-io.EOF error when their input is empty.

Yes, I'm about reading. Take a look at the playground.

All I want to know why the Read method returns the io.UnexpectedEOF?
Is it unexpected? (No-no-no-no-no-no-o-o-o-o!)
Why ioutil.ReadAll returns an error? (To be more precise, why is it normal behavior (of Readers)?)

Why the flate/gzip/snappy/etc readers break the concept of the io.Reader?
(snappy returns 'corrupt input' instead of io.UnexpectedEOF)
How do I distinguish EOF from unexpected EOF (when it's actually unexpected)?

[Peter Waller]

I think it's unexpected because you aren't closing the GZip writer. So to the reader, the output looks truncated (and hence you get an unexpected EOF). Looking at the source of Close it looks like there is normally a trailer written containing a digest and the final length:

https://tip.golang.org/src/compress/gzip/gzip.go?s=6079:6109#L237

If I modify your "Flush" call to "Close" then everything seems fine.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Waller]

Just to add to that, maybe you want to have your reader block while the writer isn't providing new bytes, rather than give EOF.

In which case you might want to use an io.Pipe() instead of a bytes.Buffer. Then you can concurrently read/write, and the reader will read bytes when the Writer flushes without needing to close the stream and without giving an UnexpectedEOF. You have to bear in mind that the bytes.Buffer can't know when EOF is (because there is no way to denote in the buffer that you're finished writing). The buffer will return EOF if you try to it when it is empty.

[Константин Иванов]

Hah, Close works like a charm. Thanks a lot.

For some reason I was sure that the Flush writes all the necessary information.

As always the answer was under my nose. Thank you for taking the time.

воскресенье, 24 января 2016 г., 19:15:45 UTC+3 пользователь Peter Waller написал: