Handling EOF when using json.NewDecoder() from named pipe

[Christopher C]

Hello all!

I'm trying to read json objects from a named pipe.  The pipe will be  filled intermittently by bash scripts.  After the Decode() of the first object, any more calls to Decode() will return EOF.  This seems proper since the script has completed, but once it errors with EOF, there doesn't seem to be a way to read any more.

Is there a way to 'reset' the decoder so when another script writes to the pipe it can process the next object, or should I be doing some pipe length validation before trying to decode?

Current read code snippet  is...

decoder := json.NewDecoder(fpipe)
for {
err := decoder.Decode(&msg)
if err != nil {
if err == io.EOF {
// how to reset this?
} else {
logger.Fatal(err)
}
} else {
// send out the msg

[Volker Dobler]

Why do you use a json.Decoder? It seems as reading

everything (io.ReadAll) until EOF and json.Unmarshal'ling

would be a cleaner/simpler solution?

V.

[Christopher C]

I was thinking partial reads could be an issue and the Decoder seemed to do the initial checking for me. Would the ReadAll() be able to recover from EOF state?

[Brian Candler]

Are you sure that the writer isn't closing the named pipe between writing JSON objects? If necessary you can check this with strace.

I'm pretty sure you'll get an EOF in the *reader* when the *writer* closes from their side. You can demonstrate this with the shell:

(In terminal 1)

mkfifo /tmp/fifo

cat /tmp/fifo

(In terminal 2)

cat >/tmp/fifo

abcd

efgh

<< wait a while >>

^D

When you hit ^D in terminal 2, the cat in terminal 1 ends.

The solution (or as you asked, "how to reset this?") is simply to re-open the fifo for reading.

[Volker Dobler]

On Tuesday, 17 October 2023 at 14:40:47 UTC+2 Christopher C wrote:

I was thinking partial reads could be an issue

In what regards are "partial reads" less well covered by io.ReadAll

than via json.Decoder?

 

and the Decoder seemed to do the initial checking for me. Would the ReadAll() be able to recover from EOF state?

io.ReadAll cannot "recover" from EOF just like an Decoder cannot.

This EOF is smth that happens during _reading_ and both, io.ReadAll

and json.Decoder do the actual read via identical methods.

If your io.Reader allows reading up to EOF and than reading

more (no idea how this is going to happen, but let's assume

your underlying Reader somehow allows this) then it doesn't

matter.

io.ReadAll reads until EOF, forget about "partial reads".

Decoding can be done on the read stuff. I think this is

clearer.

Once your io.Reader signals EOF you have to find some

way to read more (???) or reset the Reader and this has

nothing to do with _how_ you read from that Reader.

V.