unexpected EOF with fmt.Scanner

[Steven Penny]

If I want to scan through a string, I can do this:

~~~go
package main

import (
   "fmt"
   "strings"
)

func main() {
   r := strings.NewReader("west north east")
   for {
      var s string
      _, e := fmt.Fscan(r, &s)
      fmt.Printf("%q %v\n", s, e)
      if e != nil { break }
   }
}
~~~

Result:

~~~
"west" <nil>
"north" <nil>
"east" <nil>
"" EOF
~~~

I recently discovered `fmt.Scanner` [1], so I thought I would try to implement it. I came up with this:

~~~go
package main

import (
   "fmt"
   "strings"
)

type comma struct { tok string }

func (c *comma) Scan(s fmt.ScanState, r rune) error {
   tok, err := s.Token(false, func(r rune) bool {
      return r != ','
   })
   if err != nil {
      return err
   }
   c.tok = string(tok)
   if _, _, err := s.ReadRune(); err != nil {
      return err
   }
   return nil
}

func main() {
   r := strings.NewReader("west,north,east")
   for {
      var c comma
      _, e := fmt.Fscan(r, &c)
      fmt.Printf("%q %v\n", c.tok, e)
      if e != nil { break }
   }
}
~~~

Result:

~~~
"west" <nil>
"north" <nil>
"east" unexpected EOF
~~~

So the result is pretty similar, but what bothers me is the `unexpected EOF`. It seems it is due to this code:

https://github.com/golang/go/blob/3075ffc93e962792ddf43b2a528ef19b1577ffb7/src/fmt/scan.go#L956-L966

It seems like `EOF` should be valid in this case, or perhaps I dont understand the reasoning for it to be unexpected.

1. https://golang.org/pkg/fmt#Scanner

[Ian Lance Taylor]

If you want to use fmt.Scanner, the Scan method of the types you are
using should not return an acceptable io.EOF. That doesn't make sense
as an error from a Scan method. A Scan method that encounters io.EOF
at the end of valid input should store the valid result and return
nil.

Ian

[Steven Penny]

Ian, that doesnt make sense. Then how will the scanner know that the input is empty? EOF is the only graceful error I know for this case, its the only sentinel that I am aware of, to differentiate a "real error" from "input is empty".

[Ian Lance Taylor]

On Thu, May 27, 2021 at 2:25 PM Steven Penny <srp...@gmail.com> wrote:
>
> Ian, that doesnt make sense. Then how will the scanner know that the input is empty? EOF is the only graceful error I know for this case, its the only sentinel that I am aware of, to differentiate a "real error" from "input is empty".

You're right, I took a look at the regular scanners, and they call
panic(io.EOF) when they hit EOF. Try changing your Scan method to do
that and see if that works. If it does, we ought to clarify that in
the documentation of fmt.Scanner.

Ian

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/cee375a8-3b79-4cfb-96d9-cb8ba24b5a57n%40googlegroups.com.

[Steven Penny]

Did you even look at my example? If EOF is not right, then what is? How do I do what Im trying to do?

[Ian Lance Taylor]

On Thu, May 27, 2021 at 3:33 PM Steven Penny <srp...@gmail.com> wrote:
>
> Did you even look at my example? If EOF is not right, then what is? How do I do what Im trying to do?

https://play.golang.org/p/JlbEFvmsUum

Ian

[Steven Penny]

Hm interesting - so if you just panic(io.EOF) instead of return io.EOF, then it works as expected. Must be the code is recovering the panic somewhere. Thanks for the help