Issue with bufio.ReadString in Windows

[Archos]

There is a problem in Windows system with bufio.ReadString and another functions like that, which have a delimiter of one byte of like argument.

In Unix systems, the byte to indicate EOL is '\n' (1 byte). In change, you need 2 bytes in Windows systems "\r\n" but you only can use one byte, so you would use:

buf.ReadString('\r') => this gives issues because the '\n' will be added in the next reading function, if any
buf.ReadString('\n') => you finish with the string that you want plus '\r' at the end

In addition, since a lot of people use '\n' to read lines in Unix systems, I think that would be better and portable if there would be a coinstant like io.EOL set to '\n' in Unix, and windows would deppen of how is managed the anterior issue, but you see the idea.

[Rémy Oudompheng]

On 2012/9/7 Archos <raul...@sent.com> wrote:
> In Unix systems, the byte to indicate EOL is '\n' (1 byte). In change, you
> need 2 bytes in Windows systems "\r\n" but you only can use one byte, so you
> would use:
>
> buf.ReadString('\r') => this gives issues because the '\n' will be added in
> the next reading function, if any
> buf.ReadString('\n') => you finish with the string that you want plus '\r'
> at the end

bufio.Reader.ReadString always returns the delimiter in the result.
You have extra bytes in all bases.

> In addition, since a lot of people use '\n' to read lines in Unix systems, I
> think that would be better and portable if there would be a coinstant like
> io.EOL set to '\n' in Unix, and windows would deppen of how is managed the
> anterior issue, but you see the idea.

The "portable io.EOL" encourages people to produce system dependent
output, I am very firmly opposed to that.

Rémy.

[Archos]

El jueves, 6 de septiembre de 2012 23:11:19 UTC+1, Rémy Oudompheng escribió:

On 2012/9/7 Archos <raul...@sent.com> wrote:
> In Unix systems, the byte to indicate EOL is '\n' (1 byte). In change, you
> need 2 bytes in Windows systems "\r\n" but you only can use one byte, so you
> would use:
>
> buf.ReadString('\r') => this gives issues because the '\n' will be added in
> the next reading function, if any
> buf.ReadString('\n') => you finish with the string that you want plus '\r'
> at the end

bufio.Reader.ReadString always returns the delimiter in the result.
You have extra bytes in all bases.

Yes, in Windows there is to use '\n' (like in Unix) to avoid the issued showed down:

package main

import (
    "bufio"
    "fmt"
    "log"
    "os"
)

const EOL = '\r'

func main() {
    buf := bufio.NewReader(os.Stdin)

    for i := 0; i < 2; i++ {
        fmt.Print("Write: ")
        line, err := buf.ReadString(EOL)
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("\nentered: %q\n", line)
    }
}

== Windows w/ EOL = '\n'

Write: Karma

entered: "Karma\r\n"
Write: Hotel

entered: "Hotel\r\n"

== Windows w/ EOL = '\r'
Write: Karma

entered: "Karma\r"
Write: Hotel

entered: "\nHotel\r"

[Ethan Burns]

Won't something like this http://play.golang.org/p/5fhf0R1Llb work fine on both Unix and Windows?

[Ethan Burns]

On Sun, Sep 9, 2012 at 3:56 AM, justay w <wr2...@gmail.com> wrote:
> hi ,
>
> I got a other question about ReadString
> if my file have many lines, how can i read out them all using ReadString
> i try to use: ReadString('\n') just read the firstline.
> ths for help.

You can continually call ReadString('\n') to read more strings. Here
is a toy example: http://itsapad.appspot.com/288002. It reads all
lines from a file called "tmp.go" and prints them preceded by their
line number (starting with 0). The readLines function is what you
want, it returns all lines read from the given reader in a slice.
Remember to be careful when reading in entire files like this,
however, because if you don't necessarily trust the source then they
can easily eat up all of your memory.

Best,
Ethan

[justay w]

hi ,

I got a other question about ReadString 

if my file have many lines, how can i read out them all using ReadString

i try to use:  ReadString('\n') just read the firstline. 

ths for help.

BR

Jerry

2012/9/7 Ethan Burns <burns...@gmail.com>

[Archos]

It fails in Windows. There is to use:

const EOL = '\n'

If it's used '\r' then the '\n' will be added in the next string:

Write: foo

entered: "foo"
Write: bar

entered: "\nbar"

[Ethan Burns]

On Sunday, September 9, 2012 12:59:55 PM UTC-4, Archos wrote:

It fails in Windows. There is to use:

const EOL = '\n'

If it's used '\r' then the '\n' will be added in the next string:

Write: foo

entered: "foo"
Write: bar

entered: "\nbar"

I am not sure that I understand what you are saying.  I did not suggest using '\r' as the end of line delimiter; I said to use '\n' as the delimiter and then trim off \r and \n runes.  According to the documentation for ReadString:

"ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadString returns err != nil if and only if the returned data does not end in delim."

So, ReadString('\n') will read and return the string, the \r, and the \n on Windows.  The \n will not appear on the next line because it will have been read by the previous line.  You can then use strings.TrimRight to trim off any number of trailing \r and \n runes---on Unix there will be 0 \r runes and on Windows there will be 1.

Best,

Ethan

[Ethan Burns]

On Sunday, September 9, 2012 1:12:04 PM UTC-4, Ethan Burns wrote:

On Sunday, September 9, 2012 12:59:55 PM UTC-4, Archos wrote:

It fails in Windows. There is to use:

const EOL = '\n'

If it's used '\r' then the '\n' will be added in the next string:

Write: foo

entered: "foo"
Write: bar

entered: "\nbar"

I am not sure that I understand what you are saying.  I did not suggest using '\r' as the end of line delimiter; I said to use '\n' as the delimiter and then trim off \r and \n runes.

Sorry, I see you confusion.  There was a bug in my example.  I should look like this:  http://play.golang.org/p/_9N_RwmBvd.

Ethan