io.LimitedReaderAt proposal

[Aliaksandr Valialkin]

Hi.

It would be cool to have io.LimitedReaderAt similar to io.LimitedReader with the following implementation:

type LimitedReaderAt {

    R ReaderAt

    Offset int64

    N int64

}

func (l *LimitedReaderAt) Read(p []byte) (n int, err error) {

    if l.N <= 0 {

        return 0, EOF

    }

    if (len(p) > lr.N) {

        p = p[0:lr.N]

    }

    n, err = l.R.ReadAt(p, l.Offset)

    l.Offset += int64(n)

    l.N -= int64(n)

    return

}

This type could be used for concurrent reading of data chunks starting from the given offset from the underlying reader (for example, os.File) from multiple goroutines. This type also could be used by optimized io.ReaderFrom implementations such as TCPConn.ReadFrom() in order to extract information required for passing to syscall.Sendfile(). Currently TCPConn.ReadFrom() knows only how to extract this info from LimitedReader, which lacks Offset parameter.

[Rémy Oudompheng]

There is already io.SectionReader.

Rémy.

2013/9/11, Aliaksandr Valialkin <val...@gmail.com>:

> Hi.
>
> It would be cool to have io.LimitedReaderAt similar to

> io.LimitedReader<http://golang.org/pkg/io/#LimitedReader> with

> the following implementation:
>
> type LimitedReaderAt {
> R ReaderAt
> Offset int64
> N int64
> }
>
> func (l *LimitedReaderAt) Read(p []byte) (n int, err error) {
> if l.N <= 0 {
> return 0, EOF
> }
> if (len(p) > lr.N) {
> p = p[0:lr.N]
> }
> n, err = l.R.ReadAt(p, l.Offset)
> l.Offset += int64(n)
> l.N -= int64(n)
> return
> }
>
> This type could be used for concurrent reading of data chunks starting from
>
> the given offset from the underlying reader (for example, os.File) from
> multiple goroutines. This type also could be used by optimized

> io.ReaderFrom<http://golang.org/pkg/io/#ReaderFrom> implementations
> such as TCPConn.ReadFrom() <http://golang.org/pkg/net/#TCPConn.ReadFrom> in

>
> order to extract information required for passing to syscall.Sendfile().
> Currently TCPConn.ReadFrom() knows only how to extract this info from

> LimitedReader <http://golang.org/src/pkg/net/sendfile_linux.go#L24>, which
> lacks Offset parameter.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

[Aliaksandr Valialkin]

On Wednesday, September 11, 2013 1:42:19 PM UTC+3, Rémy Oudompheng wrote:

There is already io.SectionReader.

Ah, nice!

Then it would be great to have special handling for io.SectionReader in sendFile() implementation if io.File is wrapped into io.SectionReader. This will speed up sending file sections over the network.

[Aliaksandr Valialkin]

I filed the corresponding bug - https://code.google.com/p/go/issues/detail?id=6374 .

[Sokolov Yura]

Detecting io.SectionReader as well as io.LimitedReader in TCPConn.ReadFrom is the fastest option without doubt.
But still you can dup file, seek it and then wrap io.CopyN:

tmpFile := os.NewFile(syscall.Dup(myFile.Fd()))
tmpFile.Seek(pos, os.SEEK_SET)
defer tmpFile.Close() // or track it in a different way
io.CopyN(socket, tmpFile, size)

Yes, it is a bit ugly, but workable.

четверг, 12 сентября 2013 г., 20:08:08 UTC+4 пользователь Aliaksandr Valialkin написал: