encoding/binary limits(or bug?)

[Sergey Vidyuk]

Hi,

I am trying to do large read using binary.Read and getting following error

Error during vector conversion:read tcp 127.0.0.1:62423->127.0.0.1:1234: read: invalid argument, vector size:500000000, element: []int64

which comes from err = binary.Read(r, order, arr)

arr is 500M elements of int64(slice). Underlying reader is bufio.Reader around tcp socket. Is this expected behaviour?

go version go1.6.2 darwin/amd64

Regards,

Sergey

[Matthew Dempsky]

If you're able to provide a minimal reproducible test case and file an issue at golang.org/issue/new, we can take a look.

I'm a bit suspicious though because "Error during vector conversion" and "vector size" aren't strings that appear anywhere in the Go source tree.  It seems likely those errors are being caused by some other library code that you're using.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jakob Borg]

2016-07-04 17:54 GMT+02:00 Sergey Vidyuk <svi...@gmail.com>:
> Hi,
>
> I am trying to do large read using binary.Read and getting following error
>
> Error during vector conversion:read tcp 127.0.0.1:62423->127.0.0.1:1234:
> read: invalid argument, vector size:500000000, element: []int64
>
> which comes from err = binary.Read(r, order, arr)
>
> arr is 500M elements of int64(slice). Underlying reader is bufio.Reader
> around tcp socket. Is this expected behaviour?

You'll get this error for int64 slices longer than 2^31/8 items, as
the read size then exceeds an int32. At least on platforms with a 32
bit size_t. Maybe binary.Read() should handle this and split the reads
up, I don't know.

//jb

[Sergey Vidyuk]

Thanks Jakob,

I am running OSX 10.11 which is I suspect LP64. I am trying to create small reproducer.

[Sergey Vidyuk]

Filed https://github.com/golang/go/issues/16266