Casting []byte to []uint64

[Bill Katz]

Hi,

We'd like to do a zero-copy conversion of []byte to []uint64 where the underlying memory for the byte slice is properly aligned for N uint64 values.  After looking at (6) in the unsafe package documentation, I'm wondering if the code below is valid because I only use reflect.SliceHeader as pointers, and it seems to fit pattern 6.

https://play.golang.org/p/FxxKIK08pX

package main

import (

"fmt"

"reflect"

"unsafe"

)

func byteToUint64(b []byte) (out []uint64, err error) {

inhdr := *(*reflect.SliceHeader)(unsafe.Pointer(&b))

if inhdr.Len % 8 != 0 {

err = fmt.Errorf("cannot convert non-aligned []byte length (%d) to []uint64", inhdr.Len)

return

}

if inhdr.Cap % 8 != 0 {

err = fmt.Errorf("cannot convert non-aligned []byte capacity (%d) to []uint64", inhdr.Cap)

return

}

outhdr := *(*reflect.SliceHeader)(unsafe.Pointer(&out))

outhdr.Len = inhdr.Len / 8

outhdr.Cap = inhdr.Cap / 8

outhdr.Data = inhdr.Data

out = *(*[]uint64)(unsafe.Pointer(&outhdr))

return

}

func main() {

b := make([]byte, 24)

u, err := byteToUint64(b)

if err != nil {

fmt.Println(err)

} else {

fmt.Printf("b: %v\n", b)

fmt.Printf("u: %v\n", u)

fmt.Printf("len(u) = %d, cap(u) = %d\n", len(u), cap(u))

u[0] = 15

fmt.Printf("Set u[0] to 15\n")

fmt.Printf("b: %v\n", b)

fmt.Printf("u: %v\n", u)

u[2] = 255

fmt.Printf("Set u[2] to 255\n")

fmt.Printf("b: %v\n", b)

fmt.Printf("u: %v\n", u)

}

}

Outputs:

b: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
u: [0 0 0]
len(u) = 3, cap(u) = 3
Set u[0] to 15
b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
u: [15 0 0]
Set u[2] to 255
b: [15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 0 0 0 0 0 0 0]
u: [15 0 255]

[xingtao zhao]

I think you need to check if inhdr.Data is aligned with 8 bytes as well.

[Bill Katz]

I thought that's handled by the inhdr.Cap % 8 == 0 check.  Isn't inhdr.Data essentially a pointer into memory with inhdr.Cap showing the size of the allocated region?

[Ian Lance Taylor]

On Mon, Feb 27, 2017 at 11:57 AM, Bill Katz <bill...@gmail.com> wrote:
> I thought that's handled by the inhdr.Cap % 8 == 0 check. Isn't inhdr.Data
> essentially a pointer into memory with inhdr.Cap showing the size of the
> allocated region?

Yes, but that is not sufficient. Consider
b := make([]byte, 9)
u, err := byteToUint64(b[1:])
Here the len and cap of b are 8, but the data field will not be aligned.

I think your code is safe but there is a simpler way to do it:

if len(b) % 8 != 0 || cap(b) % 8 != 0 ||
uintptr(unsafe.Pointer(&b[0])) % 8 != 0 {
return errors.New("bad len or cap or alignment")
}
return (*[1<<27]uint64)(unsafe.Pointer(&b[0]))[:len(b)/8:cap(b)/8]

Ian

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Bill Katz]

Ah, of course, silly me conflating alignment with allocation size.  Thanks for the suggestion.

Best,

Bill