Slice conversion function not working on big endian platform

[Srinivas Pokala]

Hello gopher's,

I have simple go program which convert slice of one type to slice of other type using go generics for handling all the supported types.

Below is the code snippest for this:

package main

import "fmt"
import "unsafe"

type slice struct {
        ptr unsafe.Pointer
        len int
        cap int
}

func Slice[To, From any](data []From) []To {
        var zf From
        var zt To
        var s = (*slice)(unsafe.Pointer(&data))
        s.len = int((uintptr(s.len) * unsafe.Sizeof(zf)) / unsafe.Sizeof(zt))
        s.cap = int((uintptr(s.cap) * unsafe.Sizeof(zf)) / unsafe.Sizeof(zt))
        x := ([]To)(unsafe.Pointer(s))
        return x
}
func main() {
        a := make([]uint32, 4, 13)
        a[0] = 1
        a[1] = 0
        a[2] = 2
        a[3] = 0
        // 1 0 2 0
        b := Slice[int64](a)
        //Expecxted : []int64[]{0x00000000 00000001,  0x00000000 00000002}
        //Got: []int64{0x00000001 00000000, 0x00000002 0000000}
        if b[0] != 1 {
                fmt.Printf("wrong value at index 0: want=1 got=0x%x \n", b[0])
        }
        if b[1] != 2 {
                fmt.Printf("wrong value at index 1: want=2 got=0x%x\n", b[0])
        }

}

This is working fine on little endian architectures(amd64,arm64 etc), but when i run on big endian machine(s390x) it is not working , it is resulting wrong data

//Expecxted : []int64[]{0x00000000 00000001,  0x00000000 00000002}
 //Got: []int64{0x00000001 00000000, 0x00000002 0000000}

Can somepoint point me how do we write such scenario which should work on both little/endian platforms.

Any leads on this?

Thanks,

Srinivas

[Brian Candler]

That code doesn't even compile in go 1.22 or go.1.21:

https://go.dev/play/p/mPCBUQizSVo

./prog.go:20:14: cannot convert unsafe.Pointer(s) (value of type unsafe.Pointer) to type []To

What's the underlying requirement? In the test case it looks like you want to take a slice of int32's, in whatever their internal in-memory representation is, and re-represent them as a slice of half as many int64's?? Then of *course* each pair of int32's will become one int64, and the order of the hi/lo halves will depend entirely on the system's internal representation of int64's. It *is* working, in the sense that it's doing exactly what you told it to do. There's a reason why the "unsafe" package is called "unsafe"!

It would be straightforward to write a function which takes a slice containing pairs of int32's and assembles them into int64's in a consistent way. What you've not explained is:

- why you need to do this with generics (for example, what behaviour would you expect from other types?)

- why you need to do this in-place with "unsafe"

[Pokala Srinivas]

There is typo mistake while entering, below is the code snippest:

package main

import (

      "fmt"

      "unsafe"

)

type slice struct {

      ptr unsafe.Pointer

      len int

      cap int

}

func Slice[To, From any](data []From) []To {

      var zf From

      var zt To

      var s = (*slice)(unsafe.Pointer(&data))

      s.len = int((uintptr(s.len) * unsafe.Sizeof(zf)) / unsafe.Sizeof(zt))

      s.cap = int((uintptr(s.cap) * unsafe.Sizeof(zf)) / unsafe.Sizeof(zt))

      x := *(*[]To)(unsafe.Pointer(s))

      return x

}

func main() {

      a := make([]uint32, 4, 13)

      a[0] = 1

      a[1] = 0

      a[2] = 2

      a[3] = 0

      // 1 0 2 0

      b := Slice[int64](a)

      //Expecxted : []int64[]{0x00000000 00000001,  0x00000000 00000002}

      //Got: []int64{0x00000001 00000000, 0x00000002 0000000}

      if b[0] != 1 {

            fmt.Printf("wrong value at index 0: want=1 got=0x%x \n", b[0])

      }

      if b[1] != 2 {

            fmt.Printf("wrong value at index 1: want=2 got=0x%x\n", b[0])

      }

}

Please try this code

---

From: 'Brian Candler' via golang-nuts <golan...@googlegroups.com>
Sent: 08 May 2024 15:25
To: golang-nuts <golan...@googlegroups.com>
Subject: [EXTERNAL] [go-nuts] Re: Slice conversion function not working on big endian platform

 

That code doesn't even compile in go 1. 22 or go. 1. 21: https: //go. dev/play/p/mPCBUQizSVo ./prog. go: 20: 14: cannot convert unsafe. Pointer(s) (value of type unsafe. Pointer) to type []To What's the underlying requirement? In the test case it looks

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.

Report Suspicious

 

ZjQcmQRYFpfptBannerEnd

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a5b6f6fd-f3ec-4532-8579-179282610aban%40googlegroups.com.

[Pokala Srinivas]

I am using generics here to work for other types as well, it 's not only for converting int32[] slice to int64[], it is generic function to work for any conversion like BytestoInt64/32/16/8,  BytestoFloat32/64  etc.

It is failing for other conversion like BytestoInt64 , BytestoFloat64 etc on big-endian machine but working on little endian .

---

From: 'Pokala Srinivas' via golang-nuts <golan...@googlegroups.com>
Sent: 08 May 2024 15:39
To: golang-nuts <golan...@googlegroups.com>; Brian Candler <bcand...@googlemail.com>
Subject: [EXTERNAL] Re: [go-nuts] Re: Slice conversion function not working on big endian platform

 

There is typo mistake while entering, below is the code snippest: package main import ( "fmt" "unsafe" ) type slice struct { ptr unsafe. Pointer len int cap int } func Slice[To, From any](data []From) []To { var zf From var

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/SJ0PR15MB5278E4842C790EBB22976056EEE52%40SJ0PR15MB5278.namprd15.prod.outlook.com.

[Dan Kortschak]

You do need to explain why it is that you want to do this. But to
explain, look at the underlying bytes that describe the two backing
arrays. This can be seen here https://go.dev/play/p/-PemhIfLii1. Note
that (obviously) the backing data is the same, and that the backing
data shows the expected layout of bytes for a little endian array of 32
bit ints with the values: 1, 0, 2, 0. This is the same as the layout of
of a little endian array of 64 bit ints with the values: 1, 2. On s390x
the memory layout will be identical, but the interpretation of the
bytes into a 64 bit integer will be different. You should not expect
that they will be the same.

[Pokala Srinivas]

Same code posted here: https://go.dev/play/p/slgyXAo6iqq

---

From: Pokala Srinivas <Pokala....@ibm.com>
Sent: 08 May 2024 15:58
To: golang-nuts <golan...@googlegroups.com>; Brian Candler <bcand...@googlemail.com>; Pokala Srinivas <Pokala....@ibm.com>
Subject: Re: [EXTERNAL] Re: [go-nuts] Re: Slice conversion function not working on big endian platform