How to convert from []byte to []uint32?

[Christian LeMoussel]

I have a data stream of bytes and I'd like to get array of int32 (from four bytes).


func convertCharToInt32(buffer string) []uint32 {

    const SIZEOF_INT32 = 4

    var hh = make([]byte, 2)

    var cbuffer = make([]byte, len(buffer)/2)

    var hbuffer = make([]uint32, len(cbuffer)/SIZEOF_INT32)

    for i := 0; i < 28; i++ {

        hh[0] = buffer[i*2]

        hh[1] = buffer[i*2+1]

        if s, err := strconv.ParseUint(string(hh[:]), 16, 64); err == nil {

            cbuffer[i] = byte(s)

        }

    }

    for i := range hbuffer {

        hbuffer[i] = uint32(Endian.Uint32(cbuffer[i*SIZEOF_INT32 : (i+1)*SIZEOF_INT32]))

    }

    return hbuffer

}

buffer := "83f982d600c1caca7a6"
hbuffer := convertCharToInt32(buffer)



The code above seems to work, but perhaps there is a built-in function in Go that I've missed or there is a super cool hack that does that in one instruction?




[Dmitry Savintsev]

Hi Christian,
just a form note - it would be preferable and easier for the readers to put a code snippet into https://play.golang.org and share the link, similar to this one:
https://play.golang.org/p/p3TNDze923

It also helps to make sure that program is syntactically correct (not clear in your example what "Endian" refers to - binary.LittleEndian?) and the example can be run in the Playground.

Take a look at examples and API docs in https://golang.org/pkg/encoding/binary/ - they should cover what the stdlib provides for conversions.

[howar...@gmail.com]

The hack would be to use unsafe after converting it to a byte slice to cast to a Pointer and cast that back to a uint32 slice. This is bad. Do not do this.

Go's ethos is pretty strongly against 'super cool hacks' or 'code golf' style statements that perform magic to stuff a lot of code into one instruction. Indeed, Go is pretty much the opposite - plain code that does what it appears to do with minimal magic, for maximum long-term and large-group maintainability of the codebase.

This StackOverflow answer provides an example of the unsafe way: https://stackoverflow.com/questions/11924196/convert-between-slices-of-different-types

As it also says, this is strongly not recommended.

[peterGo]

Christian,

Your specialized convertCharToInt32 function, which returns []uint32, is slow in comparison to a more general HexToUint function.

BenchmarkHexToUint32-8   20000000    88.9 ns/op   16 B/op    2 allocs/op
BenchmarkCharToInt32-8    3000000   438 ns/op     96 B/op   22 allocs/op

Playground: https://play.golang.org/p/OeUDEV9Xpb

Peter

[Bryan Mills]

In this case, the "code golf" solution seems clearer: https://play.golang.org/p/Jxkf2Vheml

[peterGo]

Bryan,

Use the test case from the question: buffer := "83f982d600c1caca7a6".

https://play.golang.org/p/1gN7Y4ajOH

Peter

[Bryan Mills]

Yes, it looked like a typo to me.

Truncating the string to an even number of characters gives the same results as the original snippet:

https://play.golang.org/p/avf6xqnpEn

It's also not difficult to pad the input, if that's the desired behavior:

https://play.golang.org/p/Q6C9SBhQB5

(Padding without copying the whole string would be slightly more complex, but as Howard noted, Go style is generally to prefer simpler code.)

[Christian LeMoussel]

.Thank you Peter & Bryan for your help, it's very instructive.
I realized a Bench with all your solutions: https://play.golang.org/p/ePKkHNqTot

BenchmarkCharToInt32_0 : my original solution
BenchmarkCharToInt32_1 : Bryan "code golf" solution. https://play.golang.org/p/Jxkf2Vheml

BenchmarkHexToUint32_1 : Peter general HexToUint function. https://play.golang.org/p/OeUDEV9Xpb
BenchmarkHexToUint32_2 : Peter Truncating the string to an even number solution https://play.golang.org/p/avf6xqnpEn
BenchmarkHexToUint32_3 : Peter pad imput solution https://play.golang.org/p/Q6C9SBhQB5

BenchmarkHexToUint32_1-4         1000000              1280 ns/op              16 B/op          2 allocs/op
--- FAIL: BenchmarkHexToUint32_2
        HexToUint32_test.go:151: LittleEndian BenchmarkHexToUint32_2 failure [d682f983 cacac100] != [d682f983 cacac100 607a]
--- FAIL: BenchmarkHexToUint32_3
        HexToUint32_test.go:170: LittleEndian BenchmarkHexToUint32_3 failure [d682f983 cacac100] != [d682f983 cacac100 607a]
BenchmarkCharToInt32_0-4          500000              2250 ns/op              96 B/op         22 allocs/op
--- FAIL: BenchmarkCharToInt32_1
        HexToUint32_test.go:208: encoding/hex: odd length hex string
        HexToUint32_test.go:212: LittleEndian BenchmarkHexToUint32_3 failure [d682f983 cacac100] != []

Result :

• OK for BenchmarkHexToUint32_1
• KO for BenchmarkHexToUint32_2 & BenchmarkHexToUint32_3
• Error for BenchmarkCharToInt32_1