Raw bytes in message

[Viet Nguyen]

Hi,

In protobuf, I can use `bytes` to define a raw byte array property.
Now I'm trying flatbuffers with Go. if I use `[byte]`, I couldn't access []byte directly.

For example with this schema:

table Test {
       Data:[byte];
}

I got this generated code:

func (rcv *Test) Data(j int) int8 {
    ...
}

func (rcv *Test) DataLength() int {
    ....
}

It's weird that it returns []byte if I use type `string`

table Test {
       Data:string;
}

func (rcv *Test) Data() []byte {                                                                                                                             
    ....                                                                                                                                       
}

But in the generated code for Java or C++, it returns `String` which I think it is correct.

Did I miss something? How do you send bytes with your messages?

Thank you
--
Viet Nguyen

[mikkelfj]

Hi Nquyen,

I don't fully understand the Go typesystem, but I have used strings a bit FlatBuffers Go.

I believe the reason is that you can easily convert a byte array to a string in Go, but a string cannot be nil, while a byte array can.

Because FlatBuffers has potentially many missing or default fields, you often need to test of the string was absent.

This doesn't answer all your questions, but it explains why a string looks like a byte array.

Why byte arrays do not have the same byte array interface also puzzled me initially. But it likely has something to do with endian translation. In your case a byte array, like strings, will never have endian issues, but if you had a vector of shorts, you would need conversion. I think the Go Api ought to have an extra accessor method for byte arrays similar to strings.

In the flatcc interface for C (which I wrote), there are similar array get methods, but you can also the vector type as a C array directly, which is only safe for byte arrays, or when you know you are on little endian platforms.

[Viet Nguyen]

Hi mikkelfj

I think I know the reason returning []byte for string: to avoid memory allocation as string is immutable.

My question is why FlatBuffers doesn't return []byte for type [byte]. It is definitely possible as in string case.
Currently, in order to get []byte in Go, I have to allocate a new []byte and copy byte by byte from FlatBuffers "vector of int8".
For example:

data := make([]byte, rcv.DataLength())
for i := 0; i < rcv.DataLength(); i++ {
    data[i] = byte(rcv.Data[i])
}

And do the same thing to write bytes
That is too much overhead.

Cheers,
--
Viet Nguyen

[mikkelfj]

On Monday, September 26, 2016 at 9:48:58 PM UTC+2, Viet Nguyen wrote:

Hi mikkelfj

I think I know the reason returning []byte for string: to avoid memory allocation as string is immutable.

My question is why FlatBuffers doesn't return []byte for type [byte]. It is definitely possible as in string case.

Yes, if you make a PR you can probably get that resolved, as long as you understand it only works for 8 bit wide types. You can try to contact Rob Winslow who did the Go port and hear his opinion if he doesn't monitor this board

https://rwinslow.com/posts/use-flatbuffers-in-golang/

[Piyush Mishra]

Use [ubyte] instead.  You get a method DataBytes() which returns exactly what you want :)

I suspect there are some subtle differences between ubyte and byte. Maybe related to uint8 vs int8.

[mikkelfj]

On Thursday, September 29, 2016 at 12:42:02 PM UTC+2, Piyush Mishra wrote:

Use [ubyte] instead.  You get a method DataBytes() which returns exactly what you want :)

Great, thanks, that makes sense.