gobind generated constructor for struct containing []byte sometimes corrupts data

[Antonio Marcedone]

Hi, 

I am encountering a strange problem when using gobind. In short, initializing a struct containing a go byte array from Java and then getting such byte array sometimes returns a different value. 

I think this might be a bug with gobind.

I asked about it on stackoverflow here https://stackoverflow.com/questions/45453775/gobind-generated-constructor-for-struct-containing-byte-sometimes-corrupts-dat  yesterday, but got no suggestions. 

Perhaps someone here can take a look? 

Thanks a lot in advance,

Antonio

[Elias Naur]

[]byte arguments are passed by reference, not copied. You need to copy any byte slices you retain after the function or method call.

- elias

[Antonio Marcedone]

I am not sure I understand what is going on here.

For reference, the relevant code is:

            String s = "This is a test string";

            MyStruct struct = new MyStruct(s.getBytes("UTF-8"));
            //struct.setE(s.getBytes("UTF-8"));
            tv.append("\nFirst time: "+ new String(struct.getE(), "UTF-8"));




with corresponding go part:

package test

type MyStruct struct{
    E []byte
}

func NewMyStruct(e []byte) *MyStruct{
    return &MyStruct{e}
}




so s.getBytes allocates a new byte[] (in Java). Then the MyStruct constructor is called, and it gets a reference to such byte[]. 

Are you saying that since the Java garbage collector is not aware that a reference to that array is stored in some variable in the go code, then it just deletes it as soon as the constructor returns? 

And then the getE method just reads whatever happens to be in memory where the old array was without throwing any error despite the fact that the old byte[] has been garbage collected and might have been overwritten by arbitrary data?

Thanks a lot for your response!

Antonio

[Antonio Marcedone]

My intuition above seems to contrast with https://godoc.org/golang.org/x/mobile/cmd/gobind:

Avoid reference cycles
The language bindings maintain a reference to each object that has been proxied. When a proxy object becomes unreachable, its finalizer reports this fact to the object's native side, so that the reference can be removed, potentially allowing the object to be reclaimed by its native garbage collector. The mechanism is symmetric.

[Elias Naur]

Yes, byte slices are special and are not reference counted like other objects. You need to copy it on the Go side.

- elias

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/TLy4sa3XRec/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Antonio Marcedone]

Thanks! The only information I could find about this on the gobind documentation is (https://godoc.org/golang.org/x/mobile/cmd/gobind)

- Byte slice types. Note that byte slices are passed by reference,
  and support mutation.

However, this version of the code still shows the problem:

String s = "This is a test string";

byte[] b = s.getBytes();
MyStruct struct = new MyStruct(b);

tv.append("\nFirst time: "+ new String(struct.getE(), "UTF-8"));

tv.append("\nOriginal: "+ new String(b, "UTF-8"));

which is confusing as here b shold not be garbage collected, so even a pass by reference should work. Can you point me somewhere I can read more about this issue please?

Also, are the automatically generated setters (i.e. struct.setE() ) doing the necessary cloning, or should I implement my own?

Thanks a lot again!

 

[Elias Naur]

Den fre. 4. aug. 2017 01.26 skrev Antonio Marcedone <a.mar...@gmail.com>:

Thanks! The only information I could find about this on the gobind documentation is (https://godoc.org/golang.org/x/mobile/cmd/gobind)

- Byte slice types. Note that byte slices are passed by reference,
  and support mutation.

However, this version of the code still shows the problem:

String s = "This is a test string";

byte[] b = s.getBytes();
MyStruct struct = new MyStruct(b);

tv.append("\nFirst time: "+ new String(struct.getE(), "UTF-8"));

tv.append("\nOriginal: "+ new String(b, "UTF-8"));

Sometimes, the Java runtime will copy the byte array before passing it to Go and then copy it back just before returning from the call.

Holding on to references to Java byte arrays really is not supported. If you want to know the technical details look up GetByteArrayElements and ReleaseByteArrayElements in the JNI reference documentation.

which is confusing as here b shold not be garbage collected, so even a pass by reference should work. Can you point me somewhere I can read more about this issue please?

Also, are the automatically generated setters (i.e. struct.setE() ) doing the necessary cloning, or should I implement my own?

The setters are cloning. If not it's a bug :)