Slice header

[shan...@gmail.com]

A bit of attention has been directed at slices recently, with a few high profile blogs shining a healthy light on them (they can be traps for young players!)

I'm a bit confused about SliceHeader as found in the reflect package (https://github.com/golang/go/blob/master/src/reflect/value.go#L1986).

The comment in the code, and in the documentation at https://golang.org/pkg/reflect/#SliceHeader says
SliceHeader is the runtime representation of a slice. It cannot be used safely or portably and its representation may change in a later release. Moreover, the Data field is not sufficient to guarantee the data it references will not be garbage collected, so programs must keep a separate, correctly typed pointer to the underlying data.

My confusion is this, the comment says (very clearly)
It's not safe, nor portable, and may change (although I have had it pointed out to me that the comparability guarantee may contradict this)
Further the Data field reference won't stop the Garbage collector removing the data

So, can someone explain to me what it's purpose in life is (there's suggestion that it was a temporary fix that, like all things temporary, has become permanent, and if so, I wonder if that means the documentation needs updating)

This seems to me to be the better solution (but it's still in the pipeline?) https://github.com/golang/go/issues/19367
an unsafe, rather than reflect, access to the slice structure, which allows a user to do.. interesting things to the object. Am I correct in thinking that this change (when implemented) removes the reflect implementations, and does away with the comments?

[Jan Mercl]

On Tue, Nov 10, 2020 at 11:09 AM shan...@gmail.com <shan...@gmail.com> wrote:

> My confusion is this, the comment says (very clearly)
> It's not safe, nor portable, and may change (although I have had it pointed out to me that the comparability guarantee may contradict this)

It does not. When there exists explicit documentation for the
contrary, it means the contrary.

> Further the Data field reference won't stop the Garbage collector removing the data

It's the same as for any other uintptr anywhere else. It's not a
pointer and thus does not guarantee reachability of its "pointee". The
documentation just reminds of this.

> So, can someone explain to me what it's purpose in life is (there's suggestion that it was a temporary fix that, like all things temporary, has become permanent, and if so, I wonder if that means the documentation needs updating)

This does not make reflect.SliceHeader unusable though. The
reflect.SliceHeader is either aliased to a real slice or is later
unsafely converted to one. Then the uintptr field is no more an
uintptr, but a real pointer. The documentation points out that _in
between_ one has to guarantee the reachability by other means, the
only pseudo-reference of the uintptr value is not sufficient for that.

I see nothing in the documentation that needs to be updated. It's
possible that it could be improved to become better understood as it
seems it's not always working that way.

[Axel Wagner]

On Tue, Nov 10, 2020 at 11:09 AM shan...@gmail.com <shan...@gmail.com> wrote:

This seems to me to be the better solution (but it's still in the pipeline?) https://github.com/golang/go/issues/19367

an unsafe, rather than reflect, access to the slice structure, which allows a user to do.. interesting things to the object. Am I correct in thinking that this change (when implemented) removes the reflect implementations, and does away with the comments?

I agree that with unsafe.Slice there shouldn't be any more use cases that necessitate use of reflect.SliceHeader. I don't think it should be removed outright though. There is no need to break existing programs using it, even if we consider it allowed from the compatibility guarantee. Likewise, the comments should persist, so that readers of existing code can look up its documentation.

We can definitely add a note that unsafe.Slice and &a[0]/len/cap should be used instead, though. But, just to be clear, I also don't think anything substantially changes in terms of compatibility guarantees - unsafe.Slice will give just as little a guarantee as reflect.SliceHeader does today.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/669b67cd-6336-49ae-be04-4eade048b84an%40googlegroups.com.

[shan...@gmail.com]

On Tuesday, November 10, 2020 at 9:41:09 PM UTC+11 Jan Mercl wrote:

Thank you for taking the time to answer, I'm still a little confused and have inlined where that confusion is below.

On Tue, Nov 10, 2020 at 11:09 AM shan...@gmail.com <shan...@gmail.com> wrote:

> My confusion is this, the comment says (very clearly)
> It's not safe, nor portable, and may change (although I have had it pointed out to me that the comparability guarantee may contradict this)

It does not. When there exists explicit documentation for the
contrary, it means the contrary.

Sorry, because you quoted the whole string I am confused what you mean (on reflection I am thinking you are just saying that the documentation is contrary to the compatibility guarantee and as such the sliceheader is not subject to that guarantee)

 

> Further the Data field reference won't stop the Garbage collector removing the data

It's the same as for any other uintptr anywhere else. It's not a
pointer and thus does not guarantee reachability of its "pointee". The
documentation just reminds of this.

Ah, yes I see what you mean

> So, can someone explain to me what it's purpose in life is (there's suggestion that it was a temporary fix that, like all things temporary, has become permanent, and if so, I wonder if that means the documentation needs updating)

This does not make reflect.SliceHeader unusable though. The
reflect.SliceHeader is either aliased to a real slice or is later
unsafely converted to one. Then the uintptr field is no more an
uintptr, but a real pointer. The documentation points out that _in
between_ one has to guarantee the reachability by other means, the
only pseudo-reference of the uintptr value is not sufficient for that.

 

So, the only two uses I have found for this is

1) A Teaching aid, showing people what a slice is under the hood

2) This piece of code that I have just found that converts/moves a C array into a Go slice

```

func getArr(which int) []byte {

var theCArray *C.char = C.getTheArray(C.int(which))

length := int(C.BLOCKSIZE)

var theGoSlice []byte

sliceHeader := (*reflect.SliceHeader)((unsafe.Pointer(&theGoSlice)))

sliceHeader.Cap = length

sliceHeader.Len = length

sliceHeader.Data = uintptr(unsafe.Pointer(theCArray))

// now theGoSlice is a normal Go slice backed by the C array

return theGoSlice

}

``` 

Are there other uses?

I see nothing in the documentation that needs to be updated. It's
possible that it could be improved to become better understood as it
seems it's not always working that way.

I think it does need some attention for clarity, I'm still a little confused what exactly it's trying to convey (The way it's written, to me at least, says, don't use this because it's unsafe, non-portable, and may change)

One other bone of contention the term "sliceheader" appears only in the reflect package.
But https://blog.golang.org/slices uses the term liberally to refer to an example slice struct that appears to be like the one found in https://github.com/golang/go/blob/master/src/runtime/slice.go#L13

I *know* this is nit picking, but I feel that the blog should be clear that the term is purely informal, a number of times I have been told that that use of the term in the blog means that is what the struct is officially called

[shan...@gmail.com]

On Tuesday, November 10, 2020 at 10:09:44 PM UTC+11 axel.wa...@googlemail.com wrote:

On Tue, Nov 10, 2020 at 11:09 AM shan...@gmail.com <shan...@gmail.com> wrote:

This seems to me to be the better solution (but it's still in the pipeline?) https://github.com/golang/go/issues/19367

an unsafe, rather than reflect, access to the slice structure, which allows a user to do.. interesting things to the object. Am I correct in thinking that this change (when implemented) removes the reflect implementations, and does away with the comments?

I agree that with unsafe.Slice there shouldn't be any more use cases that necessitate use of reflect.SliceHeader. I don't think it should be removed outright though. There is no need to break existing programs using it, even if we consider it allowed from the compatibility guarantee. Likewise, the comments should persist, so that readers of existing code can look up its documentation.

We can definitely add a note that unsafe.Slice and &a[0]/len/cap should be used instead, though.

That makes sense. 

 

But, just to be clear, I also don't think anything substantially changes in terms of compatibility guarantees - unsafe.Slice will give just as little a guarantee as reflect.SliceHeader does today.

After sleeping on it I realised that that was the case (few guarantees for unsafe.Slice), but I feel that the fact that it will be in the unsafe package makes this a little more explicit