Are pointers guaranteed to keep the entire pointee alive?

[Axel Wagner]

Hi,

I am assuming this is true, but I couldn't find a definitive answer yet (or rather, the answers seems bad): Does a pointer into any part of a value keep the entire value alive? So, e.g. is this code safe?

type A struct { X int; Y int }

func F() *int {

    a := A{42, 23}

    return &a.X

}

func G() {

    p := F()

    a := (*A)(unsafe.Pointer(p))

    fmt.Println(a.Y)

}

The unsafe.Pointer rules state:

(1) Conversion of a *T1 to Pointer to *T2.
Provided that T2 is no larger than T1 and that the two share an equivalent memory layout, this conversion allows reinterpreting data of one type as data of another type.

Which doesn't apply here, as A is larger than int. I couldn't find any other rules that seemed relevant. That would seem to imply that above code is not safe. It would seem to imply that if I want to do something like that, I have to keep an actual *A alive separately, from F.

Another case where this seems relevant is unsafe.Slice. e.g. is this safe?

func F() *int {

    a := make([]int, 42)

    return &a[0]

}

func G() {

    p := F()

    s := unsafe.Slice(p, 42)

    fmt.Println(s)

}

I can't find any rules at all, about whether or not this is allowed. If it isn't, I'm left wondering what unsafe.Slice is meant for.

I was wondering about this because I considered using

type slice[T any] struct{

    ptr *T

    len int

    cap int

}

as a map key (prompted by another thread) and couldn't figure out whether that's actually safe.

As far as I know, `gc` *currently* works fine with this and treats any pointer into an object as keeping the entire object alive. But could a theoretical moving GC in the future see that only the first field of a struct, or only the first element of an array is pointed to and decide to save memory by shrinking the space allocated? Even worse, could an implementation decide that only the first field/element is used in the `return` and just not allocate the rest of the struct/slice?

[Jan Mercl]

On Sat, May 7, 2022 at 12:34 PM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:

> I am assuming this is true, but I couldn't find a definitive answer yet (or rather, the answers seems bad): Does a pointer into any part of a value keep the entire value alive? So, e.g. is this code safe?
>
> type A struct { X int; Y int }
> func F() *int {
> a := A{42, 23}
> return &a.X
> }
> func G() {
> p := F()
> a := (*A)(unsafe.Pointer(p))
> fmt.Println(a.Y)
> }
>
> The unsafe.Pointer rules state:
>
>> (1) Conversion of a *T1 to Pointer to *T2.
>> Provided that T2 is no larger than T1 and that the two share an equivalent memory layout, this conversion allows reinterpreting data of one type as data of another type.
>
>
> Which doesn't apply here, as A is larger than int.

^ Breaking the unsafe rules enables the code to behave non predictably.

AFAICT, interior pointers do and must keep the entire allocation block
alive, but a sufficiently smart compiler is IMO free to ignore setting
of the 23 value - or allocating the actual storage for .Y in F(),
because no one can observe it - unless breaking the unsafe rules.

[Axel Wagner]

On Sat, May 7, 2022 at 1:20 PM Jan Mercl <0xj...@gmail.com> wrote:

^ Breaking the unsafe rules enables the code to behave non predictably.

 

Sure, that's why I'm asking :) In the hope of either a) learning that it is, indeed, breaking the rules and thus should not be done, or b) learning that the rules are incomplete and eventually having them clarified.

 

AFAICT, interior pointers do and must keep the entire allocation block
alive,

Is there any "official" documentation on the "must"?

 

but a sufficiently smart compiler is IMO free to ignore setting
of the 23 value - or allocating the actual storage for .Y in F(),
because no one can observe it - unless breaking the unsafe rules.

If internal pointers do, indeed, *must* keep the allocation alive, I can't see how this would be allowed. After all, the code is unambiguous about allocating a large object and returning a pointer into it - and if that pointer must keep the entire object alive, surely a compiler can't just decide to allocate a smaller one.

FWIW it seems at least currently, gc is not doing doing it: https://godbolt.org/z/h74MchcT4

[Robert Engels]

The code is not compliant with the rules T2 is larger than T1 and no guarantee of the same layout either. 

On May 7, 2022, at 7:54 AM, 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEFcAWA8TN_UGv_dc8wbM%2BPZ1_ia0g18eGQ4hEju08vLg%40mail.gmail.com.

[Jan Mercl]

On Sat, May 7, 2022 at 2:53 PM Axel Wagner
<axel.wa...@googlemail.com> wrote:

>> AFAICT, interior pointers do and must keep the entire allocation block
>> alive,
>
> Is there any "official" documentation on the "must"?

I don't think there is and I don't think it's necessary. I believe
it's implied by the language semantics: The guarantees are primarily
about what is kept, not about what is collected and when. It must be
possible to dereference any non-nil pointer obtained "legally". From
that follows, for example: if both the outer and inner pointers are
reachable, the allocation block cannot be freed. If only the inner one
is reachable, it's an implementation detail if the runtime keeps the
outer alloc completely or reduces it to only the part reachable via
the interior pointer. IIRC Go runtime used to keep things simple and
just holds the whole allocation block. Might have changed meanwhile,

>> but a sufficiently smart compiler is IMO free to ignore setting
>> of the 23 value - or allocating the actual storage for .Y in F(),
>> because no one can observe it - unless breaking the unsafe rules.
>
>
> If internal pointers do, indeed, *must* keep the allocation alive, I can't see how this would be allowed. After all, the code is unambiguous about allocating a large object and returning a pointer into it - and if that pointer must keep the entire object alive, surely a compiler can't just decide to allocate a smaller one.

The outer pointer does not leave F, initialization of the .Y field
using a constant value has no side effects and the field is never read
back anywhere in F. The only pointer that survives F's execution is
the interior pointer so reducing the alloc to just a single int is not
legally distinguishable from doing the "dead" allloc+init.

> FWIW it seems at least currently, gc is not doing doing it: https://godbolt.org/z/h74MchcT4

Yep, that seems to confirm the simple approach I mentioned above. It
looks like a trivial optimization opportunity lost, but it's trivial
only in very simple cases where I'd not expect big gains for average
complexity code.