Dereferencing a pointer pointer interface value

[Peter Mogensen]

Hi,

I somehow ended up with an interface{} value being a **SomeStruct{}

... and I need to dereference it to an interface{} holding *SomeStruct{}
in a generic way (ie. without knowing about the type SomeStruct)

I'm not at all sure this is good idea or the right way for the specific
code, but out of curiosity, - can this be done in a non-unsafe way?

One would think the result is rather well defined.

In other words,

var x int = 7
xp := &x
xpp := &xp

var i1 interface{} = xpp

var i2 interface{}

i2 := SomeMagicNotSpecificToInt(i1)

yp := i2.(*int)

y := *yp // y now == 7

/Peter

[Axel Wagner]

On Fri, Jan 12, 2018 at 6:32 PM, Peter Mogensen <a...@one.com> wrote:

Hi,

I somehow ended up with an interface{} value being a **SomeStruct{}

... and I need to dereference it to an interface{} holding *SomeStruct{}
in a generic way (ie. without knowing about the type SomeStruct)

I'm not at all sure this is good idea or the right way for the specific
code, but out of curiosity, - can this be done in a non-unsafe way?

Yes, using reflection. Not using non-reflection, which is WAI. You should always

* Specialize on specific types, thus knowing them

* Not care what's in the interface

* Reflect to write code that can handle ~anything (e.g. in encoding/json and the like)

 

One would think the result is rather well defined.

It's not. The value in the interface doesn't have to be a pointer, so this is, in general, not defined at all.

 

In other words,

var x int = 7
xp := &x
xpp := &xp

var i1 interface{} = xpp

var i2 interface{}

i2 := SomeMagicNotSpecificToInt(i1)

yp := i2.(*int)

y := *yp  // y now == 7

/Peter

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Mogensen]

On 2018-01-12 19:09, 'Axel Wagner' via golang-nuts wrote:
> One would think the result is rather well defined.
>
>
> It's not. The value in the interface doesn't have to be a pointer, so
> this is, in general, not defined at all.

No, but it's not like you cannot ask about interface{} in general.
I can do a type switch, asking:

func f(i interface{}) int {
switch t := i.(type) {
case int:
return t
default:
return 0
}
}


Now I just wondered how to do something like a parameterized version of
that:

func f(i interface{}) o interface{} {
switch t := i.(type) {
case **<Something>:
o = *t
default:
o = nil
}
return
}


When you say "well defined" you refer to that an interface{} can be
anything until you do reflection or type assertion.

When I say "well defined", I speak of whether it's in general is
ambiguous what is meant.

/Peter

[Axel Wagner]

On Fri, Jan 12, 2018 at 7:23 PM, Peter Mogensen <a...@one.com> wrote:

On 2018-01-12 19:09, 'Axel Wagner' via golang-nuts wrote:
>     One would think the result is rather well defined.
>
> It's not. The value in the interface doesn't have to be a pointer, so
> this is, in general, not defined at all.

No, but it's not like you cannot ask about interface{} in general.
I can do a type switch, asking:

Yes, you can *test* if the dynamic value has a specific type (and extract it if so). That is not the same as operating on the dynamic value (e.g. by dereferencing it) *without* knowing its type.

 

Now I just wondered how to do something like a parameterized version of
that:

Go does not have parametrically polymorphic types - and even if it had, there wouldn't be any sensible type you could assume here, given that it's only known at runtime. There simply is no type that the extracted value could have. Like, what would the type of t be in your example, in the **<Something> case?

When you say "well defined" you refer to that an interface{} can be
anything until you do reflection or type assertion.

When I say "well defined", I speak of whether it's in general is
ambiguous what is meant.

Yes, it is in general ambiguous what is meant.

You can generically dereference a pointer, but you have to use reflection, which is the tool to be used on generic values that you don't know the type of: https://play.golang.org/p/73lJyOlaIdL

I consider it unlikely that you actually want to do that, though. Unless you are writing some generic encoder, that can operate on any type. But the fact that you are asking about doubly-indirect pointer and dereferencing them one layer deep, seems to suggest that you actually want to use a specific type because you know what types you can have.

[Peter Mogensen]

On 2018-01-12 22:22, Axel Wagner wrote:
> Yes, you can *test* if the dynamic value has a specific type (and
> extract it if so). That is not the same as operating on the dynamic
> value (e.g. by dereferencing it) *without* knowing its type.

I know I can test the type. But the issue was specifically to
dereference the first pointer while keeping the actual object.

 
> Go does not have parametrically polymorphic types - and even if it had,
> there wouldn't be any sensible type you could assume here, given that
> it's only known at runtime. There simply is no type that the extracted
> value could have. Like, what would the type of t be in your example, in
> the **<Something> case?

It seems we are constantly talking pass each other. There was two
questions here
1) Can it be done in Go as the Go spec is now?
2) Is it in general an ambigious task?

So let's assume we already have answered 1) with "no". (I guess)
It seems you are trying to argue that the answer to 2) is "YES", but
doing it with reference to current Go spec.

I'm perfectly aware that in a case in a type switch like this

switch t := i.(type) {
}

... t needs to have a type in each case clause, and the only type it can
have in lack of a specific type mentioned, is interface{}

But still ... that's an argument for 1) - not for 2)

> Yes, it is in general ambiguous what is meant.

No, I don't think so.

There nothing ambigious about generic operation of turning **T into *T.
The only reason it cannot be done here is because the deference operator
needs to operate on a concrete type and we have an interface{}.

It might not be meaningful to define an actual generic function
mechanism for this, since for the compiler to check the types there is
not much more than the dereferencing which can be done
But it is possible to state this in C++:

template<class T>
T *f(T **p) {
return *p;
}

... however, templates are kinda cheating.

... and that was all I was saying by it not being ambiguous.


> You /can/ generically dereference a pointer, but you have to use

> reflection, which is the tool to be used on generic values that you
> don't know the type of: https://play.golang.org/p/73lJyOlaIdL

Ahh... yes... I think I overlooked the Elem() method. It seems it allows
me to make an interface value of *T from an interface containing **T
without knowing the actual type of T.
... so the answer to the question is actually "yes" - that can be done
without unsafe.

> I consider it unlikely that you actually want to do that, though. Unless
> you are writing some generic encoder, that can operate on any type.

That's actually the case - or at least and decoder for now.

> But
> the fact that you are asking about doubly-indirect pointer and
> dereferencing them one layer deep, seems to suggest that you actually
> want to use a specific type because you know what types you can have.

No... If I knew the type, I would have said it.

However... I'm still not sure whether it actually solves what I'm trying
to do. But it did answer the question I posed out of curiosity.

Thanks,

Peter

[Axel Wagner]

On Fri, Jan 12, 2018 at 11:23 PM, Peter Mogensen <a...@one.com> wrote:

On 2018-01-12 22:22, Axel Wagner wrote:
> Yes, you can *test* if the dynamic value has a specific type (and
> extract it if so). That is not the same as operating on the dynamic
> value (e.g. by dereferencing it) *without* knowing its type.

I know I can test the type. But the issue was specifically to
dereference the first pointer while keeping the actual object.
  
> Go does not have parametrically polymorphic types - and even if it had,
> there wouldn't be any sensible type you could assume here, given that
> it's only known at runtime. There simply is no type that the extracted
> value could have. Like, what would the type of t be in your example, in
> the **<Something> case?

It seems we are constantly talking pass each other. There was two
questions here
1) Can it be done in Go as the Go spec is now?
2) Is it in general an ambigious task?

So let's assume we already have answered 1) with "no". (I guess)
It seems you are trying to argue that the answer to 2) is "YES", but
doing it with reference to current Go spec.

But the answer to 2 is "no, not in a type-safe way". It necessarily requires reflection. Even if we'd assume we somehow magically know that the dynamical value of the interface is some pointer type. That doesn't actually help us, in any reasonable way. We can dereference it, but we wouldn't know what the thing it's pointing to is. Or to put it another way: To dereference a pointer, you'd have to know - at the very least - the size of the thing it's pointing to. And to do it in a type-safe way, you'd have to know the type of the thing it's pointing to.

Yes, in theory, the compiler an inspect the type-info of the interface, see that it's a pointer-type and then construct a new interface-value, with dynamic type "type of the pointee" and dynamic value "pointee" - that's reflection. I.e. the compiler would emit code for the equivalent of

v := reflect.ValueOf(v).Elem().Interface()

That is the best you can do, but in particular, you don't have any static information about what the value is you are operating on, so you can't call any methods on it, you can't dereference it, you can't do… anything with it, really. Like, you can't actually do any better than what reflection gives you - and reflection already works.

That's why I say that, if you know the type, use that knowledge to get static assertions on it. If you don't use reflection. And that is fundamentally all you can do in a statically typed language.

Your request is essentially equivalent to reflection as a language-feature, which is essentially equivalent to making Go a dynamically typed language.

I'm perfectly aware that in a case in a type switch like this

switch t := i.(type) {
}

... t needs to have a type in each case clause, and the only type it can
have in lack of a specific type mentioned, is interface{}

But still ... that's an argument for 1) - not for 2)

> Yes, it is in general ambiguous what is meant.

No, I don't think so.

There nothing ambigious about generic operation of turning **T into *T.
The only reason it cannot be done here is because the deference operator
needs to operate on a concrete type and we have an interface{}.

It might not be meaningful to define an actual generic function
mechanism for this, since for the compiler to check the types there is
not much more than the dereferencing which can be done
But it is possible to state this in C++:

template<class T>
T *f(T **p) {
    return *p;
}

This is not the same. In the case of this C++ code, the actual type of T is statically known. In your Go case of getting passed an interface{}, you have no static information about the type (or rather: You have the information that it's an interface{} value). This is what I meant when I said, parametric polymorphism alone wouldn't even help you a priori; unless it's instantiated at runtime (say, like in Python).

I don't know enough C++ to talk about this competently, but a quick googling suggests the rough equivalent to Go's interfaces in C++ (in terms of implementation) are polymorphic objects: Objects that keep dynamic type info around. And I assume, if you'd actually try to do it using that, you'd run into similar problems and would end up having to use typeid() - which is the equivalent to reflection. But yes, as I said, I can't talk competently about C++.

[Axel Wagner]

uhm, there is a bit of a mix-up with the yes-vs-no thing and negating. I'm apparently a bit tired, disregard the actual "no" and focus on the rest of the reply, explaining why what you want can't be done :)

[Peter Mogensen]

On 2018-01-13 01:35, Axel Wagner wrote:
> But the answer to 2 is "no, not in a type-safe way". It necessarily
> requires reflection. Even if we'd assume we somehow magically know that
> the dynamical value of the interface is some pointer type. That doesn't
> actually help us, in any reasonable way. We can dereference it, but we
> wouldn't know what the thing it's pointing to is. Or to put it another
> way: To dereference a pointer, you'd have to know - at the very least -
> the size of the thing it's pointing to.

Which we do. It's a pointer.

> And to do it in a type-safe way,

> you'd have to know the /type/ of the thing it's pointing to.

Or just ensure the type T is the same for *T as it was for **T
I can't see that's ambiguous in the common language sense of the word.

> Yes, in theory, the compiler an inspect the type-info of the interface,
> see that it's a pointer-type and then construct a new interface-value,
> with dynamic type "type of the pointee" and dynamic value "pointee" -
> that's reflection. I.e. the compiler would emit code for the equivalent of
> v := reflect.ValueOf(v).Elem().Interface()
> That is the best you can do, but in particular, you don't have any
> static information about what the value is you are operating on, so you
> can't call any methods on it, you can't dereference it, you can't do…
> anything with it, really. Like, you can't actually do any better than
> what reflection gives you - and reflection already works.

My point about that the dereference operation was the only meaningful
one was exactly because there's no static type info. As I said you
probably can't write a function, but a language *could* in theory define
an operator which would turn any **T into a *T, regardless of type.

> Your request is essentially equivalent to reflection as a
> language-feature, which is essentially equivalent to making Go a
> dynamically typed language.

It was not a feature request.
I just posed the question out of curiosity. The rest of the discussion
has been about whether it was ambiguous.
If a language (in theory) provided an operator which would only act on
variables of type Pointer-To-Pointer-To-T and returned the Pointer-To-T,
I can't see how that would be ambiguous. And it could still do the same
level of static type checks as Go does with interfaces.

> template<class T>
> T *f(T **p) {
>     return *p;
> }
>
>
> This is not the same. In the case of this C++ code, the actual type of T
> is statically known.

I know .. I said it was cheating.
My point with that example was to say that it's not ambiguous (*in the
common language sense of the word*) what is meant.
You can state it and there's no doubt what is meant.

/Peter

[Peter Mogensen]

On 2018-01-13 01:37, 'Axel Wagner' via golang-nuts wrote:
> uhm, there is a bit of a mix-up with the yes-vs-no thing and negating.
> I'm apparently a bit tired, disregard the actual "no" and focus on the
> rest of the reply, explaining why what you want can't be done :)

I must admit, I'm seriously in doubt about whether we are addressing the
same questions.
It seems that you already did provide an example of how to do what I
wanted - using reflection.
I've never made it a criteria to not use reflection. Only to not use
unsafe (like using InterfaceData() to get the uintptr).

I'm perfectly happy with the .Elem().Interface() solution. AFAICS it
does what I want.

Now, regarding whether it would be ambiguous to make it a language
feature is another question.

It seems to Go compiler already has such code built in when
dereferencing pointers in selectors. However, it explicitly blocks
automatically dereferencing a pointer-to-pointer-type.

From src/cmd/compile/internal/gc/typecheck.go:

=======
} else if tt.Etype == types.Tptr && tt.Elem().Etype == types.Tptr &&
eqtype(derefall(tt), derefall(rcvr)) {
yyerror("calling method %v with receiver
%L requires explicit dereference", n.Sym, n.Left)
=======

/Peter

[Axel Wagner]

Go has this operator, it is spelled "*" and is a unary operator with even more power: It can turn a general *T into a * for any T.

Say, you'd have the function you want, called, for example "join". There are two questions we have to answer: a) How does it interact with the type system and b) how is it implemented (i.e. what are the instructions emitted by the compiler).

b) is simple to answer. The code emitted for join would be equivalent to this:

// join assumes q is a pointer to a pointer and returns it's dereference

func join(q unsafe.Pointer) unsafe.Pointer {

    return *(*unsafe.Pointer)(q)

}

Now, this is using unsafe, but that's functionally equivalent to other generic functions (e.g. append, copy…) in that the compiler checks the types for consistency and then emits the corresponding call. So, the unsafe would be hidden by the language. It would seem to me, your question about "would it be ambiguous" is "can this function be generated" and the answer is, of course, yes.

a) is about how to expose this to the language, and is a bit harder. It depends on whether you have parametric polymorphism in your type system or not.

If you don't, then `join` can not have any sensible type; it's type would be func(**T) *T, which, is parameterized. So join would have to be "magically generic", like append/copy/… which don't have a type, because they are builtins and can't be passed around. But then, this has no advantages over a simple dereference. You can't use it as a function, you always have the full types available statically, so you can also just write p := *q.

If you do have parametric polymorphism, you can of course write this function (as you've demonstrated with C++). Note, however, that if you have full-blown generics, you will likely need to add this as an optimization. A priory, type-parameters can have different sizes, so the compiler would need to figure out, that if you write that parametric type, all types that you can instantiate it with, will have the same size and thus it can reuse the code.

But note, however, that this is still not the same as writing a func(v interface{}) interface{}. Because the type of that function, by definition, boxes its value and can thus be called with non-pointers (or has to assume it could be). And thus has to manually verify the types it gets passed via reflection.

So, you end up with

1) a magically generic join, expanded by the compiler. For this to work, the compiler needs to have the full type-information available and this is thus a functional subset of normal pointer-dereference

2) a join written using compile-time generics. This is the C++ case you mention.

3) a join written using run-time generics. This is reflection.

You seem to, essentially, be asking about whether you can have parametric polymorphism in the language, but only for multi-level pointers. Which… I mean, sure, you can do that, but it doesn't seem a sensible question to me. But, yes, if you'd allowed to parameterize over pointers, you can have something like

func<P> join(q *P) P {

    return *q

}

(making up some syntax for "P is some generic pointer type", i.e. P = *T for some T). And yes, given that all type-parameters will always have the same representation, you can always check and expand this at compile-time (i.e. no boxing whatsoever required) and wouldn't need to use any reflection. It just seems like a ridiculously useless language feature to me. It would allow you to write some generic type-safe containers, as long as you don't want to store the value inline. But you couldn't write, e.g. a generic hash table (as the hash-function would need to take the actual type) without also making up some kind of notion of covariant subtyping. So you could, e.g. write something like

func<P> NewTable(hash func(P)) *Table<P>

and instantiate this with

table := NewTable(func(p *MyType) { return hashMyType(p) })

But that's yet another problem.

So in summary: The answer to your question is "yes it's possible", but the result will still either a) be a subset of the dereference operator or b) require parametric polymorphism and c) potentially only for pointer-types, which would seem mightily useless.

[Peter Mogensen]

On 2018-01-13 13:28, Axel Wagner wrote:
> Go /has/ this operator, it is spelled "*" and is a unary operator with

> even more power: It can turn a general *T into a * for any T.
>
> Say, you'd have the function you want,

Alex, This is what I want - or rather, what I'm thinking of:

https://play.golang.org/p/jpR-JqEJq-9

No more, no less.

It works. And it's unambiguous.

Now ... this is a function, so it only works for "interface{}" being the
type of both argument and result.

But I can't really see technical reasons why one could not in principle
make a language operator, which worked for any interface{..} being of
type **T for input and *T for output. - and having that compile time
checked.

/Peter

[Peter Mogensen]

On 2018-01-13 17:06, Peter Mogensen wrote:
> But I can't really see technical reasons why one could not in principle
> make a language operator, which worked for any interface{..} being of
> type **T for input and *T for output. - and having that compile time
> checked.

Hmm... actually ... no. Unless the restriction is lifted so you can do
method selection via **T on T values and not only on *T, it will not be
possible to be more specific than "interface{}"

Unfortunately (for this) a **T does not satisfy the same interface as
*T. Even though *T satisfies the interfaces of T. You get:

"calling method F with receiver c (type **S) requires explicit dereference"

So - sure... an operator doing static type checking of the
argument/result interface is not directly possible.

/Peter