How to know if interface{} data is nil w/o reflecting?

[Jonathan Gold]

I'm a bit stumped and wondering if I'm overlooking some way, besides reflection,
to determine whether the data pointed at by an interface{} is actually a nil
pointer:

http://play.golang.org/p/Isoo0CcAvr

Any ideas what I'm overlooking?

jonathan

[Kevin Ballard]

If you know the type in question then you just use a type assertion, e.g. v2.(*someType) == nil.

http://play.golang.org/p/E1tlCOb32P

If you don't know the underlying type of the interface then reflect is probably the right tool.

-Kevin

[Kyle Lemons]

Nope.

This is a common source of confusion.  The basic answer is to never store something in an interface if you don't expect the methods to be called on it.  The language may allow it, but that violates the semantics of the interface.  To expound, a nil value should usually not be stored in an interface unless it is of a type that has explicitly handled that case in its pointer-valued methods and has no value-receiver methods.

[Jesse McNelis]

On Tue, Jun 5, 2012 at 8:50 AM, Jonathan Gold <jgol...@gmail.com> wrote:
> I'm a bit stumped and wondering if I'm overlooking some way, besides reflection,
> to determine whether the data pointed at by an interface{} is actually a nil
> pointer:

You just use a type assertion. If you don't know the type, then you
don't know if nil is a valid value for it and thus have no business
checking for it.


--
=====================
http://jessta.id.au

[tmp...@hotmail.com]

I found this a little bit non sequitur - if I want to call interface function I have a perfect business to check if underlying object is not nil before call just to avoid panic on call. Besides underlying nil in interface may be used to signal condition for variety of types implementing this interface, and since there is no inheritance in Go - sometimes it's the only proper way to indicate such condition.

[Jesse McNelis]

On Tue, Oct 31, 2017 at 2:25 AM, <tmp...@hotmail.com> wrote:
> I found this a little bit non sequitur - if I want to call interface
> function I have a perfect business to check if underlying object is not nil
> before call just to avoid panic on call. Besides underlying nil in interface
> may be used to signal condition for variety of types implementing this
> interface, and since there is no inheritance in Go - sometimes it's the only
> proper way to indicate such condition.

In Go you can call a method on nil without a panic, this means that
unless you know the type you don't know if a nil value of that type is
a valid value to call a method on or not. You would be making a big
assumption about the type contained within the interface{} if you
checked that value for nil.

Of course since you don't know the type of the value inside the
interface{} it might not even be a value comparable to nil, in which
case should the comparison return false or panic?

[oju...@gmail.com]

Not being able to test for a nil interface is a Go language bug.

Humans are fallible, so are our projects. Every project has its share of errors.
Go, despite being a great tool we all love, is no exception to that universal rule.

[Ayan George]

On 10/31/2017 10:25 AM, oju...@gmail.com wrote:
> Not being able to test for a nil interface is a Go language bug.
>

Just curious: How would you do this without type assertion or reflection?

[Ian Lance Taylor]

Agreed about fallibility and errors, but Go is a programming language
so it's necessary to be precise. Go makes it straightforward to test
for a nil interface value: write `v == nil`. What you are talking
about is something different: testing whether an interface holds a
value of some type, where the value of that type happens to be nil.
Go makes a clear and necessary distinction between an interface that
is nil and an interface that holds a value of some type where that
value is nil. It would be a mistake to remove that distinction, and
once you accept that it's hard to see what we can do. See also
https://golang.org/doc/faq#nil_error .

Ian

[oju...@gmail.com]

Ian, with all due respect, I beg to differ.

Let's look at that example posted 5 years back:

http://play.golang.org/p/Isoo0CcAvr

Yes, that is the proper behavior according to the rules, we have a FAQ entry, fine, but ... put simply: that makes no sense. Why? Because we, the users, expect the code to work the other way. That is a big surprise and we don't need the tool creating surprises for us, much to the contrary.

The internal layout Go uses to store an interface should not mess with the expected behavior. If interface is two separate fields, or just a pointer, or refers to a bitmap stored on the Moon, I don't care. I shouldn't care. The fact that today we must tell every new Go user this whole story is a shame. Go is meant to be easy, after all.

The behavior of a software of mine is not what my user expect? In my book that means I have a bug. Even with a very reasonable explanation, that remains a bug. I can spend hours explaining a strange behavior to my user. He will tell me: "Oh, yes. Now I got it, thanks. Fix that".

That reminds me of an old adage:

“If the map doesn't agree with the ground the map is wrong”

I sincerely hope Go 2 will have it fixed.

[oju...@gmail.com]

Today you can't, Ayan.

[Bruno Albuquerque]

I am not sure what exactly is the issue you are complaining about. You can think of interface as a container for some data type. The container can be nil and the data contained in it can be nil. This looks reasonable to me as much as having a C++ vector that contains nil (well, nullptr) pointers.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marvin Renich]

* oju...@gmail.com <oju...@gmail.com> [171031 12:34]:

> Ian, with all due respect, I beg to differ.
>
> Let's look at that example posted 5 years back:
>
> http://play.golang.org/p/Isoo0CcAvr
>
> Yes, that is the proper behavior according to the rules, we have a FAQ
> entry, fine, but ... put simply: that makes no sense. Why? Because we, the
> users, expect the code to work the other way. That is a big surprise and we
> don't need the tool creating surprises for us, much to the contrary.

I beg to differ. I, a user, very much expect the current behavior and
would be very surprised if it worked the way you are suggesting.

Ian's msg gives good reasons why it would be wrong to change the
behavior, even if we could go back in time to before the release of Go
1 and the Compatibility Guarantee.

...Marvin

[Axel Wagner]

On Tue, Oct 31, 2017 at 5:33 PM, <oju...@gmail.com> wrote:

The internal layout Go uses to store an interface should not mess with the expected behavior. 

If interface is two separate fields, or just a pointer, or refers to a bitmap stored on the Moon, I don't care. I shouldn't care.

  

And you don't have to care. This is not an implementation detail, it is a spec-detail. It is a consequence of

a) Methods can be declared on any type, including pointers

b) Methods can be called on any value of that type, including its zero value

c) An interface is a type for "any type with the following sets of methods"

Whether you represent an interface as one word, two words, zero words or one billion words is irrelevant, as long as you implement Go, you have to adhere to these rules and they imply the criticized behavior. Personally, I have a gripe with the fact that the FAQ justifies this behavior with an implementation detail, when really it's a consequence of the language. The rules a), b) and c) make individually total sense. It makes sense to be able to declare methods on pointers, as it is expected that methods can modify state from other imperative languages. It makes sense that methods can be called on zero values, because they should be a property of the type, not the value. It makes sense to have the notion of a method set as an interface.

I sincerely hope Go 2 will have it fixed.

https://github.com/golang/go/issues/21538 has already been filed and there might be other issues amounting to the same thing. I think it is fair to say, that a) for Go1, this can't be changed either way and b) for Go2, the community is aware of the confusion.

[Jesse McNelis]

On Wed, Nov 1, 2017 at 3:42 AM, <oju...@gmail.com> wrote:
> Today you can't, Ayan.
>

It's very consistent, you can't compare an interface value reliably to
any untyped constant.
Because there is no way for the compiler to figure out what type it should take.

https://play.golang.org/p/4Fn0YNE2md

[Ayan George]

On 10/31/2017 12:42 PM, oju...@gmail.com wrote:
> Today you can't, Ayan.
>

Right -- I understand that. I'm asking how would you propose Go do
that? What changes would you make to fix this?

I'm asking: If you claim Go is fundamentally broken in this way, how
would you fix it?

-ayan

[oju...@gmail.com]

Ayan, imagine I am part of a development team. In our program I have a pointer r:

r *myType

My variable r can be nil, because that is a valid situation. It is used in dozens of places, like this:

if r != nil {
    r.doSomething()
}

That is a very usual idiom, no only in Go, but in many languages. Every programmer is acquainted to that.

Then, years later, other programmer in my team decides to create an interface to better capture a new understanding of the problem at hand, changing the type of r to:

r myInterface

Subtly, the meaning of

if r != nil {
    r.doSomething()
}

changes. Under the right circumstances our software starts to behave strangely. What?

This problem is dangerous because it is so subtle. We will read our old code time and again to no avail, because everything seems fine and no one has changed that "if r != nil r.doSomething" in ages.

As Dave Cheney said in https://github.com/golang/go/issues/21538#issuecomment-323561094 that could be solved:

"by making an nil interface and an interface which contains a type who's value is nil also equal to nil."

[Ayan George]

On 11/01/2017 07:18 AM, oju...@gmail.com wrote:
> Ayan, imagine I am part of a development team. In our program I have a
> pointer r:
>
> r *myType
>
> My variable r can be nil, because that is a valid situation. It is used
> in dozens of places, like this:
>
> if r != nil {
>     r.doSomething()
> }
>
> That is a very usual idiom, no only in Go, but in many languages. Every
> programmer is acquainted to that.
>
> Then, years later, other programmer in my team decides to create an
> interface to better capture a new understanding of the problem at hand,
> changing the type of r to:
>
> r myInterface
>
> Subtly, the meaning of
>
> if r != nil {
>     r.doSomething()
> }
>
> changes. Under the right circumstances our software starts to behave
> strangely. What?
>
> This problem is dangerous because it is so subtle. We will read our old
> code time and again to no avail, because everything seems fine and no
> one has changed that "if r != nil r.doSomething" in ages.
>

I think I understand what you're saying. The thing is: You changed the
type of r and I don't believe anyone ever promised that you can change
the type of a variable to an interface AND preserve its behavior.

Would you expect r to behave the same if you changed it from type string
to an int?

Also, you understand how interfaces work. I think our hypothetical
developer should simply understand how interfaces work as well; it is
reasonable to expect developers to know the language.

I don't think it is reasonable to introduce this kind of ambiguity to
protect developers from actually learning the language. I think
interfaces as they are are clear and consistent and making nil
interfaces suddenly equal to interfacess with nil values is more
'subtle' and harder to understand.

[Ain]

Point is, in that case compiler would catch it for you and you wouldn't be able to compile until you fix all the parts of the code which now doesn't make sense. Not so in the case JuciÊ Andrade pointed out.


ain

[Ian Lance Taylor]

A different way to state your argument is that nil is overloaded in
Go. Your argument is essentially identical to saying suppose you have
an integer r:

type MyIntegerType int
var r MyIntegerType

and a bunch of code that tests whether r is 0 or not

if r != 0 {
r.doSomething()
}

Then you change r to an interface type, and now the meaning of r != 0
has subtly changed. The code still compiles, but now it means
something different. r != 0 will be true even if the value of r
actually is 0, because in `r != 0` the `0` will be given type `int`,
but `r` is type `MyIntegerType`, so the values are not equal.

That is definitely potentially confusing, but I would like to believe
that you would not argue that we should treat r != 0 specially for
integer types. Or, at least, we shouldn't treat it any differently
than r != 1. And, obviously, `r == nil` should not be true if r
happens to be `0`.

I think the reason it feels different for pointer types is that we,
probably incorrectly, chose to use the same identifier, `nil`, to
represent the zero value both for pointers and for interfaces.
Suppose we had given the zero value of interface types the name
`nilinterface`. Then the situation for pointers would be the similar
to that for integers. Writing `r != nilinterface` would be equivalent
to `r != nil` today: it would test whether the interface itself is
nil. `r != nil` would be a compilation error, because, unlike `0`,
there is no default type for `nil`.

If we do change something in this area for Go 2, which we probably
won't, my preference would definitely be to introduce `nilinterface`
rather than to further confuse the meaning of `r != nil`.

Ian

[Jack Christensen]

Perhaps a small package and linter could solve this without requiring a
language change. The package would expose either a NilInterface value or
a IsNilInterface method. Users of the package would either compare to
the pkg.NilInterface or call the pkg.IsNilInterface method. Then the
linter could warn on all direct interface comparisons to nil.

Jack

[Axel Wagner]

It seems a very bad idea, to require the common case to go through some package or else get lint-warned.

Checking "x == nil" is the correct way to check whether a value contains a valid implementation of an interface - and a nil-pointer with the correct method set is, as far as you should be concerned, such a valid implementation.

If you indeed want to check whether the contained value is a nil-pointer (I'm having actual trouble coming up with a scenario where you'd want to do that - except cases like encoding/json, where you need to use reflection anyway), you have to check that explicitly, either by converting nil into the correct type (i.e. "x == (*T)(nil)"), or by type-asserting. That is the uncommon case, so it's fine having to jump through a couple of hoops for hat.

What *might* make more sense, is trying to lint whether a nil-pointer is used for an interface, and the corresponding methods are not nil-safe. I.e. do the "is a nil-pointer a valid value for this interface" check where it belongs: At the point where the interface-conversion happens (when assigning it to an interface-typed variable or passing it as an interface-typed parameter).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Bryan Mills]

C++ had a similar change with the `nullptr` literal in C++11: `NULL` was overloaded between pointers and integers, whereas `nullptr` can only be (implicitly converted to) a pointer.

[Volker Dobler]

On Wednesday, 1 November 2017 12:19:19 UTC+1, JuciÊ Andrade wrote:

Ayan, imagine I am part of a development team. In our program I have a pointer r:

r *myType

My variable r can be nil, because that is a valid situation. It is used in dozens of places, like this:

if r != nil {
    r.doSomething()
}

That is a very usual idiom, no only in Go, but in many languages. Every programmer is acquainted to that.

Then, years later, other programmer in my team decides to create an interface to better capture a new understanding of the problem at hand, changing the type of r to:

r myInterface

Subtly, the meaning of

if r != nil {
    r.doSomething()
}

changes. Under the right circumstances our software starts to behave strangely. What?

This problem is dangerous because it is so subtle. We will read our old code time and again to no avail, because everything seems fine and no one has changed that "if r != nil r.doSomething" in ages.

This is a nice argument. But please consider a small variation on it:

Let's start of with as you did: r is a pointer and you check its nilness:

    r *myType

    if r != nil {
        r.doSomething()
    }

"Then, years later, other programmer in my team decides to [...]

[add one more level of indirection] to better capture a new

understanding of the problem at hand, changing the type of r to:"

    r **myType

"Subtly, the meaning of" 

    if r != nil {
        r.doSomething()
    }

"changes. Under the right circumstances our software starts to

behave strangely. What?"

Of course it will! And this is (I hope undoubtedly) not a language

design error. Of course it would sometimes be handy to be able

to write

    var a ***int

    if a _!=_ nil { ... }

with an operator _!=_ with semantics like

    if a != nil && *a != nil && **a!=nil { ... }

(all the way down).

But there currently is no such deep-down-comparison operator

(and it presumably would be hard to design a consistent one).

What you have been asking for is some magic to unpeel the topmost

interface level. But where to stop? How should

    var b *myInterface

    if b != nil { ... }

behave? Check if b actually is non-nil, then check if the interface

itself is non-nil and then peek inside the interface and check if there

is a non-nil value?

Just because a single example demonstrates that some behaviour

would be helpful and help mitigate an error does not mean it does

not open a different can of worms.

V.

[oju...@gmail.com]

Could somebody please point me to any example of comparing an interface to nil, in production code? Thanks.

[Ayan George]

On 11/03/2017 05:23 AM, oju...@gmail.com wrote:
> Could somebody please point me to any example of comparing an
> interface to nil, in production code? Thanks.
>

Does checking if a variable of type error is nil count?

-ayan

[oju...@gmail.com]

Yes, Ayan, you are quite correct. There are a lot of comparisons to check if an error is nil. It has it's own FAQ entry: https://golang.org/doc/faq#nil_error

Sincerely, what I am after is an example of a situation where the solution proposed by Dave Cheney would create a problem, like "See here. In this situation the proposed change would break my code."

When comparing error to nil, Dave's change would work just fine. That FAQ entry would not be needed anymore.

[Jakob Borg]

I often do nil checks on interfaces in places like constructors that may or may not have received optional parameters. In some cases as a parameter that may be nil if no implementation is required, in some cases where a functional option setting the interface was not given. Typically this looks something like:

func Walk(ctx context.Context, cfg Config) (chan protocol.FileInfo, error) {
w := walker{cfg}

if w.CurrentFiler == nil {
w.CurrentFiler = defaultCurrentFiler{}
}

        …

}

where the field is set to some default implementation when nil. It’s also not uncommon to have nil implementations of that interface, especially in tests:

type mockCurrentFiler struct{}

func (*mockCurrentFiler) CurrentFile(string) whatever {

    return nil

}

var testCurrentFiler *mockCurrentFiler // nil

…

cfg.CurrentFiler = testCurrentFiler // == nil in the new regime

Walk(ctx, cfg)

This would break, in a surprising way as the default implementation would be used instead of the given mock implementation. Clearly the test can be refactored:

cfg.CurrentFiler = &mockCurrentFiler{} // not nil

but yes, the change would break code.

I’m not saying that having this change in Go 2 would necessarily be a bad idea, but the current Go 1 behavior makes sense in Go 1 and there is definitely code that depends on it.

//jb

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[oju...@gmail.com]

This thread helped me to understand better the current scenario and the implications of a future change.

I would be glad to recognize if this conversation had changed my mind, but it didn't.

Some programmers discovered that they could use this "valid nil interface" to do some smart tricks, as Jakob kindly has shown. While I do recognize that was indeed smart, Jakob offered another easy way of attaining the desired effect for his constructor. It would be pretty easy if he had to code that way to begin with.

I consider unfortunate the fact that I can't safely use an interface where previously I used a pointer. To me, at least, that is a natural evolutionary path for a piece of software as soon as the developer discover opportunities to leverage the commonality of an interface. I think such possibility would be more broadly useful than what we can do now.

Go has a bunch of interesting tricks and useful idioms, but this trick is proving costly.

Thanks to everyone.

[Matt Harden]

I really wish Go had not chosen to propagate Hoare's billion-dollar mistake. I do realize it's all tied up with the idea that initialization is cheap and zero values should be useful when possible, and therefore pointers, interfaces, channels, etc. need zero values.

I wonder how different Go would have been if we had required all pointers and interfaces (only) to be initialized, and made the zero value for maps a writable empty map. In cases where nil pointers and interfaces are useful, it seems to me that sentinel values would serve the purpose equally well. For example, comparing errors with (ok) would be (one character) shorter, more meaningful and less confusing than comparing with nil, which can be so confusing for newcomers that we have an FAQ for it. Few would be surprised to find (e != ok) when (e == (*myerror)(nil)) -- and if there were no nil pointers, it wouldn't even be a valid question to ask. We could still use pointers as stand-ins for Optional types, just with sentinel values like sql.NullInt64 to serve the purpose nil does.

I know this is likely a non-starter for Go2, for good reasons - virtually all Go code would need significant, probably manual refactoring, and interoperability with other languages would suffer, to name two that come to mind.

I think what I really want is Haskell plus all the benefits Go has relative to it, including Go's incredibly simple language spec, standard library, short compile times, etc. Is that too much to ask? :-)

--

[matthe...@gmail.com]

Storing a pointer in a reference type seems absurd to me.

Matt

[Matt Harden]

I don't know what you mean by "reference type" - as I understand it, that's not a meaningful phrase in Go. Did you mean "interface"? If so, we store pointers in interfaces all the time in Go. When we set an interface variable i to x, we are semantically making a copy of x and storing it in i. We won't be able to modify x using i (because it has a copy of x, not a pointer to the original). If x is a pointer to something, then we will be able to modify that something using i.

[matthe...@gmail.com]

Since an interface can be nil I’ve been assuming interface behaves like slice with a pointer to the concrete data within a reference struct (that also includes the data type) which is passed around as an interface var.

This playground shows that the interface var is a similar reference type to slice: https://play.golang.org/p/PIWpyrpwNq5

I didn’t know where to look in the Go source to find the definition of an interface (src/runtime/type.go, src/runtime/iface.go, src/go/types/type.go, and src/cmd/compile/internal/types/type.go weren't immediately clear to me). The specification says an interface stores a value; my misunderstanding is that while the interface var may refer to data elsewhere, this data is a copy of the original.

So using a pointer assigned to an interface as a way to save stack space is absurd, but using a pointer assigned to an interface as a way to modify the original data does make sense. Arguing about nil interface vs nil pointer in an interface is not absurd, I apologize for my statement.

Matt

[David Collier-Brown]

Drifting back toward the original subject, I'm reminded of the non-bsd-c idiom of

char *foo(char *p) {

    if (p != NULL && *p != NULL) {

        return some string operation...

    }

...

It seems logical to check the type of the contents of an interface

type, and its presence in a function that can take nil types and

nil contents, much as in the C idiom. 

Mind you, I do find type assertions a bit clunky. 

--dave

[Axel Wagner]

I don't understand this comparison. The C idiom you mention is concretely typed (i.e. C doesn't have interfaces, so it doesn't have dynamic types), so I fail to see what it has to do with interfaces. And it makes *far* more sense to check if you got passed a nil-pointer, than to check the concrete value of an interface -- i.e. something like this makes of course some sense, in Go:

func Foo(p *string) {

    if p != nil {

        *p = doSomeStuff()

    }

}

It makes sense to check p for nil, because you know its type and you know you can't use it, if its nil. It's thus a very different problem than what this thread is about.

Moreover, how does, what you say what was said above? i.e. a) As a user of an interface, its dynamic value shouldn't concern you, in general, b) not all zero values are nil, so checking for nil to see whether the dynamic value is the zero value doesn't make sense and c) the zero value - including nil - is a perfectly valid implementation of an interface?

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[David Collier-Brown]

I was responding to the case where one is passed an interface, expects
it to contain a typed value, and
it does not.

--dave

[scud...@gmail.com]

In my case, being a new go learner the following bit of code stumped me. I am receiving a interface{} value and I am trying to figure out the best way to print it. If the value supports the fmt.Stringer() interface then I want to use that, otherwise i can do some other things. So I came up with this:

func printer(foo interface{}) string {

     switch t := foo.(type) {

     ....

     case fmt.Stringer:

             return t.String()

      ...

}

But this crashes when foo is really a nil pointer to a type which does support Stringer. The problem is that there is no way for me to actually know foo is nil and avoid it. I can not compare foo to nil because foo is an interface so foo==nil is false, and my type assertion is converting it to an interface too so i can not compare that to nil either. Literally the only way this code can work is to just catch the error and return the string "nil". This is hugely confusing behavior especially for a new comer.

Apart for this it seems to me the natural way to implement this kind of code - for example in python the code is natural, and I wanted to carry the same general technique to go:

def printer(foo):

   if isinstance(foo, Stringer):

     return foo.String()

  ...

Thanks

Michael.

On Tuesday, June 5, 2012 at 8:50:00 AM UTC+10, jgoldbg wrote:

I'm a bit stumped and wondering if I'm overlooking some way, besides reflection,
to determine whether the data pointed at by an interface{} is actually a nil
pointer:

    http://play.golang.org/p/Isoo0CcAvr

Any ideas what I'm overlooking?

jonathan

[Jakob Borg]

On 8 May 2018, at 15:26, scud...@gmail.com wrote:

But this crashes when foo is really a nil pointer to a type which does support Stringer.

The crash isn’t on “your” side though, it’s presumably inside the String() method. Hence, the caller passed you something invalid that you can’t handle. I’d argue that avoiding this is their responsibility.

My case is made weaker by how fmt.Println and friends handle this though. They’ll call the String() method, recover from the panic, use reflect to see if the boxed value is nil, and print a “<nil>”. I guess this is friendly, but not something I think normal code should do.

(In your specific case you seem to be reimplementing fmt.Sprint; you can just use that instead. :)

//jb

[Michael Cohen]

Well no - the error I get is that I am attempting to call String() method on a null receiver - so the caller just passed me a regular nil object. The issue is that I am trying to make a generic polymorphic function which should be able to handle whatever is thrown at it - so I guess reflect is necessary.

 I think I am supposed to detect the null receiver before calling String() on it. I ended up using this little utility:

https://stackoverflow.com/questions/13476349/check-for-nil-and-nil-interface-in-go

func isNil(a interface{}) bool {
  defer func() { recover() }()
  return a == nil || reflect.ValueOf(a).IsNil()
}

But this feels really hacky when I really just want to say if value != nil { ....} .

Thanks

Michael.

[Jakob Borg]

> On 8 May 2018, at 16:19, Michael Cohen <scud...@gmail.com> wrote:
>
> Well no - the error I get is that I am attempting to call String() method on a null receiver - so the caller just passed me a regular nil object.

They passed you a totally legit fmt.Stringer which then panicked when you called String() on it. That’s their fault.

> I think I am supposed to detect the null receiver before calling String() on it. I ended up using this little utility:

I really don’t think you are. But it’s your code. :)

//jb

[Ian Lance Taylor]

On Tue, May 8, 2018 at 7:19 AM, Michael Cohen <scud...@gmail.com> wrote:
>
> Well no - the error I get is that I am attempting to call String() method on
> a null receiver - so the caller just passed me a regular nil object. The
> issue is that I am trying to make a generic polymorphic function which
> should be able to handle whatever is thrown at it - so I guess reflect is
> necessary.
>
> I think I am supposed to detect the null receiver before calling String()
> on it. I ended up using this little utility:
>
> https://stackoverflow.com/questions/13476349/check-for-nil-and-nil-interface-in-go
>
> func isNil(a interface{}) bool {
> defer func() { recover() }()
> return a == nil || reflect.ValueOf(a).IsNil()
> }
>
>
> But this feels really hacky when I really just want to say if value != nil {
> ....} .

I think there may be some confusion here. Go doesn't have a "regular
nil object." Specific types can be `nil`. In particular, pointer
types can be `nil`.

When a type implements a `String` method it is possible to call that
method with a `nil` pointer. In general it is possible to call any
method with a `nil` pointer. That is not an error.

Some specific implementations of a `String` method may panic when
called with a `nil` pointer. For better or for worse, the fmt package
has special handling for this.

What this means is that unless you have some special knowledge of the
type you are working with, you should not check for `nil` before
calling the `String` method. For some types the `String` method will
correctly handle `nil`. If you have types with a `String` method that
does not correctly handle `nil`, then it's worth pondering why and how
you got a `nil` pointer for this type in the first place. But if you
can reasonably have a `nil` pointer, and can reasonably expect that
the `String` method will panic in that case, then what you should do
is call `fmt.Sprint(v)`. That will do the right thing whether v's
`String` method handles `nil` or not.

Ian

[Michael Cohen]

Thanks Ian for the thorough explanation - again I apologize for asking noob questions :-).

The issue in this case was this error:

panic: value method github.com/shirou/gopsutil/process.Process.String called using nil *Process pointer

which seems to be coming from the go runtime itself. Certainly the String method of Process does not have a pointer receiver:

func (p Process) String() string {

        s, _ := json.Marshal(p)

        return string(s)

}

So it makes sense for the runtime to panic when we attempt to call it on a nil pointer. I looked at the code for the fmt.Sprintf method as you suggested and it seems to be doing the same thing as what my code is doing - i.e. it catches the panic and then calls reflect to check if it is because its a nil pointer. So i guess what feels hacky is really the way it is supposed to be done in golang. 

I wrote the following code to try to understand the behavior more https://play.golang.org/p/kKWGCeacAEM

package main

import (

"fmt"

)

func type_function(a *int) {

fmt.Println("type_function: ", a == nil)

 }

func interface_function(a interface{}) {

fmt.Println("interface_function: ", a == nil)

}

func main() {

type_function(nil)           // true

interface_function(nil)   // true

var foo *int = nil

fmt.Println("is foo nil: ", foo == nil)

interface_function(foo)  // false ... ?

}

It seems to me that if a function accepts a pointer type - it is ok to compare a pointer type against nil. But if a function accepts an interface as an arg it is never safe to compare an interface against nil because from inside the function you have no idea if the caller called with actual nil, or a variable who's value is nil (this difference is very weird because this is not how most languages behave - a variable is usually a full substitute to the literal value it holds).

It seems to me that the compiler should at least warn when someone is comparing an interface to nil (or maybe the linter should warn). It does not seem that this could ever be what you actually want and it is always a subtle bug just waiting to bite.

Thanks

Michael.

[Ian Lance Taylor]

On Tue, May 8, 2018 at 6:17 PM, Michael Cohen <scud...@gmail.com> wrote:
>
> It seems to me that the compiler should at least warn when someone is
> comparing an interface to nil (or maybe the linter should warn). It does not
> seem that this could ever be what you actually want and it is always a
> subtle bug just waiting to bite.

It's perfectly reasonable to compare an interface to nil if you want
to see whether it has been set to anything. In particular it's very
common in Go to write `err != nil`, so clearly warning about every
comparison of an interface value to `nil` is a non-started. See
https://golang.org/doc/faq#nil_error. But see also
https://golang.org/issue/22729.

The issue of passing a nil pointer to a value method is a good reason
to write your String methods as pointer methods.

Ian

[Louki Sumirniy]

Can you not convert it to an unsafe.Pointer and check for that being nil?

[cpu...@gmail.com]

Good morning!

I realize I'm reviving an age-old discussion here and apologize for bringing up the undead. I happend to run into this when my application panicked when some interfaces where initialized with nil mock objects instead of being left uninitialized as in production mode.

Axel said:

Checking "x == nil" is the correct way to check whether a value contains a valid implementation of an interface - and a nil-pointer with the correct method set is, as far as you should be concerned, such a valid implementation.

 

This would be an example where a nil implementing fooer is never caught:

type fooer interface {
 foo()
}

type other struct{}

func (o *other) foo() {} // implement fooer

func main() {
 var f fooer

 var p *other // nil
 f = p // it is a fooer so I can assign it

 if f == nil {
    // will not get here
 }
}

My confusion comes from the point that the nil interface is apparently not "a nil-pointer with the correct method set" while *other is even if nil.

If you indeed want to check whether the contained value is a nil-pointer (I'm having actual trouble coming up with a scenario where you'd want to do that - except cases like encoding/json, where you need to use reflection anyway), you have to check that explicitly, either by converting nil into the correct type (i.e. "x == (*T)(nil)"), or by type-asserting. That is the uncommon case, so it's fine having to jump through a couple of hoops for hat.

The above is a case where that might happen. In can be worked around but it is unexpected unless the programmer is deeply rooted in the language definition.

 

What *might* make more sense, is trying to lint whether a nil-pointer is used for an interface, and the corresponding methods are not nil-safe. I.e. do the "is a nil-pointer a valid value for this interface" check where it belongs: At the point where the interface-conversion happens (when assigning it to an interface-typed variable or passing it as an interface-typed parameter).

Seems as of today that there is no tooling to support that check. Maybe it's not a widespread issue. 

[Axel Wagner]

On Fri, Apr 10, 2020 at 7:17 PM <cpu...@gmail.com> wrote:

I realize I'm reviving an age-old discussion here and apologize for bringing up the undead. I happend to run into this when my application panicked when some interfaces where initialized with nil mock objects instead of being left uninitialized as in production mode.

Let's imagine a world in which `foo == nil` also is true if `foo` is an interface-value containing a nil-pointer. Let's say in this world, someone sends a message to golang-nuts. They wrote a mock for the same code. And since it's just a mock, they just returned static value from its methods and didn't need to care if the pointer was nil or not. They are confused, because the passed in this mock, but the code just assumed the field was uninitialized and never called into their mock. What would you tell them? Why is their confusion less valid?

This would be an example where a nil implementing fooer is never caught:

type fooer interface {
 foo()
}

type other struct{}

func (o *other) foo() {} // implement fooer

func main() {
 var f fooer

 var p *other // nil
 f = p // it is a fooer so I can assign it

 if f == nil {
    // will not get here
 }
}

My confusion comes from the point that the nil interface is apparently not "a nil-pointer with the correct method set" while *other is even if nil.

In the code you posted, even a nil *other is a perfectly fine implementation of fooer. You can call `(*other)(nil).foo()` without any problems.

So, as you illustrated, calling methods on a nil-pointer can be totally fine. A nil-interface, OTOH, doesn't have any methods to call, as it doesn't contain a dynamic value. If you write `(*other)(nil).foo()`, it is completely clear what code gets called - even if that code *might* panic. If you write `fooer(nil).foo()`, what code should be called in your opinion?

I think it's easy to see that a nil-interface and a nil-pointer stored in an interface are very different things. Even from first principles, without deep knowledge of the language. And if they are obviously different, I don't understand why you'd find it confusing that they are not the same in this particular manner.

The above is a case where that might happen. In can be worked around but it is unexpected unless the programmer is deeply rooted in the language definition.

I fully agree with that. What I *don't* agree with, is where you attribute the problem here. You say, the problem is that the nil-check is ill-behaved. I say that - if anything - the original nil-assignment is ill-behaved. Having `(fooer)((*other)(nil)) == nil` be true is semantically wrong, because by checking against `nil`, you are checking if you have a correct implementation - and you might well have a correct implementation, even if it's using a nil-pointer.

Note, that the contained pointer being nil isn't the *only* case in which calling the method might panic. For example, what about this code?

https://play.golang.org/p/lNq0qphez7v

Shouldn't the `nil`-check also catch that? After all, calling the method panics, so it's clearly not a valid implementation - even if x itself is not nil. Why is a nil-pointer more special than any other value that causes a method to panic? 

Seems as of today that there is no tooling to support that check. Maybe it's not a widespread issue.

As of today, the language also isn't changed :) Maybe someone who think this is important enough to change the language, could also feel it's important enough to write this tooling.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e0dbcd38-510e-43b9-b363-2af1c636250b%40googlegroups.com.

[Robert Engels]

I agree with the OP. The usefulness of nil interfaces is pretty limited. Show me a useful case that cant easily be implemented with non-nil interfaces. 

I would argue that allowing nil interfaces causes more subtle latent bugs and makes it harder to reason about the correctness of code when reviewing it. 

It just feels wrong. I realize I’m probably in the minority here but the OP is not alone. 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEPjcsZ3enqXyt%2BUphFJ1cNQ81cFCcjfwwkQZKHMrjSzA%40mail.gmail.com.

[Aviv Eyal]

I was trying to show that the current behavior is confusing and that fmt.Print() needing to resort to panic-and-recover is kinda code smell, but I sorts-of convinced myself that the current behavior is right, or at least consistent.

In my code, I got bit because I sometimes use v *Type to denote "I may or may not have a value here" (where Type is a value-type). 

This is probably a bad practice on my behalf, because I break the Liskov substitution principle: there is a value of `*Type` that is not a valid value of `Type`, and I let this value slip by.

In this case, `v Type` implements Stringer (i.e. valid callee for `v.String()`, but `v *Type`, in the strictest sense, does not.

The only reason we can write:

    func (Type) String() string {...}

    v *Type = &Type{...}

    _ = v.String()

and have it compile, is syntactic sugar: `v` gets implicitly de-referenced, and there's an implicit assumption that it's not nil.

And there's a matching syntactic sugar for converting `Type` to a `*Type`.

So, In the code:

    func (Type) String() string {...}

    v *Type = nil

    r interface{} = v

    _, ok = r.(Stringer)

What I really want to ask is "Can I, at runtime, call r.String()?", whereas the question Go answers is "Is any of `r`, `*r`, or `&r` defines .String()?" - which matches the static semantics of `r.String()`.

So, while I should probably not use *Type as a replacement for Optional<Type>, I think it might make sense to have some operator that can determine, at run-time, if a call `r.String()` is valid (including a nil-check).

-- Aviv

[Denis Cheremisov]

You may use something like this

        value2 := *(*uint64)(unsafe.Pointer(uintptr(unsafe.Pointer(&value)) + 8))

        if value2 == 0 {

                return true

        }

on AMD64, should work also for any 64 bit architecture (at least I believe so). Remember though this is hacky and may stop working once.

воскресенье, 23 августа 2020 г. в 22:58:51 UTC+3, Aviv Eyal:

[Ian Lance Taylor]

On Sun, Aug 23, 2020 at 1:16 PM Denis Cheremisov
<denis.ch...@gmail.com> wrote:
>
> You may use something like this
>
> value2 := *(*uint64)(unsafe.Pointer(uintptr(unsafe.Pointer(&value)) + 8))
> if value2 == 0 {
> return true
> }
>
> on AMD64, should work also for any 64 bit architecture (at least I believe so). Remember though this is hacky and may stop working once.

You could do that, but please don't.

Ian

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/c1ed2e38-6215-4ed2-8357-f8b5d83bf1a7n%40googlegroups.com.

[alex.be...@gmail.com]

Can we at least move with the https://github.com/golang/go/issues/22729 , please? Anything will help with the current mess.

> To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[targe...@gmail.com]

Not sure if it was mentioned here, but IMO the main issues isn't nil data itself, but how easy it's created. It'd be much less of a surprise if creating nil-data required explicit cast from nil struct pointer to interface pointer and resulted in just nil interface pointer in case of implicit cast. Though such change is almost certainly breaking one.

[Axel Wagner]

On Thu, Aug 27, 2020 at 10:06 AM targe...@gmail.com <targe...@gmail.com> wrote:

Not sure if it was mentioned here, but IMO the main issues isn't nil data itself, but how easy it's created. It'd be much less of a surprise if creating nil-data required explicit cast from nil struct pointer to interface pointer and resulted in just nil interface pointer in case of implicit cast. Though such change is almost certainly breaking one.

This would require to insert extra nil-checks when assigning a pointer-value to an interface, as the compiler can't know if a pointer is nil or not. Personally, I would also find it very confusing, if converting a T to a T changed program behavior (though arguably, there is one such case currently with `uintptr(uintptr(unsafe.Pointer))`. But usage of `unsafe` seems sufficiently advanced).

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/84244528-84e6-4c2e-89bf-7fbf0590e132n%40googlegroups.com.

[targe...@gmail.com]

> This would require to insert extra nil-checks when assigning a pointer-value to an interface, as the compiler can't know if a pointer is nil or not.

it would definitely. Though price for consistency looks very much acceptable.

> Personally, I would also find it very confusing, if converting a T to a T changed program behavior

Sorry, didn't get it. Are you saying that nil pointer -> nil interface is more confusing?

[Axel Wagner]

On Thu, Aug 27, 2020 at 11:10 AM targe...@gmail.com <targe...@gmail.com> wrote:

it would definitely. Though price for consistency looks very much acceptable.

I don't think "consistency" is at all the right word here. If anything, things would get *less* consistent, not more.

> Personally, I would also find it very confusing, if converting a T to a T changed program behavior

Sorry, didn't get it. Are you saying that nil pointer -> nil interface is more confusing?

I'm saying the current situation is less confusing than what you describe, yes.

AIUI, with what you describe, if I have a variable `x` of type `*T` and an interface variable `y`, then `y = x` and `y = (*T)(x)` have different semantics. I think it is strange to have a conversion of `x` *to its own type* have any sort of semantic implication. It should be a no-op.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/46d92421-a3a8-4b8a-b557-aa14d79e55b6n%40googlegroups.com.

[targe...@gmail.com]

> I'm saying the current situation is less confusing than what you describe, yes.

> AIUI, with what you describe, if I have a variable `x` of type `*T` and an interface variable `y`, then `y = x` and `y = (*T)(x)` have different semantics. I think it is strange to have a conversion of `x` *to its own type* have any sort of semantic implication. It should be a no-op.

It may be expressed in some different way. To me, if `x == nil` and then `y != nil` after `y = x` is much more confusing. If you ask my opinion, I would make interfaces compare to nil on just data pointer. If one wanted interface which doesn't require data, he could've easily created one with static stub variable. No additional checks, no "semi-nil" fat pointers, everything simple and consistent.

[Axel Wagner]

On Thu, Aug 27, 2020 at 11:39 AM targe...@gmail.com <targe...@gmail.com> wrote:

> I'm saying the current situation is less confusing than what you describe, yes.

> AIUI, with what you describe, if I have a variable `x` of type `*T` and an interface variable `y`, then `y = x` and `y = (*T)(x)` have different semantics. I think it is strange to have a conversion of `x` *to its own type* have any sort of semantic implication. It should be a no-op.

It may be expressed in some different way. To me, if `x == nil` and then `y != nil` after `y = x` is much more confusing.

And obviously you are not alone. Even though I really don't understand why this isn't just one of those "you learn about it, you know about it, you never run into any problems again" type of things. It does seem to come up sufficiently often to be a problem. And there are solutions that I think are fine. For example, using a different identifier (say `none`) to denote the zero-value of interfaces would be fine by me.

But solutions that try to give special treatment to nil-values when they are put into interfaces just seem wrong to me. They single out nil-values as somehow special or less valid than other values. They single out pointer/slice/map/chan types as somehow special over int/bool/string/… types. It just seems undeniable to me, that they make the language *less* consistent.

If you ask my opinion, I would make interfaces compare to nil on just data pointer. If one wanted interface which doesn't require data, he could've easily created one with static stub variable. No additional checks, no "semi-nil" fat pointers, everything simple and consistent.

The rule is very simple: A nil-interface is one that has no dynamic value. All values are treated the same for this purpose. All types are treated the same. I don't understand how that is anything but simple and consistent. It might be less understandable for some other reason, but I don't think it's simplicity or consistency.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/31df134b-7e55-4f32-9e1f-6d974817891en%40googlegroups.com.

[targe...@gmail.com]

> Even though I really don't understand why this isn't just one of those "you learn about it, you know about it, you never run into any problems again" type of things.

Because you then must remember about this specific case every time you cast potentially nil pointer variable to interface. Every time you must write `if x != nil { return x; } else { return nil; }` instead of just `return x;` Does this make language simpler?

> But solutions that try to give special treatment to nil-values when they are put into interfaces just seem wrong to me.

Because nils or null pointers are indeed special. They mean "Points to no data". So dereference operation on such pointer leads to exceptional state. In fact, nullable pointer in terms of richer type systems is a sum-type `nil|*T` where `*T` always points to some valid object, and thus dereferencing it is valid. map/slice/chan types are pointers in disguise. By the way, AFAIK string isn't "special" here because "" and nil are treated equally. As for comparison to int/bool, you can safely perform all the operations on 0 the same way as on non-zero integers, except dividing by it. Same for `false`. nil can be vaguely compared to floats' NaN - except NaN is a potential (albeit rare) result of any operation on floats, while pointer cannot become invalid through dereference.

> The rule is very simple: A nil-interface is one that has no dynamic value. All values are treated the same for this purpose. All types are treated the same. I don't understand how that is anything but simple and consistent. It might be less understandable for some other reason, but I don't think it's simplicity or consistency.

Please don't forget that interface pointer isn't simple. It's a so-called "fat pointer" which consists of pointer to data and pointer to methods table. Thus, we have 4 potential states - (nil, nil), (nil, pointer), (pointer, nil) and (pointer,pointer). First one is just ordinary nil interface. Last one is just ordinary interface pointer with all operations callable without issue (as long as they're properly implemented, of course). Third one with valid data pointer and nil table pointer can be AFAIK created only through reflection and is safely considered "invalid state". We're left with second state with nil data pointer and valid table pointer, making interface methods callable on it. It wouldn't be a problem - if such state couldn't be created by merely casting typed nil pointer (typed pointer variable with nil value) to interface. Why is this an issue? Because most interface implementations require valid data pointer. "Static" (i.e. not needing data) implementations can be considered corner case.

So if we're hypothetically designing language with such fat pointers, how can we solve this issue?

One way is golang's way, where interface is nil when both components are nil. In such case, implicit casts mentioned above can easily lead to invalid state. To circumvent this, such state (nil, pointer) is better created only explicitly, "I know what I'm doing".

The other way is to check interface against nil only by its data pointer. When it's created from typed pointer, it always gets valid table pointer and data pointer from original typed one. When it's compared to nil, it's compared only with its data pointer. Thus, nil translates to nil, and we avoid second Shroedinger's state. If we wanna static implementation, we just create static variable which is `struct{}` and return pointer to it. Alas, this approach is no-go because it involves breakage too deep.

[Robert Engels]

This will be a Go wart that will never go away. All languages have warts. This one just happens to be on the top of the index finger  There is so little utility in a nil interface but it’s there. 

On Aug 27, 2020, at 5:14 AM, 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGhDrbJRMr%3DtxPu_XNDTzyT7PV61Oo7kOLP5QBqg-Zaiw%40mail.gmail.com.

[targe...@gmail.com]

I know this. I'm mostly form C++ land, so you may imagine how many warts are there "on the top of the index finger" :)

I'm merely speaking against opinion that Shroedingerface is not a problem at all.

[targe...@gmail.com]

> I'm mostly form C++ land

from of course :)

[Robert Engels]

“Tip” of course :)

On Aug 27, 2020, at 6:03 AM, targe...@gmail.com <targe...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f3081a9b-bf99-41f9-81e6-62147cbb6a12n%40googlegroups.com.

[Volker Dobler]

On Thursday, 27 August 2020 11:39:11 UTC+2, targe...@gmail.com wrote:

To me, if `x == nil` and then `y != nil` after `y = x` is much more confusing.

This can happen only if x and y have different  types.

And for different types this is pretty normal as you can have

    x == 0.2   //  true

    y = int(x)

    float64(y) != 0.2  // true too

With the only difference that you have explicit type conversions

where interface assignment is implicit.

And this is not even the strangest thing that can happen:

NaN floats are especially peculiar. You can have

   x := y

   x != y  // true

   x != x  // also true

 

That is something everybody has to learn once. It is a

property of all float types. Different types behave differently.

And this is true not only for operators like +, - or / but also

for operators like =, == and !=.

For the rest of the argument: It _really_ isn't  an actual

problem. In 10 years of Go this happened maybe three

time to me and was dead simple to identify.

V.

[targe...@gmail.com]

> And for different types this is pretty normal as you can have

>    x == 0.2   //  true

>    y = int(x)

>    float64(y) != 0.2  // true too

A more correct analogy would've been:

    var x := 0.2

    var y := float32(x)

    math.IsNaN(float64(y)) // true in this analogy

> It is a property of all float types

I used floats as an example of type with such "kaboom" value.

> For the rest of the argument: It _really_ isn't  an actual problem. In 10 years of Go this happened maybe three time to me and was dead simple to identify.

I faced this issue twice. Once caused by my code and the other by not mine. Second time it wasn't that simple to identify because Schroedingerface travelled quite a long distance from its birthplace. Anyway, this issue seems to get raised repeatedly.

[Axel Wagner]

On Thu, Aug 27, 2020 at 12:53 PM targe...@gmail.com <targe...@gmail.com> wrote:

Because you then must remember about this specific case every time you cast potentially nil pointer variable to interface. Every time you must write `if x != nil { return x; } else { return nil; }` instead of just `return x;`

This is true, but that alone is not indicative of a problem. You have to remember that `select` does a pseudo-random selection of cases everytime you use it, that you can't write to nil-maps, that dereferencing a nil-pointer might panic, that an index-expression might panic, that integer-addition might overflow, that floating-point math has counter-intuitive edge-cases, that init-functions run before main… you have to remember a *lot* of things every time you need them.

I don't want to sound dismissive (as I said, I do think there is obviously *some* problem) but I don't run into cases where I would even be *tempted* to do this. Like, in your example, you'd need to a) have a declared pointer, b) would need to be oblivious to the fact on whether or not it's nil and c) it would have to be an invalid value if so and d) it would have to be a problem not caught by other means (e.g. returning a non-nil error alongside it). And when I am tempted to do this, I just know that the interface value will still be non-nil. And even when I don't, the most trivial of testing catches the problem.

Like, I'm not saying it *never* happens or even that it never happens *to me*. But it seems very rarely cause significant problems and I don't understand why people put it so high up their list of being confusing or hard to remember.

 

> But solutions that try to give special treatment to nil-values when they are put into interfaces just seem wrong to me.

Because nils or null pointers are indeed special. They mean "Points to no data". So dereference operation on such pointer leads to exceptional state. In fact, nullable pointer in terms of richer type systems is a sum-type `nil|*T` where `*T` always points to some valid object

Okay. But even in those languages, `nil` (or `None` or `Nothing` or whatever you call it is still a perfectly acceptable value, with significant semantic meaning. Yes, in those languages the compiler is able to prevent you from dereferencing it, but the value is still just a value like any other. If anything, the fact that even languages with very rich type-systems include types like `Maybe` shows how important `nil` as a possible value is. Otherwise you wouldn't have to put it in artificially.

> The rule is very simple: A nil-interface is one that has no dynamic value. All values are treated the same for this purpose. All types are treated the same. I don't understand how that is anything but simple and consistent. It might be less understandable for some other reason, but I don't think it's simplicity or consistency.

Please don't forget that interface pointer isn't simple. It's a so-called "fat pointer" which consists of pointer to data and pointer to methods table. Thus, we have 4 potential states

That's one way to look at it. And you are trying to make that way look pretty complicated. But you are talking about implementation details. It's like arguing strings are very complicated, because they can a) be string-constants held in read-only memory, b) be heap-allocated or c) be stack-allocated.

Another way to look at it is that an interface is a type, that may or may not contain a dynamic value of some other type and allows calling methods on it. If it doesn't, the interface-value is nil. If it does, it isn't. That's the way interfaces are defined in the spec.

I am still against teaching interfaces to newcomers as "two pointers, one to a type and one to a value". It leads to exactly the kinds of confusion you are expressing here, because people think about the representation and want to operate on that, instead of thinking about the semantics. It is completely unnecessary and often counterproductive to think of interfaces that way.

 

Why is this an issue? Because most interface implementations require valid data pointer. "Static" (i.e. not needing data) implementations can be considered corner case.

It's not about needing data or not needing data. It's about deriving meaning from the fact that there is none. As I said, this is clearly important and used regularly - otherwise languages wouldn't need to add a `Maybe` (or whatever) type to do it.

More importantly, if a pointer-type can't act on nil-pointers, that's a problem *orthogonal* to whether you put it into an interface or not. If you return an invalid value, it doesn't become valid just because you call it a `*T` instead of an `io.Reader`. If being able to return a `nil`-pointer as a type for which that isn't a valid value is a problem, then that should be impossible - not wrapping that value in an interface.

So if we're hypothetically designing language with such fat pointers

You are reversing the approach here. We should design a language to have certain semantics and then write an implementation for that, not the other way around. By turning this around, you are limiting the solution space you are looking at, namely:

 

One way […]. The other way […]

I gave you a third solution: Having a separate identifier for the zero value of interfaces.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4613df4c-9342-4285-9b8d-823afa82f58en%40googlegroups.com.

[Axel Wagner]

(whoops: "I don't run into cases […] very often")

[burak serdar]

On Thu, Aug 27, 2020 at 6:12 AM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> On Thu, Aug 27, 2020 at 12:53 PM targe...@gmail.com <targe...@gmail.com> wrote:
>>
>> Because you then must remember about this specific case every time you cast potentially nil pointer variable to interface. Every time you must write `if x != nil { return x; } else { return nil; }` instead of just `return x;`
>
>
> This is true, but that alone is not indicative of a problem. You have to remember that `select` does a pseudo-random selection of cases everytime you use it, that you can't write to nil-maps, that dereferencing a nil-pointer might panic, that an index-expression might panic, that integer-addition might overflow, that floating-point math has counter-intuitive edge-cases, that init-functions run before main… you have to remember a *lot* of things every time you need them.
>
> I don't want to sound dismissive (as I said, I do think there is obviously *some* problem) but I don't run into cases where I would even be *tempted* to do this. Like, in your example, you'd need to a) have a declared pointer, b) would need to be oblivious to the fact on whether or not it's nil and c) it would have to be an invalid value if so and d) it would have to be a problem not caught by other means (e.g. returning a non-nil error alongside it). And when I am tempted to do this, I just know that the interface value will still be non-nil. And even when I don't, the most trivial of testing catches the problem.
>
> Like, I'm not saying it *never* happens or even that it never happens *to me*. But it seems very rarely cause significant problems and I don't understand why people put it so high up their list of being confusing or hard to remember.

I saw this happen mostly after refactoring code to replace structs
with interfaces. Functions start returning interfaces instead of
concrete types, and there is no obvious indication in the code that
might alert the potential problem. Even though this problem is usually
discussed in the context of returning errors, this happened to me
multiple times with types other than errors, and always after
refactoring. I agree that it rarely causes problems, but when it does,
it is hard to find because there is usually a nil-check and no obvious
reason to doubt that would fail.

A go vet addition that detects and warns against this might be more
useful than a language facility. I remember discussing this before and
the concern was false-positives, so a warning instead of an error
would be useful.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfHXb16N4XcN%3Deu%3DZKMF-91RjsbEM4WRyydj-yR3BRW6xA%40mail.gmail.com.

[Marvin Renich]

* targe...@gmail.com <targe...@gmail.com> [200827 05:40]:

> On Thursday, August 27, 2020 at 12:20:59 PM UTC+3 axel.wa...@googlemail.com wrote:
>> I'm saying the current situation is less confusing than what you
>> describe, yes.
>>
>> AIUI, with what you describe, if I have a variable `x` of type `*T`
>> and an interface variable `y`, then `y = x` and `y = (*T)(x)` have
>> different semantics. I think it is strange to have a conversion of
>> `x` *to its own type* have any sort of semantic implication. It
>> should be a no-op.
>
> It may be expressed in some different way. To me, if `x == nil` and then `y
> != nil` after `y = x` is much more confusing. If you ask my opinion, I
> would make interfaces compare to nil on just data pointer. If one wanted
> interface which doesn't require data, he could've easily created one with
> static stub variable. No additional checks, no "semi-nil" fat pointers,
> everything simple and consistent.

I agree with Axel. The `y = x` operation is a conversion, not just an
assignment. You cannot assume that the transitive property of equality
holds where conversions are used. Note also that both y == nil and x ==
nil involve implicit conversion, because nil is the _untyped_ zero
value, so the value being compared to y is nil converted to the
interface type, and the value being compared to x is nil converted to
the concrete type; these are _different_ zero values.

I think there are two incorrect generalizations that lead to Go
programmers having the confusion being discussed. The first is that nil
is the "universal" zero value, which leads to the false idea that nil
of one type is the same as nil of another type. I think when a
programmer stops to think about this, they will get the correct answer,
but often this level of detail is not brought to a high enough level of
conciousness when actually writing code. The correct thinking is that
nil is the _untyped_ (rather than universal) zero value. Note that you
cannot even evaluate nil == nil!

The second is thinking of an interface value containing a concrete value
as being the same thing as the concrete value. I'm not sure why this is
such a sticking point for some, but it clearly is. It may be for some
that they think of interfaces as a "partial generic" language feature,
such that declaring a variable of an interface type is a way to directly
use any of the types that implement that interface. This is the wrong
way to think of it, as the interface value cannot be used in Go code in
the same was as the concrete value, except in the very specific case
where you are invoking a method on the interface value that is defined
_by the interface_. In particular, the interface value has its own,
distinct, zero value from the zero value of the contained concrete type.

Consider the code:

type Fooer interface {
Foo()
}

type X int

func (i X) Foo() {
}

func main() {
var i X
var f Fooer = i
fmt.Println(f == 0)
}

You would not (at least you should not) expect this to compile, much
less give you the "obvious"(?) answer. Yet, when you change

type X int
func (i X) Foo() {
...
var i X
...
fmt.Println(f == 0)

to

type X struct{}
func (i *X) Foo() {
...
var i *X
...
fmt.Println(f == nil)

you expect that the nil is being compared to the concrete value
contained by f rather than the value of f.

If you had to explicitly convert nil to the proper type before
comparison, the problem would be much more obvious.

If you think of an interface value as a container for a concrete value,
rather than the concrete value itself, then the interface value might be
empty (nil, no concrete value) or it might contain a concrete value (not
nil, but the concrete value might be its own zero value).

As Axel said, you just have to learn that interface_value == nil
determines whether interface_value has been assigned a concrete value,
not whether the concrete value is its own zero value (which might
coincide with nil converted to the concrete type).

...Marvin

[Jake Montgomery]

On Thursday, August 27, 2020 at 10:45:52 AM UTC-4, burak serdar wrote:

On Thu, Aug 27, 2020 at 6:12 AM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> On Thu, Aug 27, 2020 at 12:53 PM targe...@gmail.com <targ...@gmail.com> wrote:
>>
>> Because you then must remember about this specific case every time you cast potentially nil pointer variable to interface. Every time you must write `if x != nil { return x; } else { return nil; }` instead of just `return x;`
>
>
> This is true, but that alone is not indicative of a problem. You have to remember that `select` does a pseudo-random selection of cases everytime you use it, that you can't write to nil-maps, that dereferencing a nil-pointer might panic, that an index-expression might panic, that integer-addition might overflow, that floating-point math has counter-intuitive edge-cases, that init-functions run before main… you have to remember a *lot* of things every time you need them.
>
> I don't want to sound dismissive (as I said, I do think there is obviously *some* problem) but I don't run into cases where I would even be *tempted* to do this. Like, in your example, you'd need to a) have a declared pointer, b) would need to be oblivious to the fact on whether or not it's nil and c) it would have to be an invalid value if so and d) it would have to be a problem not caught by other means (e.g. returning a non-nil error alongside it). And when I am tempted to do this, I just know that the interface value will still be non-nil. And even when I don't, the most trivial of testing catches the problem.
>
> Like, I'm not saying it *never* happens or even that it never happens *to me*. But it seems very rarely cause significant problems and I don't understand why people put it so high up their list of being confusing or hard to remember.

I saw this happen mostly after refactoring code to replace structs
with interfaces. Functions start returning interfaces instead of
concrete types, and there is no obvious indication in the code that
might alert the potential problem.

Like Burak, once I was no longer a noob, this has rarely bitten me. In fact, I literally can't remember it being a problem since I became proficient in Go. It probably has, but not enough to stick with me. I also almost never find myself using the `if x != nil { return x; } else { return nil; }` pattern, so it is not a burden for me.

I'm not going to argue whether Go got it right, or not, since a change at this point is exceedingly unlikely. But, on a pragmatic level, two thing struck me about your response.

The first is the old adage: "Accept interfaces return concrete types." There are many good reasons for violating this rule, but I hope all this refactoring that involved changing functions that had concrete types, but choose to return them as interfaces really thought about the justification for breaking the advice.

The second is that refactoring is always tricky and requires attention to detail. A struct and interface are not the same thing, so this is not zero-change transformation by definition. It seems like you should be very aware that you are changing a function from returning a struct to returning an interface, at the time. Maybe the first time this would bite you, but after that it seems like it would become wrote to know that such a change requires a tiny bit of extra thought.

Admittedly these are not firm technical arguments ... just my experience and opinion.

 

[Robert Engels]

I think you just made the case... if you have existing code that declares a concrete return type and you change that to an interface and nothing else you are going to have problems. 

On Aug 27, 2020, at 12:11 PM, Jake Montgomery <jake...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/99a93de9-5dbc-474a-b0c9-906a8ae2cbaeo%40googlegroups.com.

[burak serdar]

On Thu, Aug 27, 2020 at 12:27 PM Robert Engels <ren...@ix.netcom.com> wrote:
>
> I think you just made the case... if you have existing code that declares a concrete return type and you change that to an interface and nothing else you are going to have problems.

I don't think anybody said "..change it to interface and nothing
else". The problem is, even if you change it to include a nil-check it
is still wrong. I agree that you should not do that. There is the
"vet" tool that tells you about things you shouldn't do, and I think
it would be useful if it warned against that.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/EEC803AB-E5A9-4274-9CE6-6DB2A340F29B%40ix.netcom.com.

[Ian Lance Taylor]

On Thu, Aug 27, 2020 at 3:53 AM targe...@gmail.com <targe...@gmail.com> wrote:
>
> Please don't forget that interface pointer isn't simple. It's a so-called "fat pointer" which consists of pointer to data and pointer to methods table. Thus, we have 4 potential states - (nil, nil), (nil, pointer), (pointer, nil) and (pointer,pointer). First one is just ordinary nil interface. Last one is just ordinary interface pointer with all operations callable without issue (as long as they're properly implemented, of course). Third one with valid data pointer and nil table pointer can be AFAIK created only through reflection and is safely considered "invalid state". We're left with second state with nil data pointer and valid table pointer, making interface methods callable on it. It wouldn't be a problem - if such state couldn't be created by merely casting typed nil pointer (typed pointer variable with nil value) to interface. Why is this an issue? Because most interface implementations require valid data pointer. "Static" (i.e. not needing data) implementations can be considered corner case.

Other people have said this too, but I want to emphasize that there
are not 4 potential states of an interface value. There are 2
potential states: 1) interface does not hold any value, and is == nil;
2) interface holds a value, and is != nil.

When you start talking about an interface as a fat pointer you are
talking about how those 2 states are implemented. There are other
possible implementations, and in fact the implementation in gccgo used
to be different. And even the implementation in the gc compiler has
changed; at one time ints and other small values were stored directly
in the interface value, but now we instead store a pointer to the
data.

Ian

[Robert Engels]

Maybe I misunderstood this

>
> The first is the old adage: "Accept interfaces return concrete types."

No interface based design would have you declare a concrete type as the return value - so it must be declaring a interface return and returning a concrete.

Which is what was being argued against.


> On Aug 27, 2020, at 1:36 PM, burak serdar <bse...@computer.org> wrote:

[targe...@gmail.com]

> Other people have said this too, but I want to emphasize that there
are not 4 potential states of an interface value. There are 2
potential states: 1) interface does not hold any value, and is == nil;

2) interface holds a value, and is != nil.

Indeed, there are 2 states of interface observable from outside. Although one of these observable states gets expressed through 2 states invisible to outer world. Which isn't a problem on its own.

> When you start talking about an interface as a fat pointer you are
talking about how those 2 states are implemented. There are other
possible implementations

Yes. And that's not an issue on its own. The issue is, such implementation detail kind of "leaks" sometimes and bites you when you don't expect. And it does so because the way it gets constructed omits one specific detail of conversion from pointer to interface.

[targe...@gmail.com]

> I gave you a third solution: Having a separate identifier for the zero value of interfaces.

It doesn't solve issue at hand. The issue is, conversion from typed nil pointer to interface results in non-nil interface with valid vtable pointer and nil receiver. And only limited set of specifically crafted implementations supports nil receiver. What would be the meaning of "none" interface? If it's nil data pointer, it would result in false-negatives for implementations which support nil receiver.

[Axel Wagner]

The clarifying question first:

What would be the meaning of "none" interface?

A `none` interface would be the zero value of an interface - an interface value with no dynamic type/value.

It doesn't solve issue at hand. The issue is, conversion from typed nil pointer to interface results in non-nil interface with valid vtable pointer and nil receiver.

I don't think this is the core of the problem.

Above, one case mentioned where this happens was refactoring a return type (or really any variable) from a pointer to an interface. This changes the semantic of `v == nil` from "is the value of v a nil-pointer" to "does v have a dynamic value (even a nil-pointer)", leading to silent bugs. That problem would be solved, because the compiler would now complain about `v == nil` if v is an interface, prompting you to change it to `v == none` - i.e. the bug would no longer be silent.

Another frequent occurrence is people *intending* to check if an interface contains a nil-pointer and then getting confused because they instead check if the interface has a dynamic value. Again, this confusion would be solved - it is now clear that you can't compare an interface to nil, so you have to use reflection to make that determination.

And only limited set of specifically crafted implementations supports nil receiver.

I would argue (without any data) that it is roughly as common for a method to not dereference a pointer-receiver, as it is sensible to check if the dynamic value of an interface is a nil-pointer. Generally, either

1. You know nothing about an interface-value whatsoever. You accept any e.g. io.Reader. In that case, introspecting the dynamic value of the interfaces makes no sense - it might not even *be* a pointer and even if it is, you have no idea whether the implementation is fine with a nil-pointer or not.

2. You know nothing about an interface-value, but need to know its shape and kind and maybe have to store something in there (so need a non-nil pointer). The `json` and `fmt` package are common examples. In this case, you need to use reflect anyway.

3. You know something about an interface-value. There's some set of supported dynamic types (e.g. ast.Node) which you know. In that case, you can add an (unexported) method to the interface to check for validity (which is even better, as it also works with non-pointers that need initialization) or use `reflect`.

I just can't think of many places where "checking if the dynamic value of an interface is a nil-pointer" is necessary or even useful, that don't need to use reflect anyway.

If you can't accidentally do it and if you can't get confused as to why "my non-nil interface contains nil", then at least it becomes obvious that you are using the wrong approach.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0ed95322-2b0c-4374-ab3c-3c68759e9ba5n%40googlegroups.com.

[Ian Lance Taylor]

On Wed, Sep 2, 2020 at 3:45 AM targe...@gmail.com <targe...@gmail.com> wrote:
>
> > Other people have said this too, but I want to emphasize that there
> are not 4 potential states of an interface value. There are 2
> potential states: 1) interface does not hold any value, and is == nil;
> 2) interface holds a value, and is != nil.
>
> Indeed, there are 2 states of interface observable from outside. Although one of these observable states gets expressed through 2 states invisible to outer world. Which isn't a problem on its own.
>
> > When you start talking about an interface as a fat pointer you are
> talking about how those 2 states are implemented. There are other
> possible implementations
>
> Yes. And that's not an issue on its own. The issue is, such implementation detail kind of "leaks" sometimes and bites you when you don't expect. And it does so because the way it gets constructed omits one specific detail of conversion from pointer to interface.

I don't think that is correct. The implementation detail never leaks.
An interface always either holds a valid value (is != nil) or does not
hold a valid value (is == nil).

I believe the confusion is due to the overloading of "nil". It means
both an invalid interface and an invalid pointer, but you are
permitted to store an invalid pointer in a valid interface.

Ian