panic: interface conversion: interface is nil, not encoding.BinaryUnmarshaler

[Jochen Voss]

Hello,

I'm trying to read gob-encoded data into a variable of interface type.  In simple cases, this works for me, but when I use a custom encoding via MarshalBinary() and UnmarshalBinary() methods, my code keeps crashing, with the error message

    panic: interface conversion: interface is nil, not encoding.BinaryUnmarshaler

Example:

- The code at https://play.golang.org/p/y8nWNhObUwb, letting gob do the en-/decoding works.

- The very similar code at https://play.golang.org/p/-zS7QjEJg9x, which uses MarshalBinary() and UnmarshalBinary() crashes with the panic shown above.

What am I doing wrong?

[I asked the same question at https://stackoverflow.com/questions/57485698/, with no answers so far]

Many thanks,

Jochen

[Marcin Romaszewicz]

Here you go:

https://play.golang.org/p/_cPmaSxRatC

You want to unmarshal into &A, not into &Duck

This means:

var duck2 A

not:

var duck2 Duck

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b19c51bc-524c-4292-915b-fc00e9289052%40googlegroups.com.

[Jochen Voss]

Dear Marcin,

My aim is to unmarshal into an interface variable, without having to know in advance which concrete type I'm receiving (the actual interface has several possible implementations, and gob data comes in over a network connection). So, while your code avoids the panic, it does not solve my problem.

Many thanks,

Jochen

On Wednesday, 14 August 2019 22:12:54 UTC+1, Marcin Romaszewicz wrote:

Here you go:

https://play.golang.org/p/_cPmaSxRatC

You want to unmarshal into &A, not into &Duck

This means:

var duck2 A

not:

var duck2 Duck

On Wed, Aug 14, 2019 at 8:46 AM Jochen Voss <joche...@gmail.com> wrote:

Hello,

I'm trying to read gob-encoded data into a variable of interface type.  In simple cases, this works for me, but when I use a custom encoding via MarshalBinary() and UnmarshalBinary() methods, my code keeps crashing, with the error message

    panic: interface conversion: interface is nil, not encoding.BinaryUnmarshaler

Example:

- The code at https://play.golang.org/p/y8nWNhObUwb, letting gob do the en-/decoding works.

- The very similar code at https://play.golang.org/p/-zS7QjEJg9x, which uses MarshalBinary() and UnmarshalBinary() crashes with the panic shown above.

What am I doing wrong?

[I asked the same question at https://stackoverflow.com/questions/57485698/, with no answers so far]

Many thanks,

Jochen

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[Axel Wagner]

I haven't really used gob much, so unfortunately I can't *really* help you. But one thing to note is that you can dump the content of the buffer and see that it doesn't actually contain the name of the type you are encoding: https://play.golang.org/p/R8HB6RP8kS0

I agree that from the documentation, it seems to me that it *should* and I can't really see what this is doing wrong. And I agree that this probably shouldn't panic, but instead return an error. But given that the encoded bytes don't contain the name of the concrete struct, there is no way the decoder can know where you want to put it, without giving it the correct type.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/042b28c0-d768-4cba-94cf-159cf4231818%40googlegroups.com.

[Jochen Voss]

Hi Axel,

On Thursday, 15 August 2019 09:13:17 UTC+1, Axel Wagner wrote:

I haven't really used gob much, so unfortunately I can't *really* help you. But one thing to note is that you can dump the content of the buffer and see that it doesn't actually contain the name of the type you are encoding: https://play.golang.org/p/R8HB6RP8kS0

You are right.  But the type is somehow represented by some index or so, instead of the name?  Since the type needs to be registered using gob.Register() beforehand, this seems not totally impossible to me.  But then, when I change your code to let gob do the encoding itself, the name is suddenly there: https://play.golang.org/p/Qz7zDKL047z, so probably the absence of the type name is part of the problem.

 

And I agree that this probably shouldn't panic, but instead return an error. But given that the encoded bytes don't contain the name of the concrete struct, there is no way the decoder can know where you want to put it, without giving it the correct type.

Yes, an informative error instead of the hard to interpret panic would be nice.  I wonder whether this is indeed intended behaviour, or whether this is some kind of bug in the encoding/gob package.

[Axel Wagner]

On Thu, Aug 15, 2019 at 11:14 AM Jochen Voss <joche...@gmail.com> wrote:

You are right.  But the type is somehow represented by some index or so, instead of the name? Since the type needs to be registered using gob.Register() beforehand, this seems not totally impossible to me.

`Register` only gets the type-name though. It somehow would have to reproducibly map from type-names to indices and back for this to work - the process doing the *decoding* isn't necessarily the same process doing the encoding (that's kind of the point :) ). So AIUI, the name still needs to be present. The documentation says so too:

Interface values are transmitted as a string identifying the concrete type being sent (a name that must be pre-defined by calling Register), followed by a byte count of the length of the following data (so the value can be skipped if it cannot be stored), followed by the usual encoding of concrete (dynamic) value stored in the interface value. (A nil interface value is identified by the empty string and transmits no value.) Upon receipt, the decoder verifies that the unpacked concrete item satisfies the interface of the receiving variable.

 

I wish I could help more :) I'd be interested to see how you get this resolved though (and maybe someone who used gob more can chime in). AIUI it *should* be possible to do what you want, so once we figured out how, it would probably be super helpful to contribute it as a runnable example.

  But then, when I change your code to let gob do the encoding itself, the name is suddenly there: https://play.golang.org/p/Qz7zDKL047z, so probably the absence of the type name is part of the problem.

 

And I agree that this probably shouldn't panic, but instead return an error. But given that the encoded bytes don't contain the name of the concrete struct, there is no way the decoder can know where you want to put it, without giving it the correct type.

Yes, an informative error instead of the hard to interpret panic would be nice.  I wonder whether this is indeed intended behaviour, or whether this is some kind of bug in the encoding/gob package.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/32c27653-c50b-42cd-a525-8c9e9298bf60%40googlegroups.com.

[Axel Wagner]

Hmmmmm, I looked a bit at the code. ISTM that if you implement BinaryMarshaler/Unmarshaler (or GobEncoder/Decoder), the values are sent as a []byte, not an interface, so the concrete type information is lost. You can kind of circumvent this, by instead passing the values as interface{}: https://play.golang.org/p/n5oxq9g2Q-9 -- As you can see, gob still uses the methods, but because you more specifically say that it should en/decode interfaces, it's passing the type-info along too. This will require that you do your own type-assertions after decoding, but maybe that's acceptable?

The panic, I think, comes from the fact that you pass a *Duck, which then gets dereferenced, pointing at a nil Duck. Even a nil-Duck *does* have a DecodeBinary method though, so gob thinks it should use that. It then type-asserts to BinaryDecoder, which panics, because it's a nil-interface, just like this example: https://play.golang.org/p/EEwOt0FQunh. Arguably, the language should allow this (returning nil from the type-assertion), but either way, it won't help you, because it would panic when calling the method anyway. So, in a way, gob behaves correctly. It says that if the value you decode into implements BinaryUnmarshaler, it will call it's UnmarshalBinary method. And the BinaryUnmarshaler you pass will panic if you actually call that method. It's still not a great failure mode.

I hope this kinda helps. Or that someone can give a more authoritative answer :)

[Robert Engels]

I think you will need to encode a struct that contains the type name and the struct. Then you decode using reflection.

-----Original Message-----
From: 'Axel Wagner' via golang-nuts
Sent: Aug 15, 2019 3:12 AM
To: Jochen Voss
Cc: golang-nuts
Subject: Re: [go-nuts] panic: interface conversion: interface is nil, not encoding.BinaryUnmarshaler

I haven't really used gob much, so unfortunately I can't *really* help you. But one thing to note is that you can dump the content of the buffer and see that it doesn't actually contain the name of the type you are encoding: https://play.golang.org/p/R8HB6RP8kS0

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b19c51bc-524c-4292-915b-fc00e9289052%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/042b28c0-d768-4cba-94cf-159cf4231818%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfFp567QVvMy4iOmYfuWzQFdbNovNAJXGu%2Bk_%3DcDKm7yBg%40mail.gmail.com.

[Axel Wagner]

Please see upthread - gob does that for you.

[Jochen Voss]

Hi Axel,

Thanks a lot for looking into this.  Your post makes things much clearer for me.  In particular, I now think that I probably should not have (Un)MarshalBinary() methods on the interface type, but have them only on the concrete type instead.  For example, this works: https://play.golang.org/p/oBgHqN74ZeM .

On Thursday, 15 August 2019 11:16:45 UTC+1, Axel Wagner wrote:

The panic, I think, comes from the fact that you pass a *Duck, which then gets dereferenced, pointing at a nil Duck. Even a nil-Duck *does* have a DecodeBinary method though, so gob thinks it should use that. It then type-asserts to BinaryDecoder, which panics, because it's a nil-interface, just like this example: https://play.golang.org/p/EEwOt0FQunh.

I had forgotten that type assertions don't allow nil values, but indeed they don't.  Thank you for reminding me.

Many thanks,

Jochen

[Ninja Ikta]

I encountered this tricky bug today. After going through all the playground examples, I finally understood what was happening.

Similar to Jochen, I was including encoding.MarshalBinary() and encoding.UnmarshalBinary() in the interface definition.

A question though. Why does this bug resolve by simply eliminating the above two method signatures from the interface definition?

1) My basic understanding was that including these method signatures just informs that the underlying concrete type will support these methods. How does their inclusion or exclusion change anything?
2) Even if the method signatures are not part of the interface definition, the problem of dereferencing a nil-Duck still exists. How does that get resolved of methods are excluded in interface definition?

Thanks!