[compilation|gob] Is it possible to have multiple gob encoding instances?

[gavraz]

Hey,

So gob has a Register function that mutates a "global" state. This implies that a single program cannot have multiple instances with different registrations.

I tried to create another module that wraps gob with its own go.mod with a different toolchain then what my program uses hoping that gob.Register != mygob.Register but no luck.

I have a few questions

1. Why didn't the gob-wrap work? 

2. Is there a proper way to get two instances of gob?

3. Why do you think gob/encoding was designed this way in the first place? 

P.S. if you are asking yourself why do I need two instance of gob, the reason is gob has been used in our project, improperly, for quite a while and I am writing something new I rather not be involved in the current mess. Apparently, it is not straight forward to achieve this.

Thanks.

[Ian Lance Taylor]

On Fri, Aug 23, 2024 at 4:26 PM gavraz <gav...@gmail.com> wrote:
>
> So gob has a Register function that mutates a "global" state. This implies that a single program cannot have multiple instances with different registrations.

True.

> I tried to create another module that wraps gob with its own go.mod with a different toolchain then what my program uses hoping that gob.Register != mygob.Register but no luck.
> I have a few questions
> 1. Why didn't the gob-wrap work?

I'm not clear on exactly what you did, but in general there can only
be one package with a given path in a binary. And an entire Go
program must be compiled by a single toolchain.

> 2. Is there a proper way to get two instances of gob?

Well, you can copy the contents of encoding/gob to a different path,
and import from that path. The two implementations will be
independent.

> 3. Why do you think gob/encoding was designed this way in the first place?

In general it doesn't matter whether different users of encoding/gob
use the gob.Register function. It's no big deal if the two users both
register the same type.

On the other hand you can get failure if different parts of the
program try to use RegisterName with overlapping names. I guess the
expectation was that that would not happen in practice. encoding/gob
was designed and implemented quite a while ago, in 2010. We might not
do it the same way today.

Ian

[gavraz]

[Thank you for taking the time to answer my questions, very much appreciated.]

1. I attempted to wrap gob with my own package and specify a distinct toolchain, hoping it would result in a fully independent copy of the gob package in the compiled code. Thanks for the clarification, this approach is a no-go.

2. The challenge with forking the gob package is the import of "internal/saferio" in decode.go, along with maintainability concerns raised by the team. Any workarounds?

3. The problem we are facing is two-fold: first, we used Register instead of RegisterName. Second, we have a primary decode flow that mistakenly uses &target where target is an interface, which requires registering even trivial structures. Developers have become accustomed to registering every structure, and now we risk breaking the decode process during refactoring due to path changes. Since I'm developing an independent module, I’d prefer to start fresh and decouple this new module from these side effects.  

By the way, my initial goal to find an encoding that offers:

1. Easy handling of interfaces (gob, zson)

2. Easy handling of maps with mixed value types (not zson)

So, I ended up with gob but realized it will be safer to separate the current usages from the new module.

Thanks.

[Ian Lance Taylor]

On Sat, Aug 24, 2024 at 2:51 AM gavraz <gav...@gmail.com> wrote:
>
> 1. I attempted to wrap gob with my own package and specify a distinct toolchain, hoping it would result in a fully independent copy of the gob package in the compiled code. Thanks for the clarification, this approach is a no-go.
> 2. The challenge with forking the gob package is the import of "internal/saferio" in decode.go, along with maintainability concerns raised by the team. Any workarounds?

Well, you could copy the internal/saferio package also. It's pretty
small and doesn't have any other dependencies. I agree that this
approach is less maintainable. Perhaps it would be an acceptable
approach if your new code will eventually displace the old code.

> 3. The problem we are facing is two-fold: first, we used Register instead of RegisterName. Second, we have a primary decode flow that mistakenly uses &target where target is an interface, which requires registering even trivial structures. Developers have become accustomed to registering every structure, and now we risk breaking the decode process during refactoring due to path changes. Since I'm developing an independent module, I’d prefer to start fresh and decouple this new module from these side effects.

It's not a problem for many different packages to call Register. They
will all get the same results. So the fact that there is a global
registry doesn't really matter as long as everybody only calls
Register, not RegisterName. So one option is to just not worry about
this.

Ian

[gavraz]

Thanks, I'll rediscuss the risk with my team, maybe we can live with it.

Btw, I prefer RegisterName, as long as we register in init, the program will crush immediately for a dup. On the other hand, Register exposes us to refactoring a type's name or to moving a type from package A to package B, implying we may fail on the decode at runtime for values encoded prior to the refactor.

gavraz