about -buildmode c-shared

[Frédéric De Jaeger]

Hi, I'm a bit curious about some constraints describe here  [Go execution Mode] (

https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ) and the fact we can't dlclose

a go plugin (https://github.com/golang/go/issues/11100)

It is said:

> ... If a C program has a C API style plugin interface, and it opens multiple plugins written in Go, all of those plugins must be built with the same version of the Go toolchain, and any shared packages must be identical.

>  all Go code shares a single runtime

Is it still valid?

I'm interested in the scenario where a C host loads some plugins (using dlopen) that happens to be written in go (buildmode c-shared).  It looks only one runtime will be instantiated and I wonder what it the reason behind that.  I can imagine that it might be cool to have two go plugin being able to communicate easily because they happen to be loaded in the same runtime. 

But IMHO, this is a nice too have.   Is there really a business for that ?  People depending on that behavior ?  Or does it happen to be easier to implement ? (I would have thought the opposite)

Naively, I would have expected that those two go plugins, loaded in the same C process using C api, would just not see each other.  Like two separate processes.  Separate heap, separate system threads, ... no easy way to communicate directly.  That way, correctly dlclosing a plugin  seems more tractable (it is perfectly ok to assume the business layer is responsible to correctly terminate all its running go-routine, and if it don't, bad things happen.  like dlclosing a C dll while there is a thread running some code of the dll.  Most hosts usually provide a business call to notify a plugin it's time to die)

Without that, this prevents the usage of go an alternative to write plugins.  Many hosts might dlclose a plugin when they don't need it.  And many independant vendors could use different versions of go to write their plugins.

So, what is the status on that ?

Thanks for your help.

Fred

[Ian Lance Taylor]

On Sun, Feb 7, 2021 at 3:08 PM Frédéric De Jaeger
<fdej...@novaquark.com> wrote:
>
> Hi, I'm a bit curious about some constraints describe here [Go execution Mode] (
> https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ) and the fact we can't dlclose
> a go plugin (https://github.com/golang/go/issues/11100)
>
> It is said:
>
> > ... If a C program has a C API style plugin interface, and it opens multiple plugins written in Go, all of those plugins must be built with the same version of the Go toolchain, and any shared packages must be identical.
>
> > all Go code shares a single runtime
>
> Is it still valid?

First let me note that this discussion should probably move to
golang-nuts. golang-dev is for discussion of the development of the
Go language and tools, not for general discussion about Go.

Second let me note that your use of the word "plugin" is ambiguous
here. We have -buildmode=plugin to produce plugins for Go programs
and we have -buildmode=c-shared to produce shared libraries that can
be opened by C programs. The latter can also be called plugins, for C
programs. I think that you are talking about the latter, not the
former.

That said, yes, it remains the case that if a C programs dlopens
multiple Go shared libraries built using -buildmode=c-shared, then
they must all be built with the same version of the Go toolchain and

any shared packages must be identical.

> I'm interested in the scenario where a C host loads some plugins (using dlopen) that happens to be written in go (buildmode c-shared). It looks only one runtime will be instantiated and I wonder what it the reason behind that. I can imagine that it might be cool to have two go plugin being able to communicate easily because they happen to be loaded in the same runtime.
> But IMHO, this is a nice too have. Is there really a business for that ? People depending on that behavior ? Or does it happen to be easier to implement ? (I would have thought the opposite)

This is a consequence of the fact that (on most systems) shared
libraries share a symbol namespace, so that all references to, say,
runtime.chansend are resolved to the same function. If different Go
shared libraries built using -buildmode=c-shared are built with
different versions of Go, then the fact that they call the same
runtime.chansend function can mean that one of the shared libraries
will not work correctly.

> Naively, I would have expected that those two go plugins, loaded in the same C process using C api, would just not see each other. Like two separate processes. Separate heap, separate system threads, ... no easy way to communicate directly. That way, correctly dlclosing a plugin seems more tractable (it is perfectly ok to assume the business layer is responsible to correctly terminate all its running go-routine, and if it don't, bad things happen. like dlclosing a C dll while there is a thread running some code of the dll. Most hosts usually provide a business call to notify a plugin it's time to die)

That's not how shared libraries work on many systems.

Ian

[Frédéric De Jaeger]

On Monday, February 8, 2021 at 12:18:43 AM UTC+1 Ian Lance Taylor wrote:

First let me note that this discussion should probably move to
golang-nuts. golang-dev is for discussion of the development of the
Go language and tools, not for general discussion about Go.

oops, sorry about that.  Stupid me.  I've answered you there https://groups.google.com/g/golang-nuts/c/fin6OT3KCOU