Go Runtime sharing with buildmode=c-shared

[Nikita Rirush]

Greetings!

I recently stumbled onto a project that implements an SQLite extension in Go as a c-shared library, and wondered if Go shares its runtime between multiple c-shared Go libraries in the same process.

I tried searching for issues on Github and through this group, but I wasn't able to find any concrete information on how this works. I found this issue (https://github.com/golang/go/issues/18976) that states that multiple Go runtimes are not supported, but I made a simple library, built it as c-shared, and linked it to a Go program, and it worked just fine, so this is probably not the case anymore.

This conversation (https://groups.google.com/g/golang-nuts/c/fin6OT3KCOU/m/ub3aWquAAAAJ) seems to hint that runtimes may be shared, but it's not 100% clear if that's the case.

So the question is, is linking multiple Go libraries in the same process and c-shared Go libraries into Go programs now supported? Do those libraries (and optionally a host program) share runtimes, or do they each start separate runtimes that somehow cooperate?

Thanks!

[Ian Lance Taylor]

On Thu, Aug 4, 2022 at 11:45 AM Nikita Rirush <raccoo...@gmail.com> wrote:
>
> I recently stumbled onto a project that implements an SQLite extension in Go as a c-shared library, and wondered if Go shares its runtime between multiple c-shared Go libraries in the same process.
>
> I tried searching for issues on Github and through this group, but I wasn't able to find any concrete information on how this works. I found this issue (https://github.com/golang/go/issues/18976) that states that multiple Go runtimes are not supported, but I made a simple library, built it as c-shared, and linked it to a Go program, and it worked just fine, so this is probably not the case anymore.
>
> This conversation (https://groups.google.com/g/golang-nuts/c/fin6OT3KCOU/m/ub3aWquAAAAJ) seems to hint that runtimes may be shared, but it's not 100% clear if that's the case.
>
> So the question is, is linking multiple Go libraries in the same process and c-shared Go libraries into Go programs now supported?

No, it is not.

> Do those libraries (and optionally a host program) share runtimes, or do they each start separate runtimes that somehow cooperate?

Separate runtimes can in some cases live entirely independently, in
which case they don't cooperate. That can be fine as long as you
don't try to call functions between them, as they just allocate memory
as needed. Exactly what is happening in your case is highly platform
specific, and not really guaranteed to continue working.

Ian

[Nikita Rirush]

Ok, thank you!

Does that mean that writing plugins, e.g. for SQLite, can cause issues when multiple Go extensions are loaded in the same process?

Or is this use-case actually safe, and issues can start to happen only when there's a Go host and a Go c-shared library in the same process?

[Ian Lance Taylor]

On Thu, Aug 4, 2022 at 12:16 PM Nikita Rirush <raccoo...@gmail.com> wrote:
>
> Does that mean that writing plugins, e.g. for SQLite, can cause issues when multiple Go extensions are loaded in the same process?
> Or is this use-case actually safe, and issues can start to happen only when there's a Go host and a Go c-shared library in the same process?

Well, plugins, as in -buildmode=plugin, are a different case. I don't
know anything about SQLite. If that is a C program, and if you try to
use it with multiple Go -buildmode=c-shared libraries, then, yes, that
can cause issues in some cases.

Ian

> On Thursday, 4 August 2022 at 21:55:45 UTC+3 Ian Lance Taylor wrote:
>>
>> On Thu, Aug 4, 2022 at 11:45 AM Nikita Rirush <raccoo...@gmail.com> wrote:
>> >
>> > I recently stumbled onto a project that implements an SQLite extension in Go as a c-shared library, and wondered if Go shares its runtime between multiple c-shared Go libraries in the same process.
>> >
>> > I tried searching for issues on Github and through this group, but I wasn't able to find any concrete information on how this works. I found this issue (https://github.com/golang/go/issues/18976) that states that multiple Go runtimes are not supported, but I made a simple library, built it as c-shared, and linked it to a Go program, and it worked just fine, so this is probably not the case anymore.
>> >
>> > This conversation (https://groups.google.com/g/golang-nuts/c/fin6OT3KCOU/m/ub3aWquAAAAJ) seems to hint that runtimes may be shared, but it's not 100% clear if that's the case.
>> >
>> > So the question is, is linking multiple Go libraries in the same process and c-shared Go libraries into Go programs now supported?
>>
>> No, it is not.
>>
>> > Do those libraries (and optionally a host program) share runtimes, or do they each start separate runtimes that somehow cooperate?
>>
>> Separate runtimes can in some cases live entirely independently, in
>> which case they don't cooperate. That can be fine as long as you
>> don't try to call functions between them, as they just allocate memory
>> as needed. Exactly what is happening in your case is highly platform
>> specific, and not really guaranteed to continue working.
>>
>> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9af821a4-401c-4fe4-b0b1-e3c234f0ee6cn%40googlegroups.com.