thread local storage of C library from go application

[Pavan]

Hi, 

We are using C library which has some data stored per thread in thread-local-storage. 

go application is using this library. as threads terminate, TLS storage was freed but now with go application, 

the TLS seems invalid or inconsistent. I mean 2 go routines can be running under same pthread, is my understanding correct?

Also looks the thread cleanup handlers, pthread_cleanup_push/pop no longer get hit in go application. 

Then i see better approach would be 

- either use go routine specific data and remove TLS from C pthreads . 

- 

Please suggest, is there other better approaches for the existing C libraries using TLS (thread local storage) 

Regards,

Pavan

[Tamás Gulácsi]

Lock C-calling goroutines to their threads with runtime LockOSThread and communicate with them through channels.

[Pavan]

Thanks Tamas , I see earlier discussion too on this

https://groups.google.com/forum/#!topic/golang-nuts/8Tdb-aCeYFY.

As I understand, the GO library should lock the current go routine to thread, if it involves C functions with TLS data and unlock after completing such C functions . 

Regards,

[Pavan]

Hi, 

putting the lockosthread from library cant match with goroutines TLS and its complex  on handling goroutines exit and mapping all the GO api's  (C fns if any with them ) called by applications to same OS thread . 

ofcourse applications cant call lockosthread too as they are not aware of the GO API internals  of c functions usage. 

I think this complexity will stay if the C library is called from GO, please suggest if i missed anything. 

Regards,

[Tamás Gulácsi]

2020. április 22., szerda 10:04:37 UTC+2 időpontban Pavan a következőt írta:

Hi, 

putting the lockosthread from library cant match with goroutines TLS and its complex  on handling goroutines exit and mapping all the GO api's  (C fns if any with them ) called by applications to same OS thread . 

ofcourse applications cant call lockosthread too as they are not aware of the GO API internals  of c functions usage. 

In your library, spawn a goroutine, which

1. calls runtime.LockOSThread,

2. and then listens to commands on a channel, 

3. calls the C library,

4. responds on a channel given beside the command

in a loop.

And hide all this, and create the API that translates direct calls to such command sending on the command channel,

waiting for the answer and returning it.

The essence is that the C-calling goroutine must be on the same thread where it has started.

This is to allow the C library to work, which assumes a lot about its calling environment when storing data in thread-local storage....

[Pavan]

Thanks Tamas for elaborating.

 So if i understand, the C library will see the calls as if application is single threaded, i mean a single goroutine/thread issues C calls.

ofcourse go library can manage to distribute to different gorotuines/threads based on the API type .  will give more thoughts on it,

Regards,