Taurus and GIL

[Giacomo S.]

Hello Carlos,

I have a technical question concerning Taurus and Python GIL.
How is multi threading managed inside Taurus?
Once I had a talk with Tiago about this topic and, as far as I can
remember, he said that, where required, the
GIL is somehow disabled by Taurus in order to achieve real multi
threaded behaviour (real POSIX/Windows threads).

Could you please confirm this and, consequently, claim that Taurus
threads are really concurrent threads?

Second question. In a situation where a third party library relies on
the GIL (being enabled), is there any
concurrency issue potentially introduced when the same library operates
in a context where the GIL is disabled?

Gracias, Giacomo.

[coutinho]

Hi Giacomo,

On 07/28/2016 05:29 PM, Giacomo S. wrote:
> Hello Carlos,
>
> I have a technical question concerning Taurus and Python GIL.
> How is multi threading managed inside Taurus?
> Once I had a talk with Tiago about this topic and, as far as I can
> remember, he said that, where required, the
> GIL is somehow disabled by Taurus in order to achieve real multi
> threaded behaviour (real POSIX/Windows threads).

The GIL only needs to be managed by PyTango (not taurus) because this
is where the blocking network calls occur.
Basically, in every place PyTango calls a Tango C++ I/O blocking call,
it releases the GIL so that other threads that may execute python code
are not blocked.

>
> Could you please confirm this and, consequently, claim that Taurus
> threads are really concurrent threads?

Taurus is based on the concept of pluggable Control Systems. By this I
mean it can work with Tango, Epics or other third-party control systems.
If the taurus plug-in that is written for a specific control system
happens to block the thread there is nothing that taurus can do about it.
It all depends on how the plug-in and the third-party libraries are written.
For the case of Tango I can tell you that the PyTango library manages
the GIL properly.

>
> Second question. In a situation where a third party library relies on
> the GIL (being enabled), is there any
> concurrency issue potentially introduced when the same library operates
> in a context where the GIL is disabled?

Since taurus does not handle the GIL by itself it does not generate any
concurrency issue.

>
> Gracias, Giacomo.
>

Thanks Giacomo for your pertinent questions. It is important that in the
context of a distributed (potentially blocking) control system, we
understand what happens in python with the GIL, who has the
responsibility to manage it and where blocking may occur.

In the case of Tango, it should be the library that handles Tango
(PyTango) who abstracts the underlying complexities of multi-threaded,
I/O blocking control system.

Taurus does not take care of the GIL because it trusts that the blocking
library does it well. Having said that, taurus does create a pool of
threads that poll the underlying attributes (tango or other) in an old
style attempt to paralyze blocking I/O calls.
We choose threads in taurus because at the time PyTango was not
coroutine friendly. Probably if it was done today we would have used
another simpler, lighter mechanism (like gevent or asyncio).

Hope it helps,

Cheers
Tiago