Global Interpreter Lock and Callbacks
Navin Keswani
I ran into a problem with the global interpreter lock (GIL) and
callbacks and think I have solved the problem but would like some
advice as I am new to this ...
The scene:
I call C function "socketPoll" from Python -- socketPoll does some I/O
stuff and then calls a callback function on the stuff its read.
The problem:
A thread with socketPoll in it didn't allow any other threads to
execute.
The solution:
Wrapped Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS around the I/O
stuff in the C socketPoll code.
A new problem:
Core dumps when the callback function is called.
Solution:
I used the code underlying the macros -- so I declare a global
variable "save" and I use
save = PyEval_SaveThread(); in place of Py_BEGIN_ALLOW_THREADS before
the socketPoll; and,
PyEval_RestoreThread( save ) in place of Py_END_ALLOW_THREADS after
the socketPoll;
I disable the global lock before the callback -- i.e
PyEvalRestoreThread( save )
and then restore the lock after the callback by using save =
PyEval_SaveThread() again. Why am I not using the macros instead --
well, because the "save" variable within the macros is not declared as
global so the nested locks and unlocks don't sit well with the
compiler.
In summary, I manage to Lock before socketPoll, unlock before the
callback, lock after returning from the callback and then unlock after
returning from socketpoll.
This works! Except that I don't know for sure that this isn't leaving
a thread state dangling precariously when a thread exits. The reason
I am nervous is that if I set up a loop to continuously socketPoll in
one thread and have a couple of other threads running with loops doing
other things, exiting the program with a ctrl-C causes core dumps but
only intermittently.
Does anyone see any obvious problems with what I am doing?
Thanks in advance--
Navin
nav...@zip.com.au