Embedded interpreter hangs when running Numba jitted functions

[Ioannis Moschakis]

Hi all,

I am facing a problem when running Numba jitted code in an embedded interpreter.

The setup is the following:

1. A Java daemon process offloads part of its calculations to a C++ library (just for reference JavaCPP is used for the mapping between C++/Java)

2. The C++ library reads data via the Java mapping and places them in NumPy Arrays (using pybind11 api) and

3. Initializes an embedded Python interpreter and passes the arrays to Python code which makes heavy use of Numba jitted functions

Steps 2 & 3 follow an iteration pattern i.e. once Java passes control over to C++, the C++ part will call the Python code a large number of times passing different data each time until the iteration finishes (data are read at the Java level via JavaCPP callbacks). Once the C++ iteration finishes control is passed back to Java. At a later time the process may be repeated by another external call, calling the same or different Python code.

Since the java process is a daemon the embedded interpreter is never actually closed once started.

After a (seemingly) random number of calls the python interpreter seems to hang while executing jitted code. CPU Util% drops to nothing and the process never finishes.

This never happens on the first call (i.e. the first C++ iteration loop (which boots the embedded interpreter) always finishes), but it may happen on the 2nd or any other call down the line.

If Numba is disabled (via NUMBA_DISABLE_JIT) Python never freezes (calc time ofc goes through the roof).

If the java daemon process is restarted before calling C++ it also never freezes (as I've noted above, it never freezes on the 1st call).

Has anyone encountered something similar before? Is there any debugging infrastructure that may be of use towards tracking the root cause of this issue?

This whole stack is part of a much larger code base so providing a working replication example is not easy.

Thanks in advance!

[Ioannis Moschakis]

Forgot to mention versions:

Python 2.7

Numba 0.39