Include paths in SharedInterpreter

[Mike]

Hi,

We have been having numerous problems with shared modules on 3.7.1 so we have a large list of modules we share.

I wanted to try out the new SharedInterpreter but I noticed there is no constructor that takes a JepConfig.

There are folders defined as part of our configuration where python scripts could be stored so I need to add these folders dynamically to the Jep instance as include paths. I currently do:

new Jep(new JepConfig().addIncludePaths(libDirs).addSharedModules(sharedModules))

How do I now let a SharedInterpreter know about the libDirs?

Regards,

Mike

[Ben Steffensmeier]

Shared interpreters do not take a JepConfig in the constructor because most of the options are invalid or they would impact all shared interpreters, not just the one being constructed. The include path in Jep just extends sys.path which would affect all shared interpreters. You can achieve the same thing by just modifying sys.path in one of the shared interpreters. You should be able to do this easily with a few Jep.eval() statements, for example this is where it is done in the Jep source with an eval.

Ben

[Mike]

Thanks for the pointer Ben, I managed to get this working using sys.path += as you do in the Jep Source.

I have a slightly stranger issue now. Our system allows users to write plugins in python and we use Jep to call the python function in java.

We allow the user to modiy their code and re-calculate on the fly.

To do this When I was using the standard Jep instances I would check the file modified time and if it had been updated since the last calculation I would use a new Jep instance to force the 'plugin' module to be re-loaded.

When switching to SharedInterpreter, creating a new SharedInterpreter does not way in the same way as the state is shared among all SharedInterpreters.

Instead I came used importlib.reload as follows:

final boolean modified = checkModifiedTime();
jep.eval("import " + getLibName());
if (modified)
{
    jep.eval("import importlib");
    jep.eval("importlib.reload(" + getLibName() + ")");
}

Where jep is a SharedInterpreter and getLibName() returns the plugin module.

When I do this though I get random errors where sys.stdout gets lost:

jep.JepException: <class 'RuntimeError'>: lost sys.stdout

I do have classes in my module that are abstract base classes and use the abc module. I found online that if you have your own module called abc.py then you get this message also.

If I comment out the if (modified) block so it doesn't reload the module and re-run the application, I don't get this error. I did notice that importlib has an abc sub module (importlib.abc) and wondered if the two modules were clashing somehow.

I tried importing the builtin abc as an alias, importing via from abc import ABC, abstractmethod but to no avail.

The biggest issue is that I can't seem to reproduce this in a cut down example to post here. I have tried to replicate as much as possible from the live system (using abc, having an abstract class with a couple of implementations, using shared interpreter and using importlib.reload).

Is there any other way I could "reload" a module when using a shared interpreter?

I am convinced it is something to do with using importlib.reload but I don't know why this works fine in the cut-down example.

Regards,

Mike

[Nathan Jensen]

I haven't used SharedInterpreter but I have been in a similar situation where users can change their Python code on the fly, without restarting the Java process.  I have not seen your particular error before.  You might want to look at this code for some inspiration: https://github.com/Unidata/awips2-core/blob/unidata_18.1.1/common/com.raytheon.uf.common.python/utility/common_static/base/python/MasterInterface.py#L166

--
You received this message because you are subscribed to the Google Groups "Jep Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jep-project...@googlegroups.com.
To post to this group, send email to jep-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jep-project/06c1fd84-d51f-4a0b-86d4-6691d8513d9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike]

Thanks for the pointer.

Before your reply, I did try just doing:

if moduleName in sys.modules:

    del sys.modules[moduleName]

 but after reading the comment at the top of your suggestion, maybe I will try the pop method instead. I never thought about clearing out the module's dictionary either.