Best way to use Jep to run multiple functions concurrently

[GianMaria Romanato]

Hello,

I am planning to use Jep to run multiple functions concurrently.

From the wiki I understood that Jep uses sub interpreters for every new Jep instance, and that such Jep instances must be accessed only by the thread that instantiated them. If I understood correctly, the recommended usage pattern consists in instantiating a Jep object, using it within the current thread, and closing it when done.

In my early tests I was able to invoke functions as follows (please advice if I am doing something wrong, I could not find a complete example in the wiki and in the Java tests);

1. Instantiate Jep passing in the includePaths populated to list the file system folders containing the custom python modules that are imported by the "top level" module where the function to be invoked is declared
2. perform a runScript() passing in the full path to the top level module
3. invoke the function

   Jep jep = new Jep(false, "/home/user/modules");
   jep.runScript("/home/user/modules/hello.py");
   Object result = jep.invoke("hello_jep");
   jep.close();

My application will be a Java Enterprise solution used concurrently by hundreds of users. Each of them will perform some action that will require invoking at least a python function to perform a numerical computation. As adviced in the wiki, I will be invoking python functions, and will not use any global variable to avoid leaks and to keep the Jep instances as much isolated as possible.

Eventually, I will have tenths, possibly hundreds, of different functions and on the live application, hundreds of users will be invoking such functions concurrently.
For example I may have f1() f2() f3() .... f100() functions and u1.... u300 users. In a particular moment in time, there may be users u1..u20 calling f1(),  u21...u70 calling f3(), and remaining users calling a different function each.

In such a scenario, the sample code I wrote above would make Jep read "hello.py" for every function invocation. This would result in a lot of disk I/O and in general a potential waste of CPU time.
Is there a way to globally preload all the python files and then re-use them in a thread-safe way?
Alternatively, would you recommend to pool Jep instances (one for each python function)?

GianMaria.

[Nathan Jensen]

Hi,

The wiki's pretty new and not far past a rough draft.  I'll use some of your questions to try and make it more clear and helpful.  I didn't intend for it to recommend one approach over another, just to explain how it works.

Your example code is fine, there's a few different ways to do it, I don't necessarily think one way is better than other, it depends on your style and what you're trying to accomplish.  An alternative, neither better nor worse, to your example code could be something like:

Jep jep = new Jep(false, "/home/user/modules");

jep.eval("import hello");

jep.eval("result = hello.hello_jep()");

Object result = jep.getValue("result");

jep.eval("del result");

If you're aiming for high performance or long-running applications (such as a server application), then yeah, I highly recommend you reuse and pool the Jep instances.  On the application I work on, we have a factory that produces and initializes the Jep instances so they're ready to go.  Then we just reuse them indefinitely and limit the pool size(s) so one component doesn't overwhelm the rest of the system.

--
You received this message because you are subscribed to the Google Groups "Jep Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jep-project...@googlegroups.com.
To post to this group, send email to jep-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jep-project/1a7c256e-b0a1-461d-bdfe-aadb85fbb85d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gurmeet.budhraja]

Can you provide an example how to pass data from Java to Py and from Py to Java back using Jep ?

[Guri]

Can you provide an example how to pass data from Java to Py and from Py to Java back using Jep ? I am actually trying to call python method using java, and return result from python to java, using the sample, i could return value from python, but not sure how to pass values to python method.

[Nathan Jensen]

Hi,

There's a few ways to do this.  If you are using the method jep.invoke(), you can pass arguments to invoke, such as

jep.invoke(methodName, arg0, arg1, arg2);

You can also do it another way with setValue(), eval(), and getValue(), such as

jep.setValue("arg0", arg0);

jep.eval("result = " + methodName + "(arg0)");

Object result = jep.getValue("result");

Also see this page: https://github.com/mrj0/jep/wiki/FAQ#whats-the-typical-usage-of-jep-code-look-like

Hope that helps.

--
You received this message because you are subscribed to the Google Groups "Jep Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jep-project...@googlegroups.com.
To post to this group, send email to jep-p...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jep-project/9e03301b-bc75-4fb1-8618-c9734745e7da%40googlegroups.com.