Python->Java call fails to return after Java process opens other port(s)

[John Collins]

Greetings - I am new to this group, I found Py4J when trying to help a student use Python to build a Power TAC broker. So I wrote a stripped down broker framework in Java that just ships all the incoming messages over to the Python process, and provides a simple API for sending outgoing messages. My goal is to keep the Python side simple, so I don't want to use multi-threading or have Java call Python. The idea is that

1. Python makes a call to log in the broker giving the standard command-line args, which simply starts a thread on the Java side and returns. In that thread, the broker logs in to the server and starts receiving and decoding messages (we're using Apache Active MQ async messaging).
2. Python makes a second call that waits for arrival of a particular message called SimStart that arrives once all the setup messages have arrived. It can then retrieve those messages and start building its models and formulating its initial tariffs and trading strategy.
3. Python then goes into a loop where it waits for the final message in each "timeslot", then pulls in the messages for thet timeslot (there can be several hundred of them in several categories), does it's analysis, formulates outgoing messages, and passes them to Java.

My problem is that step 2 never returns from Java to Python. When I do the same thing in Java, it works fine. I'm using a different monitor object for synchronization in steps 2 and 3.

I've written a simple test case where step 1 starts a thread that delays a bit before sending notifyAll() on the first monitor, then repeatedly delays a couple seconds and calls notifyAll() on the second monitor. That works with both Java and Python clients.

So I'm trying to understand what's different between the simple test case and the real broker framework. One obvious difference is that the real case opens another port and does quite a bit of communication with a server rather than just delaying. So I've checked to see whether the Py4J port gets closed when the broker's server port is opened, and it does not.

At this point I'm out of ideas as to why the second call to the Java process never returns.

Any ideas would be more than welcome. Thanks.

John Collins

[John Collins]

Update: The problem appears to be that the Java Gateway thread does not see variable updates made in another thread. See Issue #436 for details. I have browsed through the code a bit and have not yet found what's causing this. I would be grateful for any suggestions.

John

[John Collins]

Problem solved, mea culpa. I had forgotten that Spring services are not necessarily singletons, and the instance Spring was using was not the same instance my Python code was looking at. So I had to get the Spring services that Python needed to access to register themselves where Python could retrieve them and then wait for Spring to do that before I retrieved them. So now it all appears to work.

John