Exception while shutting down a socket

[Prateek Dorwal]

Hi,

I have an application in which I am using ClientServer "pinned" method for Java Python communication(Python being called from Java).

Call is initiated from UI thread, which executes both Java and Python processes in the same UI thread.

Everything runs fine if there is small number of requests, but If the number of requests per second is huge, my process break with the below exception.

And thereafter, the process stops receiving any incoming requests (Until a server restart).

I need help from experts here to understand what is going wrong here, and how to resolve this issue.

2017-12-27 14:19:40,154 - INFO - Error while python server was waiting fora message

Traceback (most recent call last):

  File "/home/anaconda2/lib/python2.7/site-packages/py4j/clientserver.py", line 510, in wait_for_commands

    proto.ERROR_RETURN_MESSAGE.encode("utf-8"))

  File "/home/anaconda2/lib/python2.7/socket.py", line 228, in meth

    return getattr(self._sock,name)(*args)

error: [Errno 32] Broken pipe

2017-12-27 14:19:40,154 - INFO - Closing down clientserver connection

2017-12-27 14:19:40,154 - DEBUG - Exception while shutting down a socket

Traceback (most recent call last):

  File "/home/anaconda2/lib/python2.7/site-packages/py4j/java_gateway.py", line 460, in quiet_shutdown

    socket_instance.shutdown(socket.SHUT_RDWR)

  File "/home/anaconda2/lib/python2.7/socket.py", line 228, in meth

    return getattr(self._sock,name)(*args)

error: [Errno 107] Transport endpoint is not connected

Regards

Prateek Dorwal

[Prateek Dorwal]

Some more information on the problem:

When everything executes properly, I see on below logs

Received command c on object id t

Closing down clientserver connection

But in erroneous execution, I get something weird (Ignore the change in logging):

INFO - Received command sMyFirstInputParameter on object id sMyOtherParameters

ERROR - Unknown command sMyFirstInputParameter

INFO - Error while python server was waiting fora message

So command c translated to some other string. No idea how.

[Barthelemy Dagenais]

Hi, it will be difficult to help you without seeing some code that
reproduces the error. This particular error is generic and occurs when
the python side receives something it does not understand. It then
tries to shut down the connection to cleanly stops the program (the
protocol is broken at this point).

In your example, it is as if a string was sent as a command, which
makes little sense unless you are directly writing on the socket.

Is it possible that the connection is shared by multiple threads (py4j
uses thread locals to prevent such cases so this should not happen) or
that you are using a clientserver on the python side and a
gatewayserver on the Java side?

Another possibility: are you creating and destroying client servers
along the way or are you always using the same clientserver instance
on both sides?

> --
> You received this message because you are subscribed to the Google Groups
> "Py4J Support and Comments" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to py4j+uns...@py4j.org.
> To post to this group, send email to py...@py4j.org.
> To view this discussion on the web visit
> https://groups.google.com/a/py4j.org/d/msgid/py4j/9b09489e-7038-4db4-a425-c7fe25cbe3c0%40py4j.org.

[Prateek Dorwal]

Hello Barthelemy,

Below is code snippet of my instance creation on Python and Java sides:

Java:

------

clientServer = new ClientServer(25316,

           GatewayServer.defaultAddress(),

           25336,

           GatewayServer.defaultAddress(),

           GatewayServer.DEFAULT_CONNECT_TIMEOUT,

           GatewayServer.DEFAULT_READ_TIMEOUT,

           ServerSocketFactory.getDefault(),

           SocketFactory.getDefault(),

           new Class[]{IRecommendation.class});

   

   

   

Python:

-------

recommendation = Recommendation()

gateway = ClientServer(

    java_parameters=JavaParameters(),

    python_parameters=PythonParameters(port = 25336),

    python_server_entry_point = recommendation )

recommendation.clientserver = gateway

I am destroying the client server on Java side after the process is complete in finally block with the below snippet:

finally {

if (null != clientServer)

clientServer.shutdown();

}

[Barthelemy Dagenais]

Hi,

where is the error occurring? Can you create a small example
reproducing the error?

Thanks,