OSCPy server forcibly closed when using different ports

[Der Achim]

I have two programs talking to each other using OSC. One foreground (on android, that would be the app) and one background (the service).

Foreground listens on port 3000, background on 3001.

With the old OSC, in kivy 1.11 discarded, this worked.

The replacement OSCPy does not go so well. It looks like it was not possible to listen on one port and send something on another port.

Here is an example program that shows this behaviour:

from oscpy.server import OSCThreadServer

def callback(*values):
    print("foreground got values: {}".format(values[0]))

osc = OSCThreadServer(encoding = 'utf8')
sock = osc.listen(address = '0.0.0.0', port = 3000, default = True)
osc.bind(b'/foreground_listening', callback)
# the following line crashes the listening thread:
osc.send_message(b'/background_listening', ["calling background!"], ip_address = '127.0.0.1', port = 3001)

input("press 'enter' ...")
osc.stop()

When run, the listing connection terminates with:

press 'enter' ...
Exception in thread Thread-1:
Traceback (most recent call last):
  File "c:\python37\Lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "c:\python37\Lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "C:\arbeit\programme\TollIna\venv\lib\site-packages\oscpy\server.py", line 335, in _listen
    data, sender = sender_socket.recvfrom(65535)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

May I not send and listen on different ports? (It worked with the old OSC).

[Gabriel Pettier]

Hello, main dev of OSCPy here.

Just tested here on linux/python37

- ran `oscli dump --host localhost --port 3001` in a terminal to receive
the orginal message from your script.

- ran your script, I did get `/background_listening: calling
background!` displayed in the first window, and "press enter..."
displayed in the second one.

- then I ran oscli send to send messages to your example server, to
check that it was working, and it did, as the messages displayed in
its console.

https://asciinema.org/a/9YcJnfMxcTy0pp2WanhzeDUzX

I'm not sure what happens for you, but I really ran your program as-is.
It could come from windows, but i don't see why windows would prevent
that, unless you have a firewall or something preventing the listening.

Does the crash only happen if you do the sending?

I do have tests in CI checking that every features is working on windows
as well as linux and osx, so i'm not sure how to reproduce this if it's
not already visible in these. Do you have any idea of what specificity
of your system could trigger that?

Best regards.

>--
>You received this message because you are subscribed to the Google Groups "Kivy users support" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to kivy-users+...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/0dd0f5da-2080-4d12-8282-1663a9193a4e%40googlegroups.com.

[Der Achim]

Yes, the crash only happens when sending. When I comment the line "osc.send_message(..." everything works as expected.


I disabled and even de-installed my anti-virus programm, the server program still crashes.

I tested this on Win7 and Win10, same behaviour.

Where I find a difference is between Python2 and Python3: When running the test suite, for Python2 everything is fine, for Python3 I get the WinError 10054 (test finishes successful, though...).

Here are the logs of the tests.

For Python2:

(venv2) C:\arbeit\oscpy-fork-heideachim\tests>pytest test_server.py
============================= test session starts =============================
platform win32 -- Python 2.7.15, pytest-4.6.5, py-1.8.0, pluggy-0.12.0
rootdir: C:\arbeit\oscpy-fork-heideachim
collected 34 items

test_server.py ..................................                        [100%]

========================== 34 passed in 0.67 seconds ==========================

For Python3:

(venv) C:\arbeit\oscpy-fork-heideachim\tests>pytest test_server.py
========================================================================================= test session starts ===========================================================
platform win32 -- Python 3.7.2, pytest-5.1.1, py-1.8.0, pluggy-0.12.0
rootdir: C:\arbeit\oscpy-fork-heideachim
collected 34 items

test_server.py .........Exception in thread Thread-10:

Traceback (most recent call last):
  File "c:\python37\Lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "c:\python37\Lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)

  File "c:\arbeit\programme\tollina\venv\lib\site-packages\oscpy\server.py", line 336, in _listen
    data, sender = sender_socket.recvfrom(65535)
ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen
.
....<socket.socket fd=596, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('127.0.0.1', 63312)>
.<socket.socket fd=616, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('127.0.0.1', 63313)>
...................                                                                                                                                                [100%]

==================================================================================== 34 passed in 69.08s (0:01:09) ======================================================

Does this give any hints? Tomorrow I will test on more Win installations.

[Der Achim]

... more testing...

Windows 10, corporate laptop:

Python 2 fails:

(venv2) C:\arbeit\oscpy\oscpy-fork-achim\tests>pytest test_server.py

============================= test session starts =============================

platform win32 -- Python 2.7.16, pytest-4.6.5, py-1.8.0, pluggy-0.12.0
rootdir: C:\arbeit\oscpy\oscpy-fork-achim
collected 34 items

test_server.py ..........Exception in thread Thread-10:

Traceback (most recent call last):

  File "c:\python27\Lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "c:\python27\Lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "c:\arbeit\oscpy\venv2\lib\site-packages\oscpy\server.py", line 335, in _listen
    data, sender = sender_socket.recvfrom(65535)
error: [Errno 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen

........................                        [100%]

========================== 34 passed in 1.13 seconds ==========================

Python 3 fails as well with two error messages from two different threads:

(venv3) C:\arbeit\oscpy\oscpy-fork-achim\tests>pytest test_server.py
================================================= test session starts =================================================

platform win32 -- Python 3.7.2, pytest-5.1.1, py-1.8.0, pluggy-0.12.0

rootdir: C:\arbeit\oscpy\oscpy-fork-achim
collected 34 items

test_server.py .......Exception in thread Thread-8:

Traceback (most recent call last):
  File "c:\python37\Lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "c:\python37\Lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)

  File "c:\arbeit\oscpy\venv3\lib\site-packages\oscpy\server.py", line 335, in _listen
    data, sender = sender_socket.recvfrom(65535)
OSError: [WinError 10038] an operation was attempted on something that is not a socket

..Exception in thread Thread-10:
Traceback (most recent call last):
  File "c:\python37\Lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "c:\python37\Lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)

  File "c:\arbeit\oscpy\venv3\lib\site-packages\oscpy\server.py", line 335, in _listen
    data, sender = sender_socket.recvfrom(65535)

ConnectionResetError: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen

.........................                                                                [100%]

================================================== warnings summary ===================================================
test_server.py:864
  C:\arbeit\oscpy\oscpy-fork-achim\tests\test_server.py:864: DeprecationWarning: invalid escape sequence \(
    match='get_sender\(\) not called from a callback'):

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================================== 34 passed, 1 warnings in 14.97s ===========================================

(I manually made the translation for WinError 10038)

Is there anything more I can do then testing on different Windows installations??

[Gabriel Pettier]

https://github.com/kivy/oscpy/pull/46

Added your example in the test suite (only replaced the "press enter"
input with a sleep(1)), and pushed, all the tests pass, no crash (though
a few tests requires rerun to pass, which is expected as osc is based on
udp, sometime messages can be lost). I'm not sure how to reproduce your
issue.

Which version of oscpy are you using, latest release (0.5.0)? master?

If you use the git version, as it seems, can you checkout my PR's
branch, and see if it crashes when running pytest?

Regards.

On Thu, Aug 22, 2019 at 11:56:01PM -0700, Der Achim wrote:
>... more testing...
>
>Windows 10, corporate laptop:
>
>
>Python 2 fails:
>
>(venv2) C:\arbeit\oscpy\oscpy-fork-achim\tests>pytest test_server.py
>============================= test session starts
>=============================
>platform win32 -- Python 2.7.16, pytest-4.6.5, py-1.8.0, pluggy-0.12.0
>rootdir: C:\arbeit\oscpy\oscpy-fork-achim
>collected 34 items
>
>test_server.py ..........Exception in thread Thread-10:
>Traceback (most recent call last):
> File "c:\python27\Lib\threading.py", line 801, in __bootstrap_inner
> self.run()
> File "c:\python27\Lib\threading.py", line 754, in run
> self.__target(*self.__args, **self.__kwargs)
> File "c:\arbeit\oscpy\venv2\lib\site-packages\oscpy\server.py", line 335,
>in _listen
> data, sender = sender_socket.recvfrom(65535)
>error: [Errno 10054] Eine vorhandene Verbindung wurde vom Remotehost
>geschlossen
>
>........................ [100%]
>
>========================== 34 passed in 1.13 seconds
>==========================
>
>
>
>
>
>

>Python 3 fails as well with *two* error messages from two different threads:

>--
>You received this message because you are subscribed to the Google Groups "Kivy users support" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to kivy-users+...@googlegroups.com.

>To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/dc78f344-cca1-4eb9-a5a8-be03435a2327%40googlegroups.com.

[Der Achim]

I do not think that will work. The exception happens in the separately spawned thread - pytest will not notice that. That is what is already happening in the current test suite, I think, it is in test_send_bundle_without_socket(): This is sending on port 1 which yields the well known error message from excpetion ConnectionResetError. But pytest continues normally and reports all tests as successfull! See my post in https://kivy.org/#?place=msg%2Fkivy-users%2FGDQr9I4DBy0%2F9HIMA1EfBAAJ, all 34 tests where passed successfully, despite the exception.

I will try to create a test that obeys that.

Maybe that is why you do not notice an error on Windows? Because tests pass successfully even though something fails?

[Der Achim]

I have created a test that checks if the server listening thread is still working, even if the server dared to send something on a different port then the one it is listening on. It's in pull request 45, you might want to have a look at that. It's my first pytest, so be patient...

[Gabriel Pettier]

Your test was passing in the CI, but failed when i tried it on windows,
so i was a bit unhappy, the CI is less useful than expected.

After some search (a lot of different thing can apparently cause the
WinError 10054 thing), i got a hint that it could be that windows
disliked having unread messages, so actually adding a server to receive
it as i did the other day would have prevented the error to happen
anyway.

But we need to be robust against such thing, and i don't see a reason
for windows to punish the listening socket when another one sends unread
messages.

Thankfully, it seems we can actually just ignore the
ConnectionResetException, and just go on to the next message waiting to
be read, which is funny because trying to process the first message
would work as well, but we would add the "b" message two times, which
also fails the test.

Anyway, i merged my fix, and all the tests still pass, so everything
should be good for you now.

>--
>You received this message because you are subscribed to the Google Groups "Kivy users support" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to kivy-users+...@googlegroups.com.

>To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/8ff01a12-8500-4ba1-bc4a-2e699e9bc832%40googlegroups.com.

[Der Achim]

Excellent! Thanks for the fix. I tried to get some information about the cause for WinError 10054 as well, but my knowledge of sockets is nil, so I failed miserably...

With your updates to oscpy, the pytests run successfully even on my Windows computer with its austere socket implementation...

Thanks for investigating and fixing this!