Autobahn/WAMP/Python: No notification of lost connections

[Nick B]

Hello, I am developing a simple WAMP application in Python using Autobahn which involves devices connecting to and communicating through a remote Crossbar router.

For the most part all of the basic features are working very well (connecting, PubSub, RPC, requesting info from the router), however, the application does not react in any way to losing the connection to the router. onLeave and onDisconnect are not called, and there isn't a single debug message printed to the console indicating a connection failure. The Crossbar router sees that the device has disconnected and notifies other devices subscribed to 'wamp.session.on_leave', however the application just hangs until I close it (Ctrl+c), at which point onLeave and onDisconnect are called and I see the debug messages in the console that I expected to see at the time of the actual connection loss. I am testing this with the most basic sample code I could find in the Autobahn Python documentation, and pulling out the network cable from the device to cause the disconnection.

from autobahn.twisted.wamp import ApplicationSession, ApplicationRunner

class MyComponent(ApplicationSession):

   def __init__(self, config = None):
      ApplicationSession.__init__(self, config)
      print("component created")

   def onConnect(self):
      print("transport connected")
      self.join(self.config.realm)

   def onChallenge(self, challenge):
      print("authentication challenge received")

   def onJoin(self, details):
      print("session joined")

   def onLeave(self, details):
      print("session left")

   def onDisconnect(self):
      print("transport disconnected")

if __name__ == '__main__':
   runner = ApplicationRunner(url = u"ws://address_of_wamp_router", realm = u"realm1")
   runner.run(MyComponent)

I did some searching and found a couple examples of setting up ReconnectingClientFactory to automatically reattempt the connection, but this unfortunately didn't work for my use case either. The best I was able to get was the application would automatically reattempt to connect when the initial connection failed, but not when an established connection had been lost.

I found this topic this topic from a few months ago from someone with this exact same problem, but unfortunately there are no replies. I have tried the code in this link, along with many modifications, but I have not been able to make it work.

If anyone could help me figure out where I am going wrong or point me to a Python WAMP code example that demonstrates either connection failure handling or automatic reconnection I would be very appreciative.

Thanks,
-Nick

[Alexander Gödde]

Hi Nick!

Trying your code and shutting down the router, both "onLeave" and "onDisconnect" are called.

Since WebSocket connections are persistent, and no data is sent when there's no need, without actual traffic it's not discovered that the underlying connection has been killed (which is what happens when you simply pull the cable) - at least not quickly.

To enable reacting quickly and deterministically to the disconnect, you can enable WebSocket auto pings - see http://crossbar.io/docs/WebSocket-Options/. When a ping is not received by within the configurable timeout, the connection is closed and you get your events called.

Automatic reconnection is not yet implemented, but currently in the works.

Hope this helps you!

Regards, 

Alex

[Nick B]

Hi Alex,

Thank you for the reply! Why is it that with my current configuration the router knows almost immediately when a device has been disconnected (via physical disconnection of the network cable), but the device itself remains unaware that anything has changed. Is there a ping/timeout configuration that needs to be set in Autobahn on the device side?

Good to know that automatic reconnection is in the works, in the meantime I can implement my own simple method for reconnection if I can get connection failures to be reliably detected.

Thanks,
-Nick

[Tobias Oberstein]

Just quickly (i am on mobile): because you need to activate ws auto ping pong in the client as well so it detects broken connection quickly

Sent from Mobile (Google Nexus 5)

--
You received this message because you are subscribed to the Google Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autobahnws+...@googlegroups.com.
To post to this group, send email to autob...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/autobahnws/90290fe4-7cab-4261-b230-269ab0ae13c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Elvis Stansvik]

On Friday, July 3, 2015 at 6:18:37 PM UTC+2, Nick B wrote:

Hi Alex,

Thank you for the reply! Why is it that with my current configuration the router knows almost immediately when a device has been disconnected (via physical disconnection of the network cable), but the device itself remains unaware that anything has changed. Is there a ping/timeout configuration that needs to be set in Autobahn on the device side?

Tobias already hinted in his reply that this is probably the case. But I'm curious: In the current configuration you speak of above, did you have WebSocket ping-pong configured on the Crossbar node? If you did not, then I'd also be puzzled as to how the router could see the disconnect so quickly, while the client could not.

Elvis

[Elvis Stansvik]

On Friday, July 3, 2015 at 7:43:39 PM UTC+2, Elvis Stansvik wrote:

On Friday, July 3, 2015 at 6:18:37 PM UTC+2, Nick B wrote:

Hi Alex,

Thank you for the reply! Why is it that with my current configuration the router knows almost immediately when a device has been disconnected (via physical disconnection of the network cable), but the device itself remains unaware that anything has changed. Is there a ping/timeout configuration that needs to be set in Autobahn on the device side?

Tobias already hinted in his reply that this is probably the case. But I'm curious: In the current configuration you speak of above, did you have WebSocket ping-pong configured on the Crossbar node? If you did not, then I'd also be puzzled as to how the router could see the disconnect so quickly, while the client could not.

Also, as a follow-up question, I'm curious: Is there really no way for Autobahn to be notified about the underlying TCP socket being disconnected without having ping-pong configured? What if I want to be able to react more quickly than a sane ping-pong timeout value?

I mean, somewhere deeper in the stack, some select() or epoll() call or something must be failing, which it should be able to detect? (sorry, my network programming knowledge is a little rusty).

Elvis

[Nick B]

Thanks for the help everyone.

I did not set up the Crossbar router that my client application is connecting to, and I don't have access to check the configuration at the moment, but it seems very likely that it was configured for websocket ping-pong whereas the client was not. This would explain the difference in connection awareness I see between the router and the client.

I will try this again next week with websocket ping-pong enabled in the protocol options of my application and report my results.

Thanks,
-Nick

[Nick B]

I found the autoPingTimeout and autoPingInterval options that can be set via setProtocolOptions on the client side, however, if I understand correctly this would initiate a second ping-pong round from the client to the router so that the client can close the connection if it doesn't receive a pong. To keep the overall network traffic down, I was hoping to use the ping-pong cycle initiated by the router to provide connection status for both the router and client.

I couldn't find an existing option for this, though using the autoPingTimeout as an example it was fairly straightforward to add another timeout in the WebSocketProtocol class that is reset whenever a ping is received. I can use this timeout to close the connection along with the ReconnectingClientFactory to automatically reconnect when something goes wrong. I'm not sure if there is a better or easier way to accomplish this, but it seems to be working very reliably now (at least for my crude cable yanking network failure test).

Thanks again for the help,
-Nick

[Elvis Stansvik]

On Tuesday, July 7, 2015 at 5:01:52 AM UTC+2, Nick B wrote:

I found the autoPingTimeout and autoPingInterval options that can be set via setProtocolOptions on the client side, however, if I understand correctly this would initiate a second ping-pong round from the client to the router so that the client can close the connection if it doesn't receive a pong. To keep the overall network traffic down, I was hoping to use the ping-pong cycle initiated by the router to provide connection status for both the router and client.

I couldn't find an existing option for this, though using the autoPingTimeout as an example it was fairly straightforward to add another timeout in the WebSocketProtocol class that is reset whenever a ping is received. I can use this timeout to close the connection along with the ReconnectingClientFactory to automatically reconnect when something goes wrong. I'm not sure if there is a better or easier way to accomplish this, but it seems to be working very reliably now (at least for my crude cable yanking network failure test).

Glad you found something that could work, but this is why I would still be interested in an answer to my follow-up question: I find it odd that just detecting a network disconnect should require a WebSocket-level ping/pong mechanism. I would think that the underlying socket layer should provide some way for Autobahn / Crossbar to detect such a disconnect (e.g. cable yanking) almost immediately, without the need for all the heartbeat traffic.

Elvis

[Tobias Oberstein]

This is how TCP works.

If you plug a cable, from an endpoint this is no different than a very, very slow connection.

These things are not specific to WebSocket, WAMP, Autobahn or Crossbar.io, but simply TCP.

[Elvis Stansvik]

2015-07-09 13:31 GMT+02:00 Tobias Oberstein <tobias.o...@gmail.com>:

This is how TCP works.

If you plug a cable, from an endpoint this is no different than a very, very slow connection.

These things are not specific to WebSocket, WAMP, Autobahn or Crossbar.io, but simply TCP.

You're right of course. *brainfart*.

Elvis
 

Am Freitag, 3. Juli 2015 19:50:46 UTC+2 schrieb Elvis Stansvik:

On Friday, July 3, 2015 at 7:43:39 PM UTC+2, Elvis Stansvik wrote:

On Friday, July 3, 2015 at 6:18:37 PM UTC+2, Nick B wrote:

Hi Alex,

Thank you for the reply! Why is it that with my current configuration the router knows almost immediately when a device has been disconnected (via physical disconnection of the network cable), but the device itself remains unaware that anything has changed. Is there a ping/timeout configuration that needs to be set in Autobahn on the device side?

Tobias already hinted in his reply that this is probably the case. But I'm curious: In the current configuration you speak of above, did you have WebSocket ping-pong configured on the Crossbar node? If you did not, then I'd also be puzzled as to how the router could see the disconnect so quickly, while the client could not.

Also, as a follow-up question, I'm curious: Is there really no way for Autobahn to be notified about the underlying TCP socket being disconnected without having ping-pong configured? What if I want to be able to react more quickly than a sane ping-pong timeout value?

I mean, somewhere deeper in the stack, some select() or epoll() call or something must be failing, which it should be able to detect? (sorry, my network programming knowledge is a little rusty).

Elvis

 

Elvis

Good to know that automatic reconnection is in the works, in the meantime I can implement my own simple method for reconnection if I can get connection failures to be reliably detected.

Thanks,
-Nick

--

You received this message because you are subscribed to the Google Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autobahnws+...@googlegroups.com.
To post to this group, send email to autob...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/autobahnws/205cbb5f-ecd6-48f6-8c2a-14d8450f91cc%40googlegroups.com.

[Priyank Kumar]

Is there a sample code on how to configure these Timeout value, are they in ms or seconds?

Crossbar listed it as ms, but there is nothing mentioned in the websocket. Kindly help.

• autoPingInterval (float or None) – Automatically send WebSocket pings every given seconds. When the peer does not respond in autoPingTimeout, drop the connection. Set to 0 to disable. (default: 0).
• autoPingTimeout (float or None) – Wait this many seconds for the peer to respond to automatically sent pings. If the peer does not respond in time, drop the connection. Set to 0 to disable. (default: 0).

transport_factory = MyClientFactory(session_factory,url = mainConfig.url, debug=True, debug_wamp=True )
# Now set the ping parameters here, factory has the protcol name.
transport_factory.autoPingInterval =5
transport_factory.autoPingTimeout =2

[Alexander Gödde]

Hi!

The values set the interval/timout in ms in both Crossbar.io and Autobahn|Python.

Regards,

Alex

[Tobias Oberstein]

Am 26.01.2016 um 17:49 schrieb Priyank Kumar:
> Is there a sample code on how to configure these Timeout value, are they
> in ms or seconds?
> Crossbar listed it as ms, but there is nothing mentioned in the
> websocket. Kindly help.

What "websocket"?

>
> * autoPingInterval (/float or None/) – Automatically send WebSocket

> pings every given seconds. When the peer does not respond in

When you read "pings every given seconds", what would you expect?
Seconds or ms? ;)

> autoPingTimeout, drop the connection. Set to 0 to disable. (default: 0).

> * autoPingTimeout (/float or None/) – Wait this many seconds for the

> peer to respond to automatically sent pings. If the peer does not
> respond in time, drop the connection. Set to 0 to disable. (default: 0).
>
> transport_factory = MyClientFactory(session_factory,url = mainConfig.url, debug=True, debug_wamp=True )
> # Now set the ping parameters here, factory has the protcol name.
> transport_factory.autoPingInterval =5
> transport_factory.autoPingTimeout =2

This code can break anytime, as we can (and likely will) rename those
private attributes.

See setProtocolOptions

http://autobahn.ws/python/reference/autobahn.websocket.html#autobahn.websocket.protocol.WebSocketServerFactory.setProtocolOptions

The values are seconds in AutobahnPython as documented in above method.

>
>
>
>
>
> On Thursday, July 9, 2015 at 7:29:58 AM UTC-7, Elvis Stansvik wrote:
>
> 2015-07-09 13:31 GMT+02:00 Tobias Oberstein <tobias.o...@gmail.com

> <javascript:>>:

> it, send an email to autobahnws+...@googlegroups.com <javascript:>.

> To post to this group, send email to autob...@googlegroups.com

> <javascript:>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autobahnws/205cbb5f-ecd6-48f6-8c2a-14d8450f91cc%40googlegroups.com

> <https://groups.google.com/d/msgid/autobahnws/205cbb5f-ecd6-48f6-8c2a-14d8450f91cc%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com

> <mailto:autobahnws+...@googlegroups.com>.

> To post to this group, send email to autob...@googlegroups.com

> <mailto:autob...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autobahnws/e26c0de6-3235-47d8-9a63-0dbbd4a3be47%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/e26c0de6-3235-47d8-9a63-0dbbd4a3be47%40googlegroups.com?utm_medium=email&utm_source=footer>.