Connection reliability / recovery

[Maarten88]

Hi,

We are now doing more testing with our application, trying mobile scenarios, using different browsers. We see several problems / unexpected behaviours in the socketapi connection.

We have programmed our SocketApi connection in javascript as a singleton, it is opened once and should then stay alive while the user is in the single page javascript app. All client/server communication uses the same connection. The user can leave the web app running for hours, or turn the phone/tablet off and back on later, etc. Both the client and the server can initiate messages at any time, in response to user activity or based on time.

What is the recommended way to get this working reliably? Is there a way to ask the client connection its status (in javascript), and reopen it on error? We have tried implementing Heartbeat, however this results in the server closing the connection after a short time. I found no documentation or sample on how to correcty implement this (in javascript).

-- Maarten

[Ulf Björklund]

Hi

The connection will be closed when inactive for x time... Where x is individual per network. This has nothing to do with XSockets.

That is where the heartbeat comes into play (as you mention).

The server-side has a heartbeat mechanism (see http://xsockets.net/docs/4/connection-lifetime-events) that will send pings, and the browser will automatically respond with a pong (according to the websocket protocol RFC6455) and that way the connection will remain open.

However, if the connection fail anyway there is a feature in the JavaScript (and C# client) that can auto-reconnect!

I do not know if the auto-reconnect is affected by your implementation? If you have changed something? But it works fine in our tests.

Sample of auto-reconnect

var ws = new XSockets.WebSocket('ws://localhost:6911', ['generic']);

ws.autoReconnect(true);

ws.onconnected = function () {

    console.log('connected');

};

ws.ondisconnected = function () {

    console.log('disconnected');

}

Then just call ws.disconnect(); from the console... a few seconds later the connection will open back up.

Regards

Uffe

[Maarten88]

That helped a lot. My original code was a little bit different on the server and I did not do the autoReconnect(true), it kept disconnecting. Now reconnect works, I can see that clients are reconnecting and things are much more stable.

More testing to do...

Thanks,

-- Maarten

[Maarten88]

More questions :-)

So I implemented autoReconnect and it helps, but it's not complete. I can now send messages after a reconnect. But I still seem to loose all subscriptions after a short period of inactivity. No matter what I do, the javascript stops receiving events from subscriptions after some time.

Do I have to recreate the subscriptions after a reconnect? I tried moving the connection setup code into onconnected, putting equivalent unsubscribe() calls in ondisconnected to avoid hanging subscriptions. Stopping and starting the server now reconnects the clients. But still, subscriptions in the browser stop working if I do nothing for some time, just keeping the server running. How do I stay subscribed? Do I need to send pings from the clientside?

A sample that implements all lifecycle events correctly and without leaks including reconnecting, subscriptions and invoke calls would help; all sample code and snippets that I found are too simplistic and incomplete.

-- Maarten

[Ulf Björklund]

We should probably have this in a new thread, but anyway :)

When you get disconnected the server will "remove" all subscriptions, so when you get back you have to create new subscriptions. Personally I like RPC better since it is less overhead in the communication and you can instead send messages based on the state of each client (kind of pub/sub with rpc).

An issue with pub/sub is when you want to receive messages directly after the connection is open, but the subscriptions is not yet bound on the server. You will then risk to miss data you send your self with pub/sub.

There is a callback for telling you that the subscription is registered in the server. See http://xsockets.net/docs/4/javascript and the section about "how to know when the subscription is ready"

I would consider using the "PersistenProperty" attribute instead so that state will be stored between connections and can then be used to target clients.

It might be wrong for you since I do not know what you are doing, but I would probably use that, maybe in combination with the IOfflineQueue plugin to not miss anything that was sent while I was away.

A bit of topic though...

/Uffe

[Ulf Björklund]

Another thing...

You could try to do the subscriptions in the OnOpen event on controller level in the client instead of in the OnConnected event on socket level.

Have not tried it, but it should work

[Maarten88]

I tried onopen too; it makes no difference.

In the meantime, I got a little bit further: I'm now convinced there is a bug somewhere in the serverside HeartBeat. I have removed the line 

this.ProtocolInstance.Heartbeat(notifyAction: (reason) => { Trace("Heartbeat: disconnected, reason = " + reason.ToString()); });

that was in my (async) OnOpened and most of the reliability problems went away. Now the controller stays connected as it should.

Somehow the controller does not get a correct reply to its HeartBeat message and disconnects. The client gets no disconnected event and continues, but never gets messages anymore from the controller. The controller does Publish them. That's is what I see.

[Ulf Björklund]

What version are you using?

[Maarten88]

I'm using 5.0 rc-2. 

I do see some (non-fatal?) assembly load warnings during initialization.

[Ulf Björklund]

Hi

I cant see any assembly load warnings that is related to XSockets specifically.

The heartbeat work fine for me, but since you have issues we will be happy to take a look. Can you provide any more information or even a repo where the issue is reproduced.

Regards

Uffe

[Maarten88]

Ok, I have created an isolated repo with code demonstrating the issues we have. Its on github at https://github.com/Maarten88/XSocketsHeartBeatTest

I think there's more than one issue here. If you run this code, you'll see the disconnection that I mentioned. Remove Heartbeat and that no longer happens. But still server initiated messages stop arriving after some time. And if you play some more (add some browser sessions, let it run for some time, refresh a browser, break the connection) it gets worse, I got ObjectDisposedException. Last week we had our servers on Azure becoming unresponsive, I think that's related.

-- Maarten