Error enabling Websockets with CometD 3.0.4 and DW 0.8.

[Dan]

Hi,

I'm new to CometD and needed some help with websockets. I enabled websocket on our js client and the browser successfully sends a Connection: Upgrade request but the response is 400 Unknown Bayeux Transport and then switches to long-polling.

The transports option in the cometD servlet is org.cometd.websocket.server.JettyWebSocketTransport followedby JSONTransport and JSONPTransport. I tried to debug where this error comes from and it turns out myServlet implements CometDServlet which calls findHttpTransport of BayeusxServerImpl.java. The 400 comes from CometDServlet since the findHttpTransport returns null. This is how it looks:

AbstractHttpTransport transport = _bayeux.findHttpTransport(request);
if (transport == null)
{
    response.sendError(HttpServletResponse.SC_BAD_REQUEST, "Unknown Bayeux Transport");
}

protected AbstractHttpTransport findHttpTransport(HttpServletRequest request)
{
    // Avoid allocation of the Iterator
    for (int i = 0; i < _allowedTransports.size(); ++i)
    {
        String transportName = _allowedTransports.get(i);
        ServerTransport serverTransport = getTransport(transportName);
        if (serverTransport instanceof AbstractHttpTransport)
        {
            AbstractHttpTransport transport = (AbstractHttpTransport)serverTransport;
            if (transport.accept(request))
                return transport;
        }
    }
    return null;
}

In case of a WS request The serverTransport is not an instance of AbstractHttpTransport and so this ends up returning null, which results in the 400 from CometDServlet. Which makes sense since JettyWebSocketTransport extends AbstractWebSocketTransport and not AbstractHttpTransport (Or AbstractHTTPStreamTransport as in case of JSONTransport where I do not see this problem)

It looks like with this implementation a WS call will always be returned a 400. Am I missing something here? Any help will be appreciated :)

The CometD version is 3.0.4, DW 0.8.1 and Jetty version is 9.2.9.v20150224

[Simone Bordet]

Hi,

On Sun, Jul 26, 2015 at 6:35 PM, Dan <sujay.bi...@gmail.com> wrote:
> It looks like with this implementation a WS call will always be returned a
> 400. Am I missing something here? Any help will be appreciated :)

A WebSocket transport never reaches the CometDServlet.

In Jetty, it is intercepted by the WebSocketUpgradeFilter, which will
handle the WebSocket upgrade.

> The CometD version is 3.0.4, DW 0.8.1 and Jetty version is 9.2.9.v20150224

Did you enable the "websocket" module in Jetty ?
Are you using Jetty standalone server, or Jetty embedded ?

--
Simone Bordet
----
http://cometd.org
http://webtide.com
Developer advice, training, services and support
from the Jetty & CometD experts.

[Dan]

Thanks Simone - I am using dropwizard so it would be Embedded Jetty. 

In my pom.xml I have included jetty websocket dependencies - 

<dependency>
    <groupId>org.eclipse.jetty.websocket</groupId>
    <artifactId>javax-websocket-server-impl</artifactId>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty.websocket</groupId>
    <artifactId>websocket-server</artifactId>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty.websocket</groupId>
    <artifactId>websocket-api</artifactId>
</dependency>

Is this enough to enable websockets? Can you please point me to some examples on what it means to enable websocket module for DW/embedded Jetty.

Thanks!

[Dan]

Hi Simone,

Besides the import I even tried explicitly adding the WebSocketUpgradeFilter - but I still see the same 400 Unknown Bayeux Transport from the CometDServlet -

environment.servlets().addFilter("wsfilter", new WebSocketUpgradeFilter());

Is there something else I need to do to make the request be intercepted by Jetty and not go to the CometDServlet.

I also tried looking at posts with people facing similar exceptions but most of them used the JSR 356 WebSocketTransport instead of Jetty specific one.

Would you have any ideas on what I could be missing/doing wrong and where I could look for similar usage examples.

Thanks for the help!

[Simone Bordet]

Hi,

On Mon, Jul 27, 2015 at 4:51 PM, Dan <sujay.bi...@gmail.com> wrote:
> Hi Simone,
>
> Besides the import I even tried explicitly adding the WebSocketUpgradeFilter
> - but I still see the same 400 Unknown Bayeux Transport from the
> CometDServlet -
>
> environment.servlets().addFilter("wsfilter", new WebSocketUpgradeFilter());
>
>
> Is there something else I need to do to make the request be intercepted by
> Jetty and not go to the CometDServlet.
>
> I also tried looking at posts with people facing similar exceptions but most
> of them used the JSR 356 WebSocketTransport instead of Jetty specific one.
>
>
> Would you have any ideas on what I could be missing/doing wrong and where I
> could look for similar usage examples.

I'm not a DW expert, but what you have to do is to call:

WebSocketServerContainerInitializer.configureContext(environment.getApplicationContext())

I don't know enough of DW to tell you where to put this call, but when
AbstractServerFactory.createAppServlet() is called should be about
right.

Let us know if it worked and how you did it.

Thanks !

[Dan]

Thanks Simone - I've made some progress on this. This is what worked for me- 

WebSocketUpgradeFilter.configureContext(environment.getApplicationContext()); 

However I'm seeing something weird now. A websocket handshake (with Connection: Upgrade) reaches this filter and in chrome I see the response as 101 - switching protocols which is great, but the Response 'Connection' header is keep-alive and the js debug console shows this error:

WebSocket connection to 'wss://mywebsite.dev/cometd/' failed: Error during WebSocket handshake: 'Connection' header value must contain 'Upgrade'

After this the communication switches back to long polling. There is nginx between the client and my web server, but I did enable ws in its config. 

Any ideas on whether the websocket connection is failing because some proxy (maybe nginx?) is modifying the response "Connection" header, or something else like maybe the bayeux handshake failing. 

Do I need any modifications in my js client code besides wsenabled = true? I'm also assuming no changes are required in cometD server code.

The 101 response also takes almost a minute to complete. Is that normal?

btw this is how I got nginx to stop fiddling with request headers incase someone might find it useful:

        proxy_set_header Upgrade $http_upgrade;

        proxy_set_header Connection "upgrade";

        proxy_set_header X-Forwarded-For   $http_x_forwarded_for;

        proxy_set_header Host              $http_host;

        proxy_pass http://mywebsite;

Thanks!

[Simone Bordet]

Hi,

On Wed, Jul 29, 2015 at 11:24 AM, Dan <sujay.bi...@gmail.com> wrote:
> Thanks Simone - I've made some progress on this. This is what worked for me-
> WebSocketUpgradeFilter.configureContext(environment.getApplicationContext());

Great.

> However I'm seeing something weird now. A websocket handshake (with
> Connection: Upgrade) reaches this filter and in chrome I see the response as
> 101 - switching protocols which is great, but the Response 'Connection'
> header is keep-alive and the js debug console shows this error:
>
> WebSocket connection to 'wss://mywebsite.dev/cometd/' failed: Error during
> WebSocket handshake: 'Connection' header value must contain 'Upgrade'
>
> After this the communication switches back to long polling. There is nginx
> between the client and my web server, but I did enable ws in its config.
>
> Any ideas on whether the websocket connection is failing because some proxy
> (maybe nginx?) is modifying the response "Connection" header, or something
> else like maybe the bayeux handshake failing.
> Do I need any modifications in my js client code besides wsenabled = true?
> I'm also assuming no changes are required in cometD server code.
> The 101 response also takes almost a minute to complete. Is that normal?
>
> btw this is how I got nginx to stop fiddling with request headers incase
> someone might find it useful:
>
> proxy_set_header Upgrade $http_upgrade;
>
> proxy_set_header Connection "upgrade";

Try to set here upper case "U": Upgrade

> proxy_set_header X-Forwarded-For $http_x_forwarded_for;
>
> proxy_set_header Host $http_host;
>
> proxy_pass http://mywebsite;

Why do you have nginx ? What does it provide for you ?

[Dan]

Even after changing to upper case I see the same error in debug logs

WebSocket connection to 'wss://mywebsite.dev/cometd/' failed: Error during WebSocket handshake: 'Connection' header value must contain 'Upgrade'

The response and request headers look like this. Do you think the Connection: keep-alive could even be causing this problem or is this something on the client side/handshake?  

1. Request URL:
   wss://mywebsite.dev/cometd/
2. Request Method:
   GET
3. Status Code:
   101 Switching Protocols
1. Response Headersview source
1. Connection:
   keep-alive
2. Date:
   Wed, 29 Jul 2015 09:49:14 GMT
3. Sec-WebSocket-Accept:
   ******
4. Server:
   nginx
5. Transfer-Encoding:
   chunked
6. Upgrade:
   WebSocket

1. Request Headersview source
1. Accept-Encoding:
   gzip, deflate, sdch
2. Accept-Language:
   en-US,en;q=0.8
3. Cache-Control:
   no-cache
4. Connection:
   Upgrade
5. Cookie:
6. Host:
7. Origin:
   https://www.mywebsite.dev
8. Pragma:
   no-cache
9. Sec-WebSocket-Extensions:
   permessage-deflate; client_max_window_bits
10. Sec-WebSocket-Key:
11. Sec-WebSocket-Version:
    13
12. Upgrade:
    websocket
13. User-Agent:
    Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36

Any suggestions on what else I could look at to debug this..?

Thanks!

[Simone Bordet]

Hi,

Remove nginx. I suspect it removes the correct headers.
Does it work without nginx ?

[Dan]

Hi Simone,

This worked for me after upgrading nginx to a version after 1.3.13 - thanks for your help!

I wanted to check if there is a standard way to keep a count of number of active websocket connections at any given time? What other metrics could be looked at to see that the transition to websockets is fine, and how many clients if any are falling back to long polling. 

For example if we transition from long-polling to websockets and we had on average 100 long polling sessions at any time, we want to be able to see how many of those are now websocket connections and how many fell back to long-polling instead. 

Are there any example usages I could look at?

Thanks!

[Simone Bordet]

Hi,

CometD does not keep these statistics yet, but it's a good enhancement.
Can you please file an issue to http://bugs.cometd.org ?

Thanks !

[Dan]

Hi Simone,

I'm trying to understand and looking for ways to tune the time it takes for cometD to fall back to long-polling when a websocket handshake fails. 

We might have a lot of instances where the websocket handshake fails initially and it might have to fallback to long-polling. In this specific case we would like to not wait and try a long-polling handshake immediately. This can be easily done by reducing the backoffIncrement parameter inside cometd.configure() to a low value. However this would also lead to failed long-polling handshakes to start retrying very fast as well, which could lead to traffic on the server if something is actually going wrong. (And I would want a larger backoffIncrement time)

So what I want is this - being able to set different retry times for the two cases (fallback trying vs retry on failing). Is that possible to do without changing the client code in cometd.js. If that needs to be done what would be the best approach to this? I'm using the advice where reconnect value is 'retry' and interval is 0, I'm using the js client library and java server library.

Thanks!

[Simone Bordet]

Hi,

On Fri, Aug 21, 2015 at 2:06 AM, Dan <sujay.bi...@gmail.com> wrote:
> Hi Simone,
>

> I'm trying to understand and looking for ways to tune the time it takes for
> cometD to fall back to long-polling when a websocket handshake fails.
>
> We might have a lot of instances where the websocket handshake fails
> initially and it might have to fallback to long-polling. In this specific
> case we would like to not wait and try a long-polling handshake immediately.

This is what CometD already does.

> This can be easily done by reducing the backoffIncrement parameter inside
> cometd.configure() to a low value. However this would also lead to failed
> long-polling handshakes to start retrying very fast as well, which could
> lead to traffic on the server if something is actually going wrong. (And I
> would want a larger backoffIncrement time)

I'm not sure I understand.

You typically want to stay connected to the server.
If the long poll fails, why you want to wait more than necessary ?

[jayamali jayawardana]

Hi,

I am using a embedded Jetty server.I have used cometD 3.0.6 and the lates jetty server dependencies.When the server is started it gives "Missing container error".If i add @endPoint  as below ,

@ServerEndpoint("/cometd")
@WebServlet(name = "CometdServlet", urlPatterns = {"/cometd/*"}, loadOnStartup = 2, asyncSupported = true ,
        initParams = {
                @WebInitParam(name = "ws.cometdURLMapping", value = "/cometd/*"),
                @WebInitParam(name = "transports", value = "org.cometd.websocket.server.WebSocketTransport," +
                        "org.cometd.server.transport.JSONTransport,org.cometd.server.transport.JSONPTransport")
        })
//@WebFilter(filterName = "cross-origin",value="org.eclipse.jetty.servlets.CrossOriginFilter",asyncSupported =true,urlPatterns ={"/cometd/*"} )
public class ZCometdServlet extends CometDServlet implements BayeuxServer.SubscriptionListener

that error goes away.

But handshake fails.

How to resolve this problem.

[Simone Bordet]

Hi,

On Wed, Nov 18, 2015 at 4:03 AM, jayamali jayawardana
<jayamal...@gmail.com> wrote:
> Hi,
> I am using a embedded Jetty server.I have used cometD 3.0.6 and the lates
> jetty server dependencies.When the server is started it gives "Missing
> container error".If i add @endPoint as below ,
>
> @ServerEndpoint("/cometd")
> @WebServlet(name = "CometdServlet", urlPatterns = {"/cometd/*"},
> loadOnStartup = 2, asyncSupported = true ,
> initParams = {
> @WebInitParam(name = "ws.cometdURLMapping", value =
> "/cometd/*"),
> @WebInitParam(name = "transports", value =
> "org.cometd.websocket.server.WebSocketTransport," +
>
> "org.cometd.server.transport.JSONTransport,org.cometd.server.transport.JSONPTransport")
> })
> //@WebFilter(filterName =
> "cross-origin",value="org.eclipse.jetty.servlets.CrossOriginFilter",asyncSupported
> =true,urlPatterns ={"/cometd/*"} )
> public class ZCometdServlet extends CometDServlet implements
> BayeuxServer.SubscriptionListener
>
>
> that error goes away.
>
> But handshake fails.
>
> How to resolve this problem.

Can you please detail the error ?

"Missing container error" is not something that CometD produces.

[jayamali jayawardana]

Hi,

I got "Missing WebSocket ServerContainer" when i used WebSocket Transport  with embedded jetty server.Then i added JettyWebSocketTransport instead of WebSocketTransport and added WebFilterUpgrade.Now it works fine.

Thanks

[Simone Bordet]

Hi,

On Wed, Nov 18, 2015 at 9:39 AM, jayamali jayawardana
<jayamal...@gmail.com> wrote:
> Hi,

> I got "Missing WebSocket ServerContainer" when i used WebSocket Transport
> with embedded jetty server.Then i added JettyWebSocketTransport instead of
> WebSocketTransport and added WebFilterUpgrade.Now it works fine.

Glad you were able to solve the problem.

Do you mind to post the code so others can benefit ?
We could include it in the documentation.

Thanks !