3.1.0-RC1 test drive questions and issues

[steven winner]

I've been trying out the RC1 for CometD 3.1.0 for the past few days, and for the most part, everything's working pretty well.  Few things I've seen:

the JS library's debug function (cometd.js:200) is throwing errors (TypeError: Cannot read property '_debug' of null) 

under still as of yet unclear situations (we use sentry https://sentry.io/welcome/ for JS error reporting, and it's showing up a decent amount, so I had to comment it out)

example error:

https://sentry.io/share/issue/37393538312e313838393731323931/

Also, been seeing these jetty websocket warnings (9.3.14.v20161028):

00:21:01,125 12-05-2016 [WARN ] org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension::failed:418()

org.eclipse.jetty.io.EofException: null

        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199) ~[rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.flush(SslConnection.java:792) ~[rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.shutdownOutput(SslConnection.java:932) ~[rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.close(SslConnection.java:972) ~[rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.flush(SslConnection.java:894) ~[rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:140) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.io.FrameFlusher$Flusher.flush(FrameFlusher.java:153) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.io.FrameFlusher$Flusher.process(FrameFlusher.java:217) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.io.FrameFlusher.enqueue(FrameFlusher.java:382) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.outgoingFrame(AbstractWebSocketConnection.java:614) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextOutgoingFrame(AbstractExtension.java:182) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.access$1000(CompressExtension.java:42) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension$Flusher.compress(CompressExtension.java:550) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension$Flusher.deflate(CompressExtension.java:451) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension$Flusher.process(CompressExtension.java:431) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.outgoingFrame(CompressExtension.java:218) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.ExtensionStack$Flusher.process(ExtensionStack.java:419) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.outgoingFrame(ExtensionStack.java:296) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.uncheckedSendFrame(WebSocketRemoteEndpoint.java:310) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.jsr356.JsrAsyncRemote.sendText(JsrAsyncRemote.java:189) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.WebSocketTransport.send(WebSocketTransport.java:110) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.WebSocketTransport.send(WebSocketTransport.java:49) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.common.AbstractWebSocketTransport$AbstractWebSocketScheduler.send(AbstractWebSocketTransport.java:196) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.common.AbstractWebSocketTransport$AbstractWebSocketScheduler$Flusher.process(AbstractWebSocketTransport.java:623) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.common.AbstractWebSocketTransport$AbstractWebSocketScheduler.send(AbstractWebSocketTransport.java:416) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.common.AbstractWebSocketTransport$AbstractWebSocketScheduler.processMessages(AbstractWebSocketTransport.java:337) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.common.AbstractWebSocketTransport$AbstractWebSocketScheduler.onMessage(AbstractWebSocketTransport.java:252) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.WebSocketTransport$WebSocketScheduler.onMessage(WebSocketTransport.java:186) [rt.runnable.ej.build.31.jar:?]

        at org.cometd.websocket.server.WebSocketTransport$WebSocketScheduler.onMessage(WebSocketTransport.java:124) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.jsr356.messages.TextWholeMessage.messageComplete(TextWholeMessage.java:56) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.jsr356.endpoints.JsrEndpointEventDriver.onTextFrame(JsrEndpointEventDriver.java:218) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:162) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:376) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:176) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:142) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.compress.DeflateFrameExtension.incomingFrame(DeflateFrameExtension.java:69) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:220) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:256) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:679) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:511) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [rt.runnable.ej.build.31.jar:?]

        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [rt.runnable.ej.build.31.jar:?]

        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

Caused by: java.io.IOException: Broken pipe

        at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_101]

        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_101]

        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_101]

        at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_101]

        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) ~[?:1.8.0_101]

        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177) ~[rt.runnable.ej.build.31.jar:?]

        ... 62 more

16:15:09,667 12-05-2016 [WARN ] org.eclipse.jetty.websocket.common.Parser::notifyWebSocketException:235()

org.eclipse.jetty.websocket.api.WebSocketException: java.lang.NullPointerException: Inflater has been closed

at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:229) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:256) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:679) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:511) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [rt.runnable.ej.build.31.jar:?]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

Caused by: java.lang.NullPointerException: Inflater has been closed

at java.util.zip.Inflater.ensureOpen(Inflater.java:389) ~[?:1.8.0_101]

at java.util.zip.Inflater.inflate(Inflater.java:257) ~[?:1.8.0_101]

at java.util.zip.Inflater.inflate(Inflater.java:280) ~[?:1.8.0_101]

at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.decompress(CompressExtension.java:175) ~[rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:79) ~[rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:220) ~[rt.runnable.ej.build.31.jar:?]

at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220) [rt.runnable.ej.build.31.jar:?]

... 15 more

not sure if those are anything to be concerned about or not.

And I've got one question regarding a deprecated jetty servlet context handler/filter org.eclipse.jetty.server.handler.IPAccessHandler.   What would be a good replacement for restricting access to servlets by IP since that has been deprecated?  I know that's a Jetty specific issue, but I know you contribute to the Jetty codebase and figured you might know (need IP restrictions on specific servlets, not the CometD servlet)

Other than those two things, RC1 seems to be working well enough.  Certainly nothing's breaking, so great job!

[Simone Bordet]

Hi,

On Mon, Dec 5, 2016 at 10:26 PM, steven winner <steven...@gmail.com> wrote:
> I've been trying out the RC1 for CometD 3.1.0 for the past few days, and for
> the most part, everything's working pretty well. Few things I've seen:
>
> the JS library's debug function (cometd.js:200) is throwing errors
> (TypeError: Cannot read property '_debug' of null)
> under still as of yet unclear situations (we use sentry
> https://sentry.io/welcome/ for JS error reporting, and it's showing up a
> decent amount, so I had to comment it out)
>
> example error:
> https://sentry.io/share/issue/37393538312e313838393731323931/

I don't understand how this is possible. Transport registration is the
first thing that happens, well before anything else, so the _cometd
object should not be null.
What binding are you using ? jQuery, Dojo, Angular1, Vanilla ?

> Also, been seeing these jetty websocket warnings (9.3.14.v20161028):
>
> 00:21:01,125 12-05-2016 [WARN ]
> org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension::failed:418()

What browser, exactly (vendor and version) ?

> not sure if those are anything to be concerned about or not.

The EofException caused by the broken pipe is the classical "the
client closed abruptly while we were writing", so in general it can be
ignored.
The NPE about the Inflater is something we should avoid in Jetty.

> And I've got one question regarding a deprecated jetty servlet context
> handler/filter org.eclipse.jetty.server.handler.IPAccessHandler. What
> would be a good replacement for restricting access to servlets by IP since
> that has been deprecated? I know that's a Jetty specific issue, but I know
> you contribute to the Jetty codebase and figured you might know (need IP
> restrictions on specific servlets, not the CometD servlet)

Use InetAccessHandler instead.

> Other than those two things, RC1 seems to be working well enough. Certainly
> nothing's breaking, so great job!

Thanks for the feedback, much appreciated !

--
Simone Bordet
----
http://cometd.org
http://webtide.com
Developer advice, training, services and support
from the Jetty & CometD experts.

[steven winner]

Simone,

> decent amount, so I had to comment it out)
>
> example error:
> https://sentry.io/share/issue/37393538312e313838393731323931/

I don't understand how this is possible. Transport registration is the
first thing that happens, well before anything else, so the _cometd
object should not be null.
What binding are you using ? jQuery, Dojo, Angular1, Vanilla ?

Using the jQuery binding FWIW... I haven't seen the JS error from that _debug function personally, but when released to the wild, I saw it was happening a fair amount.  I'll see if I can dig up some more data for you, but for the moment, I'm confused as well.

 

> Also, been seeing these jetty websocket warnings (9.3.14.v20161028):
>
> 00:21:01,125 12-05-2016 [WARN ]
> org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension::failed:418()

What browser, exactly (vendor and version) ?

I don't have that data right now, since those were just some errors coming in from the wild.  It's worth noting that it's not spamming the logs real bad, and I've always had BS websocket warnings in the logs in one form or another, so this could just be another one to ignore, but thought you'd be interested in seeing what was happening in a small-ish production environment (~3000K concurrent users, using SSL (wss/https), 2 nodes/comets using oort and custom redis user tracking over Seti)

 

> not sure if those are anything to be concerned about or not.

The EofException caused by the broken pipe is the classical "the
client closed abruptly while we were writing", so in general it can be
ignored.
The NPE about the Inflater is something we should avoid in Jetty.

> And I've got one question regarding a deprecated jetty servlet context
> handler/filter org.eclipse.jetty.server.handler.IPAccessHandler.   What
> would be a good replacement for restricting access to servlets by IP since
> that has been deprecated?  I know that's a Jetty specific issue, but I know
> you contribute to the Jetty codebase and figured you might know (need IP
> restrictions on specific servlets, not the CometD servlet)

Use InetAccessHandler instead.

Awesome, thanks!  Am I missing something here on how you only apply to specific servlets?  With the IPAccessHandler, you were able to specify the IPs or IP ranges, and the url mappings like so:

"127.0.0.1|/admin/*",
"10.2.0-255.0-255|/admin/*",
"192.168.0-255.0-255|/admin/*",

is there a way to do something similar with InetAccessHandler? (if that's beyond the scope here, I understand.  but if you know off hand, I'd definitely be interested in any assistance here)
 

> Other than those two things, RC1 seems to be working well enough.  Certainly
> nothing's breaking, so great job!

Thanks for the feedback, much appreciated !

No problem... looking forward to the release!

- Steve