SSE simulation not closing/ending
324 views
Skip to first unread message
Kenric Ashe
Jul 22, 2021, 3:48:57 AM7/22/21
to Gatling User Group
I have a script which for unknown reason is not closing the SSE connection and runs indefinitely. When I make the connection from a web browser the connection closes which seems to indicate it's not a server issue. Help?
==================================================
package quadspacechat
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
class sseSubscribe extends Simulation {
val httpProtocol = http
val scn = scenario("sseSubscribe")
.exec(
sse("GroupChat")
.connect("/subscribe/198186")
) .pause(5)
.exec(sse("Close").close)
setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}
==================================================
kenric@oryx-pro:~/Apps/gatling-charts-highcharts-bundle-3.6.0/bin$ ./gatling.sh
GATLING_HOME is set to /home/kenric/Apps/gatling-charts-highcharts-bundle-3.6.0
Choose a simulation number:
[0] quadspacechat.LoadDevHomePage
[1] quadspacechat.sseSubscribe
1
Select run description (optional)
Simulation quadspacechat.sseSubscribe started...
================================================================================
2021-07-22 00:35:37 5s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
---- sseSubscribe --------------------------------------------------------------
[--------------------------------------------------------------------------] 0%
waiting: 0 / active: 1 / done: 0
================================================================================
================================================================================
2021-07-22 00:35:42 10s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
---- sseSubscribe --------------------------------------------------------------
[--------------------------------------------------------------------------] 0%
waiting: 0 / active: 1 / done: 0
================================================================================
================================================================================
2021-07-22 00:35:47 15s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
---- sseSubscribe --------------------------------------------------------------
[--------------------------------------------------------------------------] 0%
waiting: 0 / active: 1 / done: 0
================================================================================
^C
Stéphane LANDELLE
Jul 22, 2021, 6:25:48 AM7/22/21
to gat...@googlegroups.com
We expect a 200/OK with a text/event-stream content-type, but in your case, your server never sends any response back, hence the hanging.
--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gatling/9f586bf0-2b15-423e-a9bc-78f3bea240fbn%40googlegroups.com.
Stéphane LANDELLE
Jul 22, 2021, 6:33:30 AM7/22/21
to gat...@googlegroups.com
Here's the request Gatling sends:
GET /subscribe/198186 HTTP/1.1
accept: text/event-stream
cache-control: no-cache
host: chat.quadspace.tv
accept: text/event-stream
cache-control: no-cache
host: chat.quadspace.tv
Any chance one of those headers are making your server crash?
Stéphane LANDELLE
Jul 22, 2021, 6:43:13 AM7/22/21
to gat...@googlegroups.com
Is there any chance your server implementation is broken and actually handles HTTP header names in a case sensitive fashion?
Kenric Ashe
Jul 22, 2021, 2:56:32 PM7/22/21
to Gatling User Group
My server is not crashing. My group chat app is running while I execute the Gatling script and it indicates the expected 1 additional user in the chat room which means the connection from Gatling is established.
What is the diagnostic basis for my server not responding as expected?
The very first response from my Node.js app running in a DigitalOcean droplet is:
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache'
});
My app also begins with:
const require('cors')
When loading the url directly in the Chrome browser:
I see the stream of messages and other event data, and in the Network tab the 200 response.
The Gatling script at first was ending properly and I got about a dozen html reports out of it, but at some point for unknown reason it decided to keep running indefinitely and not closing.
I previously had it set to atOnceUsers(500) which surprisingly overloaded my server (about 150 failed connections and a few 500 errors). I'm not sure if that was the point when the scenario stopped closing properly, but I changed it to atOnceUsers(1) for troubleshooting purposes. Still no joy.
I restarted my Node.js app but the Gatling scenario continues indefinitely.
The only other change before this started happening was that I upgraded from Java 8 to 11.
Any other ideas?
Kenric Ashe
Jul 22, 2021, 3:10:57 PM7/22/21
to Gatling User Group
P.S. It was running fine in Java 11 prior to when the issue started.
Stéphane LANDELLE
Jul 22, 2021, 3:13:55 PM7/22/21
to gat...@googlegroups.com
Found it.
Your chat's HTTP/1.1 support is broken. When using HTTP/1.1, it performs the handshake but then doesn't reply.
curl https://chat.quadspace.tv/subscribe/198186 -v --http1.1
vs
Now, I'm trying to understand why enableHttp2 doesn't work for SSE in Gatling.
To view this discussion on the web visit https://groups.google.com/d/msgid/gatling/3dcdcc1d-e355-4c88-9e00-8890e40284d9n%40googlegroups.com.
Stéphane LANDELLE
Jul 22, 2021, 3:38:39 PM7/22/21
to gat...@googlegroups.com
Kenric Ashe
Jul 22, 2021, 4:13:08 PM7/22/21
to Gatling User Group
HTTP/1.1 is not enabled in my server by design, to force HTTP/2 which overcomes
the old browser limit of 6 maximum connections as described here:
I thank you the awesome quick fix and I look forward to 3.6.2 availability via bundle download.
But then why did my test work the first 17 times?
Stéphane LANDELLE
Jul 22, 2021, 4:27:03 PM7/22/21
to gat...@googlegroups.com
> HTTP/1.1 is not enabled in my server by design
Your call. I wouldn't do that.
This setup will not improve anything for user-agents supporting both HTTP/2 and HTTP/1.1: they will prefer HTTP/2 anyway.
It will just block user agents not supporting HTTP/1.1.
Moreover, the way you're "not supporting" HTTP/1.1 is broken (and dangerous for you).
You should fail the TLS handshake and close the TCP connection, definitely not leave the TCP connection hanging forever.
You never close on your side and rely on the client to be a nice guy and close the connection, which is definitely not something you can expect in the jungle called the internet.
> which overcomes the old browser limit of 6 maximum connections as described here
HTTP/2 definitely makes sense for fetching static resources, typically CDNs.
But do you really expect that many concurrent requests per user-agent against the server that serves your SSE streams???
> But then why did my test work the first 17 times?
You probably ran those tests before breaking HTTP/1.1 support in your app.
There's absolutely no chance Gatling 3.6.1 is using HTTP/2 for SSE (see bug and fix).
To view this discussion on the web visit https://groups.google.com/d/msgid/gatling/9b70fc05-8643-470e-8f72-09c86c7f5a36n%40googlegroups.com.
Kenric Ashe
Jul 26, 2021, 1:49:19 AM7/26/21
to Gatling User Group
I thought that the limit of 6 connections would be an issue for load testing. Now I realize it's not a problem for your Java app, only for web browsers. Also turns out when I added http2 in the Nginx config, that didn't disable HTTP/1.1 as I assumed it would. I figured out it was the proxy headers and I fixed those so both http versions are now supported.
The other bit of good news is that after tweaking nginx.conf my group chat is now handling more then 7,000 connections! That's far more than needed. In fact, thousands of people chatting at the same time would be pretty ridiculous. ;-)
Stéphane, thank you for the education!
Stéphane LANDELLE
Jul 26, 2021, 4:48:45 AM7/26/21
to gat...@googlegroups.com
Glad you got it all sorted out :)
And thanks for your report that helped fix the SSE+HTTP/2 issue :)
Cheers,
To view this discussion on the web visit https://groups.google.com/d/msgid/gatling/f47f15b8-f2ca-4d09-a39a-65be758c7979n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages