jMeter HLS timeout over HTTP

[Doug Strick]

I'm working on load testing our Wowza 4.7.2 instance as we move from RTMP to HLS.  The wowza dev node is configured to take a local MP4 file and loop it like a livestream.  Uinsg the HLS plugin for jMeter I'm having issues where I'm getting failures when using HTTP, but HTTPS is returning success.  Below is a screenshot of the HLS plugin config.  Note that I can take that same URL and play it successfully via VLC on the same machine jMeter runs on so there are no network connectivity issues I can find.  The thread group config is just 1 thread and 1 second ramp up period.  jMeter is using Oracle java version 1.8.0_171.

Here's the result tree data.  Is there an issue with this plugin or am I just missing a config?

Sampler Result

Thread Name: Thread Group 1-1
Sample Start: 2018-06-08 13:30:47 MST
Load time: 52
Connect Time: 0
Latency: 51
Size in bytes: 663
Sent bytes:139
Headers size in bytes: 524
Body size in bytes: 139
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""):
Response code: 200
Response message: Exception: java.net.ConnectException: Connection timed out: connect


Response headers:
Accept-Ranges : bytes
null : HTTP/1.1 200 OK
Access-Control-Expose-Headers : Date, Server, Content-Type, Content-Length
Cache-Control : no-cache
Server : WowzaStreamingEngine/4.7.2
Access-Control-Allow-Origin : *
Access-Control-Allow-Methods : OPTIONS, GET, POST, HEAD
Access-Control-Allow-Credentials : true
Content-Length : 139
Date : Fri, 08 Jun 2018 20:30:47 GMT
Access-Control-Allow-Headers : Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range
Content-Type : application/vnd.apple.mpegurl




SampleResult fields:
ContentType: application/vnd.apple.mpegurl
DataEncoding:

Request

Request Headers:
GET  http://test.example.com:1935/testLive/mp4:bigBuck/playlist.m3u8

Response

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=15951424,CODECS="avc1.77.41,mp4a.40.2",RESOLUTION=1920x1080
chunklist_w383814938.m3u8

jMeter Console Log

2018-06-08 13:30:47,613 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2018-06-08 13:30:47,613 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2018-06-08 13:30:47,617 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*)
2018-06-08 13:30:47,855 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2018-06-08 13:30:47,855 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2018-06-08 13:30:47,855 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2018-06-08 13:30:47,859 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 perThread=1000.0 delayedStart=false
2018-06-08 13:30:47,859 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2018-06-08 13:30:47,859 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2018-06-08 13:30:47,863 INFO o.a.j.t.JMeterThread: Thread started: Thread Group 1-1
2018-06-08 13:31:08,988 ERROR o.a.j.s.SampleResult: sampleEnd called twice
java.lang.Throwable: Invalid call sequence
 at org.apache.jmeter.samplers.SampleResult.sampleEnd(SampleResult.java:1091) [ApacheJMeter_core.jar:4.0 r1823414]
 at com.blazemeter.jmeter.hls.logic.HlsSampler.sample(HlsSampler.java:205) [jmeter-hls-1.2.jar:?]
 at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:490) [ApacheJMeter_core.jar:4.0 r1823414]
 at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:416) [ApacheJMeter_core.jar:4.0 r1823414]
 at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:250) [ApacheJMeter_core.jar:4.0 r1823414]
 at java.lang.Thread.run(Unknown Source) [?:1.8.0_171]
2018-06-08 13:31:08,992 INFO o.a.j.t.JMeterThread: Thread is done: Thread Group 1-1
2018-06-08 13:31:08,992 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1
2018-06-08 13:31:08,992 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2018-06-08 13:31:08,992 INFO o.a.j.r.Summariser: Generate Summary Results =      1 in 00:00:21 =    0.0/s Avg:    52 Min:    52 Max:    52 Err:     1 (100.00%)
2018-06-08 13:31:08,992 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)

[Doug]

It appears the plugin may be hardcoded to use port 80 on HTTP requests.  If I remove the :1935 from the URL in the config I'm able to see a connection being made to the video.  However, the results tree is returning no responses so it's still not being of much use.

[andrei...@abstracta.com.uy]

Hi,

The plugin takes the port from the URL field. It's not hardcoded, but if you remove the port from the URL it will use the default for HTTP (80).

It's strange that the JMeter script runs ok with HTTPS and fails with HTTP since it works from VLC (I assume that the VLC playback was with HTTP protocol). Is this URL from public access? It would be great so we could debug it from here.

Thanks

[Doug]

All of the testing I do points to the port being hardcoded.  On my wowza node I have a firewall rule allowing TCP port 80 incoming traffic.  If I run jmeter HLS plugin with the address http://test.example.com:1935/testLive/bigBuck/playlist.m3u8 it's successful.  As soon as I disable the TCP port 80 firewall rule and still try with the :1935 in my URL it fails.  Testing from the same node running jMeter, VLC is able to hit the URL just fine.

[andrei...@abstracta.com.uy]

Could you send me an HTTP request-response log (for example a Fiddler record) of the workflow to see how are processed and sent the chunks of the playlist? (At least the response of the playlist request). 

Maybe the problem is in the way that the plugin parses the list of chunks to get the path of each one, but I need to have a look at the response to confirm.

Thanks.

[Doug]

Here's the initial output when I hit from the same box running jMeter in Chrome using the Native HLS Playback extensions.  Let me know if this is enough info.

playlist.m3u8 request

Request URL: http://test.example.com/testLive/bigBuck/playlist.m3u8
Request Method: GET
Status Code: 200 OK
Remote Address: 1.1.1.1:80
Referrer Policy: no-referrer-when-downgrade
Accept-Ranges: bytes
Access-Control-Allow-Credentials: true

Access-Control-Allow-Headers: Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range

Access-Control-Allow-Methods: OPTIONS, GET, POST, HEAD

Access-Control-Allow-Origin: *

Access-Control-Expose-Headers: Date, Server, Content-Type, Content-Length
Cache-Control: no-cache

Content-Length: 138
Content-Type: application/vnd.apple.mpegurl
Date: Tue, 19 Jun 2018 23:44:03 GMT
Server: WowzaStreamingEngine/4.7.2
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: test.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

playlist.m3u8 response

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=10848051,CODECS="avc1.77.41,mp4a.40.2",RESOLUTION=1920x1080
chunklist_w74636191.m3u8

chunklist request

Request URL: http://test.example.com/testLive/bigBuck/chunklist_w74636191.m3u8
Request Method: GET
Status Code: 200 OK
Remote Address: 1.1.1.1:80
Referrer Policy: no-referrer-when-downgrade
Accept-Ranges: bytes
Access-Control-Allow-Credentials: true

Access-Control-Allow-Headers: Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range

Access-Control-Allow-Methods: OPTIONS, GET, POST, HEAD

Access-Control-Allow-Origin: *

Access-Control-Expose-Headers: Date, Server, Content-Type, Content-Length
Cache-Control: no-cache

Content-Length: 187
Content-Type: application/vnd.apple.mpegurl
Date: Tue, 19 Jun 2018 23:44:04 GMT
Server: WowzaStreamingEngine/4.7.2
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: test.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

chunklist response

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:197
#EXTINF:10.0,
media_w74636191_197.ts
#EXTINF:10.0,
media_w74636191_198.ts
#EXTINF:10.0,
media_w74636191_199.ts

First .ts request

Request URL: http://test.example.com/testLive/bigBuck/media_w74636191_197.ts
Request Method: GET
Status Code: 200 OK
Remote Address: 1.1.1.1:80
Referrer Policy: no-referrer-when-downgrade
Accept-Ranges: bytes
Access-Control-Allow-Credentials: true

Access-Control-Allow-Headers: Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range

Access-Control-Allow-Methods: OPTIONS, GET, POST, HEAD

Access-Control-Allow-Origin: *

Access-Control-Expose-Headers: Date, Server, Content-Type, Content-Length
Cache-Control: no-cache

Content-Length: 8532192
Content-Type: video/MP2T
Date: Tue, 19 Jun 2018 23:44:04 GMT
Server: WowzaStreamingEngine/4.7.2
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: test.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Second .ts request

Request URL: http://test.example.com/testLive/bigBuck/media_w74636191_198.ts
Request Method: GET
Status Code: 200 OK
Remote Address: 1.1.1.1:80
Referrer Policy: no-referrer-when-downgrade
Accept-Ranges: bytes
Access-Control-Allow-Credentials: true

Access-Control-Allow-Headers: Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range

Access-Control-Allow-Methods: OPTIONS, GET, POST, HEAD

Access-Control-Allow-Origin: *

Access-Control-Expose-Headers: Date, Server, Content-Type, Content-Length
Cache-Control: no-cache

Content-Length: 10757924
Content-Type: video/MP2T
Date: Tue, 19 Jun 2018 23:44:05 GMT
Server: WowzaStreamingEngine/4.7.2
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: test.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

[andrei...@abstracta.com.uy]

Hi Doug,

Sorry for the delay.

I tested the plugin with the information that you sent me and it ran good. Also I tried it changing the port to another than the default and it ran ok too.

I checked the url's generated in my tests and they're all set with the right port.

Are you sure that is not a connectivity problem? Are you getting the timeout in the playlist request or in another later?

Have you checked that the format of the url that returns timeout is correct?

Maybe you could send me more information by private like the JMeter test results with the requests and responses or the url's sent since with this data I don't find any problem.

Thanks.