CustomTest.js - Timeout can't be longer then x seconds?

[Joralf Quist]

Hi guys,

I'm working on creating some stress tests using the Custom Tests, but I've ran into some problems. I'm running a test that is supposed to replay a log file which consists of a url and a timeout. In this test I want each user (runTest) to do 1 page request.

Here's a piece of the code that I'm running:

function MyCustomTest(redlineApi, testNum, rand, config)

{

this.testNum = testNum;

this.redlineApi = redlineApi;

this.url = 'someurl';

this.cityList = [

{id: 01, timeout: 1000},

{id: 02, timeout: 2000},
                etc. etc. list goes on to about 100 items];

}

/** Run test */

MyCustomTest.prototype.runTest = function( redlineCallback )

{

var that = this;

var testNum = that.testNum;

var testItem = that.cityList[testNum];

var url = that.url + testItem.id;

var timeout = testItem.timeout;

setTimeout(function () {

that.loadPage(url, function(err) {

if(err) {

redlineCallback(true);

}

redlineCallback(false);

});

}, timeout);

};

So this.cityList is as long as the number of users that I simulate in a test. For now the timeout will be equal to (n + 1 ) * 1000. So item 37 has a timeout of 38000 ms.

This way I hope to achieve the following: Fully replay a Varnish/Apache/Nginx log in order to run a test that is exactly the same as a real life situation from the past.

However, when running the code I run into a problem. Not all tests are executed when I simulate 49 users. The results are shown below:

If I run the same test, but with a maximum timeout of 30 seconds (if (timeout > 30000) { timeout = 30000; }), these are the results:

So I figured maybe the amount of users has influence on which item will pass and which won't, but the results with 70 users show the same behavior:

So I guess my question is: is there a limit to how long the test framework will wait for runTest to answer and if so, how can I work around this?

[Richard Friedman]

Joralf,

Those look OK.  But perhaps I am missing something.  If you send email to in...@redline13.com on the account we can take a look at the output from the specific tests. 

The reason it looks OK is on of your examples it shows the full test and for 49 user it shows about 49 seconds for the whole test.

Once again if we look at all the output could probably do a bit more analysis.

Rich

--
You received this message because you are subscribed to the Google Groups "RedLine13" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redline13+unsubscribe@googlegroups.com.
To post to this group, send email to redl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redline13/3bf4cf2a-7e7a-4e03-b0a1-dadedb3e8433%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Richard Friedman
609.314.0129

[Joralf Quist]

Hi Richard!

I've send you a mail. Just to clarify for the other readers. It's graph 1 and graph 3 that show not all requests were succeeded. Graph 2 did succeed. 

Joralf

[Richard Friedman]

Joralf,

Thanks again for raising this up and sending more information, it really helped.

The issue was us and related to the way we are checking for load tests finishing.  This specifically impacted custom tests (php, python, nodejs). The fix went up with this feature update https://www.redline13.com/blog/2016/10/update-view-running-users/ on the 27th.  

If you have any related issues please raise it up.

Thanks,

Rich

On Tue, Nov 1, 2016 at 10:45 AM, Richard Friedman <ri...@redline13.com> wrote:

Joralf,

Thanks again for raising this up and sending more information, it really helped.

The issue was us and related to the way we are checking for load tests finishing.  This specifically impacted custom tests (php, python, nodejs). The fix went up with this feature update https://www.redline13.com/blog/2016/10/update-view-running-users/ on the 27th.  

If you have any related issues please raise it up.

Thanks,

Rich

--

You received this message because you are subscribed to the Google Groups "RedLine13" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redline13+unsubscribe@googlegroups.com.
To post to this group, send email to redl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redline13/ff759ea8-8be8-474f-beb1-15b781a3ed63%40googlegroups.com.