Easy way to execute code in case of error?
Vehementi
Is there an easy way to execute some functions when LoadRunner detects
an error?
In my case, I am performance testing a webapp, and I want the VUser to
log out automatically in case it finds any error (content check, HTTP
error, etc.). Currently, it just aborts that iteration of the script
and starts it anew, but that doesn't log out properly, leaving various
resources hanging (bad).
I am aware that I could insert fun checks before each request to check
for strings and whatnot, and in case I detect an error, execute my
logoff() function, but that is very clunky and won't work for
LoadRunner-generated errors (I think).
Any help would be appreciated,
James
Floris Kraak
>
> Is there an easy way to execute some functions when LoadRunner detects
> an error?
Not with the regular web protocols as far as I know.
>
> In my case, I am performance testing a webapp, and I want the VUser to
> log out automatically in case it finds any error (content check, HTTP
> error, etc.). Currently, it just aborts that iteration of the script
> and starts it anew, but that doesn't log out properly, leaving various
> resources hanging (bad).
>
Been there before ;-)
> I am aware that I could insert fun checks before each request to check
> for strings and whatnot, and in case I detect an error, execute my
> logoff() function, but that is very clunky and won't work for
> LoadRunner-generated errors (I think).
>
What you could do is simply set some variable or parameter if you
logged in successfully, and unset it when you log out. Let's call it
"loggedIn".
Now all you need to do is check that value at the start of the
iteration. If the user is still logged in, log him out by calling your
logout() action ;-)
Like I said - I've been there before.
It's not as pretty as having some kind of event notification when
errors occur but it works.
Regards,
Floris
---
'What does it mean to say that one is 48% slower? That's like saying
that a squirrel is 48% juicier than an orange - maybe it's true, but
anybody who puts the two in a blender to compare them is kind of
sick.'
--- Linus Torvalds
Jim
vuser_end? You could put your logout into this along with some
condition so that if the iteration has completed sucessfully then this
logout doesn't happen.
Floris Kraak
>
> When your script errors and that iteration is aborted it should run
> vuser_end?
Normally it errors and starts a new new iteration immediately.
Only if the user is failed (or 'passed', 'stopped' etc.) would it run vuser_end.
( To be entirely honest, if Loadrunner fails a virtual user just
because of some error in the application under test I call it a bug.
Only if something is seriously wrong with the virtual user or the
script should the virtual user get aborted like that. )
> You could put your logout into this along with some
> condition so that if the iteration has completed sucessfully then this
> logout doesn't happen.
But that would only be done once, when the user ends, not each time
when the iteration ends ;-)
(warning: text bomb ahead)
On a sidenote, I honestly abhor putting anything in vuser_end() that
is not actually cleanup code.
I know HP/Mercury taught people to do it that way in their courses for
years (and they still presume people do things like that in their
exams) but it's just a Really Bad Idea (tm).
If you absolutely must run anything when the user is stopped put it in
an action called 'logout' or whatever it is supposed to be doing and
tell Loadrunner to run it in the 'End' block in the Run Logic section
of your run-time settings.
Most of the time there are better ways though.
Why not perform user actions in the init and end sections of your
runtime settings?
Imagine you have a long running test. Let's take the extreme example
and let's say this test is scheduled to run for days to shake out
subtle resource leak problems.
Now some scheduled maintenance hits, and one of the users has an error.
What happens then is that the iteration is aborted and the user
restarts (as above).
When we have a new iteration the default behaviour is to clear all
cookies so you will no longer be logged in. Even those settings were
changed, if your user still is logged in it will be in some unknown
state, probably staring at some error page.
Either way there is only one way to recover: Start over from a clean slate.
Now let's imagine we have some scenario in which users rarely log out
and do things the 'recommended' way: Login in vuser_init(), logout in
vuser_end().
In this case the virtual user is <expletive>. It can not possibly
recover, the user is in an unknown state. But it will try and try
again to do whatever the first step in your iteration would normally
do, as if you were logged in (but you aren't). So the user will just
start spewing errors, possibly at a very high rate as well since it
seems that pacing settings are ignored in some cases. At the end of
your long running test a number of vusers (if you're unlucky, all of
them) have been doing nothing but generating bogus load and spewing
errors, your logs are humongous and generating an analysis from it is
impossible since the log files run in the gigabytes with ease.
Test failed. Start again, please.
So what should you do?
Simple: Put the login and the logout inside the run section of your
run logic. Inside them you make a block that runs however many times
you want it to run - hunderds of cycles, if needed - and that contains
all of the things your iteration did normally. If that user gets an
error now, all that happens is that the user will be logged out
automatically (cookies are cleared, at least) and then logged in to
restart the cycle from scratch.
What happens with this kind of setup during the nightly maintenance
glitch is that you may have a number of users go through a login
cycle. If your login process is fairly hefty and the planned load high
you might still run into trouble because it could potentially overload
things. But - that's only a chance, you still have a fighting chance
that the test will survive it without any trouble.
Floris Kraak
> and starts it anew, but that doesn't log out properly, leaving various
> resources hanging (bad).
>
By the way, in case you hadn't realized this yet: If a web application
does not clean up resources if a user fails to log out then that is a
bug in the web application.
Users *will* forget to logout. In fact, the application writers should
expect about 70% to just ignore that logout button completely - it's
what happens in the real world ;-)
John Crunk
John Crunk Ph.D ABD
Sent from my iPhone
Vehementi
Thank you for your responses.
The webapp under test does indeed log the user out after some timeout;
however in the very short term it can become a problem (e.g. the
session continues to consume resources on related servers; subsequent
users may encounter "server is full" errors until the sessions time
out; cannot guarantee that the system is at the desired load
throughout the course of the test; and so on).
Checking in the first step to see whether the previous iteration
logged out could work, except I believe that each new user iteration
is set to clear all artifacts as per LR's run-time "simulate a new
user on each iteration" setting. I will look into this today, but I
suspect that if that user were to suddenly attempt to log out, the
server would be unaware of which session it were referring to since it
sees a new user altogether.
I think the approach I will have to use, as suggested by the owner of
this blog (http://mishmashmoo.com/blog/), is to just write a
my_start_transaction() function that performs error checking, and then
calls lr_start_transaction(), and then globally replace
lr_start_transaction() with mine throughout the script. Then I will
throw out all LR error handling, set all LR errors to throw warnings
instead and to continue on error, and then have my script detect page
load failures and log out and go to next iteration appropriately.
Incredibly ugly, but it's the only thing I can see actually working.
And probably the worst case for my present approach of throwaway
scripts (due to the webapp being under development).
Floris Kraak
>
> Checking in the first step to see whether the previous iteration
> logged out could work, except I believe that each new user iteration
> is set to clear all artifacts as per LR's run-time "simulate a new
> user on each iteration" setting.
>
> I will look into this today, but I
> suspect that if that user were to suddenly attempt to log out, the
> server would be unaware of which session it were referring to since it
> sees a new user altogether.
Well, there's the bit I had forgotten to mention: You will have to
change that run-time setting and do the session reset by hand after
your logout if you use this technique.
The latter is just a matter of calling the right LR functions. I don't
recall the names of those functions right now - there are 2 separate
ones I believe, one for cookies and one for the browser cache - but
they aren't very hard to find in the help.
> I think the approach I will have to use, as suggested by the owner of
> this blog (http://mishmashmoo.com/blog/), is to just write a
> my_start_transaction() function that performs error checking, and then
> calls lr_start_transaction(), and then globally replace
> lr_start_transaction() with mine throughout the script. Then I will
> throw out all LR error handling, set all LR errors to throw warnings
> instead and to continue on error, and then have my script detect page
> load failures and log out and go to next iteration appropriately.
Drastic. It would definitely work but that's a lot of code to write
for this particular problem.
You will have to make your own version of lr_end_transaction() too of course.
That having been said, I have my own versions of start and end
transaction functions - but they exist only to do automatic numbering
of transaction names and add some logging for external analysis.
> Incredibly ugly, but it's the only thing I can see actually working.
You don't have to do that though. My method works, trust me ;-)
Though I admit that might not win a prize for prettieness either..
> And probably the worst case for my present approach of throwaway
> scripts (due to the webapp being under development).
Well, you could automate your script building a bit. Write a script
that goes through recordings and fixes things up. Put all the
complicated code in functions in external .c (or .h) files and add
them later.
Of course at some point things might start to stabilize enough for you
to consider stopping with throwing your script away..
John Crunk
John
Floris Kraak
>
> You could tell it to continue on error and ignore the error or you could debug and find the cause of the error and fix it. Those are the two basic choices, the choice is yours.
>
In many cases the bug is in the application under test and you cannot
fix it yourself, only report it.
In most of those cases continue on error makes no sense either.
Usually if you hit an error on a page the next step in the script
would attempt to do something that makes no sense. Check out a
shopping cart that has nothing in there, open a page that is not
available because your login failed, etc.
Frankly, 'continue on error' in a script usually means to me 'someone
was lazy and didn't think this through' ;-)
Roland Jefferson JR
and just for extra measure through in the web kill cookies function
and the web kill connections function at the end of your action
section.
Hope this helps,
Roland