Upgrade to Performance Center Results don't match LoadRunner

[Chuck]

Just upgraded from LoadRunner 9.1 to Performance Center 9.52. We have
this one application that is written in .NET using the infragistics
grid and some Ajax. The application was poorly designed and it clearly
has performance issues. We have been using LoadRunner 9.1 for the last
year and the performance issues are clearly visible on the transaction
response time graphs where we can see the response times for some
transactions over an hour test rise from 1 second to 5 seconds. The
trend continues if the test is allowed to run. Clearly a problem and
the team is working to correct the issues.

But when we ran the same test in Performance Center 9.52 (with no
application changes) using the same scripts from LoadRunner 9.1 the
transactions that displayed issues in LoadRunner 9.1 no longer are
showing issues. We rerecord the scripts in vugen 9.52 and ran the test
with the same results, no climbing transaction response times. If we
rerun in LoadRunner 9.1 the climbing trends are still visible.

We have contacted HP support. They haven’t been much help at this
point. They tell me that there is no statistical difference (after
looking at the analysis) between the LR and PC tests even though the
(we concentrated on one transaction) transaction response time in
LoadRunner over an hour goes from 1 second to 5 seconds and the same
transaction in Performance Center doesn’t show a significant increase
in time over an hour. I’ve bee testing way too long to know when I’m
being bullcrapped.

My problem is that my customer wants me to tell them if their
application is still having issues? LoadRunner 9.1 says there is but
Performance Center says there isn’t. Evidence from production users
says there are performance issues.

We are at the end of our rope! Getting to the right people in HP
Support is so frustrating.

Any ideas out there?

[chaitanya bhatt]

1.Compare the total number of HTTP connections established between client and server. It should be almost the same in both LR and PC.

2. Compare Hits/sec graphs generated by LR and PC. It should be identical.

3. Ensure that in PC the run time settings(especially, thinktime and pacing) are properly set in the scenario. PC has known issue of not reflecting script run-time settings in the scenario.

4. Monitor your load generators. Check if the utilization of resources of LGs are almost the same when tests are run using LR and PC.

 

If you observe discrepancies in point 1, 2 and 4 then there is a good chance that the transactions fired from LR and PC are different in terms of rate and/concurrency. The work around would be to check and reduce the load on load generators and get more LGs to the test harness.

 

If none of the above mentioned points is the cause of the problem, then check the granularity of graphs, at times though you may observe spikes and increasing trends in your graph, when you decrease the granularity the spikes/bumps will disappear/gets hidden and the trend may NOT look so dramatic at all.

 

If granularity is also the same, then check the server side logs and compare statistics of HTTP connections. Check arrival rate and connection rate. It should be the same.

 

If everything is normal, then you should run a test using LR 9.52 directly(without PC). Comapare the results of LR 9.52 and 9.51 and then let us know your observations.

 

-Chaitanya M Bhatt

http://www.performancecompetence.com

 

 

2010/8/2 Chuck <chuck....@gmail.com>

--
You received this message because you are subscribed to the Google "LoadRunner" group.
To post to this group, send email to LR-Loa...@googlegroups.com
To unsubscribe from this group, send email to
LR-LoadRunne...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/LR-LoadRunner?hl=en

[isha isha]

Thanks Chaitanya. Chuck also check

If you are testing with  LG9.1 and L9.5  on the different machines, then please check the machine configuration , network and firewall settings; it must be same. Otherwise you can first test with LG9.1 and the upgrade the LG to 9.5 and test again and check if the issue comes again. Sometimes the network settings or the machine time synchronisation also effect the transaction response times.

 

Best Regards,

Isha

[Morgan]

When you run the LoadRunner 9.1 load test are you using the same Load
generators performance center, This could cause major differences if
they are not the same spec.

[James Pulley]

Isha, Chuck,

 

I am making a different assumption here.   I assume when reading this that you have two distinct sets of load generators, one for the 9.1 environment and the other for the 9.5 environment, version matched for use as appropriate.     Given this assumption I would look at differences between the generators.   Do I have an identical network path and prioritization of packets from both?   Is the hardware matched, might I have a bottleneck in one set of generators for a given load of X that I do not have in another.    In short, what are the distinct initial conditions which are different between the two environments aside from the version of the LoadRunner LG software installed?

 

If the assumption is incorrect  and you are using generators of one version matched up to a controller of a different version then I would say all bets are off for comparing the two runs.  I would tend to gravitate towards the solution with the matched version set between controller and generator for the results with higher integrity.

 

James Pulley, http://www.loadrunnerbythehour.com

[Chuck]

Thanks for the responses so far.

I'll address the load generator comments. We have 20 load generators.

All load generators are blade servers with 4GB ram running windows
2003 server SP 2

5 Generators are new and we loaded PC 9.52 LG software new on these
machines.

5 generators were left on 9.1 and 10 of the load generators were
upgraded to 9.52.

We we first witnessed this difference we ran a series of tests against
the different sets of LG's.

Ran the tests against the 10 upgraded LG that have been in operation
for 5 years so therefore the network is not a variable here. We then
reran the test against only the new LG's with the same results.
HP support did have us run loadrunner 9.1 tests against the 9.52
generators (even though they know this isn't supported) no conclusions
were drawn one way or the other.

We compared the script created in 9.1 vs 9.52 of vugen and no
differences. We have tested PeopleSoft, Siebel, SharePoint, and one
other home grown application in Performance Center 9.52 and the
results were comparable to LoadRunner 9.1. It's just this one
application.

keep the ideas coming...