Web services Response times comparison: SoapUI Vs Loadrunner

[Britto Louis]

Hi All,
I have been using LoadRunner extensively for web performance testing,
but am relatively very new to web services testing in Loadrunner.
I just wanted to find out from people on this group, if they have done
any kind of comparison between the response times obtained for web
services using SoapUI (mostly used by development teams) Vs the
response times for the same in Loadrunner.

Any pointers, experiences and advise in this regard is highly
appreciated.

Thanks in advance
Regards
Britto

[Rupesh Garg]

I would like to know same level of comparison for the Load Runner and Jmeter
Rupesh
9160002744

http://rupeshkgarg.blogspot.com/
http://www.artofliving.org

M. K. Gandhi - ""Live as if you were to die tomorrow. Learn as if you were to live forever."

--
You received this message because you are subscribed to the Google "LoadRunner" group.
To post to this group, send email to LR-Loa...@googlegroups.com
To unsubscribe from this group, send email to
LR-LoadRunne...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/LR-LoadRunner?hl=en

[James Pulley]

Define your own benchmark. Implement the benchmark in both of your
test tools. Measure not only the output but also how long it took to
generate the inputs and also to generate the output to the same level
of information. To be fairest, use an expert for each of the tools
to run the test based upon the provided documentation for the
benchmark - This way you get a feel for the most efficient delivery of
each tool. Or, conversely, you could use someone who knows neither
tool well or at all to get a feel for a worst case scenario.

LoadRunner also has the ability to measure its own API time, a concept
called wasted time, so it would be interesting to note if you could
pull an equivalent level of information out of the other tools on the
tool imposed overhead.

James Pulley, http://www.loadrunnerbythehour.com

On Sep 5, 11:38 am, Rupesh Garg <rupeshgar...@gmail.com> wrote:
> I would like to know same level of comparison for the Load Runner and Jmeter
> Rupesh
> 9160002744
>

> http://rupeshkgarg.blogspot.com/http://www.artofliving.org

>
> M. K. Gandhi - ""Live as if you were to die tomorrow. Learn as if you were
> to live forever."
>

[Britto Louis]

Thanks for the steps James.

I have made sure of all the aspects to make sure that the requests and
responses are EXACTLY the same in both soapUI and LR.
Even excluding the wasted time of LR, the difference is quite
significant, with soapUI giving significantly lesser response time.

So again, any ideas on this issue, are welcome!

Thanks
Britto


On Sep 6, 8:29 pm, James Pulley <loadrunner-li...@jamespulley.com>
wrote:

> Define your own benchmark.   Implement the benchmark in both of your
> test tools.   Measure not only the output but also how long it took to
> generate the inputs and also to generate the output to the same level
> of information.   To be fairest, use an expert for each of the tools
> to run the test based upon the provided documentation for the
> benchmark - This way you get a feel for the most efficient delivery of
> each tool.   Or, conversely, you could use someone who knows neither
> tool well or at all to get a feel for a worst case scenario.
>
> LoadRunner also has the ability to measure its own API time, a concept
> called wasted time, so it would be interesting to note if you could
> pull an equivalent level of information out of the other tools on the
> tool imposed overhead.
>

> James Pulley,http://www.loadrunnerbythehour.com

>
> On Sep 5, 11:38 am, Rupesh Garg <rupeshgar...@gmail.com> wrote:
>
>
>
>
>
>
>
> > I would like to know same level of comparison for the Load Runner and Jmeter
> > Rupesh
> > 9160002744
>
> >http://rupeshkgarg.blogspot.com/http://www.artofliving.org
>
> > M. K. Gandhi - ""Live as if you were to die tomorrow. Learn as if you were
> > to live forever."
>
> > On Mon, Sep 5, 2011 at 11:59 AM, Britto Louis <britto.lo...@gmail.com>wrote:
>
> > > Hi All,
> > > I have been using LoadRunner extensively for web performance testing,
> > > but am relatively very new to web services testing in Loadrunner.
> > > I just wanted to find out from people on this group, if they have done
> > > any kind of comparison between the response times obtained for web

> > > services usingSoapUI(mostly used by development teams) Vs the

[James Pulley]

Start documenting differences, from the location, size and configuration of
load generators to the nature of data involved in the calls. Performance
tests are the clearest example of the butterfly effect in place that I can
think of, where small differences in initial conditions make for
dramatically different results at the end of the test.

Be certain that you are deploying both tools in their optimal models.
Eliminate virtualization if you have that in place. Use exactly the same
load generators (where possible).

And I guess you are speaking of LoadUI, not SOAPUI for performance testing.

[Britto Louis]

Yes, James. I am speaking of LoadUI for load testing, but the issue
that I am talking about is just for the response time comparisons for
a single user web service test in both soapUI & LR.

On Sep 7, 9:54 pm, "James Pulley" <loadrunner-li...@jamespulley.com>
wrote:

> Start documenting differences, from the location, size and configuration of
> load generators to the nature of data involved in the calls.   Performance
> tests are the clearest example of the butterfly effect in place that I can
> think of, where small differences in initial conditions make for
> dramatically different results at the end of the test.  
>
> Be certain that you are deploying both tools in their optimal models.
> Eliminate virtualization if you have that in place.   Use exactly the same
> load generators (where possible).  
>

> And I guess you are speaking of LoadUI, notSOAPUIfor performance testing.

>
>
>
>
>
>
>
> -----Original Message-----
> From: lr-loa...@googlegroups.com [mailto:lr-loa...@googlegroups.com]
>
> On Behalf Of Britto Louis
> Sent: Wednesday, September 07, 2011 12:38 PM
> To:LoadRunner
> Subject: Re: Web services Response times comparison:SoapUIVsLoadrunner
>
> Thanks for the steps James.
>
> I have made sure of all the aspects to make sure that the requests and

> responses are EXACTLY the same in bothsoapUIand LR.

> Even excluding the wasted time of LR, the difference is quite

> significant, withsoapUIgiving significantly lesser response time.

>
> So again, any ideas on this issue, are welcome!
>
> Thanks
> Britto
>
> On Sep 6, 8:29 pm, James Pulley <loadrunner-li...@jamespulley.com>
> wrote:
> > Define your own benchmark.   Implement the benchmark in both of your
> > test tools.   Measure not only the output but also how long it took to
> > generate the inputs and also to generate the output to the same level
> > of information.   To be fairest, use an expert for each of the tools
> > to run the test based upon the provided documentation for the
> > benchmark - This way you get a feel for the most efficient delivery of
> > each tool.   Or, conversely, you could use someone who knows neither
> > tool well or at all to get a feel for a worst case scenario.
>

> >LoadRunneralso has the ability to measure its own API time, a concept

> > called wasted time, so it would be interesting to note if you could
> > pull an equivalent level of information out of the other tools on the
> > tool imposed overhead.
>
> > James Pulley,http://www.loadrunnerbythehour.com
>
> > On Sep 5, 11:38 am, Rupesh Garg <rupeshgar...@gmail.com> wrote:
>
> > > I would like to know same level of comparison for the Load Runner and
> Jmeter
> > > Rupesh
> > > 9160002744
>
> > >http://rupeshkgarg.blogspot.com/http://www.artofliving.org
>
> > > M. K. Gandhi - ""Live as if you were to die tomorrow. Learn as if you
> were
> > > to live forever."
>
> > > On Mon, Sep 5, 2011 at 11:59 AM, Britto Louis
> <britto.lo...@gmail.com>wrote:
>
> > > > Hi All,

> > > > I have been usingLoadRunnerextensively for web performance testing,

[James Pulley]

VUGEN is a development environment for script debugging. In production the
scripts run in a headless mode which is when the times would be relevant.
You can handle this a couple of ways, (1) set logging to an absolute minimum
and run from the mdrv command line or (2) run a single user from the
controller. There are many items you can do to improve the responsiveness
of VUGEN, from turning off logging to turning off animated replay to
settings an extended attribute for sensitivity to 9, but there will always
be an overhead from the VUGEN dev environment.

James Pulley, http://www.loadrunnerbythehour.com/PricingMatrix

[Aroorganesha]

Yes I too observe the same difference.

When am hitting the server though our load runner my TPS is about 25-30 for the 20 thread, whereas in soapui is around 70-75 for the same 20 thread , eventually response time significantly less in SOAP UI.

Now client is questioning me the reason ? ?? 

Any help are much appreciated.