[OBIEE EMG] Load Testing of OBIEE

[Prakash Jhunjhunwala]

Hi Everyone,

I wanted to gather user experience in how Load Testing was performed
on a new or existing OBIEE installation. Specifically, if any one had
success doing the same using tool such as Load Runner


Thanks,
Prakash

[Jeff McQuigg]

I'd be curious on this topic as well. I was with a customer doing
this back in 2002 (at 1-3am Saturday morning no less), and my client
is going to do this in a month or so on my current project.

Jeff M.

[Andriy Yakushyn]

Prakash,

I guess you could start here - http://rnm1978.wordpress.com/2009/08/19/obiee-and-load-runner-part-1/

Also, there was a white paper pdf published by Oracle where they
tested OBIEE's performance on different servers using Load Runner - it
had a very detailed methodology. I couldn't google it from first try -
but it's around.

Good luck

[Prakash Jhunjhunwala]

Thanks Andriy.

I think I have seen that document also at some point. I will try to
find it and see if that provides what I am looking for. Any other
experience with any free tool or even Oracle ATS (application testing
suite acquired in 08 or 09)

We had done the load testing manually in the past with multiple users
and multiple browsers running different queries. But recently we
noticed that queries were getting queued up on the presentation server
and we need to figure out why that is happening or if we are running
into any bottleneck. So, need to establish a repeatable methodology.

Thanks,
Prakash

On May 19, 12:25 pm, Andriy Yakushyn <andriy.yakus...@gmail.com>
wrote:
> Prakash,
>
> I guess you could start here -http://rnm1978.wordpress.com/2009/08/19/obiee-and-load-runner-part-1/

[Prakash Jhunjhunwala]

I believe you were referring to benchmark documents.

http://www.oracle.com/appserver/business-intelligence/docs/bi-suite-ee-4000-benchmark-x3755.pdf.

-Prakash

On May 19, 12:25 pm, Andriy Yakushyn <andriy.yakus...@gmail.com>
wrote:

> Prakash,
>
> I guess you could start here -http://rnm1978.wordpress.com/2009/08/19/obiee-and-load-runner-part-1/

[Jeff McQuigg]

This was an interesting read. Not so much about what the results
meant, but about how the test methodology and what it was really
testing.

The first indication I had that something was off was the HW sizes of
the BI Server and the Database - the BI Server was 4X the database!

Second, the tests they ran didn't talk about the mix from a caching
utilization perspective. How much database work was actually
happening?

Third, some of their transactions were pure catalog transactions and
not actual queries - such as "navigating catalogs".

If I had to make a guess as to what this all meant, I'd say this was a
test of the OBI layer only, particularly the UI and Cache mgmt
abilities. No real mention of the database was mentioned - size,
rows, features, pct hits, load, etc. The Load Runner box was more
powerful than the database for heaven's sake. Also, I'd reckon that
with 32GB of RAM, the cache was cached in memory not disk (meta-cache
perhaps?)

With this in mind, I can really find any value in this benchmark
except for highly cached (meaning few prompts & UI options, no data
security) system. I strongly suggest this benchmark be ignored for
any sort of real-world usage.

Jeff M.

On May 19, 10:00 am, Prakash Jhunjhunwala <prakas...@gmail.com> wrote:
> I believe you were referring to benchmark documents.
>

> http://www.oracle.com/appserver/business-intelligence/docs/bi-suite-e....

[chet justice]

My current client is using LoadRunner as well.

I'll see if I can get them to write it up to share with the group.

chet

[Christian Berg]

Hi all,

Robin actually just had an excellent presentation on this at the BI forum here. I'll ask him to join this thread

On May 19, 2010 7:22 PM, "chet justice" <chet.j...@gmail.com> wrote:

My current client is using LoadRunner as well.

I'll see if I can get them to write it up to share with the group.

chet

On Wed, May 19, 2010 at 1:00 PM, Prakash Jhunjhunwala <prak...@gmail.com> wrote:
>

> I believe y...

[Christian Berg]

...sent mail too quickly.

There's some points on perf testing and load runner: a tool is never an answer to bad performance. It's a tool. If you use it incorrectly or blindly believe what it spits out you risk missing all the gazillion of other performance-relevant aspects.

Unless you have a good and structured approach which covers all aspects of what you want to test, how, when and why installing a dedicated software like load runner will most likely completely miss the point or even lead you to wrong conclusions.

Cheers,
C.

>
> On May 19, 2010 7:22 PM, "chet justice" <chet.j...@gmail.com> wrote:
>

> My current client i...

[Kevin McGinley]

This doesn't compare to LoadRunner, but if you need something simple
and free to perform automated clicks and keystrokes to simulate a
load, there is a tool called AutoIt:

http://www.autoitscript.com/autoit3/index.shtml

When LoadRunner or other comparable tools aren't available, this can
be better than the manual alternative.

> > On Wed, May 19, 2010 at 1:00 PM, Prakash Jhunjhunwala <prakas...@gmail.com>
> wrote:
>
> > I believe y...

[Robin Moffatt]

As Christian mentioned, I've just done a presentation on Performance
Testing and OBIEE, you can find it here:
http://rnm1978.wordpress.com/2010/05/24/performance-testing-and-obiee/

I've got various posts regarding Load Runner here:
http://rnm1978.wordpress.com/category/loadrunner/
I would advise against rushing to Load Runner just because someone
says Load Testing - it may not necessarily be the best tool for the
job.

Here's the Oracle OBIEE benchmarks collated and link to each:
http://rnm1978.wordpress.com/2009/09/18/collated-obiee-benchmarks/
I'm not sure I'd discount the benchmarks as strongly as Jeff does.
Benchmarks have to start from a certain point of assumption, and a
50,000 user deployment is almost certainly not going to be generic and
repeated as standard. That the DB isn't specified (other than "this
benchmark did not use a synthetic database schema. On the contrary,
the benchmark tests were run on a full production[...] fully populated
underlying database schema.") is true and one should therefore make
sure to include that caveat when considering the benchmark in context.
However, surely a 50,000 user BI deployment would *aim* to make heavy
use of caching etc, and therefore this benchmark validates that OBIEE
can perform at this level /assuming that the DB can too/

[Prakash Jhunjhunwala]

Thanks Robin!

Excellent Presentation!