HP Performance Expert question

[Robert L. Herron]

*This article posted on Mon, 04 Mar 2002 04:49:15 GMT is resent
today

I'm involved with an application implementation that is running
into "performance issues." The application is a web-based customer
information/billing system for utilities. It is built in BEA Tuxedo,
BEA WebLogic, and an Oracle database.

We see sporadic executions times for components within the system.
Process/function A may run in less than 1 second one time and take
over 10 seconds the next time. All this while the box appears to
be idle and with plenty of "power" to spare.

The problem is not unique to a specify user, function, or dataset.
Two people sitting next to each other can perform the same function
and get dramatically different response times. User A submits first
and the application looks to be hung. User B does the exact same
function as User A (but after A is hung) and gets a response within
1-2 second (and well before A gets a response).

As best as we can tell, Glance, syslog.log, sar, vmstat, iostat,
etc look "normal."

The application vendor has been working with BEA to determine what
is happening and how to fix. Below is an excerpt from their latest
suggestion/feedback:

>The following are some recommendaytions from BEA, which I concur
with and would
>like to execute and go beyond.
>A. When we see a transaction hanging (or executing long) run the
following
>command: IPCS -aq (this will dump the IPC resources)
>B. When a transaction is hanging in Tuxedo, execute a kill -6.
This will do a
>core dump which should have indicators of what it is doing and
why it is
>hanging.
>
>To exceed these, I think we need a HP UX performance expert, that
has experience
>with KERNAL parameter settings for Oracle, and Tuxedo.
>This person should be able to do some tests, diagnostic runs,
and analysis,
>which could find the offending resources.
>

The ipcs and kill stuff is no problem. The mention of "HP UX performance
expert" in the second paragraph is my problem. They suggest we
contact HP to get someone to examine the system. Does HP offer
this type of service and how would I go about getting it?

We have a current support contract on this system (HP Support P/N:
H5355A). One line item under the P/N's description is "SW dev support."
What does this item constitute and would this type of inquiry fall
within it perview?

[Moderator's note: You could be getting stuck in a database call.
Run your servers with -r, and examine the trace file with txrpt
to see if the services have a widely varying execution time. This
will at least eliminate Tuxedo as a cause of the delay.]