Hey Joel, Tom, thanks for the feedback. What I'm looking for is something I've used in ColdFusion for some time now. (And yea, I know a J2EE app is somewhat different, but it is a web app, and damnit, I want it to be the same. ;) Given a request, CF can give you a nice report broken down to all the major aspects of that call. You get a nice tabular report with highlights on things that are especially slow.
Imagine you have a request that calls some function to generate data. You call it N times in a request. You notice that the request is taking 2-3 seconds to run so you enable the timing stuff in CF. In the report you can see the function calls along with the arguments passed to it. Imagine our function takes a number argument. You would be able to see that when you called the function with n<10, it ran pretty fast, less then 10ms. But for some reason when n=10, the function took 1000ms.
Just having that type of data then can lead me to go look at my function and do some tuning where I see how the function behaves based on the arguments passed in.
As for the database layer, there may be certain selects that are performing super slow (I used Mongo above, but I'm really thinking DB calls in general, which tend to be a slow part of web apps). Being able to see that this query ran at X ms could help me figure out which queries need tuning and which don't.
To Joel's specific question, "It seems to me that the thing you'd care about is "how long was it between when I called out to mongo and when I got the response?"", absolutely, yes.
Btw, I can share a screenshot of the CF output if you guys want to get an idea.