alert memory different from monitoring memory

[Chris]

Hi, I have Crash Protection configured to email alerts when memory
available gets below 5%. However, many times the alert comes but the
current memory usage displayed is completely different.

Alert - 97% memory used
Monitoring (Request History) - 56% memory used

Is the because the JVM is GC'ing quickly and getting memory back ...
so the alert triggered because the memory reached that point only for
a moment?

thanks,
Chris

[charlie arehart]

Well, sure, by the time you get in to view the FR interface, things could be
different than what you saw in the alert.

But you should be able to see the exact time in the alert (don't go by the
time you "received" the email but the time shown *within* the alert, which
shows when it was generated by FR), and then go to the FR interface (system
metrics or resources>memory graph), or the logs, to see what they show the
memory use was at that time.

Note, too, that you could be looking at 2 different servers: the alert could
come from one and you may be looking at the FR instance for another. Just
saying: look very closely at where the alert says it's coming from, and be
sure you see that same instance name in the top left corner of FR when
viewing it. :-)

Finally, as just a reminder for everyone, be careful also when setting the
memory alert to be sure to set the amount as the "amount of free memory
below which it should alert" (so 20, if you want an alert when free memory
falls below 20%). Some instead mistakenly think it's the "amount above which
used memory it should alert", and they would set it to 80. In that case,
you'll be getting alerts all the time, whenever the heap is above 20% used.
Very common mistake, but I'll trust, Chris, from your wording, that that's
not what's happening for you. Just pointing it out for others while we're on
the topic.

And I would note that a number like 5 (as he says he's using, for 5% free
memory) is better than most would suspect. You might think, "well I don't
want to be notified only when there's 5% left. I'd like to be told when
there's 20 or 30% left." But the problem is that the JVM (since 1.5, so in
CF 8, 9, and 10 which are on 1.6) may wait to do a full GC (if it's busy)
until as much as 95% of memory is used. As such, you may get alerts when you
set it to 80%, just to have the JVM shortly do a GC and recover.

Indeed, that could technically be happening to you, Chris. I've seen it
sometimes wait to 98% full before doing a full GC. That could then be why,
by the time you look, it's recovered. Really nothing much we can do about
this. (I mean, sure, there are some JVM tweaks one may consider.) I'd say
that as long as it DOES recover memory on a GC, that's good news. :-) And if
it doesn't, you have bigger problems than this alerting mechanism. I will
note that it's one reason I don't myself use the Memory alert that much.
Instead, I figure if there's a memory problem, the user will get an
outofmemory error and can attack it then. Not much else to do with an alert
in that case until the real problem is solved.

(To that end, I do wish FR had a 2-stage memory alert, where at stage 1, it
would do a GC if free memory falls below a given level---such as 20%--and
then if after a minute, it did not recover to now be below a higher
level--such as 30%--then it would send alert. That way, you don't get
notified if the GC solved things. But you do get the GC and an alert if
things really are starting to go bad, because memory cannot be GCed.)

More than you sought in your question, Chris, but thought it may help some
readers.

/charlie

> --
> You received this message because you are subscribed to the Google Groups
> "FusionReactor" group.
> To post to this group, send email to fusion...@googlegroups.com.
> To unsubscribe from this group, send email to
> fusionreacto...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/fusionreactor?hl=en.

[Chris]

Thanks Charlie,

I checked the request.log, and that was it -- the memory use very
quickly went up, then very quickly went down. So the alert wasn't that
helpful :-)

I've tried the alert set at 4% of memory remaining, and 3%. Now trying
1%, which may make the alert less useful, but false alerts don't help
at all.

Regards,
Chris

[charlie arehart]

Thanks for the update. :-)