Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server
sometimes experiences spikes in CF memory allocation and usage but the
memory footprint does not go back down. After a cursory examination
of the logs, I can tie the memory spikes to CPU spikes and long
running CF pages, but I am curious why the footprint stays the same
even hours after these offending pages are done processing. Is there
way to view the contents of the memory allocated to CF?
On Behalf Of hayl...@gmail.com Sent: 14 July 2008 21:09 To: FusionReactor Subject: FusionReactor Group: Sudden increase in memory used by CF
Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server sometimes experiences spikes in CF memory allocation and usage but the memory footprint does not go back down. After a cursory examination of the logs, I can tie the memory spikes to CPU spikes and long running CF pages, but I am curious why the footprint stays the same even hours after these offending pages are done processing. Is there way to view the contents of the memory allocated to CF?
Yep, file uploads are a common killer. Unfortunately, folks often miss the hot fix because they assume the cumulative hotfixes take care of all of them. Sadly, they don't, and this one needs to be added manually (as Doug notes, it's only available for 7, not 6, and it's not needed in 8 as the problem is resolved.) Also, the title of the Adobe technote and its description could lead some to presume it doesn't apply to them. Forget about CFCs (as hinted at in the title and description). If you do file uploads on CF 7, apply the hotfix.
On Behalf Of Douglas Spooner (Home) Sent: Monday, July 14, 2008 4:41 PM To: fusionreactor@googlegroups.com Cc: spoon...@rsc.org; hayl...@gmail.com Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
This sounds like the problem we have just discovered in CF 6.1 MX which is also present in CFMX 7
This explains the problem and luckily there is a hotfix for 7.1 MX, but unlucky for us there isn't one so we are moving to CF8.
I've been testing CF8 over the past few days and this problem no longer exists the memory usage barely budges even when I upload a 800mb file.
Doug
-----Original Message----- From: fusionreactor@googlegroups.com [mailto:fusionreactor@googlegroups.com] On Behalf Of hayl...@gmail.com Sent: 14 July 2008 21:09 To: FusionReactor Subject: FusionReactor Group: Sudden increase in memory used by CF
Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server sometimes experiences spikes in CF memory allocation and usage but the memory footprint does not go back down. After a cursory examination of the logs, I can tie the memory spikes to CPU spikes and long running CF pages, but I am curious why the footprint stays the same even hours after these offending pages are done processing. Is there way to view the contents of the memory allocated to CF?
Thanks to both of you for your replies! My FR memory log looked much
like the spikes on the page you linked, Doug. My first question was
going to be about whether the cumulative hotfixes including this
particular item. Thanks to Charlie for beating me to the punch. I
will install the hotfix when I get into work this morning and see what
happens.
On Jul 14, 5:07 pm, "charlie arehart" <charlie_li...@carehart.org>
wrote:
> Yep, file uploads are a common killer. Unfortunately, folks often miss the
> hot fix because they assume the cumulative hotfixes take care of all of
> them. Sadly, they don't, and this one needs to be added manually (as Doug
> notes, it's only available for 7, not 6, and it's not needed in 8 as the
> problem is resolved.) Also, the title of the Adobe technote and its
> description could lead some to presume it doesn't apply to them. Forget
> about CFCs (as hinted at in the title and description). If you do file
> uploads on CF 7, apply the hotfix.
> On Behalf Of Douglas Spooner (Home)
> Sent: Monday, July 14, 2008 4:41 PM
> To: fusionreactor@googlegroups.com
> Cc: spoon...@rsc.org; hayl...@gmail.com
> Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
> This sounds like the problem we have just discovered in CF 6.1 MX which is
> also present in CFMX 7
> This explains the problem and luckily there is a hotfix for 7.1 MX, but
> unlucky for us there isn't one so we are moving to CF8.
> I've been testing CF8 over the past few days and this problem no longer
> exists the memory usage barely budges even when I upload a 800mb file.
> Doug
> -----Original Message-----
> From: fusionreactor@googlegroups.com [mailto:fusionreactor@googlegroups.com]
> On Behalf Of hayl...@gmail.com
> Sent: 14 July 2008 21:09
> To: FusionReactor
> Subject: FusionReactor Group: Sudden increase in memory used by CF
> Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server
> sometimes experiences spikes in CF memory allocation and usage but the
> memory footprint does not go back down. After a cursory examination
> of the logs, I can tie the memory spikes to CPU spikes and long
> running CF pages, but I am curious why the footprint stays the same
> even hours after these offending pages are done processing. Is there
> way to view the contents of the memory allocated to CF?
I got the hotfix installed and not five minutes after I restarted CF I
saw a memory spike. It's related to a page that ran for almost 5
minutes which also spiked CPU during its execution. I am positive
there are no file uploads in this page, so it appears the issue is
related to something else in that page.
On Jul 15, 6:31 am, "hayl...@gmail.com" <hayl...@gmail.com> wrote:
> Thanks to both of you for your replies! My FR memory log looked much
> like the spikes on the page you linked, Doug. My first question was
> going to be about whether the cumulative hotfixes including this
> particular item. Thanks to Charlie for beating me to the punch. I
> will install the hotfix when I get into work this morning and see what
> happens.
> > Yep, file uploads are a common killer. Unfortunately, folks often miss the
> > hot fix because they assume the cumulative hotfixes take care of all of
> > them. Sadly, they don't, and this one needs to be added manually (as Doug
> > notes, it's only available for 7, not 6, and it's not needed in 8 as the
> > problem is resolved.) Also, the title of the Adobe technote and its
> > description could lead some to presume it doesn't apply to them. Forget
> > about CFCs (as hinted at in the title and description). If you do file
> > uploads on CF 7, apply the hotfix.
> > On Behalf Of Douglas Spooner (Home)
> > Sent: Monday, July 14, 2008 4:41 PM
> > To: fusionreactor@googlegroups.com
> > Cc: spoon...@rsc.org; hayl...@gmail.com
> > Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
> > This sounds like the problem we have just discovered in CF 6.1 MX which is
> > also present in CFMX 7
> > This explains the problem and luckily there is a hotfix for 7.1 MX, but
> > unlucky for us there isn't one so we are moving to CF8.
> > I've been testing CF8 over the past few days and this problem no longer
> > exists the memory usage barely budges even when I upload a 800mb file.
> > Doug
> > -----Original Message-----
> > From: fusionreactor@googlegroups.com [mailto:fusionreactor@googlegroups.com]
> > On Behalf Of hayl...@gmail.com
> > Sent: 14 July 2008 21:09
> > To: FusionReactor
> > Subject: FusionReactor Group: Sudden increase in memory used by CF
> > Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server
> > sometimes experiences spikes in CF memory allocation and usage but the
> > memory footprint does not go back down. After a cursory examination
> > of the logs, I can tie the memory spikes to CPU spikes and long
> > running CF pages, but I am curious why the footprint stays the same
> > even hours after these offending pages are done processing. Is there
> > way to view the contents of the memory allocated to CF?
I know you said there are no file uploads, but even if there were, the fix wouldn't stop the memory spike from happening. It would just stop CF from holding the memory usage indefinitely. Instead, the memory (while used) is now released for garbage collection after the request is complete.
The JVM still then is free on its own to decide when to collect that garbage (unused memory). How it works depends on the GC algorithm you may choose in the setup of CF's jvm.config (best for more to leave it as is), and it may depend also on the version of the JVM in use (1.4 by default in CF 6, 7, and 1.6 by default in CF 8). Sometimes the GC won't take place until total memory used by CF increases to a high percent of available memory.
You can see both the use and the total memory in the various memory graphs in FusionReactor. And in FR 3, you can even request a GC to take place in the System Metrics page's memory graph. (Again, it's just a request. It may not honor it.)
Bottom line: the spike itself isn't necessarily a problem. The question is does it remain spiked to the point that subsequent increases in memory use take the total memory used to or near the limit. In that case, eventually CF will die.
If you'd like, do let us know if you see memory continuing to grow, and what percent is used (that's reported on the FR memory pages as well). It will be interesting to see how things go.
On Behalf Of hayl...@gmail.com Sent: Tuesday, July 15, 2008 9:17 AM To: FusionReactor Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
I got the hotfix installed and not five minutes after I restarted CF I saw a memory spike. It's related to a page that ran for almost 5 minutes which also spiked CPU during its execution. I am positive there are no file uploads in this page, so it appears the issue is related to something else in that page.
On Jul 15, 6:31 am, "hayl...@gmail.com" <hayl...@gmail.com> wrote: > Thanks to both of you for your replies! My FR memory log looked much > like the spikes on the page you linked, Doug. My first question was > going to be about whether the cumulative hotfixes including this > particular item. Thanks to Charlie for beating me to the punch. I > will install the hotfix when I get into work this morning and see what > happens.
> > Yep, file uploads are a common killer. Unfortunately, folks often miss the > > hot fix because they assume the cumulative hotfixes take care of all of > > them. Sadly, they don't, and this one needs to be added manually (as Doug > > notes, it's only available for 7, not 6, and it's not needed in 8 as the > > problem is resolved.) Also, the title of the Adobe technote and its > > description could lead some to presume it doesn't apply to them. Forget > > about CFCs (as hinted at in the title and description). If you do file > > uploads on CF 7, apply the hotfix.
> > On Behalf Of Douglas Spooner (Home) > > Sent: Monday, July 14, 2008 4:41 PM > > To: fusionreactor@googlegroups.com > > Cc: spoon...@rsc.org; hayl...@gmail.com > > Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
> > This sounds like the problem we have just discovered in CF 6.1 MX which is > > also present in CFMX 7
> > This explains the problem and luckily there is a hotfix for 7.1 MX, but > > unlucky for us there isn't one so we are moving to CF8.
> > I've been testing CF8 over the past few days and this problem no longer > > exists the memory usage barely budges even when I upload a 800mb file.
> > Doug
> > -----Original Message----- > > From: fusionreactor@googlegroups.com [mailto:fusionreactor@googlegroups.com] > > On Behalf Of hayl...@gmail.com > > Sent: 14 July 2008 21:09 > > To: FusionReactor > > Subject: FusionReactor Group: Sudden increase in memory used by CF
> > Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server > > sometimes experiences spikes in CF memory allocation and usage but the > > memory footprint does not go back down. After a cursory examination > > of the logs, I can tie the memory spikes to CPU spikes and long > > running CF pages, but I am curious why the footprint stays the same > > even hours after these offending pages are done processing. Is there > > way to view the contents of the memory allocated to CF?
It was hard to track down, but I just happened to be looking at FR the
other day when the memory spike occurred. It turns out a long running
page was trying to loop through a massive query, generating more html
content than I can dream of, then returning a 500 code after awhile.
I had the developer make some changes to the code in that page, so we
haven't seen the spikes to the extent we have before. I still wonder
why the spike in memory use is never cleaned up by GC.
On Jul 15, 9:52 am, "charlie arehart" <charlie_li...@carehart.org>
wrote:
> I know you said there are no file uploads, but even if there were, the fix
> wouldn't stop the memory spike from happening. It would just stop CF from
> holding the memory usage indefinitely. Instead, the memory (while used) is
> now released for garbage collection after the request is complete.
> The JVM still then is free on its own to decide when to collect that garbage
> (unused memory). How it works depends on the GC algorithm you may choose in
> the setup of CF's jvm.config (best for more to leave it as is), and it may
> depend also on the version of the JVM in use (1.4 by default in CF 6, 7, and
> 1.6 by default in CF 8). Sometimes the GC won't take place until total
> memory used by CF increases to a high percent of available memory.
> You can see both the use and the total memory in the various memory graphs
> in FusionReactor. And in FR 3, you can even request a GC to take place in
> the System Metrics page's memory graph. (Again, it's just a request. It may
> not honor it.)
> Bottom line: the spike itself isn't necessarily a problem. The question is
> does it remain spiked to the point that subsequent increases in memory use
> take the total memory used to or near the limit. In that case, eventually CF
> will die.
> If you'd like, do let us know if you see memory continuing to grow, and what
> percent is used (that's reported on the FR memory pages as well). It will be
> interesting to see how things go.
> On Behalf Of hayl...@gmail.com
> Sent: Tuesday, July 15, 2008 9:17 AM
> To: FusionReactor
> Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
> I got the hotfix installed and not five minutes after I restarted CF I
> saw a memory spike. It's related to a page that ran for almost 5
> minutes which also spiked CPU during its execution. I am positive
> there are no file uploads in this page, so it appears the issue is
> related to something else in that page.
> On Jul 15, 6:31 am, "hayl...@gmail.com" <hayl...@gmail.com> wrote:
> > Thanks to both of you for your replies! My FR memory log looked much
> > like the spikes on the page you linked, Doug. My first question was
> > going to be about whether the cumulative hotfixes including this
> > particular item. Thanks to Charlie for beating me to the punch. I
> > will install the hotfix when I get into work this morning and see what
> > happens.
> > > Yep, file uploads are a common killer. Unfortunately, folks often miss
> the
> > > hot fix because they assume the cumulative hotfixes take care of all of
> > > them. Sadly, they don't, and this one needs to be added manually (as
> Doug
> > > notes, it's only available for 7, not 6, and it's not needed in 8 as the
> > > problem is resolved.) Also, the title of the Adobe technote and its
> > > description could lead some to presume it doesn't apply to them. Forget
> > > about CFCs (as hinted at in the title and description). If you do file
> > > uploads on CF 7, apply the hotfix.
> > > I've been testing CF8 over the past few days and this problem no longer
> > > exists the memory usage barely budges even when I upload a 800mb file.
> > > Doug
> > > -----Original Message-----
> > > From: fusionreactor@googlegroups.com
> [mailto:fusionreactor@googlegroups.com]
> > > On Behalf Of hayl...@gmail.com
> > > Sent: 14 July 2008 21:09
> > > To: FusionReactor
> > > Subject: FusionReactor Group: Sudden increase in memory used by CF
> > > Hello all. I am using FR 3.0.0 on CFMX7 / IIS and our server
> > > sometimes experiences spikes in CF memory allocation and usage but the
> > > memory footprint does not go back down. After a cursory examination
> > > of the logs, I can tie the memory spikes to CPU spikes and long
> > > running CF pages, but I am curious why the footprint stays the same
> > > even hours after these offending pages are done processing. Is there
> > > way to view the contents of the memory allocated to CF?
Well, again, a spike itself isn't unusual. And it could hold it long after it's no longer needed, if the GC decides it's not yet time to collect it. Of course, it could also be that the error leaves things in a state that CF doesn't mark the memory as available, so that would indeed appear to you as a "leak". You say it's "never cleaned up by GC". Are you saying that in this instance, if left on its own, it did eventually fill memory and crash? Or is this more speculation (not that there's anything wrong with that). Also, did you try to use FR's feature to request a GC? Did that help? If so, then I'd again argue it would have been cleared up at some point automatically. It would be interesting to understand this, if you have more thoughts on the matter.
On Behalf Of hayl...@gmail.com Sent: Monday, July 28, 2008 9:25 AM To: FusionReactor Subject: FusionReactor Group: Re: Sudden increase in memory used by CF
It was hard to track down, but I just happened to be looking at FR the other day when the memory spike occurred. It turns out a long running page was trying to loop through a massive query, generating more html content than I can dream of, then returning a 500 code after awhile. I had the developer make some changes to the code in that page, so we haven't seen the spikes to the extent we have before. I still wonder why the spike in memory use is never cleaned up by GC.
Agreed, spikes themselves aren't unusual... when they're spikes. The
spike caused by the failed request in question was never deallocated
from memory. I've tried the FR manual GC request a number of times to
no effect. I have a feeling that it has something to do with
ColdFusion eventually returning a 500 code, but that's just a feeling
with no supporting evidence. Really, I'm not the most conversant
person around when it comes to CF memory management. About 3 of these
"spikes" cause ColdFusion to become very slow to respond, consuming
around 600MB system memory (of 2GB total).
On Jul 29, 12:02 am, "charlie arehart" <charlie_li...@carehart.org>
wrote:
> Well, again, a spike itself isn't unusual. And it could hold it long after
> it's no longer needed, if the GC decides it's not yet time to collect it. Of
> course, it could also be that the error leaves things in a state that CF
> doesn't mark the memory as available, so that would indeed appear to you as
> a "leak". You say it's "never cleaned up by GC". Are you saying that in this
> instance, if left on its own, it did eventually fill memory and crash? Or is
> this more speculation (not that there's anything wrong with that). Also, did
> you try to use FR's feature to request a GC? Did that help? If so, then I'd
> again argue it would have been cleared up at some point automatically. It
> would be interesting to understand this, if you have more thoughts on the
> matter.
