Model-Glue memory issues

[Cheryl]

We're running CF8 on Solaris, and setting up model-glue. Not
experienced users. The translator app runs, but verrry slow. What's
most puzzling is that when looking at the CFAdmin monitor under
application scope memory, we're getting huge numbers. Is this normal
(5,646,119,943,687,573 KB)? It almost seems like there is some object
instantiation loop. Any ideas on what might be the problem would be
appreciated.
Thanks
Cheryl

[Charlie Griefer]

if you've got CF debugging enabled, especially if you have Enable Robust Exception Information selected, you'll take a significant performance hit.

The model-glue debug itself might also be consuming some resources.  You can toggle that off in your ColdSpring.xml file.
<property name="debug"><value>true</value></property>

Of course, these things would all be off in production anyway, but to get a more accurate indication of the performance, you can turn them off temporarily in dev.

Also, by default, MG will reinitialize the framework on every request.  This can also be turned off in ColdSpring.xml (and should be turned off in production).
<property name="reload"><value>false</value></property>

Hope that helps.

--
A byte walks into a bar and orders a pint. Bartender asks him "What's wrong?" Byte says "Parity error." Bartender nods and says "Yeah, I thought you looked a bit off."

[Cheryl]

Hi Folks:
I should have been more specific:
1. we've turned off CF debugging
2. we've made "false" in the coldSpring.xml
reload, rescaffold, and debubg
We're still getting a load time of 16-22 seconds for the translator
index page. When we had CF debugging on, we were getting many
coldSpring exceptions from the bean factory, but searching the web
found notes from Joe saying that these were normal. Then, in looking
at the CF8 Admin server monitor, we saw the impossibly big number for
the Application scope for this app.

We're hoping to use model-glue for a production app, and know that all
the CF/MG guru's wouldn't be using it if it took 16 seconds to load a
page, so I'm figuring I've got a configuration problem. I just can't
determine what is is. Any suggestions on where to start?
Thanks again,
Cheryl


On Jul 11, 1:53 pm, "Charlie Griefer" <charlie.grie...@gmail.com>
wrote:

> if you've got CF debugging enabled, especially if you have Enable Robust
> Exception Information selected, you'll take a significant performance hit.
>
> The model-glue debug itself might also be consuming some resources.  You can
> toggle that off in your ColdSpring.xml file.
> <property name="debug"><value>true</value></property>
>
> Of course, these things would all be off in production anyway, but to get a
> more accurate indication of the performance, you can turn them off
> temporarily in dev.
>
> Also, by default, MG will reinitialize the framework on every request.  This
> can also be turned off in ColdSpring.xml (and should be turned off in
> production).
> <property name="reload"><value>false</value></property>
>
> Hope that helps.
>

[Dan Wilson]

can you verify that the Report Execution Times setting is disabled?

And the Memory Tracking setting in the CF admin server monitor is also disabled?

Even if debugging is unchecked, these two settings can really affect how CFC based apps are processed.

DW

--
"Come to the edge, he said. They said: We are afraid. Come to the edge, he said. They came. He pushed them and they flew."

Guillaume Apollinaire quotes

[Cheryl]

Thanks Dan, we're now down to two tents of a second. Any thoughts on
the humongus reported application memory by the CFAdmin?
Cheryl

[Dan Wilson]

well, 5,646,119,943,687,573 KB represents more memory than exists in modern computing. I'd guess that is a bug. Tell you what, restart coldfusion and see if it goes back to normal.

DW

[todd sharp]

That is a known bug with the CF server monitor. Ignore it, it's wrong.

Also, no one has mentioned it yet, but what JVM are you using? There
are known slow downs with 1.6. Rolling back to 1.5 will give you
massive gains in performance with frameworks.

On Fri, Jul 11, 2008 at 2:30 PM, Cheryl <winter...@gmail.com> wrote:
>

--
Todd Sharp
to...@cfsilence.com
IM: cfsi...@gmail.com
Blog: http://cfsilence.com/blog/client
Create A SlideCast!!! http://slidesix.com
Share Your Snippets! http://cfsnippets.org

[Mike Brunt]

Cheryl, it sounds as if your performance issue is now solved am I
correct about that? I also wondered if you had applied any tuning to
the JVM? Lastly, it is true that there are instantiation gains by
rolling back to Java 1.5 however our extensive load-tests have proven
that there is a 15-20% degradation in ongoing operation in the 1.5 JVM
as opposed to 1.6. The 1.6 JVM is much more efficient in our
experience with many, clients after all classes are instantiated.

One last question, are you using Reactor or Transfer?

Kind Regards - Mike Brunt

> t...@cfsilence.com
> IM: cfsile...@gmail.com

[Cheryl]

Hi Todd:
Ignore it sounds good to me!

I'm using the 1.6 JVM and 64 bit CF8 on Solaris. I have modified the
JVM config based on info gleaned from articles, changing things such
as the heap sizes and garbage collector. I realize that this is a
starting point, and that I need to experiment. I was hoping to avoid
rolling back to 1.5, again, doing some experimentation with some of
our apps to see if it was necessary...
Thanks;
Cheryl

On Jul 11, 2:38 pm, "todd sharp" <t...@cfsilence.com> wrote:
> That is a known bug with the CF server monitor.  Ignore it, it's wrong.
>
> Also, no one has mentioned it yet, but what JVM are you using?  There
> are known slow downs with 1.6.  Rolling back to 1.5 will give you
> massive gains in performance with frameworks.
>
>

> Todd Sharp
> t...@cfsilence.com
> IM: cfsile...@gmail.com

> Blog:http://cfsilence.com/blog/client

[todd sharp]

You are probably best off taking Mr. Brunts advice on the JVM
tuning...he's MUCH smarter then me ;)

--

Todd Sharp
to...@cfsilence.com
IM: cfsi...@gmail.com

Blog: http://cfsilence.com/blog/client

[Cheryl]

Hi Mike:
Yes, problem solved for the translator index page. Now I have to get
some of our apps into model-glue and experiment. As I said to Todd,
I'm hoping to avoid rolling back to Java 1.5, especially for simpler
apps. Thanks to all you folks for getting there first and publishing.
Couldn't do it without you.
Cheryl
P.S. No, not using a Transfer or Reactor yet. We've got nasty
databases (300+ tables in some) and most queries are not re-useable,
so haven't seen a favorable cost/benefit yet. That will probably
change as I learn more!

[Dan Wilson]

That is pretty interesting info Mike!

To summarize, in 1.6 start up is slower but ongoing operations are much faster? This would really be dissuasive for downgrading to the 1.5 JVM.  I didn't know about the ongoing operational optimizations... Thanks for the info....


DW

[Elliott]

The 1.6 JVM is generally more efficent, but changes in the Class
loading system cause the slowdown people talk about. Essentially
there's some code that tracks dependencies of loaded classes across
the object graph which runs in something near quadratic time. So the
more classes you load, the slower the load process is for each
subsequent class.
<http://forum.java.sun.com/thread.jspa?threadID=5218663>

CF does a pretty significant amount of class generation and dynamic
loading (one for every udf, cfc, cfm), and frameworks that generate
code (like Transfer) really exacerbate the issue. Sometimes to the
point of making CF development near impossible.

The bug has been fixed in Java 1.6 Beta 10, and also in the 1.7 JVM.
Both of these are available for public download.
<http://java.sun.com/javase/downloads/ea.jsp>

We've been using the 1.6_b10 and 1.7 on servers and CF has been
running fine. Of course make sure you test your application, but in
general, I think upgrading to the beta 10 would be a better solution
than downgrading the 1.5 JVM.

- Elliott

[James Allen]

Great tip Elliott!

Brilliant to know there are now actual options for upgrading and sorting out
the class loading bug.. Whether there will be any knock-on effects of using
a beta is another matter, but I imagine it's pretty stable..

Thanks again for posting this...

[Brian Kotek]

This seems to come up a lot. There is a known bug in the memory monitor that will display absurd numbers for memory usage for CFC-heavy applications. I've found that if I turn off the monitor, then initialize the application (assuming you aren't reinitializing the app on every request), and then turn on the monitor. It usually reports more realistic numbers.

Your slowness issue is most likely that you have "Report Execution Times" turned on in the CF administrator. Turn that off. There is also a known bug in the Sun 1.6 JVM that causes slow initialization of CF apps. The bug is in the Java ClassLoader. The fix from sun is set for Java 1.6 update 10 but that won't be out for months. You use Java 1.5 instead, but you will trade faster app startup with a 10-20% overall performance drop vs. 1.6.

[Mike Brunt]

Elliott, thank you for this information and I fully agree with your synopsis of not downgrading to Sun Java 1.5. In the extensive load testing I have done on CF8 applications I noted a typical 15-20% degradation in load testing.  In my opinion that is too high a price to pay in the sense that if I can deliver a 15-20% improvement in performance in a client enterprise level application that is typically viewed as a substantial improvement. 

I will be downloading the 1.6 beta and trying some palpable load tests then will publish my findings; thanks again.

Kind Regards - Mike Brunt

--

Kind Regards - Mike Brunt

Senior Server Engineer
Cell: 562.243.6255
MSN: webap...@hotmail.com

[Sean Corfield]

On Fri, Jul 11, 2008 at 12:45 PM, Dan Wilson <sipa...@gmail.com> wrote:
> To summarize, in 1.6 start up is slower but ongoing operations are much
> faster? This would really be dissuasive for downgrading to the 1.5 JVM. I
> didn't know about the ongoing operational optimizations... Thanks for the
> info....

FWIW, we're running CF8.0.1 on 64-bit Linux with Java 1.6 and overall
performance is better than with Java 1.5 even tho' we take a (small)
hit at startup time. I say "small" because our Linux servers are fast
enough that startup is still sub-one-minute even with a very complex
MG-CS-Tr app.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

[Mike Brunt]

Sean, that's great to know re 1.6.  Having worked with you I know the application itself is very well written and that is a big factor too.

Kind Regards - Mike Brunt

--

Kind Regards - Mike Brunt

[denstar]

On Tue, Jul 15, 2008 at 4:42 PM, Mike Brunt wrote:
> Sean, that's great to know re 1.6. Having worked with you I know the
> application itself is very well written and that is a big factor too.

Heh, I get the same results, but I'm pretty sure it's the hardware
that's the factor. :-)

Amazing what a few quad-cores 'n gigs and gigs of ram will do for a
crappy app. *g*

--
I base my fashion taste on what doesn't itch.
Gilda Radner

[Mike Brunt]

If the app was really crappy there is not enough anything that can solve that, I know because I have seen it.  So take solace in the fact that it is not really that crappy at all.

Kind Regards - Mike Brunt

[denstar]

You, my dear sir, are too kind.

[Raymond Camden]

Is there any idea of when beta10 would not be beta? I'm sure there
aren't any hard dates or anything, but is the feeling that a final
release is imminent?

[Brian Kotek]

This FAQ (which was a TOTAL pain in the butt to find, Sun really hides this stuff) says "mid-2008", whatever that means heh.

https://jdk6.dev.java.net/6u10faq.html

[Jeremy Prevost]

Isn't MG3 targeted for 'mid-2008' too :)

This page (which was linked from the one Brian found or I never would have seen it) hints of RC being in Summer 08 and Final being Summer/Fall 08.

https://jdk6.dev.java.net/6u10ea.html

[Raymond Camden]

That seems reasonably soon. I had set my own server (and riaforge)
back to 1.5, but I think I'll risk the beta.

[Mike Brunt]

Ray, I am pretty certain I saw Sean Corfield blog-comment that the application at Broadchoice is running on 1.6.

Kind Regards - Mike Brrunt

--

Kind Regards - Mike Brunt

[Raymond Camden]

If it is good enough for my employer.... ;)

[Brian Kotek]

Right but we're talking about 1.6 update 10, which is still in beta. That is the version that is supposed to fix the ClassLoader bug.

[Sean Corfield]

On Sat, Jul 19, 2008 at 1:34 PM, Mike Brunt <go2r...@gmail.com> wrote:
> Ray, I am pretty certain I saw Sean Corfield blog-comment that the
> application at Broadchoice is running on 1.6.

Right, we're on the standard 1.6 JVM that Adobe include in the 8.0.1
CF release. We're not seeing terribly long startup times - about 30-90
seconds at most (and we have adjusted our request timeout admin
setting accordingly). Overall, the performance benefits of 1.6
outweigh the startup issues.

[Roy Martin]

If your running the MG:unity trunk you'll also want to adjust your cflock timeout for ModelGlueInit in ModelGlue.cfm. This is increased in the gesture build to 60, but the old default was 10 seconds, which can be a real problem when reiniting a large application.

<cflock name="ModelGlueInit" type="exclusive" timeout="60" throwOnTimeout="true">

[denstar]

On Wed, Jul 23, 2008 at 11:17 AM, Roy Martin wrote:
> If your running the MG:unity trunk you'll also want to adjust your cflock
> timeout for ModelGlueInit in ModelGlue.cfm. This is increased in the gesture
> build to 60, but the old default was 10 seconds, which can be a real problem
> when reiniting a large application.

Doh!

/me slaps his forehead

That would have been the elegant solution, I bet. The one time I had
a problem, I stuck a fake index.cfm file in there while the "real" one
started up. Too many people were pounding it at once. I bet the
longer timeout would have kept the lock, and prevented the endless
looping I saw!

Heh. Oh well, the fake file worked, and it was crunch time, so people
were then happy again, yay. Oddly, it was only really bad that one
time, AFAIR.
The overall speed was way more than worth the startup trouble anyways,
although I never bench tested... *blush*

Whatever it takes, neh? *sigh* :-)

--
All men are by nature equal, made all of the same earth by one
Workman; and however we deceive ourselves, as dear unto God is the
poor peasant as the mighty prince.
Plato