Object creation performance in CF 9: any better?

[Brian Swartzfager]

One of the issues that keeps coming up as part of this discussion (and
rightly so) is the performance penalty you take when you instantiate a
large number of objects in ColdFusion.

I remember when CF 8 came out, a lot of numbers were thrown out there
regarding the performance improvements (i.e. this operation now runs
43% faster, this tag now works 89% faster, etc.). But so far, I
haven't seen anyone publish such numbers for CF 9, and performance
improvements as a whole was only briefly mentioned (if at all) in
Terry's 2-hour Connect session on Monday.

So do we know if object instantiation performance has improved in CF
9, and if so by how much? Or, being that it's still in beta, is it
too soon to look at that/make a judgment about it?

[Matt Williams]

I would love to see some info on this too. It seems the question is
going unanswered lately. It was asked 2 or 3 times on Forta's blog
when he posted about ORM.
http://forta.com/blog/index.cfm/2009/7/7/ORM-Rethinking-ColdFusion-Database-Integration

Thus far I've stuck to queries for anything returning more than one
record, and seeing the code example in his blog entry:
<cfset data=EntityLoad("Books")>
immediately made me think, what about performance with regards to
object instantiation??

Perhaps because it is Hibernate creating Java objects and not CFCs we
will be okay. But the jury is out until some official test results are
in.

-Matt Williams

[Kevan Stannard]

I had a quick look at this when I first fired up CF9. I performed a test of just creating objects with only an init() function, so these numbers are around the maximum I can get out of CF on my laptop (dual core win xp - just a moderate machine). Adding functions to components will reduce the performance.

On CF8 I was able to create 4,000 objects per second.

On CF9 Beta I was able to create around 25,000 objects per second. This actually started off at a much lower number the first time I executed the code (around 10,000 per second) but after running the test a few times it found its way up to the 25,000 mark.

2009/7/16 Matt Williams <mgw...@gmail.com>

[Hal Helms]

4000 objects per second on CF 8? Wow. That doesn't at all match my
experience of working with complex objects. Perhaps the connections
(aggregation, inheritance) is a limiting factor? Just a guess.

--
Hal Helms
Training, Mentoring, Consulting

[James Allen]

That is a crazy improvement – awesome!

 

Be interesting to see this benched against Railo 3.1.

 

Hope they release soon. We are set to launch our .com on the 7^th August but are probably going to have to go CF8 as I assume CF9 won’t be out by then. Hope they offer a cheap upgrade path.

 

---

James Allen

E: ja...@jamesallen.name

Blog: http://jamesallen.name

Twitter: @CFJamesAllen (Coldfusion / Web development)

Twitter: @jamesallenuk (General)

[James Allen]

It may be worth reinvestigating it Hal. Obviously the complexity of objects
has an effect but you may find things have improved with later Java
revisions. Once Sun fixed the object instantiation bug CFC performance in
CF8 improved dramatically...

---
James Allen
E: ja...@jamesallen.name
Blog: http://jamesallen.name
Twitter: @CFJamesAllen (Coldfusion / Web development)
Twitter: @jamesallenuk (General)

-----Original Message-----
From: coldfu...@googlegroups.com [mailto:coldfu...@googlegroups.com]
On Behalf Of Hal Helms
Sent: 16 July 2009 14:47
To: coldfu...@googlegroups.com
Subject: [coldfusionoo] Re: Object creation performance in CF 9: any better?

[Hal Helms]

Definitely worth reinvestigating.

[James Allen]

I would love to hear your results.

We're launching a big .com next month which I build from the ground up in
CF8, MG2, Coldspring and Transfer. It's by no means super optimised but
after upgrading Java it runs superbly. The beta testing phase has gone well
and the app seems to be performing nicely. Just hope it scales as the user
levels increase..

[Tom Chiverton]

2009/7/16 James Allen <sling...@googlemail.com>:

> Just hope it scales as the user
> levels increase..

I assume this isn't just crossing your fingers, and you've actually
run JMeter or something against it to simulate more users ?

--
Tom

[Kevan Stannard]

Performance discussions typically focus on object creation, but I've found that function execution also carries a significant overhead.

So if you have an object with 20 setters, and all 20 setters are called when the object is created (say, from an init() function), then this will have a significant performance impact on the time to initialise the object.

From memory function calls had a similar overhead to trivial object creation, but I'd need to double check that.

2009/7/16 Hal Helms <hal....@gmail.com>

[Matt Williams]

On Jul 16, 10:12 am, Kevan Stannard <kevanstann...@gmail.com> wrote:
> From memory function calls had a similar overhead to trivial object
> creation, but I'd need to double check that.

That sounds right to me. I remember hearing that when CF compiles to
Java, each function is actually its own java class. I could be wrong
here, but if not then as CFCs have more functions you get that much
more instantiation going on.

-Matt Williams

[Henry Ho]

Latest benchmark:

http://www.objectorientedcoldfusion.org/post.cfm/coldfusion-9-object-creation-performance

http://blog.kukiel.net/2009/07/coldfusion-8-9-and-railo-object.html

Henry

[Sean Corfield]

It's important to note Adam Lehman's comments in that thread.

It's also important to notice two new Admin options in CF9:
- Cache Template In Request (each .cfc file is only checked once per request)
- Component cache (each CFC path found is cached and never looked up again)

These have a huge impact on CF9 performance if you turn them off.

Frankly, I think the performance argument against OOCF is over-used
but certainly Railo and now CF9 ought to make people reconsider...

On Thu, Jul 16, 2009 at 2:26 PM, Henry Ho<henry...@gmail.com> wrote:
> http://www.objectorientedcoldfusion.org/post.cfm/coldfusion-9-object-creation-performance
>
> http://blog.kukiel.net/2009/07/coldfusion-8-9-and-railo-object.html

[Matt Williams]

More details regarding ORM and CFCs

http://www.rupeshk.org/blog/index.php/2009/07/coldfusion-orm-and-cfc-performance/

[jmetcher]

To selectively quote Rupesh:

"But even then it can not match POJO performance and thats an
unrealistic target."

Indeed. I agree with Sean that the performance thing is a little
overamped. If you really need to crunch a lot of data, or are really
going to get lot of leverage out of an "everything is an object"
style, CF isn't going to cut it. Not with a 20% improvement, not with
a 200% improvement. But that leaves a huge set of applications where
CF is just fine.

If you like synthetic benchmarks, I tried Kevan's instantiatation test
and got CF8 = ~3500K objects/second, Java6 = ~6M objects per
second. CF doesn't, can't and shouldn't compete on raw performance.

Jaime

On Jul 18, 12:05 am, Matt Williams <mgw...@gmail.com> wrote:
> More details regarding ORM and CFCs
>

> http://www.rupeshk.org/blog/index.php/2009/07/coldfusion-orm-and-cfc-...

[jmetcher]

Hal,

On raw object instantiation my dev box gets about 3.5K objects per
second. But at the ORM level, that translates to a much lower
number. Obviously there a lot of variables, but 100 fully hydrated
domain objects per second is in the ballpark.

Jaime

[jmetcher]

Sorry, that should have been CF8 = 3.5K objects per second.

[whostheJBoss]

What Java version are you using?

[James Allen]

Hey there,

Currently running 1.6.0_14.

Cheers,
James.

[Jim]

Are you seeing any memory leak issues or anything like that? I'm finding that the creation of some simple Transfer objects (like a user with an id field, name, address fields) are eating a megabyte of memory when created. --- On Sat, 8/1/09, James Allen <sling...@googlemail.com> wrote:

[James Allen]

I haven’t noticed anything specifically but to be honest I haven’t looked at memory usage until a few minutes ago after reading your message.

 

Current memory use of JRun is 787MB and the server has been up for about 4 weeks without a single reboot.

 

---

James Allen

E: ja...@jamesallen.name

Blog: http://jamesallen.name

Twitter: @CFJamesAllen (Coldfusion / Web development)

Twitter: @jamesallenuk (General)

 

[whostheJBoss]

What kind of traffic do you get? Do you rely heavily on Transfer?

On Aug 1, 3:36 am, "James Allen" <slingsho...@googlemail.com> wrote:
> I haven't noticed anything specifically but to be honest I haven't looked at
> memory usage until a few minutes ago after reading your message.
>
> Current memory use of JRun is 787MB and the server has been up for about 4
> weeks without a single reboot.
>
> ---
>
> James Allen
>
> E: ja...@jamesallen.name
>
> Blog:http://jamesallen.name
>
> Twitter: @CFJamesAllen (Coldfusion / Web development)
>
> Twitter: @jamesallenuk (General)
>
> From: coldfu...@googlegroups.com [mailto:coldfu...@googlegroups.com]
> On Behalf Of Jim
> Sent: 01 August 2009 11:30
> To: coldfu...@googlegroups.com
> Subject: [coldfusionoo] Re: Object creation performance in CF 9: any better?
>
> Are you seeing any memory leak issues or anything like that? I'm finding
> that the creation of some simple Transfer objects (like a user with an id
> field, name, address fields) are eating a megabyte of memory when created.
>

> --- On Sat, 8/1/09, James Allen <slingsho...@googlemail.com> wrote:

> > On Thu, Jul 16, 2009 at 9:51 AM, James Allen<slingsho...@googlemail.com>

> > wrote:
>
> > > It may be worth reinvestigating it Hal. Obviously the complexity of
> > objects
> > > has an effect but you may find things have improved with later Java
> > > revisions. Once Sun fixed the object instantiation bug CFC performance
> in
> > > CF8 improved dramatically...
>
> > > ---
> > > James Allen
> > > E: ja...@jamesallen.name
> > > Blog:http://jamesallen.name
> > > Twitter: @CFJamesAllen (Coldfusion / Web development)
> > > Twitter: @jamesallenuk (General)
>
> > > -----Original Message-----
> > > From: coldfu...@googlegroups.com
> [mailto:coldfu...@googlegroups.com]
> > > On Behalf Of Hal Helms
> > > Sent: 16 July 2009 14:47
> > > To: coldfu...@googlegroups.com
> > > Subject: [coldfusionoo] Re: Object creation performance in CF 9: any
> > better?
>
> > > 4000 objects per second on CF 8? Wow. That doesn't at all match my
> > > experience of working with complex objects. Perhaps the connections
> > > (aggregation, inheritance) is a limiting factor? Just a guess.
>

> > > On Thu, Jul 16, 2009 at 9:44 AM, Kevan Stannard<kevanstann...@gmail.com>

> > > wrote:
> > >> I had a quick look at this when I first fired up CF9. I performed a
> test
> > > of
> > >> just creating objects with only an init() function, so these numbers
> are
> > >> around the maximum I can get out of CF on my laptop (dual core win xp -
> > > just
> > >> a moderate machine). Adding functions to components will reduce the
> > >> performance.
> > >> On CF8 I was able to create 4,000 objects per second.
> > >> On CF9 Beta I was able to create around 25,000 objects per second. This
> > >> actually started off at a much lower number the first time I executed
> the
> > >> code (around 10,000 per second) but after running the test a few times
> it
> > >> found its way up to the 25,000 mark.
>
> > >> 2009/7/16 Matt Williams <mgw...@gmail.com>
>
> > >>> I would love to see some info on this too. It seems the question is
> > >>> going unanswered lately. It was asked 2 or 3 times on Forta's blog
> > >>> when he posted about ORM.
>

> http://forta.com/blog/index.cfm/2009/7/7/ORM-Rethinking-ColdFusion-Da...

[James Allen]

We're only in beta at the moment so a very small amount of traffic so far.
The whole site relies on Transfer completely.

[Jim]

Ok, I am about to go into beta on something as well that relies completely on Transfer. If you hit any caveats, please post them. I will do the same. I am a little iffy about some memory usage during creation of new objects that I've been seeing.

--- On Sat, 8/1/09, James Allen <sling...@googlemail.com> wrote:

[James Allen]

Yeah no problem. We launch next Friday so it will be interesting to see how the site holds up.

 

I’ll post any issues I come across though.

[Sean Corfield]

On Sat, Aug 1, 2009 at 3:30 AM, Jim<dotf...@changethings.org> wrote:
> Are you seeing any memory leak issues or anything like that? I'm finding
> that the creation of some simple Transfer objects (like a user with an id
> field, name, address fields) are eating a megabyte of memory when created.

That's not a memory leak (unless you eventually run out of heap space
because GC can't recover the memory).

Folks seem to worry unduly about memory usage in Java based apps. Java
is designed to have lots of objects created and - eventually - garbage
collected which can mean memory swells dramatically and may not go
back down for a long time. That's part of Java's design.

I've used Transfer on some pretty high traffic sites and, with the
caveat about making sure you're using the 1.1 FINAL release build (not
the RC), it's been rock solid in terms of memory (leaks). As long as
you've load tested stuff properly - and tuned the cache - you'll be
fine.
--
Sean A Corfield -- (904) 302-SEAN
Railo Technologies US -- http://getrailo.com/
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

[Jim]

Thanks Sean, I was actually assuming the memory leak was in the virtual machine, not because of Transfer itself. I'd read that some of the jvm releases had a few bugs and some other people had been experiencing this behavior. I thought perhaps there was a crossover in what I was experiencing. I was actually advised to use u13 instead of u14 (JRE 6) by a few different people. Anyway, I've investigated more and your comments are completely accurate. I've worked with Mark directly on my Transfer issues, I'm using the 1.1 Final but I do keep hitting heap space errors and perm gen errors. Some objects knock out 3 megs of ram on creation, and they are simple objects. It eventually stabilizes, but a medium amount of traffic starts kicking out perm gen. --- On Mon, 8/3/09, Sean Corfield <seanco...@gmail.com> wrote:

From: Sean Corfield <seanco...@gmail.com> Subject: [coldfusionoo] Re: Object creation performance in CF 9: any better? To: coldfu...@googlegroups.com

[Sean Corfield]

On Mon, Aug 3, 2009 at 11:37 PM, Jim<dotf...@changethings.org> wrote:
> I've worked with Mark directly on my Transfer issues, I'm using the 1.1
> Final but I do keep hitting heap space errors and perm gen errors.
>
> Some objects knock out 3 megs of ram on creation, and they are simple
> objects. It eventually stabilizes, but a medium amount of traffic starts
> kicking out perm gen.

Yes, combined with ColdFusion, it's certainly memory hungry! I
typically run ColdFusion/Transfer apps with 2-3GB heap and at least
512MB perm space.

[whostheJBoss]

I'm running Railo, not CF ;)

On Aug 3, 11:49 pm, Sean Corfield <seancorfi...@gmail.com> wrote:

> On Mon, Aug 3, 2009 at 11:37 PM, Jim<dotfus...@changethings.org> wrote:
> > I've worked with Mark directly on my Transfer issues, I'm using the 1.1
> > Final but I do keep hitting heap space errors and perm gen errors.
>
> > Some objects knock out 3 megs of ram on creation, and they are simple
> > objects. It eventually stabilizes, but a medium amount of traffic starts
> > kicking out perm gen.
>
> Yes, combined with ColdFusion, it's certainly memory hungry! I
> typically run ColdFusion/Transfer apps with 2-3GB heap and at least
> 512MB perm space.
> --
> Sean A Corfield -- (904) 302-SEAN

> Railo Technologies US --http://getrailo.com/
> An Architect's View --http://corfield.org/

[Sean Corfield]

On Tue, Aug 4, 2009 at 12:11 PM, whostheJBoss<dotf...@changethings.org> wrote:
> I'm running Railo, not CF ;)

It's gets very confusing when you keep switching between whostheJBoss and Jim!

My point still stands: Transfer is pretty memory hungry.

--
Sean A Corfield -- (904) 302-SEAN

Railo Technologies US -- http://getrailo.com/
An Architect's View -- http://corfield.org/

[Jim]

It's not intentional :) When I reply from email it says Jim, when I post through Google Groups it puts whostheJBoss

--- On Wed, 8/5/09, Sean Corfield <seanco...@gmail.com> wrote:

From: Sean Corfield <seanco...@gmail.com> Subject: [coldfusionoo] Re: Object creation performance in CF 9: any better? To: coldfu...@googlegroups.com

[Greg]

Hey James,

How did the launch go?

On Aug 1, 6:00 am, "James Allen" <slingsho...@googlemail.com> wrote:
> Yeah no problem. We launch next Friday so it will be interesting to see how
> the site holds up.
>
> I'll post any issues I come across though.
>
> ---
>
> James Allen
>
> E: ja...@jamesallen.name
>
> Blog:http://jamesallen.name
>
> Twitter: @CFJamesAllen (Coldfusion / Web development)
>
> Twitter: @jamesallenuk (General)
>
> From: coldfu...@googlegroups.com [mailto:coldfu...@googlegroups.com]
> On Behalf Of Jim
> Sent: 01 August 2009 12:39
> To: coldfu...@googlegroups.com
> Subject: [coldfusionoo] Re: Object creation performance in CF 9: any better?
>
> Ok, I am about to go into beta on something as well that relies completely
> on Transfer. If you hit any caveats, please post them. I will do the same. I
> am a little iffy about some memory usage during creation of new objects that
> I've been seeing.
>

[James Allen]

Hey there,

Ah very well. No problems at all so far. Only bug reports I have got have
been where GoogleBot has called a page directly that it shouldn't have done.
:)

We don't have a huge volume of traffic yet but things are moving nicely.
Good speed with everything loading from the cache nicely.

The site is http://errrrrrr.com if anyone wants to take a look.

---
James Allen
E: ja...@jamesallen.name
Blog: http://jamesallen.name
Twitter: @CFJamesAllen (Coldfusion / Web development)
Twitter: @jamesallenuk (General)

Lead developer of http://errrrrrr.com