onTap under load
John Blayter
did it perform?
John Blayter
(303) 325-1979
jo...@blayter.com
http://www.blayter.com/john/
Isaac Dealey
> did it perform?
Hi John,
I'm not aware of any performance metrics with recent versions. I can
tell you that a long time ago (insert Star Wars reference here), I had
created a small blog application in the onTap framework, Mach-II,
Fusebox 3 and if I remember correctly Fusebox 4... This was before
Model-Glue and ColdBox, hence the selection.
I had done some tick-counting myself that wasn't real load testing and
discovered at the time that it didn't perform as well as I'd hoped (a
little heavier than Mach-II) and so I did some performance tuning and
brought it in line with Mach-II before I published my findings.
Someone else (and forgive me I can't remember who), put my source on a
server where they could use the Microsoft Web Stress Analyzer on it.
When he published his findings, what he found was that at the time
Mach-II scaled up in a fairly linear fashion, i.e. more traffic = more
stress on the server. The interesting result was that the onTap
framework was slow with small numbers of users, but scaled up very well
compared to Mach-II... So that by the time there were say 20 concurrent
users, Mach-II had slowed down a fair amount, but the onTap framework
version of the same app had scarcely changed at all.
The problem with using that as a metric is that was a really old version
of the framework. And I know for certain that version 3.2 performs MUCH
better with smaller numbers of users (particularly on CF8 which can take
advantage of the lazy-loading libraries). But that still leaves the
question of how well the new version scales up. I would expect it to
scale up at least equally as well as that old version, but without being
able to set up a server to perform the metrics testing I can't give any
hard data. :)
I would love to get some metrics from someone who's got a server they
can use for load testing. :) I would think the recent Galleon forums
project would be a good place to start, although it was done with
version 2.5 of the framework (no lazy-library), so I'd want to merge it
with version 3.2 of the core framework, and I would probably also want
to remove the ORM features (which is easy, 'cause it's all in the CFCs)
to get a one-to-one comparison with the others because the others
weren't using any ORM tools.
--
[ ike ] founder - onTap framework
phone: 781.769.0723
John Blayter
I am in the process of going through and evaluating various frameworks for a high availability application. I will talk to my boss and the QA manager to see if we can't load up your app and do the tests for you and publish the results to the community. I need the results from the test to make my decision and I think having an app written in onTap, Mach II, FB 5.5 and MG would be an awesome test to see how the underlying frameworks stack up.
Thanks,
Isaac Dealey
>
> I am in the process of going through and evaluating various
> frameworks for a high availability application. I will talk to my boss
> and the QA manager to see if we can't load up your app and do the
> tests for you and publish the results to the community. I need the
> results from the test to make my decision and I think having an app
> written in onTap, Mach II, FB 5.5 and MG would be an awesome test to
> see how the underlying frameworks stack up.
>
> Thanks,
Welcome. Let me know if you have any questions about how to remove the
ORM and merge the latest version of onTap back into that Galleon port...
p.s. A couple other things to note,
you might want to have a bit of a closer look at the Mach-II and
Model-Glue ports because they weren't really completed. I didn't write
real controllers / event-handlers for them, so you might want to get
some advice from the MII / MG communities about how to do that. I'd been
hoping to get some input from them to finish out those examples, but
nobody offered to help, so that's where they got left... I'd already
done 2-3 weeks or so of work on them at that point.
Also the ColdBox example was an earlier port that Luis Majano had done
and may not be the latest version of ColdBox, as a matter of fact I'm
fairly certain it's not. And the view templates for Galleon had changed
a fair amount. I don't think it will change the performance metrics much,
since that's mostly html / css, so the variations should be negligible
as long as the load-testing tool isn't running into any errors being
thrown in those views. (And errors are a good possibility because there
were some database schema changes and the like.)
Other than that, I'd like to say thank you. :) I'd like to see the
results myself. It may be proof that the framework is scalable or it may
indicate that I need to do more performance tuning. Either way I'll be
glad to know the result. :)