painfully slow development mode on large project
Tony Buser
it gets the slower development mode becomes, but runs really fast in
production mode. So I assume the slowdown at the beginning of each
request is caused by the fact that it reloads everything. It currently
has about 85 models, 75 controllers, and 11k Code LOC.
It's becoming really painful. For instance, I'm working on a page that
involves a lot of ajax calls. One call shows the following in the
development.log:
Completed in 0.38060 (2 reqs/sec) | Rendering: 0.02327 (6%) | DB:
0.06884 (18%)
But firebug timed it at 5.38 seconds. :( So that means reloading my
project (+ I guess javascript and firefox overhead, but all it's doing
is validating values on a form and enabling/disabling a button) added an
additional 5 seconds?? And it keeps getting slower over time if I don't
restart the server often. And I'm on an a MacBookPro with 2 gig of ram.
Some of the other developers here aren't quite as lucky as me and it's
even more painful for them.
At this point it's almost making more sense to run with cache_classes =
true and just restart mongrel when needed. I even considered writing a
program to monitor files in my project and restart the server when one
changes... another thought was to try and split up the app into separate
smaller apps and share things between them, however there's so much
shared logic and model relationships, that I don't think it would be
feasible.
Is there anything else I can do to optimize the development environment?
Is this a common thing with large rails projects or could there be
something I'm doing wrong? What could I do to try and track down the
slowness in development?
--
Posted via http://www.ruby-forum.com/.
Philip Hallstrom
> it gets the slower development mode becomes, but runs really fast in
> production mode. So I assume the slowdown at the beginning of each
> request is caused by the fact that it reloads everything. It currently
> has about 85 models, 75 controllers, and 11k Code LOC.
>
> It's becoming really painful. For instance, I'm working on a page that
> involves a lot of ajax calls. One call shows the following in the
> development.log:
>
> Completed in 0.38060 (2 reqs/sec) | Rendering: 0.02327 (6%) | DB:
> 0.06884 (18%)
>
> But firebug timed it at 5.38 seconds. :( So that means reloading my
What web server? It would seem that Rails is done with the request pretty
quickly... so the extra time is coming from somewhere else.
Any chance you're using webrick? I could see that happening with a lot of
ajax as webrick can only handle one request at a time...
Our app is about 43,000 lines and has 832 classes and it's not that
slow... we're running litespeed for development...
Tony Buser
> Any chance you're using webrick? I could see that happening with a lot
> of
> ajax as webrick can only handle one request at a time...
I'm using mongrel. Regular pages take about 2-3 seconds longer to load
then what is shown in development.log. The slow down occurs at the
beginning of the request before anything even shows up in the log. It's
like I hit a link, count to 3, then BAM lots of stuff shows up in log
and page comes in. On pages that might do multiple ajax calls at the
same time, it does tend to take longer. Is mongrel able to handle more
then one request at a time?
> Our app is about 43,000 lines and has 832 classes and it's not that
> slow... we're running litespeed for development...
How much of a delay for each request do you experience in development
mode?
Lionel Bouton
> Philip Hallstrom wrote:
>
>> Any chance you're using webrick? I could see that happening with a lot
>> of
>> ajax as webrick can only handle one request at a time...
>>
>
> I'm using mongrel. Regular pages take about 2-3 seconds longer to load
> then what is shown in development.log. The slow down occurs at the
> beginning of the request before anything even shows up in the log. It's
> like I hit a link, count to 3, then BAM lots of stuff shows up in log
> and page comes in. On pages that might do multiple ajax calls at the
> same time, it does tend to take longer. Is mongrel able to handle more
> then one request at a time?
>
Not exactly (the Rails processing can't be done concurrently, but
serving files in <rails_app>/public can). If there is such a delay with
near 0 CPU usage, you may want to use strace to see what syscall Mongrel
is waiting the result for, it might be a DNS request, a file opening,
... This should point you in the right direction.
Lionel
Faisal N Jawdat
very problem:
http://svn.techno-weenie.net/projects/plugins/
dev_mode_performance_fixes/
> Why don't we only reload the files when they have actually
> changed? Store the file names and their last modified time and
> then stat all the files before every request... if any are newer,
> kill the objects they originally defined and let the Dependency
> auto-loading code reload the objects from disk.
People are reporting anywhere between "4x" and "almost as fast as
production" speed improvements.
-faisal
Lionel Bouton
> As it happens, Josh Goebel just released a plugin to deal with this
> very problem:
>
> http://svn.techno-weenie.net/projects/plugins/
> dev_mode_performance_fixes/
>
>
controllers than Tony's project and webrick on a 1.06GHz Centrino Duo is
fast enough for me (nearly all actions take less than 1s and most are
around 0.3s, 3-4 whole seconds seems odd to me unless he uses something
like a Pentium MMX or less for the server), so I don't think his problem
is class reloading (or he must have a pretty peculiar code). Won't hurt
to try though.
Lionel
Philip Hallstrom
>> Any chance you're using webrick? I could see that happening with a lot
>> of
>> ajax as webrick can only handle one request at a time...
>
> I'm using mongrel. Regular pages take about 2-3 seconds longer to load
> then what is shown in development.log. The slow down occurs at the
> beginning of the request before anything even shows up in the log. It's
> like I hit a link, count to 3, then BAM lots of stuff shows up in log
> and page comes in. On pages that might do multiple ajax calls at the
> same time, it does tend to take longer. Is mongrel able to handle more
> then one request at a time?
One Rails request at a time. Static is threaded...
>> Our app is about 43,000 lines and has 832 classes and it's not that
>> slow... we're running litespeed for development...
>
> How much of a delay for each request do you experience in development
> mode?
Hardly any. Obviously it takes longer to load since it's dev and we're
not caching anything, but it still comes up...
Someone else mentioned DNS... that might be worth looking into as well.
-philip
Tony Buser
> http://svn.techno-weenie.net/projects/plugins/
> dev_mode_performance_fixes/
>
> People are reporting anywhere between "4x" and "almost as fast as
> production" speed improvements.
Great googly moogly!! I owe you and the guy that wrote that a beer...
actually I owe you 10 beers because that plugin made my development mode
at least 10x faster! It's like night and day. Practically as fast as
production mode. I need to do some more tests to see if it breaks
anything, but so far everything is passing and working great. Amazing,
why on earth doesn't rails do this by default? I'm blown away... there
must be some downside to this.
Philip: I really would like to know what you're doing differently then I
am that would allow a project as large as yours to run so fast. I
suppose it's possible I'm doing something bad in a class somewhere that
causes the slowness, but I'm having a hell of a time tracking it down.
I tried running it through strace on a linux box, but there doesn't
appear to be any one spot where it sits waiting for something. I do see
a TON of calls trying to stat files that don't exist (-1 ENOENT (No such
file or directory)) that has me confused... but I think it is rails
trying to autoload files and looking in every possible place it could
be?