auto_compile: smerl:compile is too slow

[Michael Mullis]

Hi all.
I've been digging down to the slowness when auto_compile set to true
and it looks like the call to smerl:compile is pretty slow and is
always being called even if nothing else changed.
To make matters worse, it gets called multiple times when there is a
redirect.
Its in erlyweb_compile:compile/2 near the end.

Everything else seems to be relatively fast so if we could solve this,
auto_compile would be much more tolerable.
I've not had time to figure out a solution so if anyone wants to
contribute, that would be great.

thanks,
michael.

[Yariv Sadan]

There should be a timeout interval in auto_compile so if less than
some number of milliseconds have passed it won't attempt to recompile.
This prevents multiple auto-compilations when you're loading a page
that loads multiple assets in different requests. You can try to make
it a longer interval. I don't know of any way to make auto compilation
faster. It's fast enough to not bother me though.

Yariv

[Michael Mullis]

Sorry to say, the last_compile_time check is ineffective in this
situation.

For reference, I'm running Ubuntu desktop on a Q6600 with 4 gig ram.
Erlang is version 5.6.4, [64-bit], [smp:4], [hipe].
I just started working on a simple app and there is no real code in
any of the controllers or views.

Here's all the things that seem to contribute to this issue.
1. The current code checks if last compile time was more than 3
(hardcoded) seconds ago.
2. There are 10 components - all views and controllers using
erlyweb_magic which affects size of <appname>_erlyweb_data
3. smerl:compile on <appname>_erlyweb_data takes close to 6 seconds

The long compile means it slips through the last_compile_time check
and will recompile multiple times when there are redirects - meaning
12+ second waits.

I removed all the erlyweb_magic and this drops the erlyweb_data
compile time down significantly
but that defeats the purpose. I'm trying to get a simple app up and
running and want to use
as much leverage as possible.

The erlyweb_data is dependent on whether something in the controllers/
views changed
so here's some potential solutions.
1. Track whether a controller/view was recompiled and force recompile
on erlyweb_data if needed

2. Delete the erlyweb_data beam file when a controller/view is
recompiled and then later trigger erlyweb_data recompile if the beam
isn't there

3. Check timestamps on the controller/view beam files and recompile
erlyweb_data if it doesn't have the most current timestamp

Other ideas? Did I miss something?

thanks,
michael.

[Yariv Sadan]

Turn off hipe. It has much slower compilation, and it's unnecessary in
dev mode. Does that help?

[Michael Mullis]

That's a big difference. I only played with the app compile options
since they control the options used in recompilation.
I never even thought about the native compile being an issue. Perhaps
I'm spoiled.

I implemented most of the timestamp checking approach before playing
with the compile options so now
I'm on the fence about it's value since it's unlikely anyone would use
auto_compile in production.

Thanks for the tip,
michael

On Dec 4, 1:13 pm, "Yariv Sadan" <yarivsa...@gmail.com> wrote:
> Turn off hipe. It has much slower compilation, and it's unnecessary in
> dev mode. Does that help?
>
> On Wed, Dec 3, 2008 at 7:10 AM, Michael Mullis
>