A small Perl task
Leopold Toetsch
HOW IT WORKS
The basic idea is to detect when you are compiling exactly the
same code a 2nd time and use the pre-
viously compiled output. You detect that it is the same code by
forming a hash of:
o the pre-processor output from running the compiler with -E
o the command line options
o the real compilers size and modification time
o any stderr output generated by the compiler
These are hashed using md4 (a strong hash) and a cache file
is formed based on that hash result.
When the same compilation is done a second time ccache is able
to supply the correct compiler output
(including all warnings etc) from the cache.
So you get really fast recompiles *except* for classes/*.c. What I'd
like to have is (based on ccache's philosophy) a cache for
classes/*.dump and classes/*.c files. Currently they are recreated
permanently.
NB Fink created wrong permissions for ~/.ccache/*
A:
$ sudo chown -R user:user ~/.ccache
fixed it (if your login and group name is user ;)
leo
Jeff Clites
> First, if you don't have it yet done, install ccache.
Thanks for the tip--seems awesome.
> HOW IT WORKS
> The basic idea is to detect when you are compiling exactly the
> same code a 2nd time and use the pre-
> viously compiled output. You detect that it is the same code by
> forming a hash of:
...
> So you get really fast recompiles *except* for classes/*.c. What I'd
> like to have is (based on ccache's philosophy) a cache for
> classes/*.dump and classes/*.c files. Currently they are recreated
> permanently.
I started playing with this, got confused, and finally figured out why
I'm confused.
In the dump case, you basically (it seems) need to parse the pmc file
in order to determine the parent (chain), in order to determine the
dependencies, in order to figure out what you need to put into your
digest. (In the ccache case, it's the preprocessor doing this for you.)
But by then, you've done most of the work I'd think (by parsing the pmc
file). So at that point, you'd end up calculating a digest over a bunch
of files, to avoid the Dumper overhead, and the latter may likely be
faster. So this might not be a win for us, since it doesn't look like
it would let us skip much.
I think this will only be worth doing if we have a faster way to
determine the dependencies (basically the parents of the pmc in
question). The Makefile implicitly knows this, so it might be possible
for it to pass this into the script.
But I have the caching and use-the-best-digest-module logic worked out,
I'm just stuck on figuring out what work it can skip. Let me know if
you have any ideas, or if I'm missing something.
I guess maybe the *.{c,h} case is simpler, since there you only depend
on your *.dump files (plus the script itself, etc.), though there's
more work to figure out what files you're trying to generate (in the
library case, etc.).
JEff
Leopold Toetsch
> On Oct 23, 2004, at 5:14 AM, Leopold Toetsch wrote:
>> First, if you don't have it yet done, install ccache.
> Thanks for the tip--seems awesome.
Welcome.
> In the dump case, you basically (it seems) need to parse the pmc file
> in order to determine the parent (chain), in order to determine the
> dependencies, in order to figure out what you need to put into your
> digest. (In the ccache case, it's the preprocessor doing this for you.)
> But by then, you've done most of the work I'd think (by parsing the pmc
> file).
Probably yes. But you could check all the time stamps of the parents. If
all parent pmcs are older then the cached .c and .dump files, you could
just serve from the pmc-cache. This is probably doable by Makefile-only
rules too, but then a "make clean" still does the full recreation. And
"make clean" is commonly needed due to insufficient dependencies, which
is another issue.
> JEff
leo