Building deal.II with ccache

[Jean-Paul Pelteret]

Dear all,

I've recently become aware of ccache, although I have no experience with it whatsoever. Is this something that may benefit us when it comes to recompiling deal.II during development? It looks like it might be quite easy to integrate using cmake.

Best regards,

J-P

[Matthias Maier]

On Tue, Jan 3, 2017, at 01:45 CST, Jean-Paul Pelteret <jppel...@gmail.com> wrote:

> Dear all,
>

> I've recently become aware of ccache <https://ccache.samba.org/>, although

> I have no experience with it whatsoever. Is this something that may benefit
> us when it comes to recompiling deal.II during development? It looks like
> it might be quite easy to integrate

> <https://crascit.com/2016/04/09/using-ccache-with-cmake/> using cmake
> <https://github.com/xbmc/xbmc/blob/master/cmake/modules/FindCCache.cmake>.

Yes, I have used ccache extensively in the past (well as a Gentoo Linux
user one eventually tries all that stuff). My experience with it is a
bit dated, though. If you have a fast SSD ccache might be beneficial
again - a while ago with slower HDDs it was faster to simply compile
from scartch in a tmpfs.

Due to the way how ccache works I doubt you'll gain much from it for
development, though. As soon as you change one source file in the
include chain, you'll have to recompile everything (transitively)
including it - whether you use ccache, or not.


If you want to give ccache a try, you can simply configure with

CC="ccache gcc" CXX="ccache g++" cmake -DWITH_MPI=OFF [...]


Best,
Matthias

[Jean-Paul Pelteret]

Hi Matthias,

Thanks for your reply and insight. Its a shame that its not going to be particularly useful for recompiling deal.II. I suppose I'll keep it in mind for other smaller projects then.

Cheers,

J-P

[Matthias Maier]

On Tue, Jan 3, 2017, at 06:43 CST, Matthias Maier <tam...@43-1.org> wrote:

> On Tue, Jan 3, 2017, at 01:45 CST, Jean-Paul Pelteret <jppel...@gmail.com> wrote:
>
>> Dear all,
>>
>> I've recently become aware of ccache <https://ccache.samba.org/>, although
>> I have no experience with it whatsoever. Is this something that may benefit
>> us when it comes to recompiling deal.II during development? It looks like
>> it might be quite easy to integrate
>> <https://crascit.com/2016/04/09/using-ccache-with-cmake/> using cmake
>> <https://github.com/xbmc/xbmc/blob/master/cmake/modules/FindCCache.cmake>.
>
> Yes, I have used ccache extensively in the past (well as a Gentoo Linux
> user one eventually tries all that stuff). My experience with it is a
> bit dated, though. If you have a fast SSD ccache might be beneficial
> again - a while ago with slower HDDs it was faster to simply compile
> from scartch in a tmpfs.

Due to some recent discussion, I tried ccache again and I totally
changed my opinion.

I set up a ccache backing store on an NVME SSD. With it I get the
following results:

First run:

ninja 7208.87s user 128.74s system 730% cpu 16:44.97 total

Second run:

ninja 45.14s user 8.44s system 298% cpu 17.956 total


Yes, that's 18 seconds instead of 17 minutes.

Further, configuring from scratch improved from 17 seconds to 10
seconds.


If you want to use ccache together with ninja (and for example clang),
create two files:

ccache-clang:
#!/bin/sh
exec /usr/bin/ccache clang "${@}"

ccache-clang++:
#!/bin/sh
exec /usr/bin/ccache clang++ "${@}"

and run configure with
$ cmake -DCMAKE_C_COMPILER="ccache-clang" -DCMAKE_CXX_COMPILER="ccache-clang++" -GNinja ...


Best,
Matthias

[Timo Heister]

> Due to some recent discussion, I tried ccache again and I totally
> changed my opinion.

So in what situation would you profit from using ccache? I thought it
only caches the output if none of the inputs change (which never
happens unless you do a "make clean" for whatever reason)?

--
Timo Heister
http://www.math.clemson.edu/~heister/

[Matthias Maier]

On Fri, Mar 10, 2017, at 17:02 CST, Timo Heister <hei...@clemson.edu> wrote:

>> Due to some recent discussion, I tried ccache again and I totally
>> changed my opinion.
>
> So in what situation would you profit from using ccache? I thought it
> only caches the output if none of the inputs change (which never
> happens unless you do a "make clean" for whatever reason)?

Yes.

But you might end up reusing the very same "compilate" (i.e. same header
files and source code) quite a number of times. For example in the
testsuite.

But also (at least for me) regularly during development. I have a habit
of deleting build directories, or creating multiple.

Further, we also make it unnecessarily hard by always including a
config.h file containing all configuration options (but luckily that
file tends to not change to much).

As I said - what is remarkable is the fact that the overhead ccache
creates is *much* less than I remember from ~10 years ago.

Best,
Matthias

[Martin Kronbichler]

Just to give another perspective on ccache: I also used ccache but I
have gone away from it a while ago. I realized that I had a hit rate
around 5-10% the way I was using it. Yes, it is sometimes cool to be
able to switch branches and "re-compile" deal.II in half a minute rather
than 6 minutes. But I don't do that often enough for it to pay off.
Furthermore, I don't consider ccache on an SSD particularly helpful for
the test suite: You generate 10-15 GB of code that needs to be cached.
Run the test suite in a few variants with some changes in header files
and you need 50GB+ memory (it's actually the writes that matter). Since
doing heavy development with dozens of GB written per day to the SSD is
already close to the border of what a SSD can sustain for a couple of
years, I'm not overly keen in doubling that data. I could buy a better
(enterprise-level) SSD, but I find it makes more sense to put the money
on more cores that speed up the other 90% of compilation units as well...

[Matthias Maier]

On Sun, Mar 12, 2017, at 13:21 CDT, Martin Kronbichler <kronbichl...@gmail.com> wrote:

> Just to give another perspective on ccache: I also used ccache but I
> have gone away from it a while ago. I realized that I had a hit rate
> around 5-10% the way I was using it. Yes, it is sometimes cool to be
> able to switch branches and "re-compile" deal.II in half a minute
> rather than 6 minutes. But I don't do that often enough for it to pay
> off.

Thanks for the info. 5-10% is a bit too little to be practical.

Guess I was a bit too exited by the fact that overhead induced by
the caching mechanism is almost negligible compared to what you could
got with hardware around 2006/2007.


Best,
Matthias