ccache

[Ben Goodrich]

Today I just discovered ccache

https://ccache.samba.org/

and would recommend it to anyone who tracks the git repository (including Jenkins, if we can get ccache working on 64-bit Windows). Essentially, it caches compilations so that they can be reused if the file doesn't change (even if you do make clean-all beforehand).

After installing and configuring, run it once serially so that it can cache Stan's C++ files

goodrich@CYBERPOWERPC:/opt/stan$ make clean-all && time make CC="ccache g++" O=3 test-models > /tmp/g++_caching.log
...
real    93m52.243s
user    87m29.122s
sys     4m58.650s

Now files are cached:

goodrich@CYBERPOWERPC:/opt/stan$ ccache -s
cache directory                     /home/goodrich/.ccache
cache hit (direct)                     0
cache hit (preprocessed)               0
cache miss                           158
called for link                      147
unsupported compiler option           11
files in cache                       464
cache size                         163.3 Mbytes
max cache size                       1.0 Gbytes

 
Now, you can henceforth do it in parallel much faster and with much more limited RAM spikes

goodrich@CYBERPOWERPC:/opt/stan$ make clean-all && time make CC="ccache g++" O=3 -j8 test-models > /tmp/g++_cached.log
...
real    11m48.354s
user    58m20.964s
sys     2m42.890s

goodrich@CYBERPOWERPC:/opt/stan$ ccache -s
cache directory                     /home/goodrich/.ccache
cache hit (direct)                   145
cache hit (preprocessed)               8
cache miss                           163
called for link                      294
unsupported compiler option          417
files in cache                       456
cache size                         159.2 Mbytes
max cache size                       1.0 Gbytes

We can save a couple more minutes once I make an unrelated tweak to the lsat test. But it already takes only half the time as a parallel run without caching

goodrich@CYBERPOWERPC:/opt/stan$ make clean-all && time make CC=g++ O=3 -j8 test-models > /tmp/g++_nocached.log
...
real    23m3.460s
user    140m26.479s
sys     4m8.016s

I don't think ccache works properly with clang++ yet, but clang++ is already decently fast on test-models.

Ben

[Jiqiang Guo]

I am using this for building rstan so that if I just modified part of source or only R files, it is fast to build the package and test it.

But I did not find something that works for windows.  Please let me know if someone make it work on windows.

--

Jiqiang

Ben

--
 
 

[Ben Goodrich]

Apparently, the entire world has been using it for a decade, but I didn't get the memo. I think it could also be useful for testing rstan's ability to run BUGS models, except that the .cpp file is unique each time because it uses the name of the tempfile() for the name of the inherited class. Would it be possible to use the model_name for the inherited class if it is given and only use a tempfile() when model_name = "anon_model"?

Ben

[Jiqiang Guo]

I used the tempfile() to overcome a problem in Rcpp, which seems do not allow me use the same name in different shared libraries.  So ideally I want that name to be unique.

But the issue here is the another tempfile() gets called by package inline to create the temporary cpp file to be compiled using R CMD SHLIB.

--

Jiqiang

--
 
 

[Ben Goodrich]

On Saturday, October 6, 2012 1:17:32 PM UTC-4, Jiqiang Guo wrote:

I used the tempfile() to overcome a problem in Rcpp, which seems do not allow me use the same name in different shared libraries.  So ideally I want that name to be unique.

Right. It would definitely be bad to have multiple shared libraries trying to call anon_model. But the question becomes can we largely avoid that in a more flexible manner. Maybe making the default model_name be paste0("anon_model_", tempfile()) and using the model_name for the class? I guess that is a problem if users want to have two models called "rats" in the same R session, but it seems like we could just require them to use different names.
 

But the issue here is the another tempfile() gets called by package inline to create the temporary cpp file to be compiled using R CMD SHLIB.

As far as I understand, that is not actually a problem because ccache matches on the file hash rather than the file name. So inline can randomly name the .cpp file whatever it wants, and it will still have the same hash unless the source has been changed.

Ben

[Jiqiang Guo]

I cannot ask user to use different names as I cannot check that.  It would be weird and hard to figure out problem when the model actually used is not what we are seeing.

But I can give an option, defaults to FALSE, that allow to use user supplied model name instead of randomly generated. (Actually it is the RCPP module name that need be unique, which is the model name in current Rstan. But that still means code change.)

Jiqiang

--
 
 

[Ben Goodrich]

On Saturday, October 6, 2012 2:26:26 PM UTC-4, Jiqiang Guo wrote:

But I can give an option, defaults to FALSE, that allow to use user supplied model name instead of randomly generated. (Actually it is the RCPP module name that need be unique, which is the model name in current Rstan. But that still means code change.)

Yes, I think that would work to have another argument module_name that defaults to model_name. I can currently do 77 out of 84 models in 12.5 minutes with clang++ and mclapply(). If we could get it to work with ccache and g++, it would probably be 6 or 7 minutes, which is maybe small enough to do regularly.

Ben

[Jiqiang Guo]

Well, :( I made the change (add an option obfuscate_model_name in the ... argument for stan/stanc), and found that the cached object by ccache was not used. 

The reason is that function cxxfunction in package inline we used to compile C++ code is written to create a function by specifying its body, but the name of this function in C++ is the temporary file name.  For example, this is the tiny difference for the two runs of stan():

$ diff filea6be4da0a492.cpp filea6be5ac377ae.cpp

320c320

< SEXP filea6be4da0a492( ) ;

---

> SEXP filea6be5ac377ae( ) ;

325c325

< SEXP filea6be4da0a492(  ){

---

> SEXP filea6be5ac377ae(  ){

--

Jiqiang

--
 
 

[Jiqiang Guo]

By the way, since we are the only users who might want to use the new feature I added, we can hack package inline a little bit (change the source and reinstall) so that the goal can be reached. 

--

Jiqiang