sccache as ccache

[Ted Mielczarek]

Yesterday I published sccache 0.2 to crates.io, so you can now `cargo
install sccache` and get the latest version (it'll install to
~/.cargo/bin). If you build Firefox on Linux or OS X you can (and
should) use sccache in place of ccache for local development. It's as
simple as adding this to your mozconfig (assuming sccache is in your
$PATH):

ac_add_options --with-ccache=sccache

The major benefit you gain over ccache is that sccache can cache Rust
compilation as well, and the amount of Rust code we're adding to Firefox
is growing quickly. (We're on track to enable building Stylo by default
soon, which will add quite a bit of Rust.)

On my several-year-old Linux machine (Intel(R) Core(TM) i7-3770 CPU @
3.40GHz, 32GB, SSD), if I build; clobber; build with sccache enabled the
second (fully-cached) build completes in just over 4 minutes:

4:11.92 Overall system resources - Wall time: 252s; CPU: 69%; Read
bytes: 491520; Write bytes: 6626512896; Read time: 60; Write time:
1674852

sccache still isn't completely straightforward to use on Windows[1] but
I aim to fix that this quarter so that using it there will be just as
simple as on other platforms.

-Ted

1. https://bugzilla.mozilla.org/show_bug.cgi?id=1318370

[Alex Gaynor]

If you're on macOS, you can also get sccache with `brew install sccache`.

Alex

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Kan-Ru Chen]

On Wed, Jul 26, 2017, at 09:05 PM, Ted Mielczarek wrote:
> Yesterday I published sccache 0.2 to crates.io, so you can now `cargo
> install sccache` and get the latest version (it'll install to
> ~/.cargo/bin). If you build Firefox on Linux or OS X you can (and
> should) use sccache in place of ccache for local development. It's as
> simple as adding this to your mozconfig (assuming sccache is in your
> $PATH):
>
> ac_add_options --with-ccache=sccache
>
> The major benefit you gain over ccache is that sccache can cache Rust
> compilation as well, and the amount of Rust code we're adding to Firefox
> is growing quickly. (We're on track to enable building Stylo by default
> soon, which will add quite a bit of Rust.)
>
> On my several-year-old Linux machine (Intel(R) Core(TM) i7-3770 CPU @
> 3.40GHz, 32GB, SSD), if I build; clobber; build with sccache enabled the
> second (fully-cached) build completes in just over 4 minutes:
>
> 4:11.92 Overall system resources - Wall time: 252s; CPU: 69%; Read
> bytes: 491520; Write bytes: 6626512896; Read time: 60; Write time:
> 1674852
>
> sccache still isn't completely straightforward to use on Windows[1] but
> I aim to fix that this quarter so that using it there will be just as
> simple as on other platforms.

Windows support sounds very exciting! Will it support cache sharing?

Kanru

[Ted Mielczarek]

On Wed, Jul 26, 2017, at 10:46 AM, Kan-Ru Chen wrote:
> Windows support sounds very exciting! Will it support cache sharing?

Currently sccache supports a few different cache storage backends:
* local disk
* Amazon S3
* Google Cloud Storage
* Redis

However, the cache keys currently wind up with full source paths
included, so it's hard to get cache hits across machines when using
different source directories. There's an sccache issue filed on this
(and a patch sitting in the pull requests, actually), so it might be
possible to make that work.

-Ted

[Botond Ballo]

On Wed, Jul 26, 2017 at 9:05 AM, Ted Mielczarek <t...@mielczarek.org> wrote:

> If you build Firefox on Linux or OS X you can (and
> should) use sccache in place of ccache for local development.

Can sccache be used in conjunction with icecc [1]?

I currently use the two together by having the following in my .mozconfig:

ac_add_options --with-ccache
mk_add_options 'export CCACHE_PREFIX=icecc'
mk_add_options 'export CCACHE_CPP2=yes'

Is there an equivalent / similar setup with sccache?

Thanks,
Botond

[1] https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream

[Simon Sapin]

On 26/07/2017 15:05, Ted Mielczarek wrote:
> ac_add_options --with-ccache=sccache

When used together with icecc, this appears to force all jobs to run
locally which makes icecc pointless.

I’ve ended up keeping "classic" ccache for C and C++ code and adding
'export RUSTC_WRAPPER=sccache' to my mach wrapper script in order to use
sccache for Rust code. (Having this line with or without 'export' in
.mozconfig did not appear to do anything. Can mozconfig set arbitrary
environment variables?)

--
Simon Sapin

[Ted Mielczarek]

On Wed, Jul 26, 2017, at 12:57 PM, Simon Sapin wrote:
> On 26/07/2017 15:05, Ted Mielczarek wrote:
> > ac_add_options --with-ccache=sccache
>
> When used together with icecc, this appears to force all jobs to run
> locally which makes icecc pointless.

We should figure out what's going on here and see if we can fix it. It
would be nice to make this work properly.

> I’ve ended up keeping "classic" ccache for C and C++ code and adding
> 'export RUSTC_WRAPPER=sccache' to my mach wrapper script in order to use
> sccache for Rust code. (Having this line with or without 'export' in
> .mozconfig did not appear to do anything. Can mozconfig set arbitrary
> environment variables?)

No, mozconfig variable setting is sort of restricted. Bare variable
assignments (with or without export) are evaluated in the context of
configure, but don't survive to the Makefile environment. If you write
it as `mk_add_options export RUSTC_WRAPPER=sccache` that ought to work,
since that will be set as an exported Makefile variable, meaning it will
be set in the environment for commands executed by make.

-Ted

[Ben Kelly]

On Wed, Jul 26, 2017 at 9:05 AM, Ted Mielczarek <t...@mielczarek.org> wrote:

> Yesterday I published sccache 0.2 to crates.io, so you can now `cargo
> install sccache` and get the latest version (it'll install to
> ~/.cargo/bin).
>

I tried this on my linux build machine today and got:

error: failed to run custom build command for `openssl-sys v0.9.15`
process didn't exit successfully:
`/tmp/cargo-install.FAs9llrjaqDW/release/build/openssl-sys-a543e0ede317714a/build-script-build`
(exit code: 101)
--- stdout
cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
cargo:rerun-if-env-changed=OPENSSL_DIR
run pkg_config fail: "`\"pkg-config\" \"--libs\" \"--cflags\" \"openssl\"`
did not exit successfully: exit code: 1\n--- stderr\nPackage openssl was
not found in the pkg-config search path.\nPerhaps you should add the
directory containing `openssl.pc\'\nto the PKG_CONFIG_PATH environment
variable\nNo package \'openssl\' found\n"

--- stderr
thread 'main' panicked at '

Could not find directory of OpenSSL installation, and this `-sys` crate
cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it, you can set the `OPENSSL_DIR` environment variable for
the
compilation process.

If you're in a situation where you think the directory *should* be found
automatically, please open a bug at https://github.com/sfackler/rust-openssl
and include information about your system as well as this message.

$HOST = x86_64-unknown-linux-gnu
$TARGET = x86_64-unknown-linux-gnu
openssl-sys = 0.9.15

',
/home/bkelly/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.15/
build.rs:198
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Build failed, waiting for other jobs to finish...
error: failed to compile `sccache v0.2.0`, intermediate artifacts can be
found at `/tmp/cargo-install.FAs9llrjaqDW`

[Ben Kelly]

Looks like I needed to run this as a pre-req on ubuntu:

sudo apt-get install pkg-config libssl-dev

[Ted Mielczarek]

On Wed, Aug 2, 2017, at 12:26 PM, Ben Kelly wrote:
> On Wed, Jul 26, 2017 at 9:05 AM, Ted Mielczarek
> <t...@mielczarek.org> wrote:>> Yesterday I published sccache 0.2 to crates.io, so you can now `cargo>> install sccache` and get the latest version (it'll install to
>> ~/.cargo/bin).
>
> I tried this on my linux build machine today and got:
>
> error: failed to run custom build command for `openssl-sys v0.9.15`

<...>

You need to install the `libssl-dev` package (on Ubuntu) or the
equivalent on other distros. Sorry, wish this was clearer!
-Ted

[Simon Sapin]

On 26/07/2017 19:22, Ted Mielczarek wrote:
>> I’ve ended up keeping "classic" ccache for C and C++ code and adding
>> 'export RUSTC_WRAPPER=sccache' to my mach wrapper script in order to use
>> sccache for Rust code. (Having this line with or without 'export' in
>> .mozconfig did not appear to do anything. Can mozconfig set arbitrary
>> environment variables?)

> No, mozconfig variable setting is sort of restricted. Bare variable
> assignments (with or without export) are evaluated in the context of
> configure, but don't survive to the Makefile environment. If you write
> it as `mk_add_options export RUSTC_WRAPPER=sccache` that ought to work,
> since that will be set as an exported Makefile variable, meaning it will
> be set in the environment for commands executed by make.

`mk_add_options export RUSTC_WRAPPER=sccache` causes an error:

ValueError: dictionary update sequence element #1 has length 1; 2 is
required

File "/home/simon/gecko/python/mozbuild/mozbuild/base.py", line 342,
in resolve_config_guess
make_extra = dict(m.split('=', 1) for m in make_extra)


When I removed `export` the build ran normally but didn’t use sccache.


What *did* work is mk_add_options MOZ_MAKE_FLAGS="RUSTC_WRAPPER=sccache"

I also had mk_add_options MOZ_MAKE_FLAGS="-j100" to make icecc
effective, but when specifying both separately only one would take effect.


Overall, I ended up with this:

ac_add_options --with-ccache=/usr/bin/ccache
mk_add_options MOZ_MAKE_FLAGS="-j100 RUSTC_WRAPPER=sccache"
CC="/usr/lib/icecc/bin/cc"
CXX="/usr/lib/icecc/bin/c++"

--
Simon Sapin

[Mike Hommey]

On Fri, Aug 04, 2017 at 11:25:52AM +0200, Simon Sapin wrote:
> On 26/07/2017 19:22, Ted Mielczarek wrote:
> > > I’ve ended up keeping "classic" ccache for C and C++ code and adding
> > > 'export RUSTC_WRAPPER=sccache' to my mach wrapper script in order to use
> > > sccache for Rust code. (Having this line with or without 'export' in
> > > .mozconfig did not appear to do anything. Can mozconfig set arbitrary
> > > environment variables?)
>
> > No, mozconfig variable setting is sort of restricted. Bare variable
> > assignments (with or without export) are evaluated in the context of
> > configure, but don't survive to the Makefile environment. If you write
> > it as `mk_add_options export RUSTC_WRAPPER=sccache` that ought to work,
> > since that will be set as an exported Makefile variable, meaning it will
> > be set in the environment for commands executed by make.
>
> `mk_add_options export RUSTC_WRAPPER=sccache` causes an error:

It's `mk_add_options "export RUSTC_WRAPPER=sccache"`.

Mike

[Simon Sapin]

That worked, thanks! I now have:

ac_add_options --with-ccache=/usr/bin/ccache
mk_add_options "export CCACHE_PREFIX=icecc"
mk_add_options "export RUSTC_WRAPPER=sccache"
mk_add_options MOZ_MAKE_FLAGS="-j100"

--
Simon Sapin

[ISHIKAWA,chiaki]

Does using only the two lines below

> ac_add_options --with-ccache=/usr/bin/ccache

and

> mk_add_options "export RUSTC_WRAPPER=sccache"

have the effect of using ccache for ordinary c++/cc compilation
and using sccache for rustc compilation?

I have used ccache extensively (with its setting in various scripts and
configuration files and so if I can introduce sccache in an unobtrusive
manner, I would use sccache. But come to think of it, if sscacche honors
CCACHE_DIR and other ccache environment variables, then I will have
collisons of the cachefiles, correct?
Hmm... )

TIA

[ISHIKAWA, Chiaki]

OK, as I further checked, sccache uses SCCACHE_DIR environment variable
for cache storage on local disk while ccache uses CCACHE_DIR.
So I can avoid conflict and the above two lines in my mozconfig seemed
to use sscache for rust compilation, and I see the directory
specified by SCCACHE_DIR populated by cache subdirectories.

However, I am not sure if the cache is working correctly.

With ccache, we can specify a log file in the environment variable
CCACHE_LOGFILE to specify. We can study the log file to see if
the cache is indeed working (hits, etc).

Is there an equivalent of CCACHE_LOGFILE with sccache?
There was no trace of such logfile in the directory specified by
SCCACHE_DIR.

TIA

[Chris Peterson]

On 2017-08-05 10:49 AM, ISHIKAWA, Chiaki wrote:
> However, I am not sure if the cache is working correctly.
>
> With ccache, we can specify a log file in the environment variable
> CCACHE_LOGFILE to specify. We can study the log file to see if
> the cache is indeed working (hits, etc).
>
> Is there an equivalent of CCACHE_LOGFILE with sccache?
> There was no trace of such logfile in the directory specified by
> SCCACHE_DIR.

You can check whether `sccache -s` stats show any sccache activity after
building Firefox.

Adding `mk_add_options "export RUSTC_WRAPPER=sccache"` to my mozconfig
has no effect according to my `sccache -s` stats, but adding `export
RUSTC_WRAPPER=sccache` to my .bash_profile seems to work.

Is there a way to make cargo output more verbose during mach build?
`mach build --verbose` doesn't seem to affect cargo.

[Mike Hommey]

On Wed, Jul 26, 2017 at 09:54:14AM -0400, Alex Gaynor wrote:
> If you're on macOS, you can also get sccache with `brew install sccache`.

If you're on macOS and were hitting errors building openvr, this is
fixed in sccache master.

Mike

[Ted Mielczarek]

I've also now published a 0.2.1 release to crates.io, so you can `cargo
install --force sccache` to get a release containing this fix.

-Ted

[Jonathan Watt]

On 26/07/2017 16:19, Botond Ballo wrote:
> On Wed, Jul 26, 2017 at 9:05 AM, Ted Mielczarek <t...@mielczarek.org> wrote:
>> If you build Firefox on Linux or OS X you can (and
>> should) use sccache in place of ccache for local development.
>
> Can sccache be used in conjunction with icecc [1]?
>
> I currently use the two together by having the following in my .mozconfig:
>
> ac_add_options --with-ccache
> mk_add_options 'export CCACHE_PREFIX=icecc'
> mk_add_options 'export CCACHE_CPP2=yes'
>
> Is there an equivalent / similar setup with sccache?

This is an old thread, but since I looked this up I'll circle back and note that
the issues seems to be:

https://github.com/mozilla/sccache/issues/132