ridiculous idea to speed up build

[唐佳佳]

Hi, I am a freshman on ninja, but have worked on Android for more than 5 years. 

Since Android N, ninja has been involed in AOSP, and become to be default tool now. 

Recently, I am trying to investigate if build output/targets can be shared in my team. As you know, lots of source-code in AOSP(typically external/xxxx projects) is rarely be modified/changed. So, I want to save the build-outputs/targets which locates in out/soong/.intermediates, then share them in my team. By modify these build-outputs/targets' and their dependencies' timestamp, to skipping building them.

This is different from incremental build, because incremental build happens in my local; my purpose is to share the build-outputs -- directly speed up build progress.

Does anyone have tried the same , or have the similar idea? (or this is just a ridiculous idea)

Best regards and thanks very much!

[Nico Weber]

I've been wanting to do something like this for Chromium for a long time: Most people don't hack on the blink code for example, so we could just download a prebuild libblink_core.so.

But it's not obvious (to me, at least) what a good way to implement this would be. You want something that says "if any of these build inputs have changed, we have to run this part of the build graph locally, but if not, then replace this whole subgraph with a single node that computes a hash of all the build inputs and flags and downloads the build output from somewhere if it's already been uploaded – but if it doesn't exist remotely then nevermind, I'll take the local build graph actually".

I think this can be done at the generator level, but then you'd have to do it for every linkable build target that exists, even if you only want to build a subset. Alternatively, you could make the generator a "we need to do the can-we-download-check" build rule for each linkable target that, when the target is built, does the check for that target, and then re-runs the generator.

(I don't know off the top of my head if this dynamic generator rerunning can currently be expressed; I think currently ninja only supports a static list of generator dependencies.)

So I think it's a good idea, but the implementation would need lots of prototyping to find a design that's simple.

Nico

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ninja-build/bf2b4667-2021-4417-86d3-d98ad9eb231a%40googlegroups.com.

[xxx]

Hi, i am Christophe

I am not interesting in Ninja counts, but if a developpement about build or work gestion from ide's or compil LCO is am white board i could be interested. The programm profile would be with a sqlite connection and could be functional without obligation of internet or server connection.  Typical profil: to use in a little society without leak-risks.

To view this discussion on the web visit https://groups.google.com/d/msgid/ninja-build/CAMGbLiHUN5PH4eHDWRSXok3Ukq8PsFkHwHBP4D1rKQxPzhss0Q%40mail.gmail.com.

[Charlie Dyson]

FWIW I've had some successes with ccache (but that will only cache compiler output, not link). Allegedly it works on NFS.

Cheers,

Charlie

To view this discussion on the web visit https://groups.google.com/d/msgid/ninja-build/CAJFRS3BSVV%3DiafukgJQ1NY%3D234koLzY35Yp_7NuZXB3of9rBhg%40mail.gmail.com.

[Konstantin Tokarev]

27.02.2020, 19:35, "Charlie Dyson" <cha...@charliedyson.net>:

> FWIW I've had some successes with ccache (but that will only cache compiler output, not link). Allegedly it works on NFS.

However, if several builds are running at the same time on separate machines and use same cache via NFS in read-write mode, it can lead to corruption. For this case https://github.com/mozilla/sccache should be used.


-- 
Regards,
Konstantin

[Charlie Dyson]

Top tip - thank you!

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ninja-build/5291581582821700%40sas2-a098efd00d24.qloud-c.yandex.net.