Shared directory invariants

[Jordan Woehr]

Hi Neil and Shake users,

I've been playing around with Shake's shared directory functionality, and I'm wondering what sorts of invariants need to be satisfied for it to function correctly. The context for these questions is sharing build products amongst many users.

Suppose multiple users share a machine and build a Shake project. Can multiple users perform a build safely at the same time with the --shared flag specifying the same directory? By extension, could users on several different systems share a --shared cache across an NFS mount for example. I know from the Build Systems a la Carte paper non-deterministic build results is definitely a potential gotcha, as well as environmental differences between machines (let's assume these are dealt with somehow).

The second question is around what constitutes a well-formed cache. Hypothetically, if some kind of non-filesystem key-value store were mounted at the specified --shared directory, what properties would this store need to satisfy to function correctly. Could this hypothetical key-value store evict indivudial files in a least-recently-used manner? Does a larger set of files in the cache constitute something evictible?

Finally, thank you for Shake. It's an amazing tool.

Jordan

[Neil Mitchell]

Hi Jordan,

If you haven't already seen it, https://shakebuild.com/cloud has a lot of useful information. There are also some notes in https://ndmitchell.com/downloads/paper-build_scripts_with_perfect_dependencies-18_nov_2020.pdf S2.5 around cloud builds, most of which apply to Shake.

Multiple users should be able to use the same `--shared` directory simultaneously. There is care taken to put the files somewhere else, and then atomically overwrite them. I would say it isn't the most tested code, but I'm unaware of any bugs with it. I suspect non-deterministic build results won't be a problem, other than potentially reducing caching if two users simultaneously build different variants. The code was designed to work over an NFS mount, but I don't actually have one to test, so I hope it works, but I can't promise it.

For eviction, there is `--share-list`, `--share-sanity` and `--share-remove`. I think you can evict one file at a time, and it should work, but I don't remember the details. My personal inclination would be to delete everything on Saturday evening, run the CI jobs (which will be a bit slow) and then use that as a mostly-relevant cache. I'm sure you can do smarter things, but if you can avoid it, that's always better.

Good luck! The shared cache has been used in real life, but not by that many people, so let me know how you find it.

Thanks, Neil

--
You received this message because you are subscribed to the Google Groups "Shake build system" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shake-build-sys...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/shake-build-system/1b179cc0-0ce9-46f4-9fc8-6e5e22158d82n%40googlegroups.com.