libninja.a and multiple threads/builders

[Philippe Hausler]

I am currently working on an application that uses ninja under the hood to act as a build infrastructure. So far loading/parsing single projects seems to work wonderfully. However I am running into a very strange issue where if two builds are running at the same time (granted ninja is fast enough that this isn't too big of an issue for small projects) that there are some crashes/misbehavior when the two builds happen at the same time.

What considerations should I take when using multiple parser/build pipelines? My first thought was the potentials of the current working directory getting clobbered from one build task to the other (which definitely did happen until I refactored slightly to move the working directory to each edge when parsing the manifest). But it seems that this is not the only area that has issues. Any hints on how to accomplish this would be greatly appreciated.

[Nico Weber]

Hi Philippe,

libninja is an implementation detail of ninja-the-binary, it's not designed to be a reusable library. It's probably not thread-safe since ninja doesn't use threads.

Running two builds concurrently in the same build directory likely won't work well even with ninja-the-tool. There shouldn't be crashes (if you're on 1.5.1, there were crashes in this use case in earlier versions), but "strange behavior" is kind of expected.

Why are you trying to run multiple builds in parallel? If it's for different projects, each project should have its own build directory?

Nico

On Mon, Sep 15, 2014 at 4:45 PM, Philippe Hausler <phau...@gmail.com> wrote:

I am currently working on an application that uses ninja under the hood to act as a build infrastructure. So far loading/parsing single projects seems to work wonderfully. However I am running into a very strange issue where if two builds are running at the same time (granted ninja is fast enough that this isn't too big of an issue for small projects) that there are some crashes/misbehavior when the two builds happen at the same time.

What considerations should I take when using multiple parser/build pipelines? My first thought was the potentials of the current working directory getting clobbered from one build task to the other (which definitely did happen until I refactored slightly to move the working directory to each edge when parsing the manifest). But it seems that this is not the only area that has issues. Any hints on how to accomplish this would be greatly appreciated.

--
You received this message because you are subscribed to the Google Groups "ninja-build" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ninja-build...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Philippe Hausler]

For a bit of background, I am currently working on an IDE that the basic premise is to utilize ninja to power it’s build infrastructure but also be used for the project layout (imagine if Xcode or any other IDE didn’t have funky project files, but used the build script itself to derive the structural layout of the code). Of course to keep the app responsive, I want to run the ninja build tasks in background. The other portion of this is integrating with clang’s compilation database – in order to get AST based syntax highlighting and refactoring capabilities I need to derive all of the arguments that would be invoked to compilation per file; this portion works marvelously.

I have refactored ninja itself a bit to avoid some of the assumptions of the current working directory and it is behaving much better so far; as a matter of fact the IDE can actually now self host (mostly credit there is due to ninja itself).

Is it a safe assumption to make that all file references in ninja build scripts are either a) relative to the parent directory of the script, or b) absolute? One of the changes I made unfortunately broke some of the unit tests: the edge path lookups were being assumed to paths like foo.c and with my alterations to make paths absolute in the calculations the resultant was ./foo.c instead.

If you are interested my fork is hosted on github https://github.com/phausler/ninja

Once I can get the unit tests cleaned up and the commits broken out into feature branches I will likely submit them as a set of pull requests if this type of alteration is something useful outside of just libninja.a usage.