tundra build system

[Evan Martin]

scoopr on IRC pointed out
http://deplinenoise.files.wordpress.com/2011/04/tundra43.pdf
https://github.com/deplinenoise/tundra
which is similar to Ninja in a number of ways but also quite different.
The slides are an interesting read if you're in to that sort of thing.

[Richard Geary]

Interesting!

I've written a ninja build file generator for Premake (w.i.p.) so a lot of Tundra's Lua features are captured by that, plus more. Premake sounds like it's a little bit slower than Tundra for the prebuild step, it's currently taking me ~4 seconds to build a 300kb ninja file.

One of the problems with Premake that I've had to overcome is support for complex projects & multi-level dependencies. Tundra does that out of the box.

[Konstantin Tokarev]

23.08.2012, 14:17, "Richard Geary" <richar...@gmail.com>:

> Interesting!
>
> I've written a ninja build file generator for Premake (w.i.p.) so a lot of Tundra's Lua features are captured by that, plus more. Premake sounds like it's a little bit slower than Tundra for the prebuild step, it's currently taking me ~4 seconds to build a 300kb ninja file.

But you don't need to regenerate it every time. Also, Premake surely has a room for speed optimization, and LuaJIT could be tried.

Did you compare actual build performance, i.e. generated Ninja vs Tundra?
--
Regards,
Konstantin

[Richard Geary]

True, this is comparing apples & oranges. Ninja should be faster than Tundra for incremental builds; you'd rerun Premake when you added a file. I much prefer the Premake/Ninja separation vs Tundra for that and for modularity reasons. And once you add checks for new files to the ninja build (eg. a "find src | md5sum" comparison), then you don't lose any convenience.

There's certainly room for Premake performance improvements. I hadn't heard of LuaJIT, I'll look at that.

I haven't tried porting my project to Tundra, it would take a long time. So unfortunately I have no actual numbers for comparison.

[Mathias Gaunard]

I don't understand the point of the multiple pass thing. If the
dependencies are set up properly, the generated files will be built
before the things that use them.

[Rachel Blum]

It's a common issue whenever the DAG can produce new DAG nodes during execution, and the exact number and nature of the DAG nodes is unknown. (I.e. if you don't know the exact name/amount of headers included and generated by the code generator)

Rachel

[Maxim Kalaev]

Thanks for sharing, very interesting indeed.
It seems like its a time to have a benchmark for build systems, as there are two many already. :)
The design is inspiring, but the describtion of where the implementation is parallel (multithreaded) and where is it serial were vague.
I was vondering if tundra's noop build is parallelized, and I am still is.

It was pleased to see what the author was seriously concerned about autogenerated code in his design, as my feeling till now was that noone seriously cares about the use-case.
Probably game industry is different indeed - with all the scripting engines, state machines and proprietry tools involved.
For the record, I tried author's approach (use manually defined 'barriers' as order-only deps in ninja to build files in a proper order) in our project and failed. May be this works when you start with a clean project and have an accurate view of what depends on what.
On the bright side, I've ended extending ninja to handle dynamic dependencies updates, which works pretty well for me.

[Mathias Gaunard]

On 24 août, 20:44, Rachel Blum <gr...@chromium.org> wrote:
> It's a common issue whenever the DAG can produce new DAG nodes during
> execution, and the exact number and nature of the DAG nodes is unknown.
> (I.e. if you don't know the exact name/amount of headers included and
> generated by the code generator)

If you don't know the name/amount of files generated by the code
generator in advance, then you should definitely do that code
generation at the configuration phase, not during the build phase.

[Rachel Blum]

Tundra is coming from a background in video games, where a lot of the pipeline is driven by artist input. Telling them to please run a configuration phase before the build phase is a non-starter for many reasons. (And since it needs to be run at pretty much every build, since art changes usually faster than code, building it into the build system makes sense. YMMV)

Of course, there's a good chance I'm entirely misrepresenting the reasons - I've never been involved in Tundra, just chatted with @deplinenoise a few times. You can always ask him on twitter :)

Rachel