Some ideas for future work

[Chris Hopman]

Hey all-

As people are sort of working mostly on the gyp->gn conversion, I don't know if anyone is really thinking about ways we could make gn even better in the future.

I've had a couple ideas stewing around in my head for some time now, and I thought maybe I should write them down and get other people thinking about things too.

Here's a short doc with ideas

https://docs.google.com/document/d/1DH-t-jYwQxZAXhqoBvnWGHWeF1z4Lltxny9D7lgC4HU/edit?usp=sharing

[Thiago Farina]

I'm unable to sign-in to comment.

On "gn clobber", where would we keep args.gn? 

-- 

Thiago Farina

[Chris Hopman]

You should be able to comment anonymously. 

I've updated that part to say "deletes the contents of the output directory except for args.gn". That is, if I do:

gn clobber

gn build //chrome

or (`gn build --clobber //chrome`)

I will get a clean build of //chrome with my current gn configuration.

[Brett Wilson]

There was talk of build/run/test very early but we decided it's better
to to keep GN more focused and not conflate it with the actual build
system at this point. Long term, there is interest in making ninja a
library, in which case GN would itself be the build system and these
things make more sense.

gn clean sounds good. This is implemented already in the landmines
code (it preserves the args file and writes enough of a build.ninja
file to regenerate the build) but might be useful outside of that.

Brett

[Chris Hopman]

Yeah, if you add build/run/test then gn is, to the user, the build system. I don't think that whether the underlying ninja is an executable or a library really matters to the user or that adding build/test/run would be any simpler if ninja were available as a library. In fact, run/test are things that ninja can't really do in either case, and `gn build` is straightforward either way.

There are some interesting things that could be done if ninja were a library, like doing `gn test //...` or some such, tests could start running as soon as their dependencies were built instead of having to serialize building everything with testing everything (though that optimization alone would be more complicated than the work needed to support simple build/run/test).

I think that having run+test commands in your build system is incredibly useful and I guess I just don't see the value in blocking that on ninja supporting being used as a library and gn actually using it that way.

[Chris Hopman]

I say "run/test are things that ninja can't really do in either case", but if ninja were a library, you could actually inject the run/test commands as nodes/edges in the graph of things to do. 

In the run case, I don't think that's particularly helpful because I don't foresee that being used to run multiple things. 

For the test case, I mention later on that it could be useful when you want to build+test multiple targets as you could start each test target as soon as its dependencies are ready.

[Marc-Antoine Ruel]

We're running less and less tests locally. That said, it's still important information to put in the build files but it's really more to know what to archive in a follow up step, not much to drive any test execution by ninja.

M-A

[Chris Hopman]

There's no reason `gn test` couldn't support non-local tests. Really all it is is that in the gn file you associate some command  and a bunch of dependencies with a target name.

Think about running isolated foo_unittests. Currently I do that like this:

ninja -C some/out/dir foo:foo_unittests_run

python tools/swarming_client/swarming.py run some/out/dir/foo_unittests.isolated

You could do the straightforward thing of adding a "//tools/swarming_client:swarming" and then I would do:

gn test //tools/swarming_client:swarming -- foo_unittests

That approach shares a problem with what we have now, it doesn't ensure that foo_unittests and its dependencies are properly up-to-date. So, how about automatically creating a foo_unittests_isolated runnable target for all (isolatable) unittests and then just do:

gn test //foo:foo_unittests_isolated

This is quite simpler and can ensure that the foo_unittests dependencies are up to date when running the swarming command.

Or something crazy like make the runnable bit of foo_unittests be a script that can do either a local or server based run (and the foo_unittests executable is just off somewhere where the user doesn't care about). And I can do local or server with:

gn test //foo:foo_unittests 

and

gn test //foo:foo_unittests -- --swarming

[Nico Weber]

I think it's better to focus on getting the gn build deployed before thinking about more features.

On Wed, Jan 28, 2015 at 2:07 PM, Chris Hopman <cjho...@chromium.org> wrote:

[Chris Hopman]

I think we should get the gn build deployed before adding new features.

I don't think that that should prevent us from thinking about things we can do in the future. In fact, I think that having a good idea of the future improvements that we can bring to the internal developer experience just gives more incentive to do what is needed to enable those things (in this case, getting the gn build deployed).

[Marc-Antoine Ruel]

To clarify, I did a "gn test" prototype last summer, adding necessary swarming build variables and other needed things, wrote a few CLs but never committed them because it was clearly too early and they created more noise than good. I did get an idea of what was needed and it won't be too hard to do when it's the time to do so, if this is still a desired functionality. You can look at the gn-dev@ ML archives. In the meantime, gn has to fit in the current build process, and this feature isn't needed for this specifically.

I can't talk about the other features in the document.

Thanks,

M-A 

[Nico Weber]

On Wed, Jan 28, 2015 at 7:16 PM, Chris Hopman <cjho...@chromium.org> wrote:

I think we should get the gn build deployed before adding new features.

I don't think that that should prevent us from thinking about things we can do in the future. In fact, I think that having a good idea of the future improvements that we can bring to the internal developer experience just gives more incentive to do what is needed to enable those things (in this case, getting the gn build deployed).

(For what it's worth, I disagree with this. New stuff will make gn at most 30% better, so it won't make a qualitative difference to incentive – if there isn't enough incentive now, there won't be with this new stuff. And it's a distraction from getting gn deployed.)

[Thiago Farina]

Chris, have you saw http://www.kythe.io/docs/campfire.html?

It seems to be doing exactly what you have proposed, i.e, calling ninja under the hood.

--

Thiago Farina

[Thiago Farina]

On Wed, Jan 28, 2015 at 11:06 PM, Brett Wilson <bre...@chromium.org> wrote:

gn clean sounds good. This is implemented already in the landmines
code (it preserves the args file and writes enough of a build.ninja
file to regenerate the build) but might be useful outside of that.

I tried to implement this. CL is here -> https://codereview.chromium.org/909103003/

-- 

Thiago Farina