Repercussions of GN replacing GYP upstream?

[Johan Bergström]

It's pretty clear that Google is moving away and effectively abandoning Gyp at the end of this year. That means that everything within chromium (v8 and zlib is the only stuff that would affect node, right?) will be built by gn.

Reading their roadmap (https://groups.google.com/d/msg/nodejs/Nq0AjcoEtlg/OuMlVARTZo4J), there's some overlap between the build systems - but build files and what supports what will change down the road.

Since the node ecosystem uses gyp, I thought it at least would be relevant to have a discussion about this. I'm not suggesting to switch, nor semi-maintain gyp as google drops it off its radar; but I've at least searched for more information in this regard. The last mention I saw within the node ecosystem was a reply from Ryan Dahl found here https://groups.google.com/forum/#!msg/nodejs/Nq0AjcoEtlg/OuMlVARTZo4J (not sure why he refers to gn as cancelled though).

It's somewhat unfortunate timing as "1.0" is stabilising, but it's better to at least see what node lead thinks about it since it has a pretty wide effect.

[Forrest Norvell]

I don't really have an answer for you (although I think it's a question that needs to be answered), but I wanted to offer a small correction. The link to the gn roadmap is actually https://docs.google.com/document/d/15kfxO_L3u2iJesR9RtxCDFdgzF_uqgwLHzq17CCOS5c/edit#heading=h.aav57pgqrcv7

F

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/ee7b7cf3-4c16-4388-add7-74c106a2196c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Fedor Indutny]

Some of the addon's gyp files could be translated to the GN without even requiring author to do so.

The syntax is generally pretty similar: https://code.google.com/p/chromium/wiki/GNQuickStart, https://code.google.com/p/chromium/wiki/GNCookbook .

I'm +1 for moving to GN if it is better than GYP, since GYP has quite a lot of problems and caveats.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAHG3qKr5%2BvbcxM77MwkLPVWXG8iG7jgWxN2KCi5Vs5Zig8eaOg%40mail.gmail.com.

[Johan Bergström]

..but doesn't that also imply adding ninja as dep? All npm projects using node-gyp might be affected (goes without saying, I guess). Possibly also want to rethink the 'configure' phase. I've followed GN from a distance and are willing to give it a whirl if its considered worth exploring.

[Fedor Indutny]

As far as I can see `ninja` is still one of the build tools that could be used with the build system.

I was just talking about a "transition stage", which will happen if we will decide to move to GN.

[Ben Noordhuis]

GYP is a relatively simple project. It would be possible to maintain
it and V8's GYP-based build in perpetuity with only moderate effort.
As to whether that's a good course of action, that's a separate
discussion but at least it's an option.

[Ingwie Phoenix]

GN, Generate Ninja, is a pretty cool project, actually.

I actually always wondered: We have a scripting engine. Why do we rely on Python for GYP, if we have NodeJS? Is there no gyp replacement in node itself? Its something that I just never understood… Since the only people that build from source will very likely have python installed, knowing that they will need GYP.

So I would suggest to switch to GN, or to make your fully own tool to help on that. Its not too hard either. I am working on a build tool myself, that I will very likely extend to be able to build nodejs modules, by querying GYP or a nodejs module, for the location of header files.

Just my 2 cents on that. :3

Oh, also, has anybody ever seen a stand-alone source tree of GN? I have not…

PS. The build system I mentioned: http://github.com/IngwiePhoenix/IceTea

[mscdex]

On Wednesday, July 9, 2014 2:06:53 PM UTC-4, Ingwie Phoenix wrote:

I actually always wondered: We have a scripting engine. Why do we rely on Python for GYP, if we have NodeJS? Is there no gyp replacement in node itself?

I would think the problem would be when you're building from source and and do not already have node installed. Otherwise a node executable would have to be downloaded/embedded to run the build script, at which point you may as well just use/install the downloaded/embedded node binary ;-)

I think Python is commonly already installed enough that it's rarely a problem.

[Louis Santillan]

On Wed, Jul 9, 2014 at 9:23 AM, Ingwie Phoenix <ingwi...@googlemail.com> wrote:

I actually always wondered: We have a scripting engine. Why do we rely on Python for GYP, if we have NodeJS? Is there no gyp replacement in node itself? Its something that I just never understood... Since the only people that build from source will very likely have python installed, knowing that they will need GYP.

 Build tools are (generally) less interesting than building servers, which is why there are no GYP replacements in in JS.  Big reasons why chromium/v8/nodejs changed from scons to gyp was 1) parallel builds (gyp can create makefiles which have -jX capability), 2) portability (mobile/arm/mips/non-x86 targets were becoming more important at the time of the switch) 3) scons was slow.

The GN FAQ page seems to indicate part of the driver to GN is for CI improvements [1].  At least it looks like someone at Google has done some more planning this time as was not the case with the scons->gyp transition.  Personally, I prefer the suckless approach to building software (config.mk/Makefile)[2][3].  Keeps people out of autotools/automake hell and let's you focus on writing software.

[1] https://code.google.com/p/chromium/wiki/GNFaq

[2] http://git.suckless.org/surf/tree/config.mk

[3] http://git.suckless.org/surf/tree/Makefile

[Ingwie Phoenix]

I was rather pointing towards node-gyp here, to be honest.

Node-gyp relys on Python, due to GYP. And in fact, not everyone has Python installed. Windows for one, and not all linux distributions for the other. Mac does, but I have seen python devs ranting about the delivered vesion (to be fair, OS X had PHP 5.3 still, while 5.5 was marked stable/current. :P)

I wonder if in the future, node’s default extension building will be handled thru nodejs itself.

But - when building nodejs from source - I totally would go for GN. Its lightweight and fast. And from what I remember, not bound to python…just some C++ compiler. Again, I have not come across any full GN source tree.

[Johan Bergström]

This (no touch) is my current position as well -- but whatever choice/decision is made is going to have to stick around for a long while.

Here's my thoughts so far:

 - I don't particularly care about using python or not and I don't think that discussion is beneficial to this thread.

 - node-gyp is working for a lot of packages, today. I see no reason for that to change.

 - A switch to GN needs a pretty solid documentation effort since a switch reasonably should happen ahead of  "node 1.0", whenever that is. It would also require a lot of testing.

 - Switching also 'forces' the switch from Makefiles/make/gmake. It might actually be a more intrusive change than whether gn or gyp parses the gypfile (short term) for the npm ecosystem.

 - Evaluating other build systems or growing something new seems very out of play to me. If a switch happens, the Gyp to GN transition is probably "least" effort.

[Ryan Dahl]

gn is beautifully designed progressive project (like nearly everything coming out of Google). gn is essentially gyp version 2. Indeed, I think gn will dominate build systems in the coming years. At this stage gn is under heavy development, but when/if chrome has removed gyp (predicted to be towards the end of 2014), Node should begin migration to gn. The addon story has never been good for Node and this would be a chance for the process to be cleaned up. Using gn is finally an opportunity to remove Node's python dependency.

[Alex Kocharin]

There are two different questions here:

1. What to use to build node.js itself?
2. What to use to build native modules?

For now it's GYP for both, later node.js maybe will be built with GN.

I wonder what to do with native modules though. GYP will have to be supported for a looong time anyway, and...

Is it possible/makes sense to use another build system with npm right now without explicit npm support?


09.07.2014, 22:06, "Ingwie Phoenix" <ingwi...@googlemail.com>:

> GN, Generate Ninja, is a pretty cool project, actually.
>

> I actually always wondered: We have a scripting engine. Why do we rely on Python for GYP, if we have NodeJS? Is there no gyp replacement in node itself? Its something that I just never understood... Since the only people that build from source will very likely have python installed, knowing that they will need GYP.

>
> So I would suggest to switch to GN, or to make your fully own tool to help on that. Its not too hard either. I am working on a build tool myself, that I will very likely extend to be able to build nodejs modules, by querying GYP or a nodejs module, for the location of header files.
>
> Just my 2 cents on that. :3
>

> Oh, also, has anybody ever seen a stand-alone source tree of GN? I have not...

>
> PS. The build system I mentioned: http://github.com/IngwiePhoenix/IceTea
>

> --
> Job board: http://jobs.nodejs.org/
> New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/0275E15F-E88A-45FB-8523-6D63C193C9FC%40googlemail.com.

[Fedor Indutny]

The best question to ask is - what problems do we have with GYP?

Is it hard to support it when google will drop it?

I had some problems with GYP, but generally it is very easy to use and to configure. Looks like GN is quite good too, but not that much to make all addons authors move to it.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/6306451405020601%40web20j.yandex.ru.

[Arunoda Susiripala]

I also had a few issues with GYP. It's not consistance across different boxes(even with same node version). Different python versions cause issues. Sometimes os installed gyp gives some trouble. 

If there is a standalone binary we can ship with node distribution that'd be great.

(seems like gn is a binary)

---

Arunoda Susiripala

I curate Meteor Weekly - Check it out!

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAEv2VfLymsmiPDHLR%3DcHHgZ9ziJSny92i-4KZKCv-V7UEMg5Ww%40mail.gmail.com.

[Ryan Dahl]

On Thu, Jul 10, 2014 at 3:33 PM, Fedor Indutny <fe...@indutny.com> wrote:

The best question to ask is - what problems do we have with GYP?

1. Main road block to removing python dependency. (Though gn uses python for custom actions, and the V8 build process depends on some python code, so it'd still be rather unlikely we'd be able to remove the dep for addon developers.)

2. Strange non-JSON but very JSON-like syntax. confusing to JavaScript people.
3. Undocumented, poorly understood. (Contrast this with gn's built-in documentation run 'gn help'.)

4. Slow
Problems that will be had:

5. gyp will be a major legacy project to depend on if Chrome moves to gn.
6. node will have to back port V8's build process to gyp if Chrome moves to gn.

[Johan Bergström]

On Friday, July 11, 2014 6:39:38 AM UTC+10, Ryan Dahl wrote:

Problems that will be had:

5. gyp will be a major legacy project to depend on if Chrome moves to gn.
6. node will have to back port V8's build process to gyp if Chrome moves to gn.

Your last point seems unavoidable should gyp be decided to stay. Above problems were basically the main reason I created this thread in the first place. The positive side-effect of switching gyp to gn is great but shouldn't really be a reason to explore switching.

Will ninja be added as a third party dependency (it's pretty simple to both build and install) or should a npm ecosystem also consider integrating the build tool as well? I wonder if there's been any thoughts on this matter in the npm camp. It'd be valuable feedback since they're probably involved in this as much as node itself.

[Steven R. Loomis]

If I can chip in a couple of gyp issues besides those mentioned:
7.  hard to debug issues- doesn't give detail on where the original issue was
8. lots of strange corner cases:  how and when variables expand, issues like https://code.google.com/p/gyp/issues/detail?id=232 - if you use an argument twice, gyp silently drops it.