PSA: You will need to run `gn gen out/whatever`

[Scott Graham]

After r571031, the location of the GN binary has changed. You'll need to run: `gn gen out/whatever`.

Details:

The GN binary moved from buildtools/$os/gn[.exe] => third_party/gn/gn[.exe].

This breaks GN's automatic regeneration because GN tries to re-run the binary that was used initially to generate the project. That old location has been replaced with a binary that says the same thing as this PSA and errors out, because we wanted to be sure that there were no lingering scripts pointing at the old location (which could have caused much confusion later on.)

When you manually run "gn", it will be using the wrapper script in depot_tools which will forward to the correct (new) location, but auto-regeneration doesn't do that.

Apologies for the churn.

[Morten Stenshorne]

Scott Graham <sco...@chromium.org> writes:

> After r571031, the location of the GN binary has changed. You'll need
> to run: `gn gen out/whatever`.
>
> Details:
>
> The GN binary moved from buildtools/$os/gn[.exe] =>
> third_party/gn/gn[.exe].
>
> This breaks GN's automatic regeneration because GN tries to re-run the
> binary that was used initially to generate the project. That old
> location has been replaced with a binary that says the same thing as
> this PSA and errors out

Actually, that did not happen here.

ninja: Entering directory `out/Debug/'
[1/1] Regenerating ninja files
FAILED: build.ninja
../../buildtools/mac/gn --root=../.. -q gen .
/bin/sh: ../../buildtools/mac/gn: Permission denied
ninja: error: rebuilding 'build.ninja': subcommand failed

Then I did a chmod +x on that file, and THEN I got the message.

> Apologies for the churn.

Will this happen every time we move before or past this commit
(e.g. when bisecting)?

--
Morten Stenshorne, Google, Oslo, Norway

[Bartosz Fabianowski]

This broke the Chrome ebuilds in Chrome OS for me, see http://crbug.com/855791#c18 for details. I have a partial fix at http://crrev.com/c/1118162 but there are missing dynamic libraries and I do not know how to put those in the search path correctly.

- Bartosz

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CANHK6RabRuUqNdBLWR7HHCYY3vchf1yxF48xV5yOwVXdMzB0Lg%40mail.gmail.com.

[Henrik Boström]

After doing chmod +x it says "The gn binary has moved, please re-run `gn gen <your-out-dir>`." even when I call "gn gen" as instructed.

third_party/gn/ does not exist, "which gn" says the depot_tools/gn one. I've done git pull and gclient sync several times.

Any suggestions?

I've tried:

hbos@hbos2:~/ssd/chromium/src$ gn gen out/gn

The gn binary has moved, please re-run `gn gen <your-out-dir>`.

hbos@hbos2:~/ssd/chromium/src$ ~/depot_tools/gn gen out/gn

The gn binary has moved, please re-run `gn gen <your-out-dir>`.

hbos@hbos2:~/ssd/chromium/src$ ninja -C out/gn content_unittests

ninja: Entering directory `out/gn'

[1/1] Regenerating ninja files

FAILED: build.ninja 

../../buildtools/linux64/gn --root=../.. -q gen .

The gn binary has moved, please re-run `gn gen <your-out-dir>`.

ninja: error: rebuilding 'build.ninja': subcommand failed

hbos@hbos2:~/ssd/chromium/src$ buildtools/linux64/gn gen out/gn

The gn binary has moved, please re-run `gn gen <your-out-dir>`.

On Thursday, June 28, 2018 at 12:10:56 PM UTC+2, Bartosz Fabianowski wrote:

This broke the Chrome ebuilds in Chrome OS for me, see http://crbug.com/855791#c18 for details. I have a partial fix at http://crrev.com/c/1118162 but there are missing dynamic libraries and I do not know how to put those in the search path correctly.

- Bartosz

On 28/06/18 07:22, Scott Graham wrote:

After r571031, the location of the GN binary has changed. You'll need to run: `gn gen out/whatever`.

Details:

The GN binary moved from buildtools/$os/gn[.exe] => third_party/gn/gn[.exe].

This breaks GN's automatic regeneration because GN tries to re-run the binary that was used initially to generate the project. That old location has been replaced with a binary that says the same thing as this PSA and errors out, because we wanted to be sure that there were no lingering scripts pointing at the old location (which could have caused much confusion later on.)

When you manually run "gn", it will be using the wrapper script in depot_tools which will forward to the correct (new) location, but auto-regeneration doesn't do that.

Apologies for the churn.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Marc Treib]

​I was getting the same "The gn binary has moved, please re-run `gn gen <your-out-dir>`." when trying to run "gn gen". This was caused by src/buildtools/linux64/gn still being there, which a "git clean -f" fixed.

So now I'm getting:

treib@milkman:~/work/chromium/src$ gn gen out/Release

gn.py: Could not find gn executable at: /media/treib/work/chromium/src/buildtools/linux64/gn ​

As hbos says, it seems that gn.py is looking in third_party/gn which doesn't exist.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CANHK6RabRuUqNdBLWR7HHCYY3vchf1yxF48xV5yOwVXdMzB0Lg%40mail.gmail.com.

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/426832d7-4092-42dd-b61b-6cf25d18064a%40chromium.org.

Marc Treib

Software Engineer

tr...@google.com

Google Germany GmbH

Erika-Mann-Straße 33 80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Sylvain Defresne]

Hi Scott,

This probably broke every project based on Chromium that fetches src/ via a recurse_deps due to how the gn wrapper works [1]. What those projects usually do is to fetch their repository in a sub-directory of src/, say src/$internal/, and use a recurse_deps in src/$internal/DEPS to fetch src/. Both iOS and Android internal repositories at Google are set up this way, and from the email thread, it appears this may also be the case for Chrome OS. Branch builds using buildspec are also setup this way (but this may have changed, I've not build a branch for some time).

For those affected (i.e. people working on downstream iOS, Android, Chrome OS repositories), a quick workaround is to create a symlink from src/$internal/third_party/gn pointing to src/third_party/gn.

-- Sylvain

[1]: the gn wrapper use gclient to find the path to the real binary; the heuristic used by gclient is to first look in the root of the primary repository (the first one listed .gclient), and if not found, to default to the directory containing the .gclient file

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CANHK6RabRuUqNdBLWR7HHCYY3vchf1yxF48xV5yOwVXdMzB0Lg%40mail.gmail.com.

[Marc Treib]

​For the next person running into this: It seems there is some problem with cipd dependencies not being properly updated​. I deleted "src/../.cipd" and re-ran "gclient sync", and now it's all there and works.

[Henrik Boström]

Thaaank you! <3 That seem to have done the trick!

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Henrik Boström]

That caused third_party/gn/ to reappear and compile works but that folder is listed in git status as untracked. I don't dare to do git clean -f.

[Greg Thompson]

Yup. I've noticed the same thing. Should there be an entry for it in third_party/.gitignore?

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/76ca65b8-8e93-448e-ae54-67a21f10ac1b%40chromium.org.

[A Olsen]

Based on the replies above - I tried running:

chmod +x src/buildtools/linux64/gn

rm -rf src/../.cipd

gclient sync

gn gen out/Default

Between the 4 of them, things starting working again. Very powerful incantations. Not sure if the order matters.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/76ca65b8-8e93-448e-ae54-67a21f10ac1b%40chromium.org.

[Scott Graham]

On Thu, Jun 28, 2018 at 1:17 AM Morten Stenshorne <mste...@chromium.org> wrote:

Scott Graham <sco...@chromium.org> writes:

> After r571031, the location of the GN binary has changed. You'll need
> to run: `gn gen out/whatever`.
>
> Details:
>
> The GN binary moved from buildtools/$os/gn[.exe] =>
> third_party/gn/gn[.exe].
>
> This breaks GN's automatic regeneration because GN tries to re-run the
> binary that was used initially to generate the project. That old
> location has been replaced with a binary that says the same thing as
> this PSA and errors out

Actually, that did not happen here.

ninja: Entering directory `out/Debug/'
[1/1] Regenerating ninja files
FAILED: build.ninja
../../buildtools/mac/gn --root=../.. -q gen .
/bin/sh: ../../buildtools/mac/gn: Permission denied
ninja: error: rebuilding 'build.ninja': subcommand failed

Then I did a chmod +x on that file, and THEN I got the message.

Apologies for screwing up the +x, that's been fixed now.

 

> Apologies for the churn.

Will this happen every time we move before or past this commit
(e.g. when bisecting)?

Unfortunately, yes.

[Scott Graham]

:( What a lovely mess. I'm not sure why the pull didn't work. The tracking bug related to needing to remove .cipd is https://bugs.chromium.org/p/chromium/issues/detail?id=794764, but I thought that would only happen when the cipd contents had been manually modified.

Perhaps this was due to me failing to add third_party/gn/ to .gitignore, if a git clean removed the gn binary at some point.

[Scott Graham]

On Thu, Jun 28, 2018 at 3:10 AM Bartosz Fabianowski <bar...@chromium.org> wrote:

This broke the Chrome ebuilds in Chrome OS for me, see http://crbug.com/855791#c18 for details. I have a partial fix at http://crrev.com/c/1118162 but there are missing dynamic libraries and I do not know how to put those in the search path correctly.

I'll follow up on this one on the bug.

[Scott Graham]

On Thu, Jun 28, 2018 at 3:21 AM Sylvain Defresne <sdef...@chromium.org> wrote:

Hi Scott,

This probably broke every project based on Chromium that fetches src/ via a recurse_deps due to how the gn wrapper works [1]. What those projects usually do is to fetch their repository in a sub-directory of src/, say src/$internal/, and use a recurse_deps in src/$internal/DEPS to fetch src/. Both iOS and Android internal repositories at Google are set up this way, and from the email thread, it appears this may also be the case for Chrome OS. Branch builds using buildspec are also setup this way (but this may have changed, I've not build a branch for some time).

For those affected (i.e. people working on downstream iOS, Android, Chrome OS repositories), a quick workaround is to create a symlink from src/$internal/third_party/gn pointing to src/third_party/gn.

-- Sylvain

I see, I'm sorry I wasn't aware of that behaviour. I don't have a better suggestion than your workaround for a quick fix. I filed https://bugs.chromium.org/p/chromium/issues/detail?id=857504 for following up on this.

[Sylvain Defresne]

FTR, for Chrome on iOS, we have a hook in our internal DEPS creating the symbolic link. It should be straightforward for other projects using the same setup to have a similar hook.

-- Sylvain

[Scott Graham]

The specific location of gn inside buildtools was more of an API than I was aware of for many consumers of Chrome beyond build.chromium.org (in addition to the other things that were broken described upthread).

The CL to move gn was reverted at https://chromium-review.googlesource.com/c/chromium/src/+/1118825. I am not confident in whether $YOUR_BOT will revert cleanly or if fixes made since then will interact badly, but please let me know if after the revert things are still broken and I will try to help patch things up as quickly as possible.

I will write up a postmortem doc on the myriad things that went wrong.

[Scott Graham]

Hi again chromium-dev,

I wrote up a probably-incomplete list of things that went wrong. I will attempt to address the ones that seem tractable. If you feel passionate about anything there (or anything I missed) please tag it with a bug.

My proposed next attempt will try to ensmallen the problem, by not changing the location of the gn binary.

[Scott Graham]

Me again! I know you're all absolutely fascinated to hear more about this tremendously exciting change.

I'm going to land https://chromium-review.googlesource.com/c/chromium/src/+/1119579 tomorrow morning PDT. This is a small part of what the original break-y change was doing, and in particular it uses: 1) the old download method, and 2) does not change the location of the gn binary.

The hope is that this will be uneventful, but please yell if something seems fishy.