How to inform ninja which files to recompile (and using gn gen --filters)

[Kristofer Johansson]

Hi people!

I'm completely new to this project so please bear with me. 

I have an Idea on a modification to the desktop-Notifications (that can be sent from webpages or extensions) I want to try out.

I have managed to check out and build Chromium (without Google APIs from win7x64) using the commandline tools used in the tutorial. The generated chrome.exe seem to be working as expected. So far so good. Now I want to get to work and start tweaking and here I run into trouble.

I have generated the all.sln-file for VS2015 without --filters and like the tutorial says it is a massive file to load in VS2015. I have not managed to load it all because VS2015 hangs and exits before finishing. That is when I started thinking that maybe it would work better with a less massive .sln?

So I tried using the --filters flag when running 'gn gen' . If I have hit the right target in navigating the code the src\ui\arc is the part responsible for posting these desktop notifications and hence there is the code I want to fiddle with. I tried using --filters=//ui/arc (and similar) but I only get 

"Error: No VS project generated" with no further explanation.

Then I tried --filters=//ui/* and then it indeed generated a new all.sln file. I managed to open it in VS2015 but when it finished loading this (also rather hefty) file and had status 'ready' the arc folder/sourcefiles was NOT listed under ui in the tree.

So I need some guidance here and can not seem to google my way to the information myself:

1. How can I use --filters to generate a .sln-file?

2. I don't need to use VS necessarily (and don't know if the big project is actually causing the crash) but if I am to use a simple text editor I need to know how to tell the compiler which files have been modified in order for it to know which parts to recompile, right? 

(I tried just adding some code to a cc-file, save it and run 'ninja -C out/Default chrome' hoping that it might detect the file change but it only gave me 'ninja: no work to do').

If someone could help me out with one or both of these two questions I'd be very grateful. Please pardon my extreme newbie:ism. I have coded a lot of C++ with Qt using Qt's own dev-tools but never used VS, ninja or depot_tools before.

Sincerely,

/KrisJ

[Primiano Tucci]

>2. I don't need to use VS necessarily (and don't know if the big project is actually causing the crash) but if I am to use a simple text editor I need to know how to tell the compiler which files have been modified in order for it to know which parts to recompile, right? 

This is correct. Our source of truth as regards build files lives in the BUILD.gn files.

You can read more here, the short version is that the most common pipeline chromium developers use is as follows:

BUILD.gn files ->  'gn args / gn gen' to generate .ninja files in out/Release (or whatever really) -> 'ninja -C out/Release chrome' to build the exe.

So, when you touch an existing .cc file, you just need to run ninja because you are not altering the dependency graph. ninja will figure out that the timestamp of the touched .cc file is newer than the build artifact and rebuild what necessary.

If you add/remove .cc files, you want to modify the right BUILD.gn file (typically the one in the same folder or in the closest parent.... but is a bit of a longer story), run gn gen to regenerate .ninja files and finally ninja itself.

Actually, these days ninja+gn should be smart enough, so if you previously had an output folder and change gn files, invoking directly ninja should recursively invoke gn.

If you are fine not having VS as a full IDE, using just ninja + gn might be easier to get started than using visual studio.

--
--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss

[Kristofer Johansson]

Thank you very much for your fast response!

So, when you touch an existing .cc file, you just need to run ninja because you are not altering the dependency graph. ninja will figure out that the timestamp of the touched .cc file is newer than the build artifact and rebuild what necessary.

Well, that is actually what I thought from the beginning. I see now that I did not explicitly write in my initial post that the "... tried just adding some code to a cc-file, save it and run 'ninja -C out/Default chrome' ..." actually was an existing file from the 'ui\arc' folder... But I still only got 'ninja: no work to do'. So it seems to me that ninja did not detect the change I made in the code.

Any common reason why ninja would NOT read the timestamps correctly/at all?

/KrisJ

[Torne (Richard Coles)]

Probably the file that you changed is not used in the type of build you are doing. Maybe it's for a different platform, or only used when a particular build setting is set?

[Kristofer Johansson]

That sounds reasonable. So to be more specific:

The file I edited was 'ui/ar/arc_notification_manager.cc'. I added a line: 

LOG(ERROR) << "testingtesting my mods";

in function:

void ArcNotificationManager::OnNotificationPosted(mojom::ArcNotificationDataPtr data)

With the hopes that it would log the error any time a desktop Notification was shown. After saving the file and running ninja I get this exact output:

E:\workspace\sources\chromium\src>ninja -C out/Default chrome

ninja: Entering directory `out/Default'

ninja: no work to do.

I also tried recompiling the whole thing from scratch but I can't see the error in Sawbuck when connected as tutorial shows.

Both of these facts suggest I am in the wrong place fiddling around in code that is not actually compiled for this build.

So:

1. Is there any place where I can read clearly what parts of the src is used for which kind of builds/platforms etc?

2. Any Hello World type suggestions on which file to insert such a LOG(ERROR) to try out the principle and see that ninja actually detects the new timestamp on my system setup?

3. Specifically about 'arc'. Anyone know if I am in the right place at all for what I wish to do? Is 'arc' actually sending the desktop notifications in the corner of the Windows desktop or am I way off?

Thanks!

/KrisJ

[Christian Biesinger]

ARC is not used for desktop notifications in general; it is this: https://developer.chrome.com/apps/getstarted_arc

I am not completely sure where the notification code is (and also, it is in a number of places depending on the type of notification), but I would try https://cs.chromium.org/chromium/src/chrome/browser/notifications/

-Christian

[Kristofer Johansson]

Thank you very much for clearing that up. I have now tried similar modifications in those files and ninja picks it up and compiles. It seems that the code worked well because it is leaving its prints in the log-file. I can't read them in real time in Sawbuck though but that is not a big deal.

Now my only question is regarding the compiling performance. After minor modifications to this one file the recompiling took about 5-10min (did not time it) on my rather strong computer (hexacore, 32Gb RAM, SSD). Is that normal? Anything you can do about it or is it just a fact that it takes a lot of patience to work with developing Chromium?

(I have never worked on projects anywhere near this size so I'm not sure how that usually works...)

Thank you for your guidance!

/KrisJ

[Christian Biesinger]

Windows is a bit slow in the best case, but did you enable component build?

https://chromium.googlesource.com/chromium/src/+/master/docs/component_build.md

-Christian

[Kristofer Johansson]

I assume so. I followed the tutorial:

https://www.chromium.org/developers/how-tos/get-the-code

And there was no options as far as I could see indicating that affected the "standard settings" of gn. 

And on this page:

https://www.chromium.org/developers/gn-build-configuration

It says that Debug is the default mode and is_component_build = true is standard for non iOS Debug builds.

Can I make sure somehow that I actually have component build?... because it seems remarkably slow to me. (And yes, it is the last step that takes forever...)

Thanks!

/Kris

[Christian Biesinger]

The component build is the default as of 14 hours ago, so it depends on when you updated your checkout.

Do you have a file blink_core.dll in your output directory? If yes you have a component build, no otherwise.

-Christian

[Kristofer Johansson]

Oh! I see, things move fast here, cool!

I could not find the blink_core.dll and the checkout is a couple of days old so that makes sense.

So yesterday before bedtime I did a: 

E:\workspace\sources\chromium\src\out\Default>ninja -t clean -g

in my src\out\Default folder and then back to src\ to run:

E:\workspace\sources\chromium\src>gn gen out/Default --ide=vs --is_component_build=true

Today I see it compiled without errors nut I notice no difference in load-time, compile-time after minor changes and more importantly still no blink_core.dll.

Is my ninja command correct for compiling for VS and component build? (I guess I could skip VS now if that would help)

Maybe delete the whole \out folder and start fresh from the beginning? (I used to do that in my own projects before on the rare occasion that clean did not do the job properly)

Thanks!

/KrisJ

[Christian Biesinger]

Uhm, I'm not so familiar with passing flags on the gn commandline. Usually things like is_component_build gets specified in the arguments file, as described at the top of https://www.chromium.org/developers/gn-build-configuration (by running gn args). Maybe try that?

Unfortunately I have no idea how that interacts with using IDEs.

-Christian

[Kristofer Johansson]

Hello!

Worked like a charm! I know I read that was the recommended way when I got started earlier but there is so many new things going through my mind so I actually forgot about it when it was time to recompile. Anyway I edited the args-file and cleaned and compiled. 

Now I have a huuuge out-folder, a bunch of blink_xx.DLLs and a re-compile time of about 20s as compared to 10 min before. :-)

Just want to thank you guys a lot. You have been very helpful and I really appreciate that. A week ago I did not even know Chrome was open source. Today I even have my own hacky-tacky version!

Thank you so much guys!

/Kristofer

[Christian Biesinger]

Glad to hear it is working now! You're very welcome.

-Christian

[Kristofer Johansson]

So one more thing has come up. I have googled for hours and browsed the chromium.org page to the best of my abilities but I have to ask for help on this one: How do I compile a reasonably sized "release" version?

I want to move the working version I have created to my laptop. I have compiled for release mode and component build off. My out\release folder is still 19Gb+... which is a tad to heavy for easy distribution. ;-)

I can't believe I have not found this info. Either it is less common that developers do this themselves or the answer is right under my nose and I just don't get it. I mean I have read quite a bit about gn settings and related stuff but I don't see the answer to my question.

So, how do I compile for an easy move? Which files from the out-folder are actually needed to run Chromium?

Thanks!

/KrisJ

[Torne (Richard Coles)]

There's no particular build setting for that - it's just that most of the files under out are intermediate files used only during the build.

If you're building on Windows, you can build the target "mini_installer" which will make a selfcontained installer exe you can run on another computer.

[Kristofer Johansson]

OK, Now I have managed to produce a mini_installer. I set the target_cpu to "x86" (since I'm planning to install it on an XP32 machine) set to release mode and no component build. Run ninja -C my\path mini_installer. No compile errors and I get a neat little 35Mb installer. When I run it on the target machine I get an error popup from windows: 

"ENTRY POINT NOT FOUND!"

"The procedure entry point SymSetSearchPathW could not be located in the dynamic link library dbghelp.dll".

What might that mean and how do I work it out?

I have not been able to read much about the mini_installer but it is supposed to be self contained and not need any other files to run on target machine, right?

Thanks!

/Kris 

[Torne (Richard Coles)]

I don't think we support XP any more. Does it work on a supported windows version?

[Kristofer Johansson]

Right... I actually heard about that the other day... 

Yeah, the mini_installer actually works on another win7 machine. Thanks!

(Running it on another system actually helped me confirm another very strange problem but I'll start a new thread on that as it has nothing to do with compiling.)

Is there a list somewhere covering what files are necessary if I want to just copy a compiled build to another computer?

/Kris

[Torne (Richard Coles)]

The build configuration for mini_installer presumably contains this, though perhaps not in a very easy-to-understand form :)

[Kristofer Johansson]

Haha, yeah, I guess. Thanks a lot for all your help!

/K