easychromium.sh - A bash script for installing Chromium from source on OS X

[Bobo B.]

Hello Chromium community,

I spent the last couple weeks hacking together a bash script to install Chromium from source on OS X. I was surprised thst I didn't find anything for this while googling. It seems the current options involve installing a binary from homebrew cask or from freesmug (which I believe downloads from sourceforge).

Only one of these alternatives offers a checksum for the code, and even then it's not signed. And one of them distributes via sourceforge, recently criticized for putting adware into open source projects. So I wanted a cleaner install path, so I built a script to check dependencies, download the source, and compile Chromium.

I'd appreciate any testing feedback and ideas on how to support upgrading (I have a plan around lkgr in the comments to the code, problem is that gclient sync doesn't like a safesync-url in .gclient for fresh installs). Let me know what you think, thanks!

https://github.com/the-bobo/easychromium

Best,
Bobo

PS - The documentation was challenging. It is spread all over the place, in different pages with outdated/broken links etc. I hope this code can be helpful for others just wanting to install from source on OS X!

[Erik]

• How much RAM does your machine have? Linking generally requires more than most user machines will have. I don't know of any exact requirements, but I expect 4GB is not enough. You may want to mention this in your requirements section.
  
• Why a "Debug" build rather than a "Release build?
  
• The only supported SDK for building is the 10.10 SDK. You may want to mention this in your requirements section.
  

[Bobo B.]

Hi Erik (and others who responded privately),

Thanks for the feedback! My machine has 16GB RAM.

Below are my plans for updating the script and some inline questions. This was my first project with bash, and my first exposure to Chromium, so thanks in advance for the questions.

The reason I built Debug by default is because I thought Release was something used by google employees to build Chrome, not Chromium. I have since been corrected :) Whatever documentation I was reading was confusing, I know y'all are already working on it, but multiple out of date docs where "chromium" and "chrome" are sometimes interchanged made things challenging. But, still enough for me to get something working :)

Planned modifications to script:
1) usability - make it as simple as brew cask install chromium. Just have it do all the defaults like creating a .gclient file and tweaking it and doing the fast build and actually building a release and actually building the stable version of the code.

2) command line arguments - allow bash easychromium defaults, bash easychromium beta, bash easychromium canary, these build what you would expect. I'll probably just make the first one (building a stable release build) once I figure out how to pull the source from the correct channel and leave the others as exercises for sufficiently motivated readers.

3) versioning - allow the command to respond to bash easychromium -version

4) upgrading - create a crontab to automatically upgrade the stable branch of chromium. This can serve as an example for people interested in creating cron jobs to upgrade beta or canary builds. Need to figure out if we should cp to /Applications/ and if there are other places we need to remove or generate files, like /Library/Application Support/ - does anyone know if a simple cp to /Applications/ is sufficient?

5) checking if Tree is Open - I have no idea how to do this check programmatically, and don't know how to do it programmatically via bash (can i curl something and check its value?)

6) scrape API keys - correctly scrape config.txt for API keys (do these need to be passed in as part of GYP_DEFINES?)

7) update README.md - with new requirements, per Erik's post

Thanks,
Bobo

[Anand]

I've built on multiple platforms by pretty much copy-pasting the instructions at https://www.chromium.org/developers/how-tos/get-the-code. I'm curious what you found confusing and what you're like to see clarified.

[Stuart Morgan]

On Tue, Jan 19, 2016 at 1:14 PM Bobo B. <the....@gmail.com> wrote:

5) checking if Tree is Open - I have no idea how to do this check programmatically, and don't know how to do it programmatically via bash (can i curl something and check its value?)

If the goal is to switch to building from a stable branch (or beta or canary branch), this is probably not especially useful. Tree closure is often about the state of trunk rather than an infrastructure problem that would affect branches.

The overlap of people who want to build a random-and-quite-possibly-broken snapshot from trunk, but who don't have any interest in actually learning how to check out and build Chromium code (e.g., to make changes locally), is likely vanishingly small.

-Stuart

[Bobo]

Hello, thanks for your patience. Wanted to get a response out before the week ended. I really appreciate the help and guidance I'm receiving, even though I'm a less-experienced user :)

@Stuart - thanks, so that means that stable releases are housed in a branch somewhere? I've got a query to search the dev mailing lists for that Nico provided regarding how to pull stable releases, that might turn this up. Thanks for your patient explanation, it makes sense.

@Anand - Absolutely, I'm really glad to hear you raise the documentation question. The main challenge for me was that I had what felt like a simple task (build Chromium on OS X) but felt like the information I needed was spread in multiple places.

I had a workflow going into the project, so I had an idea of what I needed to figure out how to do: check dependencies, pull the code, build it. For an idea of the "spread out" factor, here's the main documentation I consulted:
1) http://dev.chromium.org/developers/how-tos/get-the-code
2) https://chromium.googlesource.com/chromium/src/+/master/docs/mac_build_instructions.md
3) https://sites.google.com/a/chromium.org/dev/developers/how-tos/api-keys
4) http://dev.chromium.org/developers/how-tos/install-depot-tools
5) https://www.chromium.org/developers/gyp-environment-variables
6) https://chromium.googlesource.com/chromium/src/+/master/docs/ccache_mac.md
7) https://www.ulyaoth.net/resources/tutorial-install-chromium-from-source-on-mac-os-x.43/

If I were writing a tutorial on how to do this I would host it all on one page, with a gentler intro for people whose only knowledge of Chromium is superficial.

Some of the pain points are below:

• not knowing syntax for certain commands (e.g. fetch, does the --no-history option go in the same place as the --nohooks or somewhere else?)
• not understanding the relationship between commands (after doing the ./build for editing my GYP_DEFINES I need to run gclient runhooks)
• not clearly understanding the difference between a Debug and Release build (this is most likely my own newbishness, Nico was very kind in explaining this to me)
• as well as a weird error message I got telling me that fetch was outdated and only for svn (this may have been because I incorrectly configured my .gclient config file following link #7 to use the svn repo, I later fixed this)
• also the inability to have a safesync-url in .gclient on first fetch (I forgot where I tracked down that bug, it's linked somewhere on a chromium bug tracker).

Seven pages is a lot to consult, but I think probably the issue here is that this documentation is targeted at more experienced devs and/or people more experienced with Chromium. That's not necessarily bad, it just means the on-ramp is harder for individuals not matching those descriptions.

Probably this reflects the fact that my use-case -- simply building a stable version of Chromium for personal use -- is not one the documentation targets.

Thanks,

Bobo

[Reilly Grant]

Link 1 is supposed to be that single page you were looking for, and links to 3 and 4. Everything else likely made the process more complicated and confusing for you to figure out. Notably, link 2 should probably be renamed to something like "OS X Build Tips and Tricks" because it doesn't really contain easy step by step build instructions like link 1 does.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Stuart Morgan]

On Fri, Jan 22, 2016 at 10:47 AM Bobo <the....@gmail.com> wrote:

@Stuart - thanks, so that means that stable releases are housed in a branch somewhere?

Releases are all built from branches, yes.

 

If I were writing a tutorial on how to do this I would host it all on one page

The flipside of this is that about a third of the pages you mentioned are mac-specific, and several more are optional things that not everyone needs to do. If we inlined all the instructions for all platforms and all optional configurations into a single page, I guarantee that the result would not be a simpler or more friendly experience for anyone :)

Probably this reflects the fact that my use-case -- simply building a stable version of Chromium for personal use -- is not one the documentation targets.

I wouldn't say it's a non-goal, in that if there are ways to improve that without making other things worse, they are certainly worth looking at. Obviously though the priority where there is a tradeoff is going to be toward maximizing usefulness to people who want to be actively involved in contributing to the project.

A top-level page that gives a big-picture overview of what's different when building a release version from a stable tag, as a starting point for people trying to do what you are doing, probably wouldn't be a bad idea. It can't realistically inline all instructions though, since if it did it would duplicate 90+% of the build process—and if you have two duplicate sets of instructions, one of them is going to get out of date quickly (and if one is used by developers and the other isn't, you can probably guess which it's going to be...)

-Stuart

[Bobo B.]

Thanks Stuart and Reilly, that makes a lot of sense. Documentation for multiple audiences without redundancy (which quickly becomes out-of-date/stale) is tricky. The top-level page sounds like a good idea though, that could be helpful!

I'd love some advice on the following two questions :)

A) Pulling the latest stable Chromium release programmatically - I found https://groups.google.com/a/chromium.org/d/msg/chromium-dev/VTOniO05UDc/nG3F2e67_4sJ while searching the group, but the instructions appear to require explicit declaration of the <version> to git checkout -b with.

1. Is there a "latest" branch or something similar I can default to?
2. Are these instructions still correct?
3. Currently I set GYP_DEFINES as follows, do I need to make any changes (like adding -Dbuildtype=Official)?

./src/build/gyp_chromium -Dfastbuild=1 -Dmac_strip_release=1

B) Upgrading in OS X - I found this link http://en.flossmanuals.net/chromium/ch014_removing-chromium-from-mac-os-x/ which seems to suggest I can just /bin/cp Chromium.app /Applications/ and that will leave the old version of Chromium's user profiles and bookmarks/preferences intact. Can anyone confirm or suggest best practice here?

Thanks a bunch everyone, looking forward to getting Stable building down, and then implementing the crontab for auto-updating.

Best,
Bobo

[Thiago Farina]

On Sun, Jan 24, 2016 at 10:25 PM, Bobo B. <the....@gmail.com> wrote:

1. Currently I set GYP_DEFINES as follows, do I need to make any changes (like adding -Dbuildtype=Official)?

I don't think you can set buildtype=Official, if I'm not wrong that will require internal source code that is not available externally.

--

Thiago Farina

[Torne (Richard Coles)]

No, you can and should set buildtype=Official for build intended to be for end users. Internal code is only supposed to be required for branding=Chrome. If you have trouble using official it's a bug we should fix.

--

[Sylvain Defresne]

On Mon, 25 Jan 2016 at 01:26 Bobo B. <the....@gmail.com> wrote:

Thanks Stuart and Reilly, that makes a lot of sense. Documentation for multiple audiences without redundancy (which quickly becomes out-of-date/stale) is tricky. The top-level page sounds like a good idea though, that could be helpful!

I'd love some advice on the following two questions :)

A) Pulling the latest stable Chromium release programmatically - I found https://groups.google.com/a/chromium.org/d/msg/chromium-dev/VTOniO05UDc/nG3F2e67_4sJ while searching the group, but the instructions appear to require explicit declaration of the <version> to git checkout -b with.

1. Is there a "latest" branch or something similar I can default to?
2. Are these instructions still correct?
3. Currently I set GYP_DEFINES as follows, do I need to make any changes (like adding -Dbuildtype=Official)?

./src/build/gyp_chromium -Dfastbuild=1 -Dmac_strip_release=1

B) Upgrading in OS X - I found this link http://en.flossmanuals.net/chromium/ch014_removing-chromium-from-mac-os-x/ which seems to suggest I can just /bin/cp Chromium.app /Applications/ and that will leave the old version of Chromium's user profiles and bookmarks/preferences intact. Can anyone confirm or suggest best practice here?

The profiles/bookmark/preferences/etc are saved in "${HOME}/Library/Application Support/Chromium" so copying the Chromium.app to /Applications should work (though not sure you can do it with just cp, I would use "rsync -ac --delete Chromium.app /Application/Chromium.app").

Thanks a bunch everyone, looking forward to getting Stable building down, and then implementing the crontab for auto-updating.

Best,
Bobo

On Friday, January 22, 2016 at 2:16:05 PM UTC-5, Stuart Morgan wrote:

On Fri, Jan 22, 2016 at 10:47 AM Bobo <the....@gmail.com> wrote:

@Stuart - thanks, so that means that stable releases are housed in a branch somewhere?

Releases are all built from branches, yes.

 

If I were writing a tutorial on how to do this I would host it all on one page

The flipside of this is that about a third of the pages you mentioned are mac-specific, and several more are optional things that not everyone needs to do. If we inlined all the instructions for all platforms and all optional configurations into a single page, I guarantee that the result would not be a simpler or more friendly experience for anyone :)

Probably this reflects the fact that my use-case -- simply building a stable version of Chromium for personal use -- is not one the documentation targets.

I wouldn't say it's a non-goal, in that if there are ways to improve that without making other things worse, they are certainly worth looking at. Obviously though the priority where there is a tradeoff is going to be toward maximizing usefulness to people who want to be actively involved in contributing to the project.

A top-level page that gives a big-picture overview of what's different when building a release version from a stable tag, as a starting point for people trying to do what you are doing, probably wouldn't be a bad idea. It can't realistically inline all instructions though, since if it did it would duplicate 90+% of the build process—and if you have two duplicate sets of instructions, one of them is going to get out of date quickly (and if one is used by developers and the other isn't, you can probably guess which it's going to be...)

-Stuart

--

[Thiago Farina]

On Mon, Jan 25, 2016 at 11:58 AM, Torne (Richard Coles) <to...@chromium.org> wrote:

No, you can and should set buildtype=Official for build intended to be for end users. Internal code is only supposed to be required for branding=Chrome. If you have trouble using official it's a bug we should fix.

Thanks for the correction. Sorry for my confusion.

--

Thiago Farina

[Bobo B.]

Hey Folks,

Working on getting the correct build (building from latest stable for mac) working. Had a weird error pop up that I wanted to flag. gclient sync degrades disgracefully when attempting to rebase fails (it just hangs).

The current workflow I'm testing is to update the Chromium install on a user's computer. Currently I'm assuming they've already built Chromium once,
though I don't think that assumption matters. Here are the commands I'm running in the order I'm running them:
#    gclient sync --nohooks --no-history --with_tags --verbose --verbose --verbose
#    git checkout -b new_release tags/$TARGET
#    gclient sync --no-history --verbose --verbose --verbose
#    ./src/build/gyp_chromium -Dfastbuild=1 -Dmac_strip_release=1 -Dbuildtype=Official
#    gclient runhooks

$TARGET refers to the latest stable version for mac, which is 48.0.2564.109 right now (grabbing this from https://omahaproxy.appspot.com/all)

The first time I ran the gclient sync after the git checkout it hung for 35 minutes saying:
"still working on
src"

After I did ctrl+c to interrupt it I found the last command it was trying to run where it hung:
Attempting rebase onto origin...

It just hung there until I interrupted. Running it again after I interrupted it, it ran fine, with echo $? returning 0.

I've put the tail of the (very long) output from the first run below, and put what seems like the relevant portion from the second attempt afterwards.

FIRST RUN OF GCLIENT SYNC:
...
[0:00:11]  = [up to date]      9.0.597.94 -> 9.0.597.94
[0:00:11]  = [up to date]      9.0.597.96 -> 9.0.597.96
[0:00:11]  = [up to date]      9.0.597.97 -> 9.0.597.97
[0:00:11]  = [up to date]      9.0.597.98 -> 9.0.597.98
[0:00:11]  = [up to date]      9.0.597.99 -> 9.0.597.99
[0:00:11]  = [up to date]      9.0.598.0  -> 9.0.598.0
[0:00:11]  = [up to date]      9.0.599.0  -> 9.0.599.0
[0:00:11]  = [up to date]      9.0.600.0  -> 9.0.600.0
[0:00:11]  = [up to date]      pre_blink_merge -> pre_blink_merge
[0:00:24] Attempting rebase onto origin...
----------------------------------------
interrupted

SECOND (SUCCESSFUL) RUN OF GCLIENT SYNC:
...
[0:00:11]  = [up to date]      9.0.597.94 -> 9.0.597.94
[0:00:11]  = [up to date]      9.0.597.96 -> 9.0.597.96
[0:00:11]  = [up to date]      9.0.597.97 -> 9.0.597.97
[0:00:11]  = [up to date]      9.0.597.98 -> 9.0.597.98
[0:00:11]  = [up to date]      9.0.597.99 -> 9.0.597.99
[0:00:11]  = [up to date]      9.0.598.0  -> 9.0.598.0
[0:00:11]  = [up to date]      9.0.599.0  -> 9.0.599.0
[0:00:11]  = [up to date]      9.0.600.0  -> 9.0.600.0
[0:00:11]  = [up to date]      pre_blink_merge -> pre_blink_merge
Checked out revision 7a5332d762625352a252355a3d7735dd2de4fbea
[0:00:16] Finished.
----------------------------------------
gclient(412) requirements:Dependency(src/breakpad/src).requirements = ('src',)
gclient(412) requirements:Dependency(src/breakpad/src).requirements = ('src',)

...

It looks like there is some error handling missing for when attempting a rebase fails, but i'm not sure.

Hope this helps, and will continue to try and finish my project. Thanks for all your help!

Best,
Bobo

[Bobo B.]

Second related question that just occurred to me - is there a way to programmatically check if my depot_tools is up to date and update if not? Does gclient sync already do this?

[Nico Weber]

Just running `gclient` will autoupdate depot_tools.

--

[Bobo]

Thanks Nico, any advice on catching this rebase error? I feel like setting a timer that aborts and retries is too sensitive to differences in the underlying machines people will be using. On the other hand, non-deterministic errors must be effectively dealt with for the script to be sufficiently usable, hmm...