Faster Chrome releases in M114+

[Harry Souders]

Hi all,

Earlier today we announced that we are reducing the time between a milestone's branch and stable release. Note that this does not change our planned stable release dates.

We will also be delaying M115 stable release to accommodate the July freeze, and we're shifting dates after this to account for the delay.

The Chromium Dash Schedule is updated with these revised dates.

If you have questions or feedback, let us know by sending a note to bran...@chromium.org.

Thanks,

Harry

[Yngve N. Pettersen]

Hi,

Sorry, but I don't really like this change.

The previous flow, with the announcement of the new branch on a Friday evening meant that the new branch would have been configured and fetched into our Git mirror, and ready for our 4-5 day rebase+compilefixing update process  by Monday morning (CET). For more about the process in case you are interested, please see my article https://vivaldi.com/blog/lets-go-under-the-browser-hood-with-vivaldis-yngve/ .

This new timing means that, assuming the branch announcement is made Wednesday evening CET (not Tuesday), means that we will lose three days that could be used to fix issues  before our release based on the new version. Normally, we would be starting compilation on Windows sometime Wednesday (a process that usually takes two days), now we will be lucky if we can start *after* the weekend, on Monday (although, given the May 1 public holiday, it would be Tuesday). After it (finally) builds, we still need two+ weeks to resolve issues introduced by upstream changes; essentially this change means that we lose most of one week to fix issues.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAOC0RHBVoe0sA3LEh-AMXo4N2r_fU1%3DM%3Dr88TyGfJ_SsK87QEQ%40mail.gmail.com.

-- 
Sincerely,
Yngve N. Pettersen
Vivaldi Technologies AS

[Yngve N. Pettersen]

Hi,

Yes, we are using the Extended Stable branch, and our usual schedule is to release the desktop version the week after (Extended) Stable release.

As an example of how long an upgrade takes: Chromium 112 branch email was sent out Friday February 24, we started the upgrade process Monday 27, and integration into main codebase was March 13 (and that was a fairly quick update, there have only been a couple of cases of the project being completed in a shorter interval, usually it takes another week), and we still had a major issue with Android AAB bundles that took another week and a partial revert of upstream code to fix The first public desktop snapshot with 112 wasn't released until March 22 at the end of the week after due to various regressions.

So, that was 4 weeks between upstream branch-off and the first public snapshot; just 10 days before upstream final release. Assuming we manage to keep the same schedule with 114, we would get a snapshot out 5 days (including two weekend days) before 114 final. And, as mentioned our plan is to usually release desktop within a week of the upstream release.

The initial upgrade usually cause rebase merge errors in about 10% of our 1400 patched files (the last couple of updates have had 200+ conflicted files), and the process resolving those takes at least two days. The compilation on Windows then usually take another two days, with 150+ cc files having compilation errors of various severity, plus another for Mac and Linux. (Getting Android and IOS to build takes another couple of days each after desktop compiles), in total this process takes 4 or 5 days before the branch can be made available to other developers in the team. Afterwards there is usually two to three weeks fixing issues and evaluating conflict/compile resolutions that significantly changed the code.

On Thursday, 6 April, 2023 00:35:47 (+02:00), Harry Souders wrote:

Hi Yngve,

Thanks for the feedback and the link.

When you say you lose a week, is this because you time your stable releases with Chrome Extended Stable releases?

(Assuming you are still syncing with Extended Stable releases every 8 weeks?)

Just trying to understand a bit more about the concern.

Thanks,

Harry

[Harry Souders]

Hi Yngve,

Thanks for the feedback and the link.

When you say you lose a week, is this because you time your stable releases with Chrome Extended Stable releases?

(Assuming you are still syncing with Extended Stable releases every 8 weeks?)

Just trying to understand a bit more about the concern.

Thanks,

Harry

On Wed, Apr 5, 2023 at 1:07 PM Yngve N. Pettersen <yn...@vivaldi.com> wrote:

[Harry Souders]

Hi Yngve,

Sorry for the late reply.

Without knowing more about your development practices, it's difficult to offer any suggestions. Have you considered upstreaming more of your changes or perhaps integrating before branch and more often (maybe every Dev release?). I'm sure you've considered these options before, so I doubt I'm offering any new insights.

We are trying to accelerate development velocity, and we plan to continue evolving the release cycle to that end -- this means there will likely be less time between branch and stable release in the future.

Thanks,

Harry

[Yngve N. Pettersen]

Hi again,

A few can likely be easily upstreamed, mostly null pointers checks, the problem is rediscovering them among the 1444 patched files.

A large number of changes is addition of our features. Examples are: 

  * Two tab bars (android)

  * Our own editable search engines, which is a major rewrite of the relevant code in chromium.

  * Speed dial on mobile 

  * Our own synced feature (Notes), which causes one set of irritating merge conflicts in the static assert for number of sync types)

  * Added new fields to bookmark items

  * Use platform media to play back video (originally from Opera)

Quite a lot of the changes we have made are workarounds of how we want to stuff, including:

  * changing the UI theme of the Android and IOS

  * How to handle tabs in our own JS-based GUI

Some patches are fixes to make building and our own features work, e.g.:

  * we disabled a recent remote-build action addition in build/config/BUILDCONFIG.gn, It broke our updates of targets

  * freezing the Mac SDK version (occasionally needed; we don't use an hermetic SDK for Mac/IOS)

  * make message_compiler.py, midl.py work

  * Use Vivaldi's version number a lot of places, not Chromium's

  * Changing application names from "chrome" to "vivaldi"

  * Had to make extensive changes in the tools/grit code to replace Google, Chromium etc product names and company names with "Vivaldi", including in translations We have our own system to override strings and translations in the GRD and XTB files. Changing the files themselves is too costly, especially wrt to merge conflicts; we tried it. I would prefer that you remove all product names from the strings and translations, and replace with an argument based on the branding. You can keep strings like "Google Pay" and similar ones that mention specific products.

  * Have to hook in our own extra functionality, like replacing ChromeMainDelegate with our own VivaldiMainDelegate; make sure Vivaldi code is disabled in the unit tests, etc.

to mention a few.

We currently also have two reverts of upstream changes, related to https://crbug.com/1426950 and https://crbug.com/1337134; the latter is likely to be a permanent revert, since the devs have so far refused to undo the change. The other has been fixed AFAICT, but not in my current branch point.

In a number of cases we could probably reduce patches slightly, provided that we could add our own override classes that are produced by our own factories that override the standard ones, e.g. the Sync Engine (we use a hack inside the factory to build our own Sync Engine object). In a number of cases that is currently completely out of the question because the classes are NOT created by a factory, but directly with new/make_unique allocations in the code, meaning it is not practical to override the classes.

I do have several active upstream reviews, most of which have been sitting idle for over two *years* because the reviewers have not decided to accept them. That does not give much confidence that anything major we should choose to upstream will be accepted.

Two of those upstreams are an improvement to the hermetic SDK system and a proof-of-concept suggestion for making it easier to customize some parts of the build, such as application name.

  

At present I have started a new multistage update process, by first rebasing to main as of last Wednesday (5711). It took 3.5  days to complete the rebase (185 conflicted files in 600 commits, the largest conflict batch had 19 broken files, a most less than 5), another 2 plus to get it compiling on Windows (still working on Linux and Mac).

However, something that changed in the Extension system completely broke our GUI on Windows (and presumably Lin/Mac), so the GUI does not launch, at all.

We will probably spend all of this week fixing issues, and into next week.

Later, once you branch, I will rebase up on top of that.

FYI, a few stats: As mentioned there are 1444 updated files, in ~600 commits (and those have been repeatedly squashed to reduce the multi-conflict potential, last time was last week). The -U0 diff is 44K lines, 2 MB; with context 72K lines, 3MB. At all of this is after as much as practical have been moved up into our own module, mostly new functions.

If you want to have a closer look, you can download our source code from https://vivaldi.com/source/