Future of L10n Releng Automation -- Open for Requests/Use-Cases

[Justin Wood]

Hey Everyone,

I have recently decided to take ownership of Localization from a
Release Engineering perspective. Specifically what this means is that
I'll own most bustages in the process as it relates to our
infrastructure, as well as work on systemic improvements and overall
long-standing desires in the system.

I have recently taken Nick's initial work for L10n on Try (c.f.
footnote 1) And created a similar Taskcluster L10n-on-try job, (c.f.
footnote 2); as part of the Taskcluster work I have helped round out
using the in-tree compare-locales instead of the external repo {older}
copy.

I initially had a draft going on where I hoped to head next, however I
think getting your input will help me formulate concrete
thoughts/plans better.

Two things I do know for sure:
* Make L10n always depend on an en-US build that is based on the
changeset we're using. (This allows us to more-easily [re]-trigger
l10n on older nightly changesets, and can improve the coverage
potential with l10n-on-try)
* Make Android L10n testable on try.

What I am wondering is what sort of use-cases do you want/need from
releng with regard to l10n. What cool thing have you always hoped
Release Engineering could provide that we have never had the resources
for.

What use-cases do you have for L10n builds provided by us currently?
What things could we do that would make your lives easier with regard
to l10n.

(Feel free to think outside the box, or even propose things like "it'd
be awesome if I could click on any UI and do a live translation", or
"It'd be awesome if I could select any string, and see how it would
look live in Firefox on each platform" -- while both of those are cool
ideas, they are outside the realm of what I envision being done
anytime soon, but ideas *you* have can be simpler than you think, or
at least help me to architect around improvements we want, rather than
improvements I *think* we want).

By all means this is an open invitation to tell me both what has been
wrong with Relengs L10n support and what you want from Relengs L10n
Support.

(Due to the nature of the system and lots of technical debt I don't
promise any timelines yet, and will hesitantly promise any actual
features)

Thank You,
~Justin Wood (Callek)

footnotes:
1 - https://wiki.mozilla.org/ReleaseEngineering/TryServer#Desktop_l10n_jobs
2 - force-buildable with `-p linux-l10n,linux64-l10n` or with `-p
linux,linux64` if you touch one of the known-affected files for l10n.

[Axel Hecht]

Hi Justin,

the two you listed sound great, thanks.

I have a few other things:

* l10n (on a small subset) for all jobs on automation (try, integration,
central)
* A well-defined sequence of scopes for taskcluster/buildbot,
mozharness, mach, make.
* Unification of artifact builds and l10n repacks.
* Visibility on the treeherder level which locales are in a build/update
job, even before task completion. I'd love to see on treeherder how the
german build is doing, for example. Maybe also on the pulse level.
* Run-time tests for l10n.
* Run-time tests for l10n on l10n commits.

I'll stop here ;-)

Axel

[Nicholas Alexander]

Hi Callek,

On Mon, Jul 18, 2016 at 6:48 AM, Justin Wood <jw...@mozilla.com> wrote:

> Hey Everyone,
>
> I have recently decided to take ownership of Localization from a
> Release Engineering perspective.

\o/ I'm glad to see clear ownership of this underloved area.

> Specifically what this means is that
> I'll own most bustages in the process as it relates to our
> infrastructure, as well as work on systemic improvements and overall
> long-standing desires in the system.
>
> I have recently taken Nick's initial work for L10n on Try (c.f.
> footnote 1) And created a similar Taskcluster L10n-on-try job, (c.f.
> footnote 2); as part of the Taskcluster work I have helped round out
> using the in-tree compare-locales instead of the external repo {older}
> copy.
>
> I initially had a draft going on where I hoped to head next, however I
> think getting your input will help me formulate concrete
> thoughts/plans better.
>
> Two things I do know for sure:
> * Make L10n always depend on an en-US build that is based on the
> changeset we're using. (This allows us to more-easily [re]-trigger
> l10n on older nightly changesets, and can improve the coverage
> potential with l10n-on-try)
>

It's my opinion that Releng needs to figure out the "task B depends on
build A depends on pre-step Z" flow. When I last checked, ~3 months ago,
we had one case (test B depends on build A) and it was not handled
generally. Defining and documenting this (if it hasn't already happened!)
unlocks the value in TC.

> * Make Android L10n testable on try.
>

Axel brings it up below, but making l10n repacks work like artifact builds
would go a long way to improving turn-around times. It's not clear this is
in Releng's court -- I think the build system needs to change to make this
happen.

Nick

[Gregory Szorc]

On Mon, Jul 18, 2016 at 6:48 AM, Justin Wood <jw...@mozilla.com> wrote:

> Hey Everyone,
>
> I have recently decided to take ownership of Localization from a

> Release Engineering perspective. Specifically what this means is that

> I'll own most bustages in the process as it relates to our
> infrastructure, as well as work on systemic improvements and overall
> long-standing desires in the system.
>
> I have recently taken Nick's initial work for L10n on Try (c.f.
> footnote 1) And created a similar Taskcluster L10n-on-try job, (c.f.
> footnote 2); as part of the Taskcluster work I have helped round out
> using the in-tree compare-locales instead of the external repo {older}
> copy.
>
> I initially had a draft going on where I hoped to head next, however I
> think getting your input will help me formulate concrete
> thoughts/plans better.
>
> Two things I do know for sure:
> * Make L10n always depend on an en-US build that is based on the
> changeset we're using. (This allows us to more-easily [re]-trigger
> l10n on older nightly changesets, and can improve the coverage
> potential with l10n-on-try)

> * Make Android L10n testable on try.
>

Thank you for taking ownership of this, Justin! You definitely have more...
courage... than me :)

Here's my wish list:

* Visibility. Right now I believe a lot of l10n jobs report to some random
buildbot server. I want to see the results on Treeherder like everything
else.
* In-tree, TaskCluster jobs. I can't remember if things are all in
mozharness now or if there are lingering jobs still in buildbot. Either
way, moving everything to in-tree (first step) as TaskCluster jobs (second
step) would be a huge win.
* Testing l10n/packaging configurations against any build. When the build
system changes, we often bust l10n or packaging but don't find out for
weeks until a job runs on e.g. Beta. I'd like to see *some* l10n packaging
occur as part of regular CI. This includes e.g. performing a Beta or
Release packaging simulation against a commit still on central. Detecting
bustage weeks after something lands is not tenable.
* l10n repo management. Having to "fork" l10n repos for each release we do
is silly for the same reasons that having multiple repos for Firefox is
silly. I prefer we didn't have to do this. To be clear, the volume or size
of these repos is not posing a scaling concern to hg.mozilla.org. I just
wish we could be a little bit smarter about not creating so many. My
understanding is changing things would require localizers to change
workflows, so this probably isn't trivial to do. It is certainly low on my
priority list.

[Axel Hecht]

This is fixed,
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-job_group_symbol=L10n&filter-job_group_symbol=Update-3&filter-job_group_symbol=Update-1&filter-job_group_symbol=Update-2&exclusion_profile=false
shows things that are more or less l10n related these days. Which is
l10n nightlies and the update job queue.

> * In-tree, TaskCluster jobs. I can't remember if things are all in
> mozharness now or if there are lingering jobs still in buildbot. Either
> way, moving everything to in-tree (first step) as TaskCluster jobs (second
> step) would be a huge win.

I agree with the goal. The way you phrased this also says to me that we
need a clearly defined role for mozharness.

> * Testing l10n/packaging configurations against any build. When the build
> system changes, we often bust l10n or packaging but don't find out for
> weeks until a job runs on e.g. Beta. I'd like to see *some* l10n packaging
> occur as part of regular CI. This includes e.g. performing a Beta or
> Release packaging simulation against a commit still on central. Detecting
> bustage weeks after something lands is not tenable.

Agreed.

> * l10n repo management. Having to "fork" l10n repos for each release we do
> is silly for the same reasons that having multiple repos for Firefox is
> silly. I prefer we didn't have to do this. To be clear, the volume or size
> of these repos is not posing a scaling concern to hg.mozilla.org. I just
> wish we could be a little bit smarter about not creating so many. My
> understanding is changing things would require localizers to change
> workflows, so this probably isn't trivial to do. It is certainly low on my
> priority list.

It's more about changing how we develop Firefox, actually. The localizer
workflow will get a little bit of change, but most localizers won't need
to change a lot.

The change on the Firefox side is how we change strings, and at which
point we can remove strings from the tree. I wrote most of that on
https://wiki.mozilla.org/L10n:Firefox_in_2016.

I had been working on testing which strings are actually used in our
code, and it's a mess. Such a mess that I'm optimistic that we'll get to
a better place faster by including these changes into the rewrite to
l20n, than trying to fix it on the chrome://*/locale/* side. We just
have a lot of unexpected content behind chrome urls, sadly, and that
makes it really hard to reason about runtime behavior based on our
source code.

Axel