Re: OPEN QUESTION: Porting L10n Nightlies to TaskCluster

[Justin Wood]

Bump (I got one, in private reply here) and also + dev-l10n incase
there is any outside-of-employee use of these.

~Justin Wood (Callek)

On Fri, Sep 9, 2016 at 1:00 PM, Justin Wood <jw...@mozilla.com> wrote:
> Hey Everyone,
>
> tl;dr How do we handle Taskcluster Index's for L10n repacks for
> nightlies. (and does anyone use the current ones?)
>
> So longer story:
>
> Finding artifacts (binaries) and builds/jobs in taskcluster is
> different than buildbot. Not everything is on artifacts.mozilla.org.
>
> In order to facilitate multiple methods to find things, there is what
> we call the "Taskcluster Index" - using this requires an associated
> "Task" that has binaries "uploaded" to it (usually from within the
> very task)
>
> In BUILDBOT world we have a seperate "taskcluster upload" step/section
> in many jobs, that attaches the binaries that buildbot generated to a
> taskcluster task, and moves on.
> e.g.: Nightly: gecko.v2.mozilla-central.nightly.latest.firefox [1]
> L10n: gecko.v2.mozilla-central.latest.firefox-l10n [2]
>
> For l10n these indexes expand to stuff like
> "gecko.v2.mozilla-central.latest.firefox-l10n.linux-opt.es-ES"
>
> However, in buildbot, what roughly happens is
> * We do a bunch of repacks, at once, in a "chunk" (same
> machine/object directory/etc)
> * We then upload all those repacks to archive.mozilla.org
> * To support the taskcluster index, we upload them, one at a time,
> to individual index entries.
> * CON: These binaries are not in a guessable-from-outside name,
> since they contain version number (not-a-new-issue)
>
> With taskcluster things get harder, because:
> * Uploading to a `new` task an artifact we'll already be uploading
> to our own task is harder (and a resource drain)
> * While we can mark multiple indexes for the same task, these
> indexes have to be defined at task definition time, so defining
> {locale} for a chunk is harder than I'd like
> * Even if we do mark multiple locales to point at one chunk, the
> location of the binaries will be different (and could end up being
> named differently due to simplifying package naming) so its no longer
> as simple for consumers of the index (give me the only .exe there)
> * Thus far we have strived hard to keep indexes used while the jobs
> were on buildbot and indexes used while the jobs are migrated to
> taskcluster the same without changing.
>
> If you note above the existing buildbot index's are not even using
> `.nightly.` for L10n either though, which does give us some room to
> play here.
>
> So my overall ask, is Does anyone have any hard requirements, explicit
> non requirements (in their own mind) and/or know of *anyone* who uses
> the existing indexes for L10n?
>
> (Please CC anyone who does *use* these, since their use case will be
> what I am trying most to keep working, if possible)
>
> Thank You,
> ~Justin Wood (Callek)
>
>
> [1] - https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-central.nightly.latest.firefox/gecko.v2.mozilla-central.nightly.latest.firefox
> [2] - https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-central.latest.firefox-l10n/gecko.v2.mozilla-central.latest.firefox-l10n

[Axel Hecht]

I'd be very surprised if anybody used indexes.

If anything, the lastest-nightly ftp view is what I look at.

CCing Pascal for localized nightlies and the page, though I suspect that
that's just product-details.

Axel