Mozbase, mozilla-aurora, and you!

[Jonathan Griffin]

Traditionally, the work done by the a-team has primarily been done in
trunk branches, and we've been able to ignore mozilla-aurora and
mozilla-beta. Well, those days are coming to an end!

B2G development has moved to mozilla-aurora in order to catch the gecko
18 release train[1]. This will make our lives a little bit more
complicated.

What it means for us:

1 - If you make a change to some test code that is used in B2G but not
by Firefox in mozilla-aurora (Marionette, B2G test runners), you need to
merge the change to mozilla-aurora using the process described at [1].
2 - If you make a change to some test code (like devicemanagerADB.py, or
the SUTAgent, etc) that is used by some of the above, but is also used
by desktop Firefox, you need to merge the change to mozilla-aurora using
the process described at [2].

The tricky part will be dealing with mozbase. Currently, mozbase
changes are checked in to github, and then merged en-masse to
mozilla-central once a week, plus additionally as needed. We'll need to
figure out a strategy for merging changes that affect B2G into
mozilla-aurora. I think we have two options:

Option A: For every merge of mozbase to m-c, perform another merge from
m-c to aurora. This will likely work until we make some mozbase change
that requires updating other affected in-tree consumers. At this point,
we'd need either to update the affected consumers in aurora as well, or
move to Option B.

Option B: Manually create new patches to cherry pick changes from m-c
that are necessary for B2G automation to work in mozilla-aurora. This
is obviously a lot more work, and is more error-prone, as there is no
mozilla-aurora-try.

I think we should plan to go with Option A until and unless we run into
problems with that approach, unless someone has a better idea. With
this plan, I volunteer to shepherd the 'mirror mozbase to m-c' bugs from
m-c to aurora, after Jeff lands them in m-c.

Jonathan



[1]
https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.b2g/dAyDcpYO58c
[2] https://wiki.mozilla.org/Tree_Rules#mozilla-aurora

[William Lachance]

I agree with this; perhaps as a corollary we should try to keep the
number of patches/commits to mozbase modules with large potential side
effects (e.g. mozdevice should always use exceptions, not return codes
to communicate errors:
https://bugzilla.mozilla.org/show_bug.cgi?id=795456) to what's
absolutely necessary until we've got B2G out the door.

(FWIW, I certainly don't have any more changes of that scope in my
pipeline, nor do I have plans to add any)

Will

[Andrew Halberstadt]

Another possible solution is to modify our b2g automation to expect
devicemanager to be available in a virtualenv rather than have the file
copy/pasted into the same directory. There is already a virtualenv in
m-c and the change should be as simple as:

- import devicemanagerADB
+ import mozdevice

This way the mozharness script can use its own version of mozbase that
can be updated separately from m-c/aurora. I kind of hate suggesting
using a separate copy, but I foresee lots of headaches if we need to
wait for blocking-basecamp and approval-aurora/beta every single time
some minor thing needs fixing in mozbase. Obviously this would be
temporary until B2G is shipped.

I wouldn't exactly be thrilled with doing this, but I think it may be
the lesser of two evils.

[Jeff Hammel]

This sounds like it is pointing towards a correct solution to me. In
general, if we can peg b2g to package versions I think we win. If
mozharness is the only concern (which I'm blindly doubting) it already
has the ability to install packages from an internal find-links page
which we can upload to (or, more literally, write a bug to upload to).

[Jonathan Griffin]

I also think this approach makes sense, and have filed
https://bugzilla.mozilla.org/show_bug.cgi?id=800111 to get us to switch
over to using mozbase packages from puppetagain (our "internal pypi")
instead of tests.zip for B2G mozharness scripts.

Note that for changes to non-mozbase code, we'll still have to merge to
aurora.

Jonathan

> _______________________________________________
> tools mailing list
> to...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/tools

[Andrew Halberstadt]

On 10/10/2012 04:19 PM, Jonathan Griffin wrote:
> I also think this approach makes sense, and have filed
> https://bugzilla.mozilla.org/show_bug.cgi?id=800111 to get us to switch
> over to using mozbase packages from puppetagain (our "internal pypi")
> instead of tests.zip for B2G mozharness scripts.
>
> Note that for changes to non-mozbase code, we'll still have to merge to
> aurora.
>
> Jonathan

So we'll need some way to resolve bug 800116 first. Doing so properly
would involve moving all of the fennec automation to use mozharness, but
that might be getting out of scope of what we are trying to accomplish.

A second way would be to do:
try:
import mozdevice
except ImportError:
import devicemanager

In all the harness files. Otherwise we might be stuck mirroring mozbase
to aurora.

[Jonathan Griffin]

An update:

We now have the ash branch to use as a sort of mozilla-aurora try
branch. For any automation patches that need to be landed on aurora,
land on ash first, make sure that the resulting run on TBPL doesn't
include any inexplicable failures, and then land directly on aurora with
a=NPOTB.

Or, add the whiteboard entry [automation-needed-in-aurora] and I'll take
care of it for you.

Per agreement with Alex Keybl and other stakeholders, we don't need to
request approval-mozilla-aurora any longer for automation patches. But
we do need to be responsible for making sure that we don't break any of
the automation presently running on aurora.

Jonathan

On 10/11/12 9:26 AM, Andrew Halberstadt wrote:
> On 10/10/2012 04:19 PM, Jonathan Griffin wrote:

>> I also think this approach makes sense, and have filed
>> https://bugzilla.mozilla.org/show_bug.cgi?id=800111 to get us to switch
>> over to using mozbase packages from puppetagain (our "internal pypi")
>> instead of tests.zip for B2G mozharness scripts.
>>
>> Note that for changes to non-mozbase code, we'll still have to merge to
>> aurora.
>>
>> Jonathan
>

> So we'll need some way to resolve bug 800116 first. Doing so properly
> would involve moving all of the fennec automation to use mozharness,
> but that might be getting out of scope of what we are trying to
> accomplish.
>
> A second way would be to do:
> try:
> import mozdevice
> except ImportError:
> import devicemanager
>
> In all the harness files. Otherwise we might be stuck mirroring
> mozbase to aurora.
>
>
>