SDK compatibility with Firefox versions

[Myk Melez]

Rocketeers!

Until recently, the Firefox minVersion hadn't been updated for a long time because we weren't paying enough attention to its value. Recently, though, @warner and I started thinking about what it should be, and we came to the tentative conclusion that it should represent the oldest version of Firefox that we actually test against and can thus verify that the SDK is compatible with.

Thus we set the Firefox min/maxVersion on the SDK's development branch to 9.0a2/10.0a1, because we automatically test that SDK branch against Firefox's aurora and central branches, which represent the Firefox 9 and 10 releases and are currently set to versions 9.0a2 and 10.0a1, respectively.

The following text on the Development Process page specifies the policy (although we haven't actually gotten all the necessary automated testing stood up yet):

Rel maintains compatibility with Firefox's release and beta branches. Stab maintains compatibility with Firefox's beta and aurora branches. Dev maintains compatibility with aurora and central branches.

However, both @warner and I are unsure that this is the right policy. In particular, it means that addons built by developers who track the SDK's development closely (i.e. by pulling the code from the repository) will be marked as incompatible with the current and upcoming release versions of Firefox, and those developers might want to use development versions of the SDK to ship stable releases of their addons targeted to the stable release of Firefox.

One option for addressing this issue would be to keep the SDK compatible with all four branches of Firefox (central, aurora, beta, and release) simultaneously, doing the necessary engineering work to maintain that compatibility and the necessary release engineering work to prove it via testing, i.e. automatically testing the SDK's development branch against all four Firefox branches (and other SDK branches against the Firefox branches as appropriate).

That seems like a heavy burden, though, for SDK engineers, Release Engineering, and our continuous integration infrastructure.

Note the simple (although unsupported) workaround: change the minVersion in the python-lib/cuddlefish/app-extension/install.rdf file in your SDK installation to the value you want it to be.

Another option would be to make it easier (and supported) to make a minVersion change for your own addon, which would address the problem for individual addons, provided the SDK code your addon accesses is actually compatible with the older version.

And a third, longer-term option would be to land more of the SDK's code into Firefox, reducing the burden of maintaining compatibility between the two products (although potentially creating other compatibility problems).

None of these options are mutually exclusive. And I'm keen to keep the SDK working well for folks who track its development closely, because y'all're the ones filing bugs and contributing fixes!

So if you are one of the addon developers who tracks SDK development closely, let us know how you use the SDK and ship software from it, so we can figure out a policy that works well for you while also being workable for the various engineering teams and infrastructure.

-myk

[David Clarke]

So what is the takeaway here.. I am not sure what we should do to respond to this? What is the proposed workaround, should we change our build script to search / replace across that one file ? Or is there going to be a revert on github, until a longer term solution has been developed. 

[Alexandre poirot]

It sounds like we are unplugging one wagon from the rapid release train, but this wagon has still many people inside!!
I already mentioned during our weekly meeting that we should really support current release version.
I know that we were discussing about stabilization branch, here, we are talking about development.
I'm used to suggest to developers coming in irc channel or group to use development branch as it contains fixes improvement they were looking for and it will be frustrating for them if it wasn't working on current firefox release. It would prevent them to release an addon *now* using a master.
I don't think that adding an option to force compatibility in cfx makes sense. Addon developers won't have any idea of the result if they start using it!

I don't think we have real limitation from the code itself, the only limitation I'm aware of is this old bug that has been fixed in a beta of 4.0. So it is more about automatic testing because we just suppose it is working on old 4.0bx+ version. Then it sounds more like a continuous integration issue than anything else. I mean, if our continuous integration works fine and is simple to run/maintain, this kind of testing won't cost much.
I know that we have lot of issues around testing on another subject (enabling jetpack in platform tinderbox dashboard), but I was hoping it doesn't prevent us from running more simple jetpack tests.

I may have simplify things too much and in that case we still have to round corners of this change.
Because it is far from being clear that "cfx xpi" build an addon that won't work on the same firefox binary you used during "cfx test/run".

2011/10/21 Myk Melez <m...@mozilla.org>

--
You received this message because you are subscribed to the Google Groups "mozilla-labs-jetpack" group.
To post to this group, send email to mozilla-la...@googlegroups.com.
To unsubscribe from this group, send email to mozilla-labs-jet...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mozilla-labs-jetpack?hl=en.

[Myk Melez]

On 2011-10-20 5:03 PM, Anant Narayanan wrote:
> It should be fairly easy to specify a particular branch or tag when pulling in addon-sdk as a dependency for apps. If it makes it easier to maintain the master branch by only supporting a few versions of Firefox at at time, I would recommend maintaining a set of tags that point to an earlier version of the addon-sdk that were tested an known to work with earlier versions of Firefox.
In addition to the master branch, we also have stabilization and release
branches, and the release branch is always marked compatible with the
current stable release of Firefox, so if you specify that branch, you'll
always be able to test your addon with that release. However, then you
are doing periodic integration rather than continuous integration, as
the release branch only gets updated once every six weeks.

-myk

[Matěj Cepl]

Dne 24.10.2011 22:25, Myk Melez napsal(a):

> always be able to test your addon with that release. However, then you
> are doing periodic integration rather than continuous integration, as
> the release branch only gets updated once every six weeks.

Also, IIRC, release and stabilization branches don't cover all four
branches of Firefox either. Which was the only thing I was missing from
whole current setup.

Matěj

--
http://www.ceplovi.cz/matej/, Jabber: mcepl<at>ceplovi.cz
GPG Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC

In those days spirits were brave, the stakes were high, men were
real men, women were real women and small furry creatures from
Alpha Centauri were real small furry creatures from Alpha
Centauri.
-- Douglas Adams

[Myk Melez]

On 2011-10-21 2:59 PM, David Clarke wrote:

So what is the takeaway here.. I am not sure what we should do to respond to this? What is the proposed workaround, should we change our build script to search / replace across that one file ? Or is there going to be a revert on github, until a longer term solution has been developed.  --

Given that y'all're continuously integrating Apps with the SDK master branch, which is a great thing... yes, I'm inclined to revert the change on that branch, setting the Firefox minVersion to the current release version of Firefox (currently 7.0.1, although I'd probably set minVersion to 7.0) on the SDK master branch while leaving minVersion on the SDK stabilization and release branches at their current values, as determined by the current policy.

We aren't actually integrating the SDK master branch with the Firefox release and beta branches, however, so it's possible that a change to the SDK or Firefox on those branches will break compatibility, and we won't know about it.

Lukas: how difficult would it be to add such integration? In one direction, it would mean turning on the tests that use /testing/jetpack/jetpack-location.txt on the beta and release branches, just like we currently have them for the central and aurora branches. In the other direction, it would mean adding test runs against the latest beta and release (and aurora, for that matter) builds to the Jetpack tree.

-myk

[Myk Melez]

On 2011-10-21 4:43 PM, Alexandre poirot wrote:

I don't think we have real limitation from the code itself, the only limitation I'm aware of is this old bug that has been fixed in a beta of 4.0.

I don't know of any actual issues at the moment either, but we'll certainly encounter them and break older versions unless we intentionally make sure to support them, as I did in the fix for bug 695499, whose original fix didn't support older versions of Firefox until its reviewer noticed the bustage.

So it is more about automatic testing because we just suppose it is working on old 4.0bx+ version. Then it sounds more like a continuous integration issue than anything else. I mean, if our continuous integration works fine and is simple to run/maintain, this kind of testing won't cost much.

Our current continuous and periodic integration works fairly well at the moment for the SDK master branch, modulo intermittent test failures and the other issues that are dependencies of bug 629263. However, I'm not sure how easy it is to expand it to the Firefox beta and release branches (and, for the Jetpack tree, also the Firefox aurora branch).

Lukas would know, though.

I may have simplify things too much and in that case we still have to round corners of this change.
Because it is far from being clear that "cfx xpi" build an addon that won't work on the same firefox binary you used during "cfx test/run".

We disabled compatibility checks for "cfx test/run" to remove the danger of busting automated tests because we fail to bump maxVersion in time for the regular Firefox version bump. And that's a worthy cause. But it does indeed create this issue.

We could address it by warning users about the compatibility mismatch on `cfx run/test`. Or we could revert to the previous behavior of failing to run/test unless an "ignore compatibility" flag is set, which the test automation could then set while running tests.

Overall, I'm inclined to make `cfx run/test` fail to run/test if a compatibility issue is detected, but allow the user/automation to override that with a flag.

-myk

[Irakli Gozalishvili]

To me it would make most sense to have min version set to the current stable version of FF (in the master branch) & update min version on next release of FF. (In fact that matches my default test environment as well). Min version in stabilization branch will end up being whatever happens to be on the master during merge, which is exactly right IMO. In addition to that, I think we should allow devs to override compatibility range if they decide so by setting appropriate values in package.json.  

Regards
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/
Address: 29 Rue Saint-Georges, 75009 Paris, France

[Myk Melez]

On 2011-10-24 13:43, Myk Melez wrote:
> Given that y'all're continuously integrating Apps with the SDK master
> branch, which is a great thing... yes, I'm inclined to revert the
> change on that branch, setting the Firefox minVersion to the current
> release version of Firefox (currently 7.0.1, although I'd probably set
> minVersion to 7.0) on the SDK master branch while leaving minVersion
> on the SDK stabilization and release branches at their current values,
> as determined by the current policy.

Ok, I've gone ahead and done this:

https://github.com/mozilla/addon-sdk/commit/72a0e5795be44c8dab6511791aff868193adf28e

David: are your tests passing again after this change? And is there a
dashboard where we can see the results of your tests?

-myk

[KWierso]

So, AMO's list of accepted minVersions don't include wildcards. It
probably should be set to 7.0 before the next release, just to be
safe.

On Oct 25, 4:50 pm, Myk Melez <m...@mozilla.com> wrote:
> On 2011-10-24 13:43, Myk Melez wrote:> Given that y'all're continuously integrating Apps with the SDK master
> > branch, which is a great thing... yes, I'm inclined to revert the
> > change on that branch, setting the Firefox minVersion to the current
> > release version of Firefox (currently 7.0.1, although I'd probably set
> > minVersion to 7.0) on the SDK master branch while leaving minVersion
> > on the SDK stabilization and release branches at their current values,
> > as determined by the current policy.
>
> Ok, I've gone ahead and done this:
>

> https://github.com/mozilla/addon-sdk/commit/72a0e5795be44c8dab6511791...

[Myk Melez]

On 2011-10-27 00:53, KWierso wrote:
> So, AMO's list of accepted minVersions don't include wildcards. It
> probably should be set to 7.0 before the next release, just to be
> safe.

Hmm, but that's exactly what I set it to
<https://github.com/mozilla/addon-sdk/commit/72a0e5795be44c8dab6511791>.
I didn't include a wildcard.

Nor is there a wildcard in the minVersion value on the stabilization
branch
<https://github.com/mozilla/addon-sdk/blob/stabilization/python-lib/cuddlefish/app-extension/install.rdf>,
from which we'll spin the next stable release.

There is a wildcard in the maxVersion on that branch. But that's
expected and accepted by AMO.

-myk

[KWierso]

Oh. Looks like the Planning meeting just got it wrong in the notes:
https://wiki.mozilla.org/Firefox/Planning/2011-10-26#Add-on_SDK

Disregard, then. :)

On Oct 27, 11:24 am, Myk Melez <m...@mozilla.org> wrote:
> On 2011-10-27 00:53, KWierso wrote:> So, AMO's list of accepted minVersions don't include wildcards. It
> > probably should be set to 7.0 before the next release, just to be
> > safe.
>
> Hmm, but that's exactly what I set it to
> <https://github.com/mozilla/addon-sdk/commit/72a0e5795be44c8dab6511791>.
> I didn't include a wildcard.
>
> Nor is there a wildcard in the minVersion value on the stabilization
> branch

> <https://github.com/mozilla/addon-sdk/blob/stabilization/python-lib/cu...>,

[Myk Melez]

On 2011-10-27 09:35, KWierso wrote:
> Oh. Looks like the Planning meeting just got it wrong in the notes:
> https://wiki.mozilla.org/Firefox/Planning/2011-10-26#Add-on_SDK

Fixed!

-myk

[David Mason]

Yeah - that was me, I put the splat there because I was unsure (when I
wrote it) if myk had gone with 7.0 or 7.0.1. I should have updated it
when he let us know in the meeting!

Thanks for noticing that though. Its good to be correct