dev process improvements

[Myk Melez]

Rocketeers!

As we prepare to release SDK 1.1, let's look back on its development cycle and think about process improvements to make for the following ones. Here are several I have identified:

Update Version On Development Branch

We didn't update the version identifier on the development branch for the 1.1 cycle (nor for the 1.2 cycle currently underway), so folks who cloned the repository didn't have a good indicator of the approximate version they were using.

I propose we update the version on the development branch at the beginning of each cycle to be [major].[minor]a0, where [major].[minor] represents the identifier with which we intend to ship the code under development, f.e. 1.3a0 when the development branch represents the code we intend to ship as SDK 1.3, which indicates:

1. via [major].[minor], the version under development;
2. via "a", that the code is at an alpha level of quality;
3. via "0", that the code is unreleased and precedes any test build we may spin.

Another option would be to use the similar [major].[minor]a1pre, which we've used before (as has Firefox and other XUL apps). But that seems more complicated without a commensurate improvement in clarity.


Spin Only Beta Test Builds

For 1.1, we spun two alphas, two betas, and a release candidate. Besides being complicated, it isn't clear how the quality of the alphas differed from the betas, as both were high quality (indeed, we landed very few bug fixes on the stabilization branch).

Also, such a progression suggests that the code on the development branch is pre-alpha, whereas in fact the development branch maintained at least an alpha level of code quality throughout the 1.1 cycle (and over the last six weeks of the 1.2 cycle).

So I propose that we dispense with spinning alphas and spin only betas off the stabilization branch, spinning the first beta one week after we merge to stabilization, which gives us a week to fix or back out any changes that prevent that first test build from being beta quality.


Release Three Weeks Before Firefox

Our current schedule has us releasing two weeks before each Firefox release. And that seems like enough time for us to repack addons (we'll know for sure once we finish the 1.0 -> 1.1 repacks). But sometimes our repackaging service will fail to repack an addon, either because it was created with a non-release version of the SDK or because of an unidentified issue in the SDK or the service.

In those situations, the best we can do is notify authors that they need to repack their addons manually for the addons to be compatible with the upcoming Firefox release. And an extra week seems like useful breathing room for authors to do manual repacks.

Nor does it suggest we'll stabilize the SDK too soon. By the time we spin the final test build, in the fourth or fifth week of the stabilization period, the target version of Firefox will already be in beta and should have long eschewed breaking API changes.

And a three week offset, which puts our merges exactly in between Firefox releases, seems like it'll make it easier to reason about the various combinations of Firefox and SDK branches we should be testing automatically (the plan for which is still not completely morphous).

So I propose we move to a three-week offset by shortening the 1.2 stabilization and 1.3 development cycles by one week, shipping 1.2 on Tuesday, October 18 (instead of October 25), three weeks before Firefox 8, and 1.3 on Tuesday, November 29 (instead of December 6), three weeks before Firefox 9.


Thoughts?

-myk

[dcm]

To me, the proposal for version schemes makes a lot of sense - and will be nice, once learned, to see at a glance where we are. I also agree about skipping alphas as, on this release model, they did seem rather needless.

However, the proposal to move to 3 weeks before a Fx release is something I think I need to think about more myself. I certainly understand the reason behind your proposal, but it seems early to basically release halfway through the beta cycle for Fx. I think the thing I'm really missing here is the repack process and how difficult or easy its going to be.

At this point, I'd be behind implementing the first two items (versions and alphas) and devoting more time to evaluate the third item (3 week pre Fx)



Dave

[Myk Melez]

On 2011-09-20 8:45 AM, dcm wrote:
> I certainly understand the reason behind your proposal, but it seems
> early to basically release halfway through the beta cycle for Fx.

That depends on when we can reasonably determine that a version of the
SDK will be compatible with the version of Firefox it targets.

Firefox versions go through a 24-week cycle, including 18 weeks in
development and six weeks as stable releases. The 18 weeks of
development comprise a six week pre-alpha period on the central branch,
a six week alpha period on the aurora branch, and a six week beta period
on the beta branch.

SDK versions, on the other hand, go through an 18-week cycle, including
12 weeks in development and six weeks as stable releases. The 12 weeks
of development comprise a six week alpha period on the development
(master) branch and a six week beta period on the stabilization branch.

The SDK targets Firefox for compatibility, and an SDK version currently
merges to stabilization two weeks before its targeted Firefox version
merges to beta. The SDK then releases two weeks before Firefox releases.

My proposal is to move up both dates by one week, so the SDK merges to
stabilization three weeks before Firefox merges to beta and releases
three weeks before Firefox releases.

If Firefox intends to stop making incompatible changes before or upon
merging to beta, then both the current and the proposed process seem
reasonable, since they both give enough time (at least four and three
weeks, respectively) to resolve any late-breaking bustage before
shipping the SDK.

(Indeed, if the cutoff for Firefox breaking changes is no later than the
merge to beta, then the SDK ship date could theoretically move up even
further.)

If, on the other hand, Firefox intends to make incompatible changes
during the beta cycle, then the later we ship the SDK, the better.

(Either way, it's always possible that some late breaking change occurs,
triggering a jetscramble. Neither the current process nor the proposed
one is intended to prevent such situations; that's what hotfix releases
are for.)

In making this proposal, I assume that Firefox does indeed intend to
stop making incompatible changes by the merge to beta at the latest.
Indeed, I suspect that such changes are frowned upon even during the
aurora period, although they certainly seem possible, since aurora can
include backouts of incompatible changes made during the central period,
and backing out an incompatible change is itself an incompatible change.

But it would make sense to confirm this.

dcm: perhaps you could ping the product manager for Firefox about it?

-myk

[KWierso]

There's also now the ESR branch/cycle to keep in mind. Are we going to
keep a SDK branch compatible with the full length of an ESR build
while keeping the SDK compatible with Nightly?

[David Mason]

That's a good point Wes, but we don't fully know what parts of the ESR
proposal will be implemented at this point. It's definitely something
to talk about though!

myk: I'll check in with Asa and others on intents for Fx processes
regarding breakages in Aurora & Beta

Dave

[davidi...@gmail.com]

Hi. Glad to see this being so thoroughly thought through. My question is about the Firefox minVersion for each version of the SDK. 1.1 seems to still set 4.0b7. I'd be surprised if anyone is testing with levels going back that far?

I've got a problem with panels with SDK 1.1 and Firefox 6 (that seems fine in Nightly) - I assume it's fair game to raise a bug on that? If it was only a problem in 5, I assume that wouldn't? If so, shouldn't the minVersion be 6?

Cheers,

David

--
You received this message because you are subscribed to the Google Groups "mozilla-labs-jetpack" group.
To post to this group, send email to mozilla-la...@googlegroups.com.
To unsubscribe from this group, send email to mozilla-labs-jet...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mozilla-labs-jetpack?hl=en.

[KWierso]

That was actually a platform bug that was fixed for the Firefox 7
release (tomorrow!), and they didn't deem it important enough to
backport to 6 or earlier.

See https://bugzilla.mozilla.org/show_bug.cgi?id=638430 for details.

[Jeff Griffiths]

If ESR is implemented as-is, it strikes me that we should maintain
compatibility with the earliest currently active ESR release. Under the
ESR proposal there is an overlap period after we release during which
the newer release is 'qualified', and after which there will be at least
one more older ESR release:

https://wiki.mozilla.org/Enterprise/Firefox/ExtendedSupport:Proposal

Practical example:

* the second ESR release is Fx 13
* there will be a 12-week qualifying period for this release before we
stop releasing the previous ESR release based on Fx 8
* during this period we should continue to support Fx 8.x, but we could
phase out support for Fx 8.x afterwards.

Of course, our minimum supported version might be informed by any number
of things, but I think this neatly captures how ESR might impact it.

Jeff

[Myk Melez]

On 2011-09-26 12:10 PM, David Mason wrote:
> That's a good point Wes, but we don't fully know what parts of the ESR
> proposal will be implemented at this point. It's definitely something
> to talk about though!

Right. The ESR is still just a proposal, so it's worth tracking but not
over-rotating or gating this decision on.

-myk

[Myk Melez]

On 2011-09-26 12:22 PM, davidi...@gmail.com wrote:
> Hi. Glad to see this being so thoroughly thought through. My question
> is about the Firefox minVersion for each version of the SDK. 1.1 seems
> to still set 4.0b7. I'd be surprised if anyone is testing with levels
> going back that far?

That's a very good point. minVersion shouldn't be set lower than a
version we consistently test and explicitly promise to support.

I have been manually testing SDK releases against the targeted Firefox
release (i.e. Firefox 7 for SDK 1.1), and other manual testers like Jeff
have reported results against older releases (i.e. Firefox 6 for SDK
1.2b1), but no one has reported testing anything older.

Nor is it clear that we should support anything older, given Firefox's
version support schedule. Since the SDK is a distinct product, we could
promise compatibility with Firefox versions that the Firefox team
doesn't itself support (f.e. we could make sure SDK 1.2 is also
compatible with Firefox 6 instead of just 7 and 8). But should we?

Note that Brian and I have been working with the Release Engineering
team to stand up automated testing of multiple SDK branches against
multiple Firefox branches, and our current model assumes that each SDK
release needs to be compatible with only the current and upcoming
Firefox releases at the time we release the SDK (i.e. for SDK 1.2 that
would be Firefoxes 7 and 8, since 1.2 will be released after 7 but
before 8).

To address the problem, I propose we update minVersion on the
stabilization branch to the version of the next release of Firefox after
merging development to stabilization. Under this policy, we would have
updated minVersion to 7 after merging two weeks ago in preparation for
SDK 1.2, since the next release version of Firefox is 7 (and that is the
version that the current beta builds advertise). And we will update
minVersion to 8 after merging in a few weeks for SDK 1.3.

A related problem is maxVersion. Our current policy is to update
maxVersion on the development branch in conjunction with Firefox version
number bumps on its central branch. So tomorrow, when Firefox's central
branch version changes to 10a1, I'll update the SDK's maxVersion to
10a1. But this means that after we merge to stabilization, that branch's
maxVersion will claim support for a version it doesn't target. For
example, the stabilization branch for the 1.3 cycle will claim support
for 10a1, although 1.3 targets Firefox 9.

To address this problem, I propose that we update maxVersion on the
stabilization branch to the target version of Firefox (appending ".*" to
cover Firefox hotfix releases) after merging development to
stabilization. Under this policy, we would have changed maxVersion to
8.* after merging two weeks ago in preparation for SDK 1.2, since
Firefox 8 is the target version of Firefox for that SDK release. And we
will change maxVersion from 10a1 to 9.* after merging in a few weeks for
SDK 1.3.

(It's a little weird for maxVersion to decrease after the merge. But we
want the SDK development branch to be compatible with the Firefox
central branch. And we want the stabilization branch to target a
particular Firefox version. So it seems like the best thing to do.)

> I've got a problem with panels with SDK 1.1 and Firefox 6 (that seems
> fine in Nightly) - I assume it's fair game to raise a bug on that? If
> it was only a problem in 5, I assume that wouldn't? If so, shouldn't
> the minVersion be 6?

Yes, please do report the bug! SDK 1.1 is targeted at Firefox 7, which
is about to be released, so if the problem is only with Firefox 6, as
Wes suggests in a later post, then we might not do a hotfix update of
SDK 1.1 to address it at this point. But it's still very helpful to know
about the issue; at the very least, knowledge of such issues will help
to inform these plans!

-myk

[Myk Melez]

Rocketeers!

Based on your feedback in this thread and the weekly meeting, I have finalized the proposals for updating the version on the development branch and spinning only beta test builds, updating the Development Process page to incorporate the changes.

I also noted the three week offset proposal on that page but didn't finalize the proposal, as I'd first like to hear back from dcm about when in Firefox's cycle developers intend to stop making incompatible changes.

I didn't mention the minVersion/maxVersion proposals, as I'd like to wait another day or two for any additional feedback on them first.

-myk

[Myk Melez]

On 2011-09-26 1:48 PM, Myk Melez wrote:
> To address the problem, I propose we update minVersion on the
> stabilization branch to the version of the next release of Firefox
> after merging development to stabilization.

...

> Our current policy is to update maxVersion on the development branch
> in conjunction with Firefox version number bumps on its central branch.

...

> I propose that we update maxVersion on the stabilization branch to the
> target version of Firefox (appending ".*" to cover Firefox hotfix
> releases) after merging development to stabilization.

We should also update minVersion on the development branch in
conjunction with Firefox version number bumps to the version on aurora,
so it tracks the versions on the Firefox branches against which it is
tested.

-myk

[M.-A. DARCHE]

Le 26/09/2011 22:48, Myk Melez a �crit :

> On 2011-09-26 12:22 PM, davidi...@gmail.com wrote:
>> Hi. Glad to see this being so thoroughly thought through. My question
>> is about the Firefox minVersion for each version of the SDK. 1.1 seems
>> to still set 4.0b7. I'd be surprised if anyone is testing with levels
>> going back that far?
>
> That's a very good point. minVersion shouldn't be set lower than a
> version we consistently test and explicitly promise to support.
>
> I have been manually testing SDK releases against the targeted Firefox
> release (i.e. Firefox 7 for SDK 1.1), and other manual testers like Jeff
> have reported results against older releases (i.e. Firefox 6 for SDK
> 1.2b1), but no one has reported testing anything older.
>
> Nor is it clear that we should support anything older, given Firefox's
> version support schedule. Since the SDK is a distinct product, we could
> promise compatibility with Firefox versions that the Firefox team
> doesn't itself support (f.e. we could make sure SDK 1.2 is also
> compatible with Firefox 6 instead of just 7 and 8). But should we?
>

I've read all this thread and sorry not to have reacted right away.
But it's when I've read today that SDK 1.2b2 has minVersion set to 7
that I got scared!

There must be a right balance between the way to old 4.0b7
and the brand new 7.

As I've already posted here I'm in the process of converting
a XUL-chrome.manifest extension to the Addon-SDK. But the current
extension has (for various reasons [*]) many users still with Firefox 5
and some that will stay with Firefox 6 for some time after Firefox 7
is out.

I'm convinced by the new rapid-cycle of Firefox but I don't think
it's necessary the Addon-SDK's job to enforce it. By doing this
it would simply limit the interest of switching over to the Addon-SDK!
For one if the minVersion is too high a number I will have to stop
the porting to the Addon-SDK since our users won't ever have the chance
to be able to use the new developments :-(

Here is a proposed pragmatic addition to the dev process:

Addon-SDK will only officially support the targeted Firefox release
but for giving time to the addon users to switch to the current
Firefox release and still not stopping to use the new developments
and bug fixes of their addon, the Addon-SDK minVersion will be
set to "targeted Firefox version - 2".

This proposal might be revised in a near future (for example
moving to target version - 1), when more people and organizations
have caught up with the new rapid cycles. But the context is not
ready yet. The Addon-SDK was with 4.0b7 just some time ago,
so increasing this value to only 5 won't do any harm.

And finally, as I've done with the old extension I'm still doing
support for, I'll have tests for all versions of Firefox, so I can
contribute results and report here as I have done with Mozmill tests:
http://hg.mozilla.org/qa/mozmill-tests/file/1bfa0d025cc6/tests/addons/fidesfit-client%40fidesfit.org
But please just give me the time to finish the porting and have unit
tests so that I can contribute to!

[*] : Users using the Firefox versions coming with their GNU-Linux
distribution, users having a Firefox version installed by their company
that has not caught up yet with the new rapid cycle of development.

Thank you in advance,

--
Marc-Aur�le DARCHE https://developer.mozilla.org/User:madarche
AFUL http://aful.org/
Association Francophone des Utilisateurs de Logiciels Libres
French speaking Libre Software Users' Association

[Myk Melez]

On 2011-09-28 1:24 AM, M.-A. DARCHE wrote:
> There must be a right balance between the way to old 4.0b7
> and the brand new 7.

By the time we ship SDK 1.2, 1.7 will be three weeks old, so it won't
actually be brand new anymore. In fact, given the current Firefox
release schedule, it will be middle-aged and halfway to retirement!

> As I've already posted here I'm in the process of converting
> a XUL-chrome.manifest extension to the Addon-SDK. But the current
> extension has (for various reasons [*]) many users still with Firefox 5
> and some that will stay with Firefox 6 for some time after Firefox 7
> is out.

You can easily set minVersion to an older version of Firefox for your addon.

> I'm convinced by the new rapid-cycle of Firefox but I don't think
> it's necessary the Addon-SDK's job to enforce it. By doing this
> it would simply limit the interest of switching over to the Addon-SDK!
> For one if the minVersion is too high a number I will have to stop
> the porting to the Addon-SDK since our users won't ever have the chance
> to be able to use the new developments :-(

This isn't about enforcing Firefox's rapid-release cycle, and it isn't
even really about the value of minVersion. It's about which versions of
Firefox the SDK engineers can maintain proven compatibility with, where
by "proven" I mean tested automatically.

I too would like the SDK to maintain proven compatibility with older
versions of Firefox. But right now we're struggling to stand up test
automation that proves compatibility even with newer versions of Firefox
(limping along in the meantime with manual testing). And in the absence
of such proof, it is misleading to continue to claim such compatibility.

> Here is a proposed pragmatic addition to the dev process:
>
> Addon-SDK will only officially support the targeted Firefox release
> but for giving time to the addon users to switch to the current
> Firefox release and still not stopping to use the new developments
> and bug fixes of their addon, the Addon-SDK minVersion will be
> set to "targeted Firefox version - 2".
>
> This proposal might be revised in a near future (for example
> moving to target version - 1), when more people and organizations
> have caught up with the new rapid cycles. But the context is not
> ready yet.

I understand your motivation, and I agree in principle, but I don't
think we have the justification to make this claim at the moment. The
current proposal is to support "targeted version - 1", and I can
certainly see moving to "targeted version - 2" at some point, but I
don't think we're there yet.

> The Addon-SDK was with 4.0b7 just some time ago,
> so increasing this value to only 5 won't do any harm.

Setting minVersion to any value we can't justify does harm by setting
people's expectations incorrectly.

> And finally, as I've done with the old extension I'm still doing
> support for, I'll have tests for all versions of Firefox, so I can
> contribute results and report here as I have done with Mozmill tests:
> http://hg.mozilla.org/qa/mozmill-tests/file/1bfa0d025cc6/tests/addons/fidesfit-client%40fidesfit.org
> But please just give me the time to finish the porting and have unit
> tests so that I can contribute to!

I very much want you to continue porting your addon! So bear in mind
that I'm not proposing intentionally breaking compatibility with older
versions of Firefox. If the SDK currently works with those versions,
it'll continue to do so after the minVersion change, if you change back
the minVersion when packaging your addon.

All I'm proposing is that the minVersion reflect the versions of Firefox
that we've actually tested and proved are compatible. Once we finish
getting test automation up and running that proves such compatibility
with the current and targeted versions of Firefox, if we can extend that
automation to support older versions, I'm all for doing so. But until
then, we're doing our users a disservice by making a claim we cannot
justify.

-myk

[Myk Melez]

On 2011-09-27 4:46 PM, Myk Melez wrote:

I also noted the three week offset proposal on that page but didn't finalize the proposal, as I'd first like to hear back from dcm about when in Firefox's cycle developers intend to stop making incompatible changes.

I have now finalized the three week offset proposal, as all the information I have indicates that Firefox's developers stop making incompatible changes well before the proposed earlier ship dates, giving us plenty of time to maintain compatibility with Firefox changes.

The change is reflected in the Development Process page's Release Schedule section.

SDK 1.2 will now ship on Tuesday, October 18, three weeks before Firefox 8 ships on November 8. SDK 1.3 will now ship on Tuesday, November 29, three weeks before Firefox 9 ships on December 20.

Note: in the process, I also simplified the color scheme for the graph depicting Firefox and SDK releases, but I would love to see an even better graph take its place. Anyone with some graphic design skilz want to give this a go?

-myk

[Myk Melez]

On 2011-09-27 4:46 PM, Myk Melez wrote:

I didn't mention the minVersion/maxVersion proposals, as I'd like to wait another day or two for any additional feedback on them first.

I've now finalized this proposal, and it is reflected in the Development Process and Release Process pages.

Note that finalization doesn't mean this policy will never change. It's always possible to change any part of the process by which we develop and ship the SDK, so we can revisit this and change it as appropriate and warranted.

-myk

[M.-A. DARCHE]

Hello,

Le 04/10/2011 18:42, Myk Melez a écrit :
> On 2011-09-28 1:24 AM, M.-A. DARCHE wrote:
>> There must be a right balance between the way to old 4.0b7
>> and the brand new 7.
> By the time we ship SDK 1.2, 1.7 will be three weeks old, so it won't
> actually be brand new anymore. In fact, given the current Firefox
> release schedule, it will be middle-aged and halfway to retirement!
>
>> As I've already posted here I'm in the process of converting
>> a XUL-chrome.manifest extension to the Addon-SDK. But the current
>> extension has (for various reasons [*]) many users still with Firefox 5
>> and some that will stay with Firefox 6 for some time after Firefox 7
>> is out.
>
> You can easily set minVersion to an older version of Firefox for your
> addon.
>

That may be the part I'm missing!

So please, where is the documentation explaining how to do what
you are suggesting?

I couldn't find it on
https://addons.mozilla.org/en-US/developers/docs/sdk/1.1/
and especially on
https://addons.mozilla.org/en-US/developers/docs/sdk/1.1/dev-guide/addon-development/package-spec.html

On a traditional XUL-chrome.manifest extension one has full control
over the install.rdf and update.rdf files and I know how to change
those minVersion and maxVersions values. But with the Addon-SDK,
the install.rdf file is a generated one.

> This isn't about enforcing Firefox's rapid-release cycle, and it isn't
> even really about the value of minVersion. It's about which versions of
> Firefox the SDK engineers can maintain proven compatibility with, where
> by "proven" I mean tested automatically.

> [...]

> I understand your motivation, and I agree in principle, but I don't
> think we have the justification to make this claim at the moment. The
> current proposal is to support "targeted version - 1", and I can
> certainly see moving to "targeted version - 2" at some point, but I
> don't think we're there yet.

> [...]
>

I take your very nice open-minded answer as a very good
signal for the future, and feel much reassured.

Thank you!

--
Marc-Aurèle DARCHE https://developer.mozilla.org/User:madarche

[Hernan Rodriguez Colmeiro]

You can always unzip the xpi, modify the install.rdf and rezip all
back into an xpi :)

Hernan

[M.-A. DARCHE]

Hello,

Le 08/10/2011 13:53, Hernan Rodriguez Colmeiro a �crit :

> On Oct 8, 2011, at 4:39, "M.-A. DARCHE" <moz...@cynode.org> wrote:
>

>> Le 04/10/2011 18:42, Myk Melez a �crit :

Sure :-)

Was it what you were suggesting Myk?

Thanks!

--
Marc-Aur�le DARCHE https://developer.mozilla.org/User:madarche

[Myk Melez]

On 2011-10-08 12:39 AM, M.-A. DARCHE wrote:
> So please, where is the documentation explaining how to do what
> you are suggesting?

There isn't any documentation for it, since it isn't explicitly
supported, but the install.rdf file is located in
python-lib/cuddlefish/app-extension/. If you modify that file and then
repackage the addon, its XPI will be updated to use the new version of
the file.

-myk

[M.-A. DARCHE]

Le 10/10/2011 18:35, Myk Melez a �crit :

I have tested it and it's very easy to change that way.
And gives all the flexibility needed. I am delighted.

Thanks a lot Myk!

--
Marc-Aur�le DARCHE https://developer.mozilla.org/User:madarche