Firefox Development/Release Process Specifics [DRAFT]
Christian Legnitto
--------------------------
* Check out http://mozilla.github.com/process-releases/draft/development_overview/
Details
--------------------------
* Rob's previous document[1] dealt with an overall framework to think about development/releases in an accelerated release world
* We now need to start dealing with specifics to unblock various teams
* There is a draft document available at http://mozilla.github.com/process-releases/draft/development_overview/ which deals with specifics
* This is BY NO MEANS FINAL, but we need something concrete to play around with and attack. I expect RelEng specifically to weigh in heavily
* Please discuss the document here and submit specific changes as pull requests[2]
Thanks,
Christian
[1] http://mozilla.github.com/process-releases/draft/development_overview/
[2] https://github.com/mozilla/process-releases/tree/gh-pages
Christian Legnitto
http://mozilla.github.com/process-releases/draft/development_specifics/
Thanks,
Christian
On Mar 29, 2011, at 11:38 AM, Christian Legnitto wrote:
> Summary
> --------------------------
>
> * Check out http://mozilla.github.com/process-releases/draft/development_specifics/
>
>
> Details
> --------------------------
>
> * Rob's previous document[1] dealt with an overall framework to think about development/releases in an accelerated release world
>
> * We now need to start dealing with specifics to unblock various teams
>
> * There is a draft document available at http://mozilla.github.com/process-releases/draft/development_overview/ which deals with specifics
>
> * This is BY NO MEANS FINAL, but we need something concrete to play around with and attack. I expect RelEng specifically to weigh in heavily
>
> * Please discuss the document here and submit specific changes as pull requests[2]
>
> Thanks,
> Christian
>
> [1] http://mozilla.github.com/process-releases/draft/development_overview/
> [2] https://github.com/mozilla/process-releases/tree/gh-pages
>
>
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
Robert Kaiser
> * Please discuss the document here and submit specific changes as pull requests[2]
1) I know someone else already mentioned this somewhere, but why is
mozilla-release not just being done as named relbranches in the
mozilla-beta repo? That would basically match current practice and we
already have all the instrumentation etc. to release chemspills off such
a relbranch. (Of course, that might also open a naming discussion on
mozilla-beta itself - welcome to the bikeshed!)
2) Versioning: I see problems with both models, which make me happy with
neither of them, though both of my itches (I really want a better word
than "problem") are solved in one of the models.
The standard model gives us no way to find out what channel someone is
one when he gives us a version number, and so things can easily become
confusing on support forums, journalist reports, etc. ("I have Firefox
5.0" - er, the beta or release channel one?)
The Chrome model makes us end up with the release channel having the
weirdest numbers that don't look like finals to anyone who usually
understands version numbers (it's confusing enough that Google's finals
are hard to identify, doesn't need to be the same with us) -
5.10.45.233? Er, was that a beta or a final? I certainly can't remember
even after a day...
I don't really want to go into fully bikeshedding here, so don't
consider this one seriously, but I once dreamed up we could go with
5.0a1...5.0aXXX on central, 5.0b1...5.0bXXX on the next one but then the
question of what the other channel has would come up - though maybe
central could just stay with 5.0a1pre, and experimental would go 5.0a1
etc. and beta would go 5.0b1 etc.? well, as I said, probably too much
bikeshedding to even bring this up here.
3) L10n: I hope there will still be a possibility for localizations to
follow mozilla-central and have localized bleeding-edge development
nightlies. We need to catch issues as early as possible, esp. when we
might be changing L10n infrastructure (which is an ongoing plan).
4) Firefox 6: I wonder how 2 times 5 weeks and 2 times 6 weeks leaves a
gap of a week in the merge from experimental to beta... but maybe I'm
misunderstanding something there. ;-)
The rest of this sounds good, I'm looking forward to seeing this getting
deployed!
Robert Kaiser
--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community needs answers to. And most of the time,
I even appreciate irony and fun! :)
Alexander Limi
> ("I have Firefox 5.0" - er, the beta or release channel one?)
FWIW, we've already discussed this in bug on the channel
switcher<https://bugzilla.mozilla.org/show_bug.cgi?id=644517>,
and the consensus so far is that we'll massage the version numbers reported
a bit, to make it very clear what channel you're currently on — e.g.
"Firefox 6.x.y (experimental)" or "Firefox 7.x.y (beta)".
But yeah, bikeshed alert! ;)
--
Alexander Limi · Firefox UX Team · @limi <http://twitter.com/limi> ·
limi.net
Benjamin Smedberg
> Summary
> --------------------------
>
>
> * Please discuss the document here and submit specific changes as pull requests[2]
Specific issues that need to be addressed:
* How often will builds be made on the experimental and beta channels?
* In the 8 weeks before Firefox 5 goes to the beta channel, are we going
to use the beta channel to do testing of the 4.0.x builds?
* Will builds on the experimental or beta channels be signed? How does
this affect the release cadence? (Signing requires manual intervention
from releng)
* I'm not sure I understand the relationship between this page and the
https://wiki.mozilla.org/RapidRelease page which JPR has created. Can
these be unified, or at least clarify the difference?
* I think we very soon need to have lots of specifics about how l10n
branching will interact with mozilla-central branching. Since many
language teams will be localizing solely on experimental, and not on
trunk, we may need branch the localization repositories differently than
mozilla-central, or even have different branching plans per-language
depending on whether each language does work against mozilla-central.
* I think we need to develop (probably over time as we develop
experience) our extension compatibility story. There needs to be mention
of not breaking jetpack, as well as being a lot more proactive about the
API and XUL changes which are made between these shorter releases.
--BDS
Justin Wood (Callek)
repo to use for .x releases. Because this new model doesn't have .x
releases (only chemspill/rapid response updates) we don't need to copy
to the releases directory and deal with the version naming. That being
said, we could still do so if we wanted to but there doesn't appear to
be much benefit/point. Please give feedback.
My comment is that doing it will be a large benefit for projects outside
of Firefox, but not for Firefox. Which includes benefiting distros.
My reasoning:
It allows a single place for projects/distro's to pull from, expecting
tip to always work for the release they are trying to build.
This could be for an XULRunner based project, ICEWeasel, etc. But means
that if there is a chemspill on a 3rd-party application (which USES
Gecko) that they don't accidentally break by pulling tip of
Firefox.next, and/or don't accidentally MISS a chemspill change on the
Firefox.ver they had first built with.
I'm not sure if that reason is compelling enough to do the magic here.
>>Version Scheme<<
I really dislike the chrome style, severely, for our version numbering.
The largest reason I have not seen expressed [I could just have missed
it] is that it bloats the changelog, and with hg that is copied to the
users computer on every checkout. It also means that the version number
will be different with _every_ nightly. And have at least a checkin a
day, even if no other changes.
I am concerned of the "Standard" scheme in terms of identifying which
channel a user is on, so I'm inclined to like KaiRo's bikeshedding of
version number better :-)
>>Gecko Version<<
We did this with Mozilla Suite, do we _really_ feel the need to have the
Gecko version tied so tightly with the product version, especially in
such a rapid release process? I personally think it is a bad idea, and
would rather avoid it.
--
~Justin Wood (Callek)
On 3/29/2011 2:38 PM, Christian Legnitto wrote:
Chris AtLee
> Doh, the link to the specifics is:
>
> http://mozilla.github.com/process-releases/draft/development_specifics/
>
> Thanks,
> Christian
A few quick questions:
- are we going to be doing nightly builds (with nightly update channels)
on the experimental, beta and release branches? "nightly-experimental",
"nightly-beta" and "nightly-release"?
- you mention that the anticipated update rate for beta users is weekly,
does this means we'll be doing weekly release builds to the beta channel?
- will there be anything other than dep and nightly builds on the
experimental stream?
Regarding automated merging...If we only have one repo per stream
(instead of a repo per version like dbaron suggests), then I don't think
blowing away the repo and starting fresh is a great idea because it
invalidates history, links to revisions in bugmail, etc.
Would a true merge be that hard to do?
You *could* automate the a fake merge like this:
rsync -a --exclude .hg --delete mozilla-central/ mozilla-experimental/
hg -R mozilla-experimental addremove
Robert Sayre
>
> Specific issues that need to be addressed:
>
> * How often will builds be made on the experimental and beta channels?
This question has come up a bunch. The intent was for them to be
nightly. Build frequency should probably be clearer in my document.
> * Will builds on the experimental or beta channels be signed? How does
> this affect the release cadence? (Signing requires manual intervention
> from releng)
Is signing a requirement for unobtrusive updates?
> * I'm not sure I understand the relationship between this page and the
> https://wiki.mozilla.org/RapidRelease page which JPR has created. Can
> these be unified, or at least clarify the difference?
JPR's document is tracking the work needed to solidify both release
management documents.
> * I think we need to develop (probably over time as we develop
> experience) our extension compatibility story. There needs to be mention
> of not breaking jetpack, as well as being a lot more proactive about the
> API and XUL changes which are made between these shorter releases.
Fully agree.
- Rob
Robert Sayre
>
>
> * How often will builds be made on the experimental and beta channels?
This question has come up a bunch. The intent was for them to be
nightly. Build frequency should probably be clearer in my document.
> * Will builds on the experimental or beta channels be signed? How does
> this affect the release cadence? (Signing requires manual intervention
> from releng)
Is signing a requirement for unobtrusive updates?
> * I'm not sure I understand the relationship between this page and the
> https://wiki.mozilla.org/RapidRelease page which JPR has created. Can
> these be unified, or at least clarify the difference?
JPR's document is tracking the work needed to solidify both release
management documents.
> * I think we need to develop (probably over time as we develop
> experience) our extension compatibility story. There needs to be mention
> of not breaking jetpack, as well as being a lot more proactive about the
> API and XUL changes which are made between these shorter releases.
Fully agree.
- Rob
Chris AtLee
one more thing...
what's the benefit of re-organizing existing hg repos? You mention that
it facilitates automated tools, but I think any tools are going to need
a canonical list of release and project branches anyway. We may as well
record the existing location for e.g. tracemonkey instead of moving it
into projects/js/tracemonkey.
L. David Baron
> You *could* automate the a fake merge like this:
> rsync -a --exclude .hg --delete mozilla-central/ mozilla-experimental/
> hg -R mozilla-experimental addremove
There are probably better ways to do a merge that blows away all
the changes on one side. In particular, quoting from
http://mercurial.selenic.com/wiki/TipsAndTricks#Keep_.22My.22_or_.22Their.22_files_when_doing_a_merge
# Occasionally you want to merge two heads, but you want to throw
# away all changes from one of the heads, a so-called dummy merge.
# You can override the merge by using the ui.merge configuration
# entry:
#
# $ hg --config ui.merge=internal:local merge #keep my files
# $ hg --config ui.merge=internal:other merge #keep their files
#
# Here local means parent of working directory, other is the head
# you want to merge with. This will leave out updates from the
# other head.
#
# However, note that files added in the other head wont cause a
# conflict, and therefore no merging will be done. To merge X into
# the current revision without letting any of the changes from X
# come through, do:
#
# hg --config ui.merge=internal:fail merge X
# hg revert --all --rev .
#
# This will ensure that only changes from the current working copy
# parent revision are committed when you commit the merge.
#
# Using internal:fail will fail the merge - this is useful if you
# want to prevent Mercurial from starting a merge tool after a
# merge with conflicts.
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Christian Legnitto
On Mar 29, 2011, at 12:57 PM, Robert Kaiser wrote:
> Christian Legnitto schrieb:
>> * Please discuss the document here and submit specific changes as pull requests[2]
>
> 1) I know someone else already mentioned this somewhere, but why is mozilla-release not just being done as named relbranches in the mozilla-beta repo? That would basically match current practice and we already have all the instrumentation etc. to release chemspills off such a relbranch. (Of course, that might also open a naming discussion on mozilla-beta itself - welcome to the bikeshed!)
It could be done that way. I feel it is cleaner to keep release-related metadata next to the released code, which is easiest in its own repo (older releases can still exist as tags, heads, or branches.)....
* You are assuming we will do chemspills off relbranches. I think that is most likely a faulty assumption (but I do note there is nothing in that section of the doc yet! Hang tight)
* 3rd party developers developing on top of mozilla releases only need to pull from mozilla-release tip to get the code for the latest release. They only have to hg pull to slurp in any chemspill or later release changes they may not even be aware of after cloning. I know I would find having a repo preferable to having to hunt around in hg for the proper relbranch name, determine if it is the latest, etc. Sure it'd probably be pretty easy...but not as easy as just pulling mozilla-release
* Similar to the last point, I'd say over 90% of the time people want code for the latest release. Using a repo allows us to hide the 10% info and default to the 90%. With relbranches hunting for an older release is as much work as hunting for the latest release
* Similar to the last two points, if a developer needs to look at the source for the latest release to investigate a bug there is virtually no chance they will get confused and look at the wrong code
* There is no potential for tagging or branching at the wrong changeset (hey, mistakes happen). We merely take what is on mozilla-beta tip at the specified date and dump it into mozilla-release
* It fits in with the rest of the repo/channel scheme nicely. There is something to be said for consistency, especially when automating
* We can have hg hooks and tools that trigger off landings on mozilla-releases. For example, we can edit the "Releases" wiki page in a hg hook (or a tool triggered by a hook) to update the actual release date automatically. We can comment in bugs saying "This bug fix has been released in Firefox X" for all bug numbers mentioned in the last merge. We can add entries to product details automatically. Sure, this can be done with relbranches but it would be more work, a lot less clear, and have potential to impact general mozilla-beta development
As an aside, repo names are not finalized (and they are part of the proposal so I don't feel it is bikeshedding). But, I feel it's best to not argue about them until we have a specific proposal from Product Marketing. That being said, I would love feedback on the whole [product]-[channel] format as I feel it has a lot of underlying power and has the potential to reduce a lot of logic for cross-product tools.
> 2) Versioning: I see problems with both models, which make me happy with neither of them, though both of my itches (I really want a better word than "problem") are solved in one of the models.
Ok, this is great feedback!
> The standard model gives us no way to find out what channel someone is one when he gives us a version number, and so things can easily become confusing on support forums, journalist reports, etc. ("I have Firefox 5.0" - er, the beta or release channel one?)
I don't see how this would be a problem. With silent auto-updates we are assuming 98% of each channel audience is on the latest build for that channel after 24 hours (based on comments from Google, trying to find the source). Thus, when a build is 5.0 on the release channel 98% of users on the beta channel already have something called 6.0 within 24 hours. So if we are told they have "Firefox 5.0" and we have pushed 5.0 to the release channel we know they are on the release channel rather than the beta channel (as they would be telling us they have "Firefox 6.0" instead).
> The Chrome model makes us end up with the release channel having the weirdest numbers that don't look like finals to anyone who usually understands version numbers (it's confusing enough that Google's finals are hard to identify, doesn't need to be the same with us) - 5.10.45.233?
We are demoting version numbers everywhere. The whole version won't be reflected in the UA (my understanding from bsmedberg is it would stay at Firefox 5.0 throughout 5's development) or UI.
> Er, was that a beta or a final? I certainly can't remember even after a day...
Well, we need a marker for "this version is the latest on the release channel". From that it is easy to determine based on the version number, assuming again that 98% of a channel is updated to the latest version on that channel in 24 hours.
Additionally, we can tell just by looking at the version number what repo a build came from:
* 5.0.0.[not zero] ? You know that build was created out of the mozilla-central repository during Firefox 5 development
* 5.0.[not zero].[not zero] ? You know that build was created out of the mozilla-experimental repository during Firefox 5 development
* 5.[not zero].[not zero].[not zero]. ? You know that build was created out of the mozilla-beta repository during Firefox 5 development
> I don't really want to go into fully bikeshedding here, so don't consider this one seriously, but I once dreamed up we could go with 5.0a1...5.0aXXX on central, 5.0b1...5.0bXXX on the next one but then the question of what the other channel has would come up - though maybe central could just stay with 5.0a1pre, and experimental would go 5.0a1 etc. and beta would go 5.0b1 etc.? well, as I said, probably too much bikeshedding to even bring this up here.
I'd love to see this proposal properly written up with the intricacies hashed out. Just because there are two proposals in the document doesn't mean those are the only two that are valuable to choose from.
> 3) L10n: I hope there will still be a possibility for localizations to follow mozilla-central and have localized bleeding-edge development nightlies. We need to catch issues as early as possible, esp. when we might be changing L10n infrastructure (which is an ongoing plan).
Yes, I believe that is expressed in the document already. Axel has made it very clear having more localization involved earlier is better for the product overall. That being said, he mentioned a lot of localizers like to set their calendars, do their work, and then go away until their calendar says it is time again.
> 4) Firefox 6: I wonder how 2 times 5 weeks and 2 times 6 weeks leaves a gap of a week in the merge from experimental to beta... but maybe I'm misunderstanding something there. ;-)
It's 2 gaps...1 week on mozilla-experimental and 1 week on mozilla-beta. So 2*5+2 = 2*6. If you stare at my graphic long enough it'll make sense ;-) I'll try to come up with a clearer explanation and/or graphic...but no promises.
> The rest of this sounds good, I'm looking forward to seeing this getting deployed!
I really appreciate the detailed feedback!
Thanks,
Christian
Christian Legnitto
On Mar 29, 2011, at 2:23 PM, Benjamin Smedberg wrote:
> On 3/29/2011 2:38 PM, Christian Legnitto wrote:
>> Summary
>> --------------------------
>>
>> * Check out http://mozilla.github.com/process-releases/draft/development_overview/
>>
>> * Please discuss the document here and submit specific changes as pull requests[2]
> Can I just provide feedback here and let you incorporate it?
Sure.
>
> Specific issues that need to be addressed:
>
> * How often will builds be made on the experimental and beta channels?
mozilla-experimental and mozilla-beta will be built nightly and the resulting builds will be placed on the experimental and beta channels respectively. Ideally the builds would only happen if changes have come into the mozilla-* or l10n-* central repository, but we are willing to have nightly updates as the base case. This is why silent updates are critical to the whole process.
> * In the 8 weeks before Firefox 5 goes to the beta channel, are we going to use the beta channel to do testing of the 4.0.x builds?
Still figuring that out, but most likely yes. I'm not sure we need the whole 4.0 beta audience (~2.7 million) but don't think it would hurt.
> * Will builds on the experimental or beta channels be signed? How does this affect the release cadence? (Signing requires manual intervention from releng)
Not sure we need to sign them. I don't think this will affect the release cadence at all.
> * I'm not sure I understand the relationship between this page and the https://wiki.mozilla.org/RapidRelease page which JPR has created. Can these be unified, or at least clarify the difference?
Looks like this has been answered elsewhere, but I'll restate it here. The wiki page is to organize what work is being done and what is remaining, the proposal pages delve into specifics about the work.
> * I think we very soon need to have lots of specifics about how l10n branching will interact with mozilla-central branching. Since many language teams will be localizing solely on experimental, and not on trunk, we may need branch the localization repositories differently than mozilla-central, or even have different branching plans per-language depending on whether each language does work against mozilla-central.
This is the underlying context for the note in the localization section of the proposal. Essentially (correct me if I am wrong Axel) l10n wants to take the current l10n-central repo and have it track mozilla-experimental. A new repository (let's call it l10n-noname) with a subset of locales would be created which would then track mozilla-central. When mozilla-central merges to mozilla-experimental l10n-noname merges to l10n-central.
I expressed my desire for the naming and content to match mozilla-*'s for clarity but it may cause tons of tooling changes due to the assumption that l10n-central have everything (which breaks in this scheme as it would only have the ~8 locales while l10n-experimental has all the locales).
Admittedly l10n stuff is fuzzy to me but Axel is aware of the proposal and active in discussions.
> * I think we need to develop (probably over time as we develop experience) our extension compatibility story. There needs to be mention of not breaking jetpack, as well as being a lot more proactive about the API and XUL changes which are made between these shorter releases.
I agree. Luckily, as a release manager this is something I get to gloss over and let other people figure out the details :-)
Thanks,
Christian
Mook
<snip>
> * 3rd party developers developing on top of mozilla releases only need to pull from mozilla-release tip to get the code for the latest release. They only have to hg pull to slurp in any chemspill or later release changes they may not even be aware of after cloning. I know I would find having a repo preferable to having to hunt around in hg for the proper relbranch name, determine if it is the latest, etc. Sure it'd probably be pretty easy...but not as easy as just pulling mozilla-release
As a 3rd party developer developing on top of mozilla releases: nope, so
far both places (covering all of gecko 1.8(.1) through 2.0) I've worked
at involve pulling a specific gecko branch, not "whatever the latest is"
- there's enough code that there's always an effort to move across
branches. If mozilla-release becomes the generic "current release" in
the way latest/ is on releases.m.o (i.e. rolling, currently 4.0), I'd
expect to have to pull by tag instead. Finding the latest release it
usually trivial anyway - just look for the largest number, there isn't
that many of them :)
> * Similar to the last point, I'd say over 90% of the time people want code for the latest release. Using a repo allows us to hide the 10% info and default to the 90%. With relbranches hunting for an older release is as much work as hunting for the latest release
Depends on who "people" are; for "people wanting to build Firefox",
probably yes (or m-c). For people who want to hack on Firefox,
definitely m-c (or a project branch). For building off of Mozilla... to
start off, certainly yes, but (assuming 6 week schedules, or even the 3
months version), probably not by the time they want to release their app.
> * Similar to the last two points, if a developer needs to look at the source for the latest release to investigate a bug there is virtually no chance they will get confused and look at the wrong code
Wouldn't they be looking at m-c instead? After, of course, testing that
it still exists on a nightly. It's not like they can get a patch
committed anywhere other than m-c, anyway.
<snip>
> I don't see how this would be a problem. With silent auto-updates we are assuming 98% of each channel audience is on the latest build for that channel after 24 hours (based on comments from Google, trying to find the source). Thus, when a build is 5.0 on the release channel 98% of users on the beta channel already have something called 6.0 within 24 hours. So if we are told they have "Firefox 5.0" and we have pushed 5.0 to the release channel we know they are on the release channel rather than the beta channel (as they would be telling us they have "Firefox 6.0" instead).
This assumes looking at the number ~ instantly. For a bug report that's
a few months old, figuring out whether the old bug was in the release
version may be harder (or this would get encoded in the chrome-style
version numbers, or something).
--
Mook
Christian Legnitto
On Mar 29, 2011, at 8:18 PM, Justin Wood (Callek) wrote:
> >Why not copy to releases/mozilla-X.Y? We currently do so to have a repo to use for .x releases. Because this new model doesn't have .x releases (only chemspill/rapid response updates) we don't need to copy to the releases directory and deal with the version naming. That being said, we could still do so if we wanted to but there doesn't appear to be much benefit/point. Please give feedback.
>
> My comment is that doing it will be a large benefit for projects outside of Firefox, but not for Firefox. Which includes benefiting distros.
> My reasoning:
>
> It allows a single place for projects/distro's to pull from, expecting tip to always work for the release they are trying to build.
>
> This could be for an XULRunner based project, ICEWeasel, etc. But means that if there is a chemspill on a 3rd-party application (which USES Gecko) that they don't accidentally break by pulling tip of Firefox.next, and/or don't accidentally MISS a chemspill change on the Firefox.ver they had first built with.
>
> I'm not sure if that reason is compelling enough to do the magic here.
I don't see how any of this argues against one mozilla-release repo. 3rd parties can choose to check out a specific release (however we mechanically mark it using tags, branches, heads, etc). Additionally, if they want to track our stable releases they can by simply pulling to their existing local repo from mozilla-release default.
In the releases/mozilla-5, releases/mozilla-6 world the option to track the latest released bits isn't there without additional manual work to enumerate the available repos and reclone. So as far as I can see the mozilla-releases covers the releases/mozilla-x use case and enables an additional one.
Perhaps I am missing something? Perhaps you were assuming the proposal meant mozilla-release would only have the source from the latest release and would be recreated for every release with the final bits from mozilla-beta?
> >>Version Scheme<<
>
> I really dislike the chrome style, severely, for our version numbering. The largest reason I have not seen expressed [I could just have missed it] is that it bloats the changelog, and with hg that is copied to the users computer on every checkout. It also means that the version number will be different with _every_ nightly. And have at least a checkin a day, even if no other changes.
This assumes that the version number is stored in the repo rather than some versioning system incorporated at build time. We already have something similar that changes every build...the build id. There's nothing to say we can't fix the leftmost digit in hg and let the rest be handled by the build process outside hg. Of course, if it is a good idea is a whole other question.
Also, it may be beneficial to see where content for each build starts and stops just by looking at hg log. I'm not so sure but it is plausible.
But yes, these are definitely some concerns. I'm just not sure they are dealbreakers or entirely negative...the tradeoffs certainly are different though.
> I am concerned of the "Standard" scheme in terms of identifying which channel a user is on, so I'm inclined to like KaiRo's bikeshedding of version number better :-)
Please see my reply to KaiRo.
>
> >>Gecko Version<<
>
> We did this with Mozilla Suite, do we _really_ feel the need to have the Gecko version tied so tightly with the product version, especially in such a rapid release process? I personally think it is a bad idea, and would rather avoid it.
What do we gain from keeping them separate? If there is no driving need either way, why would we not align the version numbers for clarity? FWIW, as a new employee the gecko vs Firefox version mismatch was extremely annoying to say the least.
Thanks again for the feedback,
Christian
Christian Legnitto
On Mar 30, 2011, at 12:02 AM, Mook wrote:
> On 3/29/2011 11:20 PM, Christian Legnitto wrote:
> <snip>
>
>> * 3rd party developers developing on top of mozilla releases only need to pull from mozilla-release tip to get the code for the latest release. They only have to hg pull to slurp in any chemspill or later release changes they may not even be aware of after cloning. I know I would find having a repo preferable to having to hunt around in hg for the proper relbranch name, determine if it is the latest, etc. Sure it'd probably be pretty easy...but not as easy as just pulling mozilla-release
> As a 3rd party developer developing on top of mozilla releases: nope, so far both places (covering all of gecko 1.8(.1) through 2.0) I've worked at involve pulling a specific gecko branch, not "whatever the latest is" - there's enough code that there's always an effort to move across branches. If mozilla-release becomes the generic "current release" in the way latest/ is on releases.m.o (i.e. rolling, currently 4.0), I'd expect to have to pull by tag instead. Finding the latest release it usually trivial anyway - just look for the largest number, there isn't that many of them :)
>
This is very, very good feedback. It appears my underlying assumptions are largely invalid, so thank you for pointing that out.
>> * Similar to the last point, I'd say over 90% of the time people want code for the latest release. Using a repo allows us to hide the 10% info and default to the 90%. With relbranches hunting for an older release is as much work as hunting for the latest release
> Depends on who "people" are; for "people wanting to build Firefox", probably yes (or m-c). For people who want to hack on Firefox, definitely m-c (or a project branch). For building off of Mozilla... to start off, certainly yes, but (assuming 6 week schedules, or even the 3 months version), probably not by the time they want to release their app.
Again, thanks for your insight.
>
>> * Similar to the last two points, if a developer needs to look at the source for the latest release to investigate a bug there is virtually no chance they will get confused and look at the wrong code
> Wouldn't they be looking at m-c instead? After, of course, testing that it still exists on a nightly. It's not like they can get a patch committed anywhere other than m-c, anyway.
Good point. My feeling is that people refer to released code a bit, but that could be a consequence of having maintenance releases which don't exist in the new process.
I can think of contrived examples such as security researchers checking the release code to confirm they can safely blog about a security issue they found...but nothing that sounds plausible and common. I need to think about this a bit more.
>
> <snip>
>> I don't see how this would be a problem. With silent auto-updates we are assuming 98% of each channel audience is on the latest build for that channel after 24 hours (based on comments from Google, trying to find the source). Thus, when a build is 5.0 on the release channel 98% of users on the beta channel already have something called 6.0 within 24 hours. So if we are told they have "Firefox 5.0" and we have pushed 5.0 to the release channel we know they are on the release channel rather than the beta channel (as they would be telling us they have "Firefox 6.0" instead).
> This assumes looking at the number ~ instantly. For a bug report that's a few months old, figuring out whether the old bug was in the release version may be harder (or this would get encoded in the chrome-style version numbers, or something).
Great point, but I'm not entirely convinced it is a major issue. I need to think about this a bit more as well.
Thanks,
Christian
Christian Legnitto
On Mar 29, 2011, at 9:01 PM, Chris AtLee wrote:
> On 29/03/11 11:36 PM, Chris AtLee wrote:
>> On 29/03/11 02:42 PM, Christian Legnitto wrote:
>>> Doh, the link to the specifics is:
>>>
>>> http://mozilla.github.com/process-releases/draft/development_specifics/
>>>
>>> Thanks,
>>> Christian
>>
>> A few quick questions:
>>
>> - are we going to be doing nightly builds (with nightly update channels)
>> on the experimental, beta and release branches? "nightly-experimental",
>> "nightly-beta" and "nightly-release"?
There will be nightly builds, but there is no separate nightly channel. Nightly builds off the mozilla-central repo go on the central channel. Nightly builds off the mozilla-experimental repo go on the experimental channel, etc. This is why silent updates are critical...we will quickly lose our experimental and beta audience if they are nagged to update every day.
Sure, we could prepend "nightly-" to the channel...but because it isn't adding any additional information it is just as easy to not include it. Also, having channel names of nightly-[whatever] sort of implies to me that there is a channel for [something other than nightly]-[whatever], which is not the case in this proposal.
FWIW this is why I don't want the channel for mozilla-central builds to be "nightly" (as stated in the proposal). When we talk about nightly updates we have to qualify it with which repo we are referring to. Talking about nightly updates off mozilla-experimental (which are actually published to the experimental channel) when the channel for mozilla-central builds is named nightly feels very "Who's on first" to me.
>> - you mention that the anticipated update rate for beta users is weekly,
>> does this means we'll be doing weekly release builds to the beta channel?
This part is confusing I admit. Ideally we'd only spin a build on the repo at night if the mozilla-* repo or corresponding l10n repo had changed. Because we don't expect the rate of change to be high on *-beta the updates would probably be weekly if such a scheme was put in place. But, I understand there are likely complications and additional tool work to enable "nightly only if content has changed" updates....so nightly updates with small patch updates is just fine as a stopgap.
I should clarify this in the proposal.
>>
>> - will there be anything other than dep and nightly builds on the
>> experimental stream?
No.I would imagine the first build after merging from mozilla-central to mozilla-experimental will be a clobber though.
>>
>> Regarding automated merging...If we only have one repo per stream
>> (instead of a repo per version like dbaron suggests), then I don't think
>> blowing away the repo and starting fresh is a great idea because it
>> invalidates history, links to revisions in bugmail, etc.
Good point.
>>
>> Would a true merge be that hard to do?
I don't think so. This is something we need to fumble through manually a couple of times and figure out the best way to go. I'm perfectly content to manage the merge manually as it is only once every 6 weeks.
>>
>> You *could* automate the a fake merge like this:
>> rsync -a --exclude .hg --delete mozilla-central/ mozilla-experimental/
>> hg -R mozilla-experimental addremove
>
> one more thing...
>
> what's the benefit of re-organizing existing hg repos? You mention that it facilitates automated tools, but I think any tools are going to need a canonical list of release and project branches anyway. We may as well record the existing location for e.g. tracemonkey instead of moving it into projects/js/tracemonkey.
Why have all the user repos under users/? What happens if the js team wants to bucket some project repos further (js/beat-crankshaft/foundation, js/beat-crankshaft/frame, js/beat-crankshaft/walls, etc)? Why would we let the structure stay flat only to come back in a year or so and propose a reorganization due to it getting out of hand? I also bet that people would naturally create structure with prefixes and postfixes (ux-progressbar, ux-tabcleanup, ux-purplefirefoxbutton, etc). Why not get out in front of that so there is consistency across all the project repos and teams?
Personally, I would love my tools to instantly know what project repos are available, grouped by what team, grouped by sub projects just by looking at the repository structure. Maintaining lists is a pain.
The structure, as the proposal states, also makes it so the channel names off project branches are easy to infer and have no collisions.
As the document states, this hasn't been circulated widely so I'd like to hear your thoughts on it...those are mine.
Thanks,
Christian
Axel Hecht
I think that
hg merge --config ui.merge=internal:other (or internal:local, depending
on order) is likely the most fail-safe and hg graph nice solution. That
is, if a merge would be appropriate.
That also should take care of any back and forth with flag switches.
Axel
Axel Hecht
here's my take on the l10n story for the new development process:
- the long tail of localizers need a simple story, dead on simple, really.
- our development community needs help to not break l10n early.
Let me start with the latter:
We already have a few localizers that work very close to mozilla-central
[1], and interact with our development community in bugs. We need to
keep this up, and find and fix issues for l10n (and rtl) before we put
the work out to the long tail of our l10n community. I plan to formalize
this a bit, and have only those locales bother with mozilla-central at
all. That may be a hand-full or two. Going forward, I also would like to
get this community to respond to project branches, maybe even localize
project branches. Also, I'm part of that group, at least as far as
responding to requests in bugs go.
=> a limited number of locales track mozilla-central. The rest doesn't
exist there at all (?).
On to the dead-simple story for the majority of our localizers, that
should hopefully turn out to be:
wake up on calendar alarm, and
pull mozilla, pull l10n/ab-CD,
hit whichever l10n tool
land, push
get builds, with nightlies to testers
rinse, repeat.
The hope is that for the most part, localizers won't need to bother with
repos on an earlier stream, or a later stream. They'll just stay put
like a sea-devil and bite any string that swims along.
The whole cycle should be doable in a weekend, aka, 2-3 days. That as a
boundary condition to how/why/when we generate and publish localized builds.
The proposed procedure also has the benefit that it's dead simple to
document, and that the documentation doesn't change. Easier for me (and
the communications guy we want to hire), for localizers, for external
infrastructure like narro or ANLoc's pootle.
The right channel for this procedure seems to be -experimental, after
talking to legneato.
=> All locales work on experimental to get their localizations ready to
ship. Sign-offs stay, and happen here.
Now a few fall-out items:
=> There shouldn't really be all that much traffic on beta at all.
We should probably take fixes, still, but whether those should be
marshaled by a release driver or if that tree is owned by localizers, no
idea. Might be worth to reconsider here at some point down the road.
=> Beta-only locales don't end up on release.
Right now, we have that funky setup where new locales in Beta are
regular release builds, just with a different flag on the website. I'd
like to get rid of that. Just have them in Beta, and once they mature,
pull them into the next release, perhaps even platform dependent?
Comments?
Axel
[1] http://hg.mozilla.org/l10n-central/fr even shows the en-US check-in
messages, for example.
Henri Sivonen
> Doh, the link to the specifics is:
>
> http://mozilla.github.com/process-releases/draft/development_specifics/
> There are currently two version proposals for Firefox. Please give
> feedback or an alternate approach.
(Sorry about being such a broken record about UA string versioning
but...)
I think we should do what IE does about versioning--not what we've done
before or what Chrome does.
IE6 with a decade of patches still only says it's "6.0" as far as *Web
sites* can find out. IE9 beta said "9.0" to sites without revealing it
was beta. The real version is available to the *user* in the about box.
Thus, when a user installs a security patch for IE, sites never fail due
to bad UA sniffing because the UA string doesn't change.
I think Firefox 5 should say Firefox/5.0 (without .x, "b" or "pre") to
sites. When Firefox 5 moves to experimental, I think m-c should start
saying Firefox/6.0 (without .x, "b" or "pre") to sites. This would
maximize the testing and evangelism period for the upcoming UA string we
are going to ship. It would minimize the ability of sites to do silly
things that sniff on "pre", "b" or chemspill patch level while still
revealing the major version so that parties other than Mozilla can
convince themselves independently of Mozilla's ADU claims by examining
their own HTTP logs that Firefox users update their browser and new Web
features can be used.
If https://*.mozilla.org/ needs more precision, let's expose more
precision only to https://*.mozilla.org/.
I don't have a strong opinion on what kind of versioning scheme should
be used as the "real" version in the about box (or exposed to
https://*.mozilla.org/ for bug reports, support or add-on compat).
However, it would seem reasonable to use the local (to hg.mozilla.org)
hg changeset integer id to get a number that increases with every
changeset. E.g. 5.0.63875 when 5.0 is reported to general Web sites or
6.0.72383 when 6.0 is reported to general Web sites. I think this scheme
would be simpler than Chrome's scheme, but this scheme doesn't have the
property of showing in the version number which channel the build came
from. However, at any one time, the major version will be different on
each channel, so overlapping numbers wouldn't get minted for MoCo
builds. (However, anyone building Firefox locally would get a number
that can/will overlap with a number assigned to a MoCo build. Linux
distros should probably pull from repo, freeze the number and then apply
their own patches so that those patches don't increment the number for
extension compat.)
> More discussion is needed for the merging mechanics.
The plan from the telecon yesterday that involved merging
mozilla-central into mozilla-experimental in a way that'd leave the
backouts from the previous cycle in force sounded painful. In that
model, someone could land a changeset in m-c, back it out in
experimental and then build upon the changeset in m-c in ways that would
break the build when a merge at start of the next cycle removes the
changeset from a stream of changes that had built upon it.
I think it would be vastly preferable for the tip of
mozilla-experimental to have the same code as the tip of mozilla-central
immediately after mozilla-central gets pulled/merged/copied into
mozilla-experimental. (Likewise for the later transitions.)
I think it's vastly preferable to have to back out a change n times if
the feature is not ready over n release trains than to have to reland
stuff.
--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/
Mike Hommey
> (However, anyone building Firefox locally would get a number
> that can/will overlap with a number assigned to a MoCo build. Linux
> distros should probably pull from repo, freeze the number and then apply
> their own patches so that those patches don't increment the number for
> extension compat.)
Project branches will have overlapping changeset ids.
Mike
Chris AtLee
>>> - are we going to be doing nightly builds (with nightly update
>>> channels) on the experimental, beta and release branches?
>>> "nightly-experimental", "nightly-beta" and "nightly-release"?
>
> channel. Nightly builds off the mozilla-central repo go on the
> central channel. Nightly builds off the mozilla-experimental repo go
> on the experimental channel, etc. This is why silent updates are
> critical...we will quickly lose our experimental and beta audience
> if they are nagged to update every day.
What's included in the "etc." here? If our nightly mozilla-beta builds
are going on the beta channel, that complicates (from an updates point
of view) doing any kind of more official beta release build that also
goes to the beta channel.
> Sure, we could prepend "nightly-" to the channel...but because it
> isn't adding any additional information it is just as easy to not
> include it. Also, having channel names of nightly-[whatever] sort of
> implies to me that there is a channel for [something other than
> nightly]-[whatever], which is not the case in this proposal.
>
> FWIW this is why I don't want the channel for mozilla-central builds
> to be "nightly" (as stated in the proposal). When we talk about
> nightly updates we have to qualify it with which repo we are
> referring to. Talking about nightly updates off mozilla-experimental
> (which are actually published to the experimental channel) when the
> channel for mozilla-central builds is named nightly feels very
> "Who's on first" to me.
The nightly prefix makes it clear (to me at least!) where the builds are
coming from, and how updates get handled.
>>> - you mention that the anticipated update rate for beta users is
>>> weekly, does this means we'll be doing weekly release builds to
>>> the beta channel?
>
> This part is confusing I admit. Ideally we'd only spin a build on
> the repo at night if the mozilla-* repo or corresponding l10n repo
> had changed. Because we don't expect the rate of change to be high
> on *-beta the updates would probably be weekly if such a scheme was
> put in place. But, I understand there are likely complications and
> additional tool work to enable "nightly only if content has changed"
> updates....so nightly updates with small patch updates is just fine
> as a stopgap.
Yes, we can trigger nightly builds only if something has changed. the
last time we proposed this idea, there was some concern about getting
enough perf data[1] on branches with low activity and no regular nightlies.
> I should clarify this in the proposal.
>
>>>
>>> - will there be anything other than dep and nightly builds on the
>>> experimental stream?
>
> No.I would imagine the first build after merging from
> mozilla-central to mozilla-experimental will be a clobber though.
That's fine; dep builds are the same as clobber builds for us. Somewhere
we should do a breakdown of how dep, nightly and release builds work,
and how they're different...
>>> Regarding automated merging...If we only have one repo per stream
>>> (instead of a repo per version like dbaron suggests), then I
>>> don't think blowing away the repo and starting fresh is a great
>>> idea because it invalidates history, links to revisions in
>>> bugmail, etc.
>
> Good point.
>
>>>
>>> Would a true merge be that hard to do?
>
> I don't think so. This is something we need to fumble through
> manually a couple of times and figure out the best way to go. I'm
> perfectly content to manage the merge manually as it is only once
> every 6 weeks.
>
>>>
>>> You *could* automate the a fake merge like this: rsync -a
>>> --exclude .hg --delete mozilla-central/ mozilla-experimental/ hg
>>> -R mozilla-experimental addremove
>>
>> one more thing...
>>
>> what's the benefit of re-organizing existing hg repos? You mention
>> that it facilitates automated tools, but I think any tools are
>> going to need a canonical list of release and project branches
>> anyway. We may as well record the existing location for e.g.
>> tracemonkey instead of moving it into projects/js/tracemonkey.
>
> Why have all the user repos under users/? What happens if the js
> team wants to bucket some project repos further
> (js/beat-crankshaft/foundation, js/beat-crankshaft/frame,
> js/beat-crankshaft/walls, etc)? Why would we let the structure stay
> flat only to come back in a year or so and propose a reorganization
> due to it getting out of hand? I also bet that people would
> naturally create structure with prefixes and postfixes
> (ux-progressbar, ux-tabcleanup, ux-purplefirefoxbutton, etc). Why not
> get out in front of that so there is consistency across all the
> project repos and teams?
>
> Personally, I would love my tools to instantly know what project
> repos are available, grouped by what team, grouped by sub projects
> just by looking at the repository structure. Maintaining lists is a
> pain.
Sure, organization is better, but there are lots of other types of repos
than clones mozilla-central. Your tools need to know how to find the
repos you care about without having to inspect hundreds of different
repos on hg.m.o.
> The structure, as the proposal states, also makes it so the channel
> names off project branches are easy to infer and have no collisions.
If the channel name is a transformation of the repo path, you wouldn't
have collisions regardless of how things were organized. 'tracemonkey'
wouldn't collide with 'users-catlee-tracemonkey'
Perhaps I'm arguing too strongly for something I don't feel that
strongly about. It feels unnecessary and creates a bunch of extra work
for little benefit, but if the repository owners don't mind...
> As the document states, this hasn't been circulated widely so I'd
> like to hear your thoughts on it...those are mine.
>
> Thanks, Christian
Thanks for opening this up to everybody. I feel much better now that I
can see the whole plan.
Cheers,
Chris
Henri Sivonen
Is that a problem if the policy is not to have nightly users on project
branches?
Mike Hommey
Is that not a problem to have users (nightly users, or developers, or
testers, or whatever) use completely different builds with the same
version ? But then, there's also the buildid to help differentiate,
though that's still far from useful IMHO.
Maybe major.minor.changesetid.changesetsha1 would be a solution (where
changesetsha1 would be the changeset sha1 stripped to, say, 3 or 4
characters).
Mike
Henri Sivonen
> On Wed, Mar 30, 2011 at 04:44:37PM +0300, Henri Sivonen wrote:
> > On Wed, 2011-03-30 at 15:31 +0200, Mike Hommey wrote:
> >
> > >
> > > On Wed, Mar 30, 2011 at 04:23:04PM +0300, Henri Sivonen wrote:
> > > > (However, anyone building Firefox locally would get a number
> > > > that can/will overlap with a number assigned to a MoCo build.
> > > > Linux
> > > > distros should probably pull from repo, freeze the number and
> > > > then
> > > apply
> > > > their own patches so that those patches don't increment the
> > > > number
> > > for
> > > > extension compat.)
> > >
> > > Project branches will have overlapping changeset ids.
> >
> > Is that a problem if the policy is not to have nightly users on
> > project
> > branches?
>
> Is that not a problem to have users (nightly users, or developers, or
> testers, or whatever) use completely different builds with the same
> version ?
If it's a problem, it's not a new problem.
> But then, there's also the buildid to help differentiate,
> though that's still far from useful IMHO.
A date as a build id has always been bogus between different build streams.
> Maybe major.minor.changesetid.changesetsha1 would be a solution (where
> changesetsha1 would be the changeset sha1 stripped to, say, 3 or 4
> characters).
WFM if not exposed to Web sites (other than https://*.mozilla.org/).
Axel Hecht
sha1's are not incrementing. no idea how much of our infra relies on that.
Also, as legneato mentioned, there's no requirement to have the version
in the repo, local builds, developer or linux distros (that don't use
our update system) could just default to a "5.devel" or something.
Axel
Mike Hommey
> Mike Hommey wrote:
> > On Wed, Mar 30, 2011 at 04:44:37PM +0300, Henri Sivonen wrote:
> > > On Wed, 2011-03-30 at 15:31 +0200, Mike Hommey wrote:
> > >
> > > >
> > > > On Wed, Mar 30, 2011 at 04:23:04PM +0300, Henri Sivonen wrote:
> > > > > (However, anyone building Firefox locally would get a number
> > > > > that can/will overlap with a number assigned to a MoCo build.
> > > > > Linux
> > > > > distros should probably pull from repo, freeze the number and
> > > > > then
> > > > apply
> > > > > their own patches so that those patches don't increment the
> > > > > number
> > > > for
> > > > > extension compat.)
> > > >
> > > > Project branches will have overlapping changeset ids.
> > >
> > > Is that a problem if the policy is not to have nightly users on
> > > project
> > > branches?
> >
> > Is that not a problem to have users (nightly users, or developers, or
> > testers, or whatever) use completely different builds with the same
> > version ?
>
Certainly not, but this is a good opportunity to fix these old problems.
> > But then, there's also the buildid to help differentiate,
> > though that's still far from useful IMHO.
>
> A date as a build id has always been bogus between different build streams.
>
> > Maybe major.minor.changesetid.changesetsha1 would be a solution (where
> > changesetsha1 would be the changeset sha1 stripped to, say, 3 or 4
> > characters).
>
> WFM if not exposed to Web sites (other than https://*.mozilla.org/).
I'm not even sure *.mozilla.org needs it in the UA at all. At least some
will take the version from the URL and/or parameters, fed by the
urlformatter on the browser end.
Mike
Benjamin Smedberg
>
>> * Will builds on the experimental or beta channels be signed? How does this affect the release cadence? (Signing requires manual intervention from releng)
> Not sure we need to sign them. I don't think this will affect the release cadence at all.
when dealing with interactions with antivirus software. I really think
that the builds on the beta channel should be fully signed builds.
And if they are signed, this definitely affects whether and how we do
"nightly" builds on this channel, because signing requires intervention
from release engineering.
--BDS
Chris AtLee
We could purchase and maintain a separate nightly signing key. Then I'd
need to finish up some work to enable us to do signing as part of the
packaging step on the build machines...
Robert Kaiser
> * You are assuming we will do chemspills off relbranches. I think that is most likely a faulty assumption (but I do note there is nothing in that section of the doc yet! Hang tight)
I assumed we were doing chemspills off whatever the release is built on,
with only spot-fixes applied for whatever the chemspill is - tzhat would
be a relbranch if final releases come off a relbranch or it will be the
mozilla-release repo as a "constant relbranch" if you will. :)
> * 3rd party developers developing on top of mozilla releases only need to pull from mozilla-release tip to get the code for the latest release.
You assume their code is always compatible with "the latest release",
which I'm not sure is true. They always need to go back to their own dev
mode when they pull from there unless there's only chemspill spot-fixes
on there. A pull that takes them inadvertently from Mozilla 7 to Mozilla
8 (or 9, or whatever, as they might not be on 6-week dev cycles) is
almost sure to break their software in some way.
Still, you brought in one POV I missed when thinking about this myself,
and it surely has merit.
> I don't see how this would be a problem. With silent auto-updates we are assuming 98% of each channel audience is on the latest build for that channel after 24 hours (based on comments from Google, trying to find the source). Thus, when a build is 5.0 on the release channel 98% of users on the beta channel already have something called 6.0 within 24 hours. So if we are told they have "Firefox 5.0" and we have pushed 5.0 to the release channel we know they are on the release channel rather than the beta channel (as they would be telling us they have "Firefox 6.0" instead).
A lot of people cannot install that rapidly as their computers or
browser installations might be locked down in some way without
themselves or even their software having a way to rapidly update.
OK, that scenario probably doesn't affect users on anything but release,
of course. Still, it's also not uncommon to have people who are testing
different setups to have an older beta or dev version around and "just
try something" there are report in with a result.
And, of course, when I look at a journalist's report of some testing he
did on "Firefox 7.0" I don't even know if he used an early beta or a
final as I have no idea if he did those tests a few weeks ago or so and
the article just took some time to be published.
>> The Chrome model makes us end up with the release channel having the weirdest numbers that don't look like finals to anyone who usually understands version numbers (it's confusing enough that Google's finals are hard to identify, doesn't need to be the same with us) - 5.10.45.233?
>
> We are demoting version numbers everywhere. The whole version won't be reflected in the UA (my understanding from bsmedberg is it would stay at Firefox 5.0 throughout 5's development) or UI.
We need some way for various support sites etc. to get at an
identification of the build, but I'm fully OK if that's not the UA
string itself. We really need to figure out something there, e.g. having
that info from the navigator object in JS, possibly with a prompt to the
user similar to what geolocation has, but we shouldn't over-engineer the
problem.
>> Er, was that a beta or a final? I certainly can't remember even after a day...
>
> Well, we need a marker for "this version is the latest on the release channel".
Right, and usually, i.e. for almost everything out there except Chrome,
that's the .0 in the version number. Having it be, say, .4.67.233 is
hard for a human to remember (machines have no problem with that, of
course).
> * 5.0.0.[not zero] ? You know that build was created out of the mozilla-central repository during Firefox 5 development
> * 5.0.[not zero].[not zero] ? You know that build was created out of the mozilla-experimental repository during Firefox 5 development
> * 5.[not zero].[not zero].[not zero]. ? You know that build was created out of the mozilla-beta repository during Firefox 5 development
And how to I easily see that it's a final release and not a beta channel
build? That one is missing from the proposal. ;-)
>> I don't really want to go into fully bikeshedding here, so don't consider this one seriously, but I once dreamed up we could go with 5.0a1...5.0aXXX on central, 5.0b1...5.0bXXX on the next one but then the question of what the other channel has would come up - though maybe central could just stay with 5.0a1pre, and experimental would go 5.0a1 etc. and beta would go 5.0b1 etc.? well, as I said, probably too much bikeshedding to even bring this up here.
>
> I'd love to see this proposal properly written up with the intricacies hashed out. Just because there are two proposals in the document doesn't mean those are the only two that are valuable to choose from.
OK, how about this one:
* 5.0a1pre for all builds from mozilla-central, distinguished by build
ID just like they are now. Bumped to 6.0a1pre etc. after merges to
experimental.
* 5.0a1 for the first build off experimental, with the number after the
"a" increased for every build with code changes "shipped" from there,
i.e. the first one with a different revision after 5.0a1 is 5.0a2, etc.
If we have 6 weeks on there and assume daily builds with revision
changes every day (we may not have that, actually), we end up with
5.0a42, that's not really bad.
* 5.0b1 for the first build off beta, number after "b" increases for
every build with code changes shipped there. For 6 weeks with weekly
"beta"s shipped, that gives us 5.0b6 or so, if we give that one a no-go
for going up to release, that makes 5.0b12 or so at the most - nothing
we didn't have before. ;-)
* 5.0 for the release off this. If we need to chemspill on that, 5.0.1
can easily be used.
This proposal of course has the charm of being near to what we have been
using so far, so it might fit into minds of people in the project more
easily. Final versions are easy to spot, beta channel builds have an
already-known beta-marker in their name, experimental becomes an "alpha"
channel version-wise, and "pre" (not in UA, only in version) remains
attached to bleeding-edge development nightlies.
Automation for the version bumps should be easy, as automation can just
look if there was any non-tagging changeset after the last version
commit and if there was, it increases the after-a/b number, checks that
in and build off that revision.
Looking at it again, I guess you could call that a compromise between
the standard and Chrome models you presented, with a small injection
from current practice. ;-)
>> 3) L10n: I hope there will still be a possibility for localizations to follow mozilla-central and have localized bleeding-edge development nightlies. We need to catch issues as early as possible, esp. when we might be changing L10n infrastructure (which is an ongoing plan).
>
> Yes, I believe that is expressed in the document already. Axel has made it very clear having more localization involved earlier is better for the product overall. That being said, he mentioned a lot of localizers like to set their calendars, do their work, and then go away until their calendar says it is time again.
Yes, a lot of the localizers do that, but some follow development very
closely (e.g. French, IIRC) or somewhat closely (e.g. German) and we
still need those around working against central. Axel seems to have that
in his reply here, so that's good, I was just a bit worried as he
dropped somewhere that "l10n-central might be going away" and I was not
so sure that would be good - but looks like this is clearing up. :)
>> 4) Firefox 6: I wonder how 2 times 5 weeks and 2 times 6 weeks leaves a gap of a week in the merge from experimental to beta... but maybe I'm misunderstanding something there. ;-)
>
> It's 2 gaps...1 week on mozilla-experimental and 1 week on mozilla-beta. So 2*5+2 = 2*6. If you stare at my graphic long enough it'll make sense ;-) I'll try to come up with a clearer explanation and/or graphic...but no promises.
Oh, hmm, I guess I need to stare for a bit longer, then. ;-)
> I really appreciate the detailed feedback!
Thanks for the chance to weigh in!
Robert Kaiser
--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community needs answers to. And most of the time,
I even appreciate irony and fun! :)
Robert Kaiser
> What's included in the "etc." here? If our nightly mozilla-beta builds
> are going on the beta channel, that complicates (from an updates point
> of view) doing any kind of more official beta release build that also
> goes to the beta channel.
From how understand things, those "nightly" (actually, done at one time
of a day if the code has changed since the last build) beta builds _are_
the "official beta release" builds and the "nightly beta channel" is the
official beta channel.
LegNeato might correct me if I'm wrong, but that's how I take it. Same
would be true for experimental, of course.
Robert Kaiser
> Why not get out in front of that so there is consistency across all the project repos and teams?
As long as links to revisions and other hgweb stuff in bugs etc. keep
working, I guess that's a good idea. We just need to make sure of that
tiny thing of keeping existing links working, esp. in places where they
can't be changed after the effect, like Bugzilla, mailing lists, etc.
Justin Wood (Callek)
>> 4) Firefox 6: I wonder how 2 times 5 weeks and 2 times 6 weeks leaves a gap of a week in the merge from experimental to beta... but maybe I'm misunderstanding something there. ;-)
>
> It's 2 gaps...1 week on mozilla-experimental and 1 week on mozilla-beta. So 2*5+2 = 2*6. If you stare at my graphic long enough it'll make sense ;-) I'll try to come up with a clearer explanation and/or graphic...but no promises.
>
KaiRo is not wrong... lets think of it this way.
|----|-----|-----|-----|
|----||-----|-----|
|----|?|-----|
I'm not sure if my ascii graph will get through, but think this way:
m-c
5+6+6 (5=initial, 6=firefox 6, 6=firefox 7)
-experimental
<none=5>+5+<gap=1>+6 (none=initial [5], 5=Firefox 5, gap of 1 week,
6=Firefox 6)
-beta:
<none=5>+<none=5>+5+<gap=2>
This is the path, with missing-weeks, that gets the plan properly into
shape. The math can be deceiving at first.
--
~Justin Wood (Callek)
Robert Kaiser
> This is the underlying context for the note in the localization section of the proposal. Essentially (correct me if I am wrong Axel) l10n wants to take the current l10n-central repo and have it track mozilla-experimental. A new repository (let's call it l10n-noname) with a subset of locales would be created which would then track mozilla-central. When mozilla-central merges to mozilla-experimental l10n-noname merges to l10n-central.
ugh, I don't think -central tracking anything else than -central is a
good idea. No problem with creating a new l10n-experimental (or whatever
matches the mozilla-* tree), seeding with copies of all the l10n-central
ones and HTTP redirect all l10n-central ones that are not following
mozilla-central to the new one to keep links working. But let's please
keep names consistent. :)
Justin Wood (Callek)
> Thanks! Comments inline...
>
> On Mar 30, 2011, at 12:02 AM, Mook wrote:
>
>> On 3/29/2011 11:20 PM, Christian Legnitto wrote:
>> <snip>
>>
>>> * 3rd party developers developing on top of mozilla releases only need to pull from mozilla-release tip to get the code for the latest release. They only have to hg pull to slurp in any chemspill or later release changes they may not even be aware of after cloning. I know I would find having a repo preferable to having to hunt around in hg for the proper relbranch name, determine if it is the latest, etc. Sure it'd probably be pretty easy...but not as easy as just pulling mozilla-release
>> As a 3rd party developer developing on top of mozilla releases: nope, so far both places (covering all of gecko 1.8(.1) through 2.0) I've worked at involve pulling a specific gecko branch, not "whatever the latest is" - there's enough code that there's always an effort to move across branches. If mozilla-release becomes the generic "current release" in the way latest/ is on releases.m.o (i.e. rolling, currently 4.0), I'd expect to have to pull by tag instead. Finding the latest release it usually trivial anyway - just look for the largest number, there isn't that many of them :)
>>
>
> This is very, very good feedback. It appears my underlying assumptions are largely invalid, so thank you for pointing that out.
>
Since Mook elaborated here, I won't re-hash the same things in my
sub-thread. But this is what I was getting at regarding specific release
repos.
--
~Justin Wood (Callek)
Robert Kaiser
> IE9 beta said "9.0" to sites without revealing it
> was beta.
That probably made a few web sites create workarounds for it by
detecting that version and be broken again by final being different
again. :)
Just for illustration that you can _never_ do The Right Thing (TM) for
sniffing. ;-)
> I think Firefox 5 should say Firefox/5.0 (without .x, "b" or "pre") to
> sites.
I thought we agreed to that about a hundred times already?
We still need a mechanism to give support and similar sites (no,
mozilla-provided sites are NOT the only ones who can make things way
easier for a user if they know an exact version) some way to figure out
the actual version, but it might be through a JS property (e.g. on the
navigator object) and a whitelist, user prompt (like geolocation) or
something like that. Without such a mechanism, our own people like SUMO
and AMO folks, but also probably people giving support elsewhere will
probably hate us, and we should avoid that.
Mike Hommey
> On 30.03.11 16:16, Mike Hommey wrote:
> >On Wed, Mar 30, 2011 at 04:44:37PM +0300, Henri Sivonen wrote:
> >>On Wed, 2011-03-30 at 15:31 +0200, Mike Hommey wrote:
> >>
> >>>
> >>>On Wed, Mar 30, 2011 at 04:23:04PM +0300, Henri Sivonen wrote:
> >>>>(However, anyone building Firefox locally would get a number
> >>>>that can/will overlap with a number assigned to a MoCo build. Linux
> >>>>distros should probably pull from repo, freeze the number and then
> >>>apply
> >>>>their own patches so that those patches don't increment the number
> >>>for
> >>>>extension compat.)
> >>>
> >>>Project branches will have overlapping changeset ids.
> >>
> >>Is that a problem if the policy is not to have nightly users on project
> >>branches?
> >
> >Is that not a problem to have users (nightly users, or developers, or
> >testers, or whatever) use completely different builds with the same
> >version ? But then, there's also the buildid to help differentiate,
> >though that's still far from useful IMHO.
> >
> >Maybe major.minor.changesetid.changesetsha1 would be a solution (where
> >changesetsha1 would be the changeset sha1 stripped to, say, 3 or 4
> >characters).
>
> sha1's are not incrementing. no idea how much of our infra relies on that.
That's why I put the changeset id before the sha1.
Mike
Steve Wendt
>> We did this with Mozilla Suite, do we _really_ feel the need to
>> have the Gecko version tied so tightly with the product version,
>> especially in such a rapid release process? I personally think it
>> is a bad idea, and would rather avoid it.
>
> What do we gain from keeping them separate? If there is no driving
> need either way, why would we not align the version numbers for
> clarity? FWIW, as a new employee the gecko vs Firefox version
> mismatch was extremely annoying to say the least.
It's easier to figure out the marketing version late in the game, if
they are separate. If Firefox "7" doesn't have enough changes since
Firefox "6", a decision could be made to call it "6.1" or "6.5" instead.
Having to change the underlying Gecko version at that stage wouldn't
be so nice, I would think. Gecko != Firefox, isn't that why this site
exists? http://geckoisgecko.org/
I don't know if Trident or WebKit use version numbers, but certainly in
the latter case it can't be tied to a browser version.
Matt Brubeck
> It's easier to figure out the marketing version late in the game, if
> they are separate. If Firefox "7" doesn't have enough changes since
> Firefox "6", a decision could be made to call it "6.1" or "6.5" instead.
Actually, changing the app version late in the game could be very bad for extension compatibility (minVersion/maxVersion). The app version is not just a "marketing" number - it is actually used for practical purposes at least as much as the Gecko version.
Aakash Desai
> and AMO folks, but also probably people giving support elsewhere will
> probably hate us, and we should avoid that.
I'd also suggest adding input.mozilla.com to that mix as well. TO get user feedback right across channels and find trends across differing versions, the website will need a method to determine what build id the user is on. The JS Property idea sounds like a good idea and its worth bringing this up to like-minded parties.
-- Aakash
Robert Kaiser
_______________________________________________
dev-planning mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-planning
Nick Thomas
> => Beta-only locales don't end up on release.
>
> Right now, we have that funky setup where new locales in Beta are
> regular release builds, just with a different flag on the website. I'd
> like to get rid of that. Just have them in Beta, and once they mature,
> pull them into the next release, perhaps even platform dependent?
This sounds like good news to me. Are you thinking we'd change
shipped-locales after merging from beta to release ?
-Nick
Axel Hecht
I'm expecting fall-out for both all-locales and shipped-locales. The
shipped-locales piece has a proof-of-concept in fennec, where it's just
not existing and merged with l10n-changesets.
In the interim, yeah, we should patch shipped-locales between beta and
release.
For all-locales, I'm not sure which way to go, but it may be a good call
to move the information on which locales to build out of the repo.
Before having a uniform solution for that, we can probably just
hard-code a subset of locales to use for mozilla-central in the build
setups. That's me hoping that the releng infra still supports that, the
dashboard would.
Axel
Christian Legnitto
> On 3/30/2011 12:14 AM, Christian Legnitto wrote:
>
>>> We did this with Mozilla Suite, do we _really_ feel the need to
>>> have the Gecko version tied so tightly with the product version,
>>> especially in such a rapid release process? I personally think it
>>> is a bad idea, and would rather avoid it.
>>
>> What do we gain from keeping them separate? If there is no driving
>> need either way, why would we not align the version numbers for
>> clarity? FWIW, as a new employee the gecko vs Firefox version
>> mismatch was extremely annoying to say the least.
>
> It's easier to figure out the marketing version late in the game, if they are separate. If Firefox "7" doesn't have enough changes since Firefox "6", a decision could be made to call it "6.1" or "6.5" instead. Having to change the underlying Gecko version at that stage wouldn't be so nice, I would think. Gecko != Firefox, isn't that why this site exists? http://geckoisgecko.org/
We're setting a stake in the ground that the next version of Firefox is called 5, the one after that is 6, etc. If Firefox 5 is merely Firefox 4 + the pgo fix + a version bump we are fine with that. This is admittedly a radical shift from the way we currently do development and market releases.
I'd be interested in hearing other pros and cons but marketing flexibility is a non-goal.
Thanks,
Christian
Adam Kowalczyk
While I am not necessarily opposing this, it's worth pointing out that
it's not just about marketing as in promotion, but also about
communication. Historically with Firefox, as with most applications,
users have learned that the increase in version number indicates what
kind of changes to expect - if it is a maintenance release or if it
might bring a significant upheaval of their experience. Version number
is the very first thing they look at when deciding whether to upgrade
(or whether to give Firefox another try, in case of users of competing
browsers).
- Adam
Christian Legnitto
Yep, this is going to be different / a potential pain point for a bit. There will be no .x releases other than to fix rapid response / chemspill issues. We likely won't even bump up the user-visible version number. Additionally, with silent updates there is no choice by default...users get the latest and greatest and should only know they are running "Firefox" in the most general case.
Thanks,
Christian
Steve Wendt
> There will be no .x releases other than to fix rapid response /
> chemspill issues. We likely won't even bump up the user-visible
> version number.
That sounds like a bad idea - "OMG, the news says the old version is
fubared, but I can't tell if mine is updated!"
Unless the goal is to cater to the "ignorant masses" who don't pay any
attention to the news, anyway? They are probably still running IE...
Ehsan Akhgari
> This assumes that the version number is stored in the repo rather than some versioning system incorporated at build time. We already have something similar that changes every build...the build id. There's nothing to say we can't fix the leftmost digit in hg and let the rest be handled by the build process outside hg. Of course, if it is a good idea is a whole other question.
Another point to note here is that nightly builds are not necessarily
nightly, we can trigger then at any time. This may complicate things
here (or it might not).
> Also, it may be beneficial to see where content for each build starts and stops just by looking at hg log. I'm not so sure but it is plausible.
Good point. I've never thought about that, but we currently do have the
problem that by looking at a changeset, there is no easy way to tell
exactly which nightly version it was included in (or first appeared in).
The other side of the problem is an easier question to answer (by
looking at about:buildconfig) but it still requires one to download the
exact build and run it, so it's not ideal.
Ehsan
Dao
> The other side of the problem is an easier question to answer (by
> looking at about:buildconfig) but it still requires one to download the
> exact build and run it, so it's not ideal.
There's
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-4.2a1pre.en-US.win32.txt,
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-4.2a1pre.en-US.linux-i686.txt
etc.
Armen Zambrano Gasparnian
>> Also, it may be beneficial to see where content for each build starts
>> and stops just by looking at hg log. I'm not so sure but it is plausible.
>
> Good point. I've never thought about that, but we currently do have the
> problem that by looking at a changeset, there is no easy way to tell
> exactly which nightly version it was included in (or first appeared in).
> The other side of the problem is an easier question to answer (by
> looking at about:buildconfig) but it still requires one to download the
> exact build and run it, so it's not ideal.
>
> Ehsan
and the changeset. Is this good enough to not have to download the build?
e.g.
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/firefox-4.2a1pre.en-US.win32.txt
Armen Zambrano Gasparnian
>> and stops just by looking at hg log. I'm not so sure but it is plausible.
>
> Good point. I've never thought about that, but we currently do have the
> problem that by looking at a changeset, there is no easy way to tell
> exactly which nightly version it was included in (or first appeared in).
> The other side of the problem is an easier question to answer (by
> looking at about:buildconfig) but it still requires one to download the
> exact build and run it, so it's not ideal.
>
> Ehsan
Armen Zambrano Gasparnian
Ehsan Akhgari
Do we have any way to get this information for arbitrary builds at an
arbitrary point back in time?
Cheers,
Ehsan
Armen Zambrano Gasparnian
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/1301574863/firefox-4.2a1pre.en-US.win32.txt
even debug builds!
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32-debug/1301581854/firefox-4.2a1pre.en-US.win32.txt
Jesper Kristensen
> => a limited number of locales track mozilla-central. The rest doesn't
> exist there at all (?).
Will there be central builds of the locales which do not follow central?
I would like to follow central as my browser, but I would also like to
use the localized version, so that I can proofread the (partial)
localization as I browse.
> => All locales work on experimental to get their localizations ready to
> ship. Sign-offs stay, and happen here.
Will that mean that after experimental has branched to beta, that
localizations can no longer be corrected? In the current (Firefox 4)
process, we allow localization changes right up to RC.
I have an idea that it might be possible to have one l10n repo used
actively for both the experimental and beta dev repo. Compare-locales
could then use the union of the en-US strings, marking an string as
missing when it is missing in one of the versions, and obsolete only
when it is obsolete in both. That would allow 12 weeks of l10n instead
of 6, but still with only one repo. The downside would be that when a
string changes its key, the new key would be added, and the old one
would only be deleted 6 weeks later, instead of at the same time.
-
Jesper Kristensen
Axel Hecht
> Den 30-03-2011 13:40, Axel Hecht skrev:
>> => a limited number of locales track mozilla-central. The rest doesn't
>> exist there at all (?).
>
> Will there be central builds of the locales which do not follow central?
> I would like to follow central as my browser, but I would also like to
> use the localized version, so that I can proofread the (partial)
> localization as I browse.
I don't expect to have builds for locales that don't follow central.
Where would you get the "danish" strings from?
>> => All locales work on experimental to get their localizations ready to
>> ship. Sign-offs stay, and happen here.
>
> Will that mean that after experimental has branched to beta, that
> localizations can no longer be corrected? In the current (Firefox 4)
> process, we allow localization changes right up to RC.
I think we should figure out how this works in practice, but in general,
I'd endorse folks to work on experimental only.
> I have an idea that it might be possible to have one l10n repo used
> actively for both the experimental and beta dev repo. Compare-locales
> could then use the union of the en-US strings, marking an string as
> missing when it is missing in one of the versions, and obsolete only
> when it is obsolete in both. That would allow 12 weeks of l10n instead
> of 6, but still with only one repo. The downside would be that when a
> string changes its key, the new key would be added, and the old one
> would only be deleted 6 weeks later, instead of at the same time.
I'll ponder about this a few.
Axel
Jesper Kristensen
> On 05.04.11 00:54, Jesper Kristensen wrote:
>> Den 30-03-2011 13:40, Axel Hecht skrev:
>>> => a limited number of locales track mozilla-central. The rest doesn't
>>> exist there at all (?).
>>
>> Will there be central builds of the locales which do not follow central?
>> I would like to follow central as my browser, but I would also like to
>> use the localized version, so that I can proofread the (partial)
>> localization as I browse.
>
> I don't expect to have builds for locales that don't follow central.
> Where would you get the "danish" strings from?
From the l10n-(experimental/aurora) repository, with English strings
merged in when Danish strings are missing, like they have always been.
Robert Kaiser
> We still need a mechanism to give support and similar sites (no,
> mozilla-provided sites are NOT the only ones who can make things way
> easier for a user if they know an exact version) some way to figure out
> the actual version, but it might be through a JS property (e.g. on the
> navigator object) and a whitelist, user prompt (like geolocation) or
> something like that. Without such a mechanism, our own people like SUMO
> and AMO folks, but also probably people giving support elsewhere will
> probably hate us, and we should avoid that.
Filed https://bugzilla.mozilla.org/show_bug.cgi?id=648444 on this, btw.
Axel Hecht
I don't think we'll do this. I don't think there are tons of
localizations that need that, and for the few that do, you can just keep
builds going and push to both aurora and central and get basically the
same thing.
If this turns out to be a major requirement from a ton of folks at some
point, I'll be happy to reconsider, but for now, I guess this is simple
enough.
Axel