Firefox 3.5 release candidate builds: new and exciting!
Mike Beltzner
Some of you may have noticed that we're doing something a little bit
different with our release candidate milestones for Firefox 3.5. I'd
like to take a brief moment to explain what we're doing, why, and what
the implications are for release dates.
// Who are our beta testers?
Our beta testing audience (defined as users who have downloaded a beta
release of Firefox and thus put themselves on the "beta" update
channel) is big; over 800,000 active daily users. These individuals
have accepted the risks of running beta software, agreeing to help us
by testing the breadth of the web with a stable version of the browser
that, potentially, contains a few bugs.
// How do we make a development milestone, anyway?
Historically, we prepare development milestones (betas or release
candidates) more or less as follows:
1. "freeze" code development
2. identify the last code change and use it as the basis for milestone
builds
3. build the milestone on all operating systems
4. sign the builds on windows
5. generate update snippets
6. run a set of quality assurance tests on the builds
7. ship partial updates to beta-channel users and change the beta
download page to point to the new development milestone
// So, what's new and exciting?
We've realized that we can run this process in a more parallel fashion
by splitting step 7 into:
7.1 ship partial updates to beta-channel users
7.2 change the beta download page to point to the new development
milestone
We'll be doing this for the Firefox 3.5 release candidate builds. As
soon as QA performs basic smoketests and update tests on the
milestone, we'll ship the partial updates to our existing beta-channel
users. We feel confident in doing so since our nightly testing
audience and unit test infrastructure have shown that by this point,
the vast majority of serious stability bugs have already been
identified and mitigated, so we know we're not putting our beta
audience at any increased risk.
Then, while our quality assurance team finishes more in depth
functional testing, our large beta audience will be helping us by
using the builds across the breadth of the web. We'll get wider
feedback earlier, giving us even more confidence before we declare a
milestone as final. Once QA - and in effect our beta-audience - sign
off on this testing phase, we can update the beta download page.
The release drivers feel that this parallelization helps us get the
most out of our beta community, increasing the quality of our
development milestones faster. Further, at this release candidate
stage, it allows users to decide if they want to be beta testers or
not (release candidate builds use the standard product update channel,
not the beta channel)
// What does this mean for the upcoming Firefox 3.5 release candidate?
We've just released partial updates for Firefox 3.5 beta users which
will update them to an early version of the release candidate (the
exact version reference is rc1build2: this means it is the first
revision of the release candidate for which we'll ship updates, and it
required two build attempts). This build is *not* being released on
the beta download page; if someone out there wants to get at it, they
should first become a beta tester by downloading a beta and then they
will receive the updates.
As we find issues with the release candidate, we will fix and rebuild.
We're already generating a new version (rc2build2) with several small,
localized fixes. We can do this because the small-scope nature of the
fixes means that we can re-run a segment of the quality assurance
tests instead of starting over from scratch.
Our target is still to release the most up to date version of the
release candidate for download on our website this Friday. It really
depends on what we find between now and then, but with 800,000 people
looking, it means that if nothing is found, we can be extremely
confident of the quality of the release candidate code!
// I'm confused: what's the schedule?
June 16th: Firefox 3.5 beta users get updated to an early release
candidate version
June 19th: target public ship date for Firefox 3.5 Release Candidate 1
end of June: target ship date for Firefox 3.5
cheers,
mike
Frédéric Buclin
> June 16th: Firefox 3.5 beta users get updated to an early release
> candidate version
> June 19th: target public ship date for Firefox 3.5 Release Candidate 1
> end of June: target ship date for Firefox 3.5
So if RC1 is released in two days and 3.5 final at the end of the month,
when will RC2 be released?? Are you saying that the current rc2buildX
will never end as RC2 but as final directly?
Mike Beltzner
> Le 17. 06. 09 11:16, Mike Beltzner a écrit :
I wouldn't worry too much about the build reference versions as I've
published them above in this note. Version numbering here gets
confusing, really.
On June 19th, we hope to release (on our website, as a full download)
a release candidate. Right now that would be the rc2build2 version, as
it's the latest thing we have. So, in other words:
June 16th: Firefox 3.5 beta users get updated to an early release
candidate (rc1build2)
June 19th: Firefox 3.5 beta users get updated to the official release
candidate, which is also put up for direct download (right now this
would be rc2build2, but that may yet change!)
end of June: target ship date for Firefox 3.5
cheers,
mike
David Tenser
> On 17-Jun-09, at 2:27 AM, Fr�d�ric Buclin wrote:
>
>> Le 17. 06. 09 11:16, Mike Beltzner a �crit :
>>> June 16th: Firefox 3.5 beta users get updated to an early release
>>> candidate version
>>> June 19th: target public ship date for Firefox 3.5 Release Candidate 1
>>> end of June: target ship date for Firefox 3.5
>>
>> So if RC1 is released in two days and 3.5 final at the end of the
>> month, when will RC2 be released?? Are you saying that the current
>> rc2buildX will never end as RC2 but as final directly?
>
> I wouldn't worry too much about the build reference versions as I've
> published them above in this note. Version numbering here gets
> confusing, really.
Indeed; I thought I was reading that we already know there will be an
RC2, but I see now that RC1 >= rc2build2. Thanks for clearing that up!
John J. Barton
...
... This build is *not* being released on the beta download
> page; if someone out there wants to get at it, they should first become
> a beta tester by downloading a beta and then they will receive the updates.
Perhaps some can explain this sentence to me. To me it says: "The build
cannot be downloaded; if you want to download it, first download it."
I don't get the whole idea anyway. If I have a beta, I get the updates
asap. If I don't have a beta, I can't get the updates asap (duh). So the
only thing new here is that if I don't have a beta I also can't get one?
How is that new and exciting?
jjb
Mike Beltzner
> Mike Beltzner wrote:
> ...
> ... This build is *not* being released on the beta download
>> page; if someone out there wants to get at it, they should first
>> become a beta tester by downloading a beta and then they will
>> receive the updates.
> ...
>
> Perhaps some can explain this sentence to me. To me it says: "The
> build cannot be downloaded; if you want to download it, first
> download it."
It means that the build which we are shipping as an update to existing
beta users (via the built in Firefox software update mechanism) is not
yet available at http://www.mozilla.com/firefox/all-beta.html
The primary reason we're doing things this way, as explained in the
original post, is to get our 800,000+ beta users helping us on daily
testing as part of our QA phase on a milestone
> I don't get the whole idea anyway. If I have a beta, I get the
> updates asap. If I don't have a beta, I can't get the updates asap
> (duh). So the only thing new here is that if I don't have a beta I
> also can't get one? How is that new and exciting?
The new thing here is that while (in the scenario you posit) you would
get an update asap, you would not be able to download it from the
Firefox Beta Download web page, yes.
I find it new and exciting as a process change. Others find it
slightly odd. I'm glad that you find it warm, comforting and not
disturbing. It means we can move on.
cheers,
mike
John J. Barton
> On 17-Jun-09, at 8:31 AM, John J. Barton wrote:
>
>> Mike Beltzner wrote:
>> ...
>> ... This build is *not* being released on the beta download
>>> page; if someone out there wants to get at it, they should first
>>> become a beta tester by downloading a beta and then they will receive
>>> the updates.
>> ...
>>
>> Perhaps some can explain this sentence to me. To me it says: "The
>> build cannot be downloaded; if you want to download it, first download
>> it."
>
> It means that the build which we are shipping as an update to existing
> beta users (via the built in Firefox software update mechanism) is not
> yet available at http://www.mozilla.com/firefox/all-beta.html
>
> The primary reason we're doing things this way, as explained in the
> original post, is to get our 800,000+ beta users helping us on daily
> testing as part of our QA phase on a milestone
FF3.5b4 and FF3.5b99 did not work with Firebug. So we found a nightly
build from June 5 that worked and directed our users to that build. I
would like to direct them to a newer build. I tested the RC1 build2 last
night and it looks great. So I posted the link in our blog. Then I was
told that I should not do that. So I am disappointed that there is not
a way for Firebug users to get a newer beta as soon as possible. If
800,000 is good, wouldn't a few more just be better?
jjb
Mike Beltzner
> FF3.5b4 and FF3.5b99 did not work with Firebug. So we found a
> nightly build from June 5 that worked and directed our users to that
> build. I would like to direct them to a newer build. I tested the
> RC1 build2 last night and it looks great. So I posted the link in
> our blog. Then I was told that I should not do that. So I am
> disappointed that there is not a way for Firebug users to get a
> newer beta as soon as possible. If 800,000 is good, wouldn't a few
> more just be better?
I don't know who told you not to point your own testing audience to a
specific build. We ask that people don't reference a specific build on
the FTP site and say "this is a beta, go get it!" or the like, but
that's mostly to protect our servers from overload. I don't think that
will be a problem in the case you mention, though.
Be aware that if you point people to a nightly build, they'll start
getting nightly updates. If you point people to that RC build, they
won't be on the beta channel, they'll be on the release channel.
For your general user audience, I'd suggest you wait a couple of days
until all-beta is updated with a pointer to the release candidate. Or,
if you want to advise that they become Firefox beta testers, have them
download Beta 4 and check for updates.
Neither of these things seem particularly hard to do, if you are
willing.
cheers,
mike
John J. Barton
I updated my advice with this information.
>
> Neither of these things seem particularly hard to do, if you are willing.
I know all of this stuff seems easy for you. But I could not figure out
from your original post how to get a copy of the release candidate. Now
I see that you can update 800,000 users in few minutes but to update the
web page takes several days. To avoid waiting on the web page, users can
load the old build and update it.
It's funny that browsers and software updates are getting faster but web
page updates are getting slower.
jjb
prmatt11
wrote:
Got it. Now this all makes sense. I will mainly be testing -moz-border-
radius and-moz-box-shadow, so I will report with any bugs.
Jesse Ruderman
more frequently. But why not also put those builds on the web page?
Is putting new builds on the web page way more work than it should be?
Mike Beltzner
The release candidate hasn't finished the QA cycle yet, so we're not
comfortable publishing it as a completed milestone.
cheers,
mike
_______________________________________________
dev-planning mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-planning
Jesse Ruderman
beta, but not to the next 200,000 users who come by to download the
latest beta? This doesn't make sense to me. Is it just a PR thing?
Mike Beltzner
expectation of quality, and more importantly, when we publish such a
release we are effectively saying: "this is it; this is the version
that we believe is Firefox 3.5"
So, yes, we're unwilling to take that step until we've finished all QA
on the product.
Further, we're comfortable sending an incremental update to our beta
testing audience, who have effectively agreed to receive in-
development software; we do believe that the update we delivered this
morning is at the quality level of our previous beta releases. There
is additional quality assurance testing, though, that we perform on a
release candidate as compared to a beta. We would like our beta
audience to help with that additional testing.
Also, we already know that the final version of the release candidate
(rc2build2 is presently the latest revision) will be slightly
different than the update we just shipped (rc1build2). So I don't
think we can state "this is our release candidate of Firefox 3.5." We
know that it isn't yet. What we can, and have stated, is that this is
an early version of our release candidate.
All of this - all of it - was in the original email I sent. I
encourage you to re-read it.
cheers,
mike
ps: I find your comment about this new mechanism being "just a PR
thing" incredibly offensive. Not just to me, but to our PR team at
Mozilla, who are the most open, honest and forthright group you'll
ever care to meet.
Michael Lefevre
> It means that the build which we are shipping as an update to existing
> beta users (via the built in Firefox software update mechanism) is not
> yet available at http://www.mozilla.com/firefox/all-beta.html
>
> The primary reason we're doing things this way, as explained in the
> original post, is to get our 800,000+ beta users helping us on daily
> testing as part of our QA phase on a milestone
At least the terminology, if not the idea, seems a bit odd here (which I
thought when I saw the first-run page text as well). "Our basic QA
isn't done yet, but we've automatically updated you to this pre-release
release candidate build anyway."
If you've shipped it to 800,000 users, is it really not a "milestone" or
a "release"? (Although I guess maybe the idea is to avoid having
"release candidate 1" on a web page which will get slashdotted and have
3,000,000 people run it, is that it?)
You said in the top post that the thing that is shipped in a couple of
days will be "release candidate 1". How does that make sense when it
will come after (or be identical to) "release candidate 2 build 2"?
If rc1build2 was a released release candidate, then wouldn't the next
release have to be "release candidate 2". And if rc1build2 wasn't a
release, then shouldn't rc2build1 have been rc1build3?
I guess the answer may be that the process was worked out while it was
happening and the numbering happened before the sequence was decided, in
which case I think I understand everything now you've posted about it,
so that's fine.
Michael
Jesse Ruderman
to existing beta users but don't put it on the beta page. Giving it
to existing beta users is a clear win; I think we all agree about
that. But listing it on the beta page would cause confusion:
* Calling it "beta 7" would make some people expect new features, make
others wonder why we're having so many betas, and confuse everyone
when we recast the build as a release candidate.
* Calling it "release candidate 1 build 1" or something along those
lines is confusing, and prevents users from joining the beta channel.
* Calling it "the latest beta" is vague.
* Additionally, it has been through slightly less QA (so far) than a
normal beta, so it might be ok to give it to existing beta users (e.g.
because they opted for an early beta) but not new beta users.
So, we don't list it on the beta page. This causes some confusion of
its own, and some people end up testing an older build than we'd like,
but neither of these problems is huge. It's also a great chance to
shake out updater bugs: if updating fails for a small percentage of
users, we can be pretty sure they'll complain.
John J. Barton
> Ok, I think I understand why ship this almost-release-candidate build
> to existing beta users but don't put it on the beta page. Giving it
> to existing beta users is a clear win; I think we all agree about
> that. But listing it on the beta page would cause confusion:
>
> * Calling it "beta 7" would make some people expect new features, make
> others wonder why we're having so many betas, and confuse everyone
> when we recast the build as a release candidate.
Next time consider calling it beta 7. Then you can educate users that
'beta' means "fixing bugs, not adding features", and "more betas is
better than less betas, because it means the final release will have
high quality". Plus no one gets confused because there is no release
candidate.
>
> * Calling it "release candidate 1 build 1" or something along those
> lines is confusing, and prevents users from joining the beta channel.
So don't.
> * Calling it "the latest beta" is vague.
So don't.
> So, we don't list it on the beta page.
But if was called b7 you could list it. And not have all of this.
jjb
Mike Shaver
Barton<johnj...@johnjbarton.com> wrote:
> Next time consider calling it beta 7. Then you can educate users that 'beta'
> means "fixing bugs, not adding features", and "more betas is better than
> less betas, because it means the final release will have high quality". Plus
> no one gets confused because there is no release candidate.
We *did* consider calling it a beta, and decided not to. One reason
is that we don't want people evaluating it as "beta software" -- they
should be holding it to the same standards as released software in
terms of what they would report as problems, and we have seen an
unfortunate number of "I would have reported it, but I figured it was
beta so I'd see if it just got fixed next time" on bugs we would have
rather heard about earlier.
Linus talks about this phenomenon periodically, as it affects the
Linux kernel -- you don't get that last push of testing until it's
declared as "done", so we're trying to walk the right line between
"testers, please be very picky" and "world, we're not yet ready to
call it Firefox 3.5". I think we've made a *huge* step forward in
letting our beta testers be closer to our key endgame QA cycle here,
naming nitpicks aside.
Mike
John J. Barton
> On 17-Jun-09, at 8:56 AM, John J. Barton wrote:
> For your general user audience, I'd suggest you wait a couple of days
> until all-beta is updated with a pointer to the release candidate. Or,
> if you want to advise that they become Firefox beta testers, have them
> download Beta 4 and check for updates.
I've read all of this thread again. I still can't make out what the deal
is.
If we go to the all-beta site now we become beta testers but we get the
RC build by check-for-updates. That much I tried and it works. A bit of
silly extra pointless work but whose counting.
If we wait a few days and go to the all-beta site we get the RC build
but don't become beta-testers? So the all-beta becomes the site for RC
download at that point and the beta program closes? This does not mesh
with the whole story of how great the beta program is, so I don't think
that is what you meant to say. I guess.
jjb
Mike Beltzner
> I've read all of this thread again. I still can't make out what the
> deal is.
Believe it or not, I'm ok with that. You understanding isn't a pre-
requisite. Sorry that you're not grokking it, let's please move on.
cheers,
mike
Peter Weilbacher
> On 18-Jun-09, at 12:05 AM, John J. Barton wrote:
>
>> I've read all of this thread again. I still can't make out what the
>> deal is.
>
> Believe it or not, I'm ok with that. You understanding isn't a
Yeah, just title your next post with "new and confusing" and we'll all
be OK with that.
P.
John J. Barton
> On 18-Jun-09, at 12:05 AM, John J. Barton wrote:
>
>> I've read all of this thread again. I still can't make out what the
>> deal is.
>
> Believe it or not, I'm ok with that. You understanding isn't a
Good call, thanks.
jjb
Mithgol the Webmaster
MB> June 16th: Firefox 3.5 beta users get updated to an early release
MB> candidate version
MB> June 19th: target public ship date for Firefox 3.5 Release Candidate 1
MB> end of June: target ship date for Firefox 3.5
What was at http://tinyurl.com/Firefox-3-5rc1-win32-en-US yesterday then?
An early release candidate version?
With best Fidonet 2.0 regards,
Mithgol the Webmaster.
.. 199. I will not make alliances with those more powerful than myself.
Daniel Veditz
> If we wait a few days and go to the all-beta site we get the RC build
> but don't become beta-testers? So the all-beta becomes the site for RC
> download at that point and the beta program closes? This does not mesh
> with the whole story of how great the beta program is,
The beta program is great, but it's not for everyone. Since the point of
a release "candidate" is to become the actual release (if successful)
then the candidate bits can't have "beta" baked into the
channel-prefs.js file.
The beta program continues for those already signed up for betas --
we'll use them for testing early releases of security updates. But
signing up for the betas at that point will be more cumbersome, you'll
have to really mean it. This isn't any different from how we're handling
Firefox 3.0 -- you can still get onto the Firefox 3.0.x beta program,
but you have to either go download a released beta, manually edit your
channel-prefs.js file, or install the update channel selector addon.
John J Barton
Thanks, but we've already determined this stuff is too complicated for
me to understand.
All I really want to know is what link I should give to Firebug users
when I ask them to try 3.5. I want to do that as soon as possible so we
can get some testing of Firebug 1.4 on Firefox 3.5. I thought that was
possible now, but I'll just wait.
jjb
Daniel Veditz
> And about 13:16 17 Jun 09 it was written from Mike Beltzner to dev. planning:
>
> MB> June 16th: Firefox 3.5 beta users get updated to an early release
> MB> candidate version
>
> MB> June 19th: target public ship date for Firefox 3.5 Release Candidate 1
>
> An early release candidate version?
broken link (was the real link so much harder to use?), but assuming you
meant releases.mozilla.org then rc1 was a "release candidate candidate".
Just relax and trust that if you're on the beta update channel the right
thing will happen. If you're not on the beta channel then trust that the
right bits will eventually show up at the download link.
In no case should anyone be referring people to get builds from places
like http://tinyurl.com/Firefox-3-5rc1-win32-en-US -- not even if it worked.
Ken Saunders
for a long time and I've started to get my feet wet, but I'm
wondering, since betas only advance, and then progress into release
candidates from nightlies, should I focus my attention and time on
nightly builds rather than just on betas and release candidates?
Or am I waaayyy off?
Can you point me to documentation on Mozilla's product life cycles and
how they evolve and from what?
I'm not a coder by nature but I can certainly offer an extra set of
eyes and run full functional tests on litmus and hopefully, the
Mozilla Testers Learn As You Go program that I bought on Amazon will
pay off. :)
To show you how green that I am, I always thought that I had to keep
downloading nightly builds to get the latest. Thanks to this post,
I've learned about the update channels.
Thanks for your time.
Ken
Gervase Markham
On 20/06/09 09:15, Ken Saunders wrote:
> I've been interested in helping out with testing Mozilla's products
> for a long time and I've started to get my feet wet,
Great :-) mozilla.dev.quality is probably the newsgroup to hang out in;
Moving the conversation there.
> but I'm
> wondering, since betas only advance, and then progress into release
> candidates from nightlies, should I focus my attention and time on
> nightly builds rather than just on betas and release candidates?
> Or am I waaayyy off?
At the moment (and it would be good if someone could say where this is
documented in easy-to-understand fashion) we have the Firefox 3.5
branch, which has produced alphas, betas and release candidates as well
as nightlies, and the trunk, which produces only nightlies. So "nightly"
could refer to a build from either.
> I'm not a coder by nature but I can certainly offer an extra set of
> eyes and run full functional tests on litmus and hopefully, the
> Mozilla Testers Learn As You Go program that I bought on Amazon will
> pay off. :)
:-)
> To show you how green that I am, I always thought that I had to keep
> downloading nightly builds to get the latest. Thanks to this post,
> I've learned about the update channels.
If you are on trunk nightlies, update will give you the latest trunk
nightly. If you are on branch nightlies, you will get the latest branch
nightlies. But if you manually install a beta or RC, you'll just get
future betas/RCs (and final).
At least, I _think_ that's how it works :-)
Gev
Mike Beltzner
> If you are on trunk nightlies, update will give you the latest trunk
> nightly. If you are on branch nightlies, you will get the latest
> branch nightlies. But if you manually install a beta or RC, you'll
> just get future betas/RCs (and final).
>
> At least, I _think_ that's how it works :-)
Almost! :)
There are three update channels:
nightly - updates you every day to the latest nightly build
beta - updates you every time we release a beta milestone
release - the default, updates to official releases
On top of that, the channels are unique per major version. So, someone
on the 3.5 beta channel will receive updates from beta 1 to beta 2,
and once Firefox 3.5 is released, they'll also receive betas of 3.5.1,
3.5.2, etc. However, someone on the 3.0 beta channel will not receive
3.5 betas, as that's a major version jump.
Finally, release candidates are set to use the release channel. So if
you've downloaded the Firefox 3.5 RC you'll be updated to each
subsequent RC, and then to final, and then to final versions of the
security and stability releases (3.5.x) without getting betas.
It's not the easiest soup to see through, but it works!
cheers,
mike
John J. Barton
Seems pretty clear. And, hoping not to offend by reopening this issue:
"new and exciting" meant: "We are sending the RC through the beta
channel so we get another chance to find bugs".
If correct, then I know you said that. The part that threw us off was
not updating the all-beta page to the latest thing sent through the beta
channel.
How can a user determine which update channel they are on? If you are
concerned that users can be 'stuck' on inappropriate channels, then how
about making the channel more visible? Maybe title bar:
Firefox Nightly-Channel web browser
Firefox Beta-Channel web browser
Firefox web browser
jjb
Philip Chee
> There are three update channels:
>
> nightly - updates you every day to the latest nightly build
> beta - updates you every time we release a beta milestone
> release - the default, updates to official releases
>
> On top of that, the channels are unique per major version. So, someone
> on the 3.5 beta channel will receive updates from beta 1 to beta 2,
> and once Firefox 3.5 is released, they'll also receive betas of 3.5.1,
> 3.5.2, etc. However, someone on the 3.0 beta channel will not receive
> 3.5 betas, as that's a major version jump.
Ah, does that mean that if the trunk version changes from 3.6a1pre to
(say) 4.0a1pre, those on the 3.6x nightly channel will stop getting updates?
Phil
--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
Robert Strong
> On Mon, 22 Jun 2009 07:07:46 -0700, Mike Beltzner wrote:
>
>> There are three update channels:
>>
>> nightly - updates you every day to the latest nightly build
>> beta - updates you every time we release a beta milestone
>> release - the default, updates to official releases
>>
>> On top of that, the channels are unique per major version. So, someone
>> on the 3.5 beta channel will receive updates from beta 1 to beta 2,
>> and once Firefox 3.5 is released, they'll also receive betas of 3.5.1,
>> 3.5.2, etc. However, someone on the 3.0 beta channel will not receive
>> 3.5 betas, as that's a major version jump.
>>
> Ah, does that mean that if the trunk version changes from 3.6a1pre to
> (say) 4.0a1pre, those on the 3.6x nightly channel will stop getting updates?
>
Cheers,
Robert