Re: Adjustments to the Desktop QA Beta cycle testing schedule
Ryan VanderMeulen
What are we doing?
QA will execute week long test runs, focused on test importance, rather than execute test runs per Beta build, that were based on feature priority.
Why?
We want to ensure that our most critical tests, for every feature of Firefox, are executed earlier in the Beta cycle.
How?
In preparation for this change. the QA team has triaged our tests, and reconfigured them by test type. The team analyzed what testing workload can be handled in each week of a Beta cycle and then developed a schedule to accommodate the new process for stable feature functional testing, new feature testing and any additional required testing.
When?
As of Fx120 Beta cycle, week of Oct 23rd, we will use this scheduling template. In practice, Fx 120 Beta cycle schedule. Note that dots indicate testing in progress and filled in squares indicate intended report day.
What to expect:
- QA will no longer be “signing-off” on individual Beta builds, with the exception of Beta 1. Instead we will send functional testing report summaries each Friday of weeks 1-3.
- Smoke testing of the entire browser will occur during week 1 of Beta.
- Broader functional testing will progress through weeks 2 and 3 of the Beta cycle, during which, QA will use each new beta build as it becomes available.
- Bugs will be reported when discovered and any high impact bugs found will be escalated immediately.
- We will continue to send test reports specific to new feature testing as well as other planned testing efforts.
- Testing and reporting during RC week remains largely unchanged.
- We will no longer send the “Bugs logged by Desktop QA in the last week” email report, as that information will be folded into the end of week reports.
Please direct any questions or concerns to me, here or on Slack (@tracy)
Thank you,
Tracy Walker
Desktop QA Technical Lead.