Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.

Dismiss

QAC Redesign

4 views

Skip to first unread message

Clint Talbert

unread,

Jun 25, 2009, 6:07:35 PM6/25/09

to Onno Steenbergen, Heather Arthur, Aaron

Hello!

AaronMT, Harthur, Onno and I are working on a redesign to the QA
Companion extension. The working document on our design plans is on the
wiki here: https://wiki.mozilla.org/QA/QAC. The wiki document will be
our official working document, but I've reproduced it below for easy
inline commenting.

We'd love to hear your thoughts.

Thanks,

Clint

== Wiki Doc Reproduction =

The current Quality Assurance Companion extension is pretty broken in
terms of functionality and usabillity. We have had a "redesign" effort
under way for many months now with only slight progress to show for it.
It is time to change this and make using the QAC into a really
enjoyable, useful experience for our community testers. On Monday of
this week, Heather, Aaron and I hashed through some use cases and took a
hard look at the QAC, tab by tab. What follows is our lists of
improvements, and it will be easier to follow along if you start up the
QAC right now so you can look at the current interface. I'll get a cup
of coffee while you do that.

Have you launched the QAC? Do you have a cup of coffee? Good, good.
Let's dig in.

* Primary Design Considerations *
The primary audience for the QAC is a new user to Mozilla QA. In the
light of that the more 'user friendly' we can make these interfaces the
better. Additionally, the more streamlined all these interfaces can be,
the better experience that person will have in terms of not being
confused and being able to easily and quickly contribute their time to
working on testing.

We still believe that the original purpose of the QAC - to help people
run litmus without impacting the browser (or the application under test)
is a valid one. So, we are still going to be our own free standing
window, and we will continue to constrain ourselves to the smallest
possible screen real estate so that the application under test can take
up more real estate. We hope to allow the user the ability to run the
QAC and the application under test side by side on their screen.

* The Startup Experience *
The multiple panels that firs launch when a user starts up QAC for the
first time are a completely broken experience. The only useful thing
those panels do is alert the user to some security flaws in the product
(which we will simply fix) and obtain a login/password for the Litmus
web application. The operating system/platform selection can be
reasonably guessed and does not need to be displayed. So, in light of
these thoughts our propsal for the "first run" is to bring up the QAC,
but overlay a little dialog over the "Q" tab to allow people to create a
login. It would look something like this:

======================================
| |
| | Q | Litmus | Bugs | |
| |
| ------------------------------X- |
| | Enter/create your login | |
| | | |
| | Login |________________| | |
| | Password |_____________| | |
| | | |
| | [Submit] | |
| |_______________________________| |
|_____________________________________|

Now, this is a little simplistic but you get the idea. What we really
want this UI to do is two things: create an account for you and log you
in to an existing account. If it is creating an account for a new user
we want to automatically create a login for the top three sites that a
new user will need:

* Litmus.mozilla.org
* Quality.mozilla.org (QMO)
* bugzilla.mozilla.org

In order to work for existing users who already have accounts for these
items, then the "log in" section will probably default to logging you in
to litmus and not the other accounts. Obviously we'll need some
messaging around this so that people will know that they are now the
proud owner of three happy QA related accounts. There is a whole host
of complexitiy here, especially with existing users who may have
different addresses for their user names on each of these three services.

* Overall UI *
You might have noticed in my little mockup above that the "Chat",
"Settings" and "Help" tab are removed. We decided that this material
doesn't constitute enough value to merit screen realestate on their own.

The Chat and Help items will be moved into the Q tab, the Settings tab
will be moved into the standard application preference pane where it
will take up its own tab there. Those preferences will be accessible
from both the "Preferences" pane window as well as the "Preferences"
button off the Add-Ons dialog.

* The Q tab *

The Q tab is the news and events area of the tab and serves as a little
mini-feed reader for the content hosted in the quality.mozilla.org (QMO)
calendar and blog. We will add in the "Chat" functionality as a link on
this page and the help button as well. It might look something like this:

So, this is pretty similar to what we currently have. The "Chat button"
will start up either the Chatzilla instance we currently ship with the
product or will create a mibbit window and log you into #qa on
irc.mozilla.org. There is also thought that the "Chat" button might be a
multi-faceted control so that the user can choose whether to use IRC or
simply send an email form to a known address if they are too timid to
jump on IRC.

The [?] button will be avaiable at the bottom of every tab and will
cause a link to be opened in your browser to a help page hosted on QMO.
For non-browser applications, we'll have to figure out what to do, but
we can probably open a <window> with a <browser> element that points to
the content, so that should be ok. The first version of the redesigned
QAC will be firefox specific, then we'll work on ports.

The events and news items will also have scrollbars so that their
content space if fixed but you can still read the items in that content
space.

* The Litmus Tab *

This is by far everyone's favourite tab. It is difficult, complex and
often overflows off people's screens. So, we need to fix it. The first
thing we are going to do is remove the "list of tests" box. That will
free up quite a bit of screen real estate. The end design we are
thinking of will look something like:

======================================
| |
| | Q | 'Litmus' | Bugs | |
| |
| | Current Test Run Foo [Change] | |
| |
| [<] [>] Test: Bar 4/5 Test Baz |
| -------------------------------- |
| | 1. These are the test steps | |
| | 2. They will go here | |
| | 3. And they will have a | |
| | scrollbar | |
| |______________________________| |
| |
| ------------------------------- |
| | Expected Results will go | |
| | here and will have a scroll | |
| | bar too. | |
| |_____________________________| |
| |
| X Passed Comment (optional): |
| _ Failed | | |
| _ Unclear |_________________| |
| [SUBMIT RESULT] |
|_____________________________________|
|_________________________________[?]_|

So, when you click on the Litmus tab for the first time ever, you will
be presented with a floating dialog over the tab that allows you to
choose a test run to run from. This will be the same floating dialog
that you will get if you click the [Change] button to change the test
run and it will probably look a lot like the one you currently get
except that it will hover over the center of the Litmus tab rather than
being a true new window.

Once you have selected a test run, the next time you log in and go to
the Litmus tab, you will automatically be presented with your last
state. This includes if you were last on test X and you restarted the
browser, then when you come back and log in, test X will be displayed
once more with your comment replaced. It's session restore for litmus
tests :). The Rest of the dialog is pretty self explanatory. The [<]
and [>] buttons allow you to skip backward and forward through the tests
in your current selected run.

What happens if the user gets to the end of that selected run? How
would we handle an "random test" type of scenario? That is entirely up
to you.

* The Bugzilla Tab *

And now the bugzilla tab. So, if you declare a test as failed, we are
thinking of showing you another of these floating dialogs and asking you
to submit a bug. And if you say "yes, I'll submit a bug", then you come
to this nifty bugzilla tab.

The current bugzilla tab does three things:
* Search for bug by ID
* Search for bug by text.
* Show selected bug X in bugzilla.

New users do not really need to search bugzilla. Finding duplicate bugs
is not something that users who are new to the project are good at doing
and it is a very high barrier to getting involved. All too often bugs
do not get submitted because we tell people that they have to first
search for dupes to their bugs, or the QA team has to help search for
dupes during a test day. To address that, we are going to retool the
entire thinking of the Bugzilla tab.

We want the new and improved bugzilla tab help people file that first
bug. So it needs to:
* Log you into Bugzilla.
* Provide the easiest possible interface to filing a bug.
* Automatically do a very simple search for dupes and show possible
duplicates to this bug that is being filed.

Let's talk about the bug that the tool will enter first. We don't have
much screen real estate and we don't want to duplicate bugzilla. So, we
will pre-fill a bunch of the fields for the user. We will fill in the
OS, Platform, Component and Whiteboard fields for the user. We will put
all the "QAC generated" bugs into Firefox/General with a
<strike>whiteboard flag of "[qac triage me]"</strike> and they will be
filed as Unconfirmed bugs. Actually, we discovered that there is no way
to add a whiteboard flag to the bug entry through bugzilla's RPC
interface. So we will hve to appened this QAC tag to the summary. This
will be transparent to the user. After the user enters their summary,
we will prepend [QAC Generated] to the summary. This way, the QA team
can easily query for any QAC filed bugs after a testday and triage them
appropriately.

The user will type in a Summary and a Description field for the bug.
We'll prepopulate the description with some helper text to get them to
write a decent report, and we will append the link to the Litmus test
case if the bug is being filed from a test case that was marked
"failed". One thing to consider is that if we are filing from a failed
litmus test case we may not need to ask for things like "steps to
reproduce" because those steps are in the litmus test case itself.

Once the user inputs a bug, we'll show a quick float over dialog (or
change the main UI) that says "success" and gives them the link to the
bug report, which they can then click to load into bugzilla.

Now, the duplicate search ability is going to be very basic.
Essentially if a new user is filing a bug from a failure in a litmus
test case, then chances are good that this bug is already filed. So, we
will check for duplicates using the following query:
* take the summary, query for summaries with "any of these words"
* in components firefox/general, core/general (other high-landing areas
that should be checked?)
* that have been filed in the last four weeks.

We will take a set of those bugs and bring up an info bar or some such
UI to ask the user if his bug matches one of these existing ones. If
the user clicks on one of the bugs then a bugzilla tab will be opened
for that bug in bugzilla. We need to think out the rest of the
interaction here to advise people how to add more information to
existing bugs - ideally, we would have a system to easily instruct the
user to leave a comment on said bug with the Litmus test case URL that
we supplied them.

Therefore, we propose to do away with all searching in the bugzilla tab.
If you find the QAC bugzilla searching useful, please let us know why
and how, and we can reconsider. But at this time, we are thinking that
the searching is simply too basic to be useful, and it is far more
useful to provide a simple way for people to file bugs that they see in
the product with the fewest number of hurdles in doing so.

Therefore, the proposed bugzilla tab UI looks something like this:

======================================
| |
| | Q | Litmus | 'Bugs' | |
| |
| Bug Summary: |
| -------------------------------- |
| | This is the summary | |
| |______________________________| |
| |
| Bug Description: |
| ------------------------------- |
| | = Steps to Reproduce = | |
| | 1. | |
| | 2. | |
| | 3. | |
| | = Expected Behavior = | |
| | | |
| | = Actual Behavior = | |
| | | |
| | Litmus test: foo | |
| | http://litmus.mozilla.org/foo| |
| |_____________________________| |
| |
| [ENTER BUG] |
|_____________________________________|

* Some Other Thoughts *
The tab names should be what people are doing and not the web sites that
they are using for it. So Litmus should probably become "Run Test" and
Bugzilla should become "Enter Bug", so some such. The OS/Platform stuff
should be guessed, but should be configurable in the QAC tab of the
preferences pane, along with the notification settings and any other
preferences we need. The "Logout" button will also be located there,
and QAC will simply keep you logged in to Litmus until its cookie
expires, which is what it does now.

We might want to surface the statistics from Litmus so that users will
know what tests have been run, which areas have coverage, etc. It's
really not clear how to do that just yet.

* Moving Forward *
So, I encourage feedback and debate about all these ideas, nothing here
is final, but we are going to start on the re-work of the UI and the
backend changes needed to support these UI changes now. So, if you have
thoughts get them in soon. Up next, I want to propose a schedule for
getting each of these tabs done in parallel sprints of code. So stay
engaged and stay tuned, and let us know what you think.

0 new messages