Remote Usability by Nate Bolt #SXSW 2011

[AJK]

Source: Optimal Usability Newsletter

 

Stop listening to your customers - Nate Bolt, Bolt|Peters and Mark Trammell, Twitter.

 

This was a big push for remote usability. We are certainly advocates at Optimal Usability, but Nate Bolt is an evangelist.

 

Here are some key points:

• Don't ask people what they want, watch what they do
• The key strengths of remote usability are that it is Contextual, Time Aware, and Behaviour-Based
• Remote usability must be taken just as seriously as an in-person test:
• Work closely with the team, watch the sessions together (bring snacks!)
• "If your designers and developers are not beating down the door to attend user testing, fire them"
• Remote usability helps you test at a time convenient to your participants, so work around them, not vice versa
• Don't bore people with long, detailed surveys. Instead, ask a few broad open-ended questions, from which you can create summary categories. In future studies, use these summary categories as multi-choice options
• Sprint to a functional prototype, so that you can test early and iterate While I agree in principle, I'm not so sure about this one. The risk is getting stuck down a design path too early as a result of not enough research, which may exclude other, better suited, options

Check out this cool new toy that the presenters mentioned at www.sifteo.com

When user testing, either remote or in-person, follow Kuniovsky's seminal book Observing the User Experience, which asks researchers to:

• Define the audience and goals
• Create tasks that address those goals
• Get the right people
• Watch them try to perform the key tasks

To provide an example of how remote testing was used effectively, the presenters mentioned a case study from Twitter where researchers:

• Defined the design principle as "emphasize the moment" and wrote tasks that tested this
• Tested four people per round over 15 rounds of user tests
• Used friends and family for the recruit. The researchers accepted that there would be a bias with this type of recruit, but were satisfied with knowing that they had a good sense of what this bias would be

Myths and truths of good design

• Myth: Genius design bubbles up from the imaginative ether of the design team
• Truth: Imaginative research facilitates invention

When thinking about design, consider this baseball metaphor: the pitcher is the designer, the batter is the design, the catcher is the researcher, and the umpire is metrics

• The pitcher and catcher are trying to get the batter out. The catcher (researcher) is watching the batter (design) and looking for weaknesses, then calls the pitch he thinks will get the batter out. The pitcher can choose to work as a team and listen to the catcher's advice, or go it alone. The umpire (metrics) is in the background, always there and keeping an eye on the game to make sure everything is OK. If anyone has a problem, they can go to the umpire

And, finally, the presenters talked about these cool looking mobile user testing tools

http://www.dscout.com/

http://www.revelationglobal.com/mobile

[TomLessing]

Don't agree with:

"If your designers and developers are not beating down the door to
attend user testing, fire them"

TL: If you consider that finding good developers is hard, finding
great ones is even harder then firing is a pretty stupid action if
they don't entirely behave as the mantra/religion/process dictates.
However if a your development team does not have a serious interest in
usability in the products they produce then you have a serious
problem. Seems management recruited the wrong people to begin with.

User testing is at best a flawed process. Any serious developer will
probably yawn at being forced to act stupidly sitting through
incredibly boring sessions at times. (If your development cycle is
right your developer will be not very far from the user anyway so
interaction will happen naturally).

What is the difference in output between an average developer rated at
1.0 and a good developer and the best?

On May 10, 10:57 am, AJK <ajk...@gmail.com> wrote:
> Source: Optimal Usability Newsletter
>
> Stop listening to your customers - Nate Bolt, Bolt|Peters and Mark Trammell,
> Twitter.
>
> This was a big push for remote usability. We are certainly advocates at
> Optimal Usability, but Nate Bolt is an evangelist.
>
> Here are some key points:
>

>    - Don't ask people what they want, watch what they do
>    - The key strengths of remote usability are that it is Contextual, Time
>    Aware, and Behaviour-Based
>    - Remote usability must be taken just as seriously as an in-person test:
>    - Work closely with the team, watch the sessions together (bring snacks!)
>    - "If your designers and developers are not beating down the door to

>    attend user testing, fire them"

>    - Remote usability helps you test at a time convenient to your

>    participants, so work around them, not vice versa

>    - Don't bore people with long, detailed surveys. Instead, ask a few broad

>    open-ended questions, from which you can create summary categories. In
>    future studies, use these summary categories as multi-choice options

>    - Sprint to a functional prototype, so that you can test early and
>    iterate *While I agree in principle, I'm not so sure about this one. The

>    risk is getting stuck down a design path too early as a result of not enough

>    research, which may exclude other, better suited, options*

>
> Check out this cool new toy that the presenters mentioned atwww.sifteo.com
> When user testing, either remote or in-person, follow Kuniovsky's seminal
> book Observing the User Experience, which asks researchers to:
>

>    - Define the audience and goals
>    - Create tasks that address those goals
>    - Get the right people
>    - Watch them try to perform the key tasks

>
> To provide an example of how remote testing was used effectively, the
> presenters mentioned a case study from Twitter where researchers:
>

>    - Defined the design principle as "emphasize the moment" and wrote tasks
>    that tested this
>    - Tested four people per round over 15 rounds of user tests
>    - Used friends and family for the recruit. The researchers accepted that

>    there would be a bias with this type of recruit, but were satisfied with
>    knowing that they had a good sense of what this bias would be
>
> Myths and truths of good design
>

>    - Myth: Genius design bubbles up from the imaginative ether of the design
>    team
>    - Truth: Imaginative research facilitates invention

>
> When thinking about design, consider this baseball metaphor: the pitcher is
> the designer, the batter is the design, the catcher is the researcher, and
> the umpire is metrics
>

>    - The pitcher and catcher are trying to get the batter out. The catcher

[AJK]

I think what was intended with the comment was that programmers should care how user interact with their programs. Attending a user testing or two or even just attending the summary presentation will suffice.

I was recently at a meeting where programmers just "suggested" alternatives because they "couldn't (not willing, don't want to spend the time/money, too much effort, don't have the skills) present a display in a certain way. This was after user testing was done. Now this kind of thing happens all of the time, but the difference between someone that does respect users input and those that don't, can be seen in the following example.

A) It is not possible to do point 3. Just move the navigation over here and take that stuff away.

B) It is not possible to do point 3. We can suggest the following alternatives. How will this affect the user experience or which solution will have the least negative effect on the user experience?

I think it is pretty obvious who understands the importance of the user experience. You don't have to be an expert. You just have to respect the process and acknowledge that your actions affect it.

[TomLessing]

Saying what you mean is exactly the big challenge with UX, a good
interface is just so much easier to work when it gets simple things
like this right. This statement doesn't mean what you explained it to
be. I often make the same mistake, say one thing when I mean another.
This is one of the big contradictions within UX, do you practice what
you preach? Do we even know what we are preaching? Even more painful.

In SuperFreakonimcs they talk about how doctors discovered a hundred
years or so ago that bad hand hygiene are killing their patients in
hospitals. It is a gruesome indictment for doctors. Instead of hailing
the discoverer of the cause a hero he was branded an outcast. Doctors
were literally murdering their patients with their lack of knowledge
and failed to realize the real problem. (The discoverer connected the
dots when a professor died after being cut in an autopsy, the
professor contracted a whole range of diseases). (See,
http://en.wikipedia.org/wiki/Ignaz_Semmelweis)

If you think doctors have learned ... then you will be a bit
optimistic, hand hygiene is probably behind a lot of the super bugs
doing the rounds in hospitals and some studies put doctors in top
hospitals washing their hands at about 9% of what it should be. So if
doctors who took an oath not to harm can still struggle to wash their
hands as often as they should then what does it say about UX?
Different field maybe but UX is probably no more mature than doctoring
was a hundred years ago.

So if doctors unwittingly kills? how many good software programs have
been killed by people who knew better or worse should have known
better?

When you get to the end of your knowledge it is time to make
mistakes.....

[AJK]

I completely agree with you on saying what you mean, but we are reading a summary of the talk and it might be out of context.