Some things I learnt running my first informal Guerrilla user tests

[Shaun O'Connell]

Hi Gang,

As part of a Job Application recently, I had to complete a practical assessment, which I interpreted as an opportunity to do some Guerrilla User Testing.

Here are some of the things I learned along the way:

• Recruiting Users is not as easy and successful as I hoped.
  
• I learned a lot just from the casual exchanges with my potential Users, however subjective it was.
  
• Scheduling 10-minute screening calls requires a lot of time and patience.
• Screening can take a lot longer than expected when the user is relaxed and talking their mind.  (I need to find a way to keep them on track, and in time, or allow for more time)
  
• Chaining Interviews and Testing Sessions together in one day is a recipe for lateness or postponements on my behalf.
• Testing family members could lead to loaded results.
  
• The potential cost of Guerrilla Testing is minimal.  Barring paying for small gifts for my Test Users, the only true expense was my time.  I imagine the more time I spend planning and researching, the higher the quality of results I will get back.
• Any sort of User Testing will make the User quite nervous.  (I still need to find a way to make my user's relax)
• I must have a screen, keyboard and mouse for the User Test.  Testing from a laptop, especially a Mac with a small screen, is quite finnicky for some users.
  
• Going into a User Test without a task-plan is not as productive as it should be.  Next time I'll plan a storyboard of Tasks for the user to complete.
  
• When user testing from someone's own house: Remember to turn off the TV, Radio, Cellphones etc.  Don't forget to let the dogs out of the room! haha.
  
• Silverback takes about a minute to render every minute of Screen-Casting recorded.  If the User Testing session is 30 minutes long, the exporting of that session will take 30 minutes.
• Silverback eats away at your free harddrive space, like a ravenous... err, Gorilla. :)
  

What have your experiences been with Guerrilla Testing?

Cheers,
Shaun

[Nick]

Thanks for your lessons, I'm sure they'll come in handy.

[supermonkey]

It's really great that you took on something like this! thanks for
sharing.

It's the only way to get first hand experience no matter how much
theory you cover.
people remain odd, varied and unpredictable beings - which means
dealing with the people in the moment is the nr 1 thing you end up
doing (making some notes on the side and having to burn your brain to
control 3 things at the same time!).

The things I learned is
1. that it's a LOT more tiring than I thought initially, so limiting
the amount of people per day keeps you sharper - else you tend to
start leading and answering for them to speed along cause you simply
can't focus after seeing 5 users for 90minutes a pop.
2nd thing I've learned.. review and summarise your findings
immediately after the test / test day, no matter how tired you are -
else you WILL forget the context and situation of the result.
3. don't design in your mind when testing - you will end up seeing and
steering users to point to your design as a solution = no no.


I'd like to hear a little more detail... as in,
how many users did you test?
how dit you recruit your users?
did you have a screening criteria?
what did you use as gifts that didn't cost you money!?
sounds like you did interviews and test with the same people back to
back? how did that play out? started with an interview and then onto
the test? how long was the interview bit?
did you get some solid results / patterns?

my 2 cents on what works for me to relax people... (I've found that
they tend to relax after the first 10minutes when they see there's no
stupidity meter, exploding laptops or any other freaky stuff)

ask them to tell you a little about themselves, what they did before -
or doing after the test, , what they like about the net / computers
trying to find something in the answer you can small talk around for
couple of minutes. make friends before jumping into the test, and then
ease into the test bit by making it a part of the making friends
conversation (this is part bit where you reciprocate and tell a bit
about yourself ending with.. I design usable stuff)... I tend to try
and make them feel more important than me right from the start -
ultimately the user's behavior IS the most important thing. tell them
that you're not testing them, but testing the system and their input
is needed to make the system better - you cannot do it without them...
and as a new friend you're going to test the system with them now.