SMS usability

[kieran sharpey - schafer]

Hey guys,

Was just looking at creating a few prototypes here so stakeholders can
understand what can work.

So a child is already registered via SMS with a unique ID. If I want
to capture a malnutrition consultation (bearing in mind, other
indicators could also be possible), would the SMS look like:

ID        MUAC     Hight     Weight  Oedema
1234   +M 10.1     98         41.5    Y
(probably least effort to type, maybe easy to mess up all numbers)

or
1234   +M 10.1   +H 98  +W 41.5    +O Y
(clearer and works if you only have a subset of indicators, but
probably more effort)

or
1234   +M 10.1cm   98cm   41.5kg    Oedema severe
(follows paper well, but a bit longer to type switching between nums & letters).

or perhaps just a dedicated message for malnutrition
MUAC +1234 10.1 98 41.5 Y

What are peoples experience of the pro's & cons of the various approaches?

My gut feels like things should be as natural language as possible -
and then have someone daily validate those that couldn't be parsed. Or
is that just crazy talk. Maybe users are fine with the fastest option
(#1)?

Any thoughts (or ridicule) welcome.
Cheers,
Kieran

----------------------------------------------------------------------
Kieran Sharpey-Schafer
Project Coordinator
UNICEF

[Nic Pottier]

This is one particular subject that I would love to hear people's
experience on. Field experience is king there.

One piece of feedback we got from Chris from using XForms in the field
is that '+' is a horrid separator to use, it can be really burdensome
to find on some cell phones. '.' is probably the most universally
accessible 'command' delimiter we could use, though it might collide
with other things.

The command syntax works well for cases where fields are optional, but
I think it should be avoided unless needed. So in your case, if you
know you will always be recording all those fields, the message should
just be an ordered set of fields. That is easier to type out and
explain given a good paper form analogue.

So maybe something like this:

Cmd ID        MUAC    Height     Weight  Oedema
NUT 1234   10.1        98         41.5         Y

Looking forward to seeing what other people say as I think this is an
especially interesting subject.

-Nic

[kieran sharpey - schafer]

> One piece of feedback we got from Chris from using XForms in the field
> is that '+' is a horrid separator to use, it can be really burdensome
> to find on some cell phones.  '.' is probably the most universally
> accessible 'command' delimiter we could use, though it might collide
> with other things.

Understood but I wouldn't rule out the + just yet as people only need
to learn it once, and if they care enough and have enough peer support
I reckon they can get over the initial hurdle. Though it certainly is
worth doing a broader user test (which we intend to do at some
point!).

> The command syntax works well for cases where fields are optional, but
> I think it should be avoided unless needed.  So in your case, if you
> know you will always be recording all those fields, the message should
> just be an ordered set of fields.  That is easier to type out and
> explain given a good paper form analogue.
>
> So maybe something like this:
>
> Cmd   ID        MUAC    Height     Weight  Oedema
> NUT   1234      10.1        98           41.5         Y

Good point about command fields. I think ChildCount has lots of
optional fields so I'm intersted in following up what their thinking
is.

Cheers,
K

----------------------------------------------
Kieran Sharpey-Schafer

mobile: +27 72 371 6469 (SA)
mobile: +1 347 703 9280 (US)
skypeid: kieran.sharpey.schafer

[Evan Wheeler]

On Tue, Aug 24, 2010 at 10:43 AM, kieran sharpey - schafer
<kieran....@gmail.com> wrote:
>> One piece of feedback we got from Chris from using XForms in the field
>> is that '+' is a horrid separator to use, it can be really burdensome
>> to find on some cell phones.  '.' is probably the most universally
>> accessible 'command' delimiter we could use, though it might collide
>> with other things.
>
> Understood but I wouldn't rule out the + just yet as people only need
> to learn it once, and if they care enough and have enough peer support
> I reckon they can get over the initial hurdle. Though it certainly is
> worth doing a broader user test (which we intend to do at some
> point!).

Its not merely a learning thing. It takes around 8 key presses to get
a '+' to show up no matter how many times youve done it before. On
most handsets (like nokias) '.' takes only one long press.

>> The command syntax works well for cases where fields are optional, but
>> I think it should be avoided unless needed.  So in your case, if you
>> know you will always be recording all those fields, the message should
>> just be an ordered set of fields.  That is easier to type out and
>> explain given a good paper form analogue.
>>
>> So maybe something like this:
>>
>> Cmd   ID        MUAC    Height     Weight  Oedema
>> NUT   1234      10.1        98           41.5         Y
>
>  Good point about command fields. I think ChildCount has lots of
> optional fields so I'm intersted in following up what their thinking
> is.

Yes. Command fields are a very good idea. Helps ensure your system is
extensible -- you can easily add other commands/forms that dont
conflict with each other. They can also aid in parsing/handling
schemes.

[David McCann]

YES!  Someone else that hates '+' as much as I do!  I was working on another SMS campaign (that will remain nameless) that wanted to use equal signs for parameters, even worse:

MUAC ID=1234 HGT=98 WGT=41.5 OE=Y

Or similar.

For me, the bottom line is that if your requirements are:

1) to collect a fair amount of data (something more complicated than a single number or yes/no answer),

2) without having to prompt the user, and

3) to automatically parse said data in a way that's fairly robust to type-os and a low potential for ambiguity

You're going to be stuck with a format that's fairly distant from anything natural, and so you'll have to train (and probably distribute laminated cheatsheets) regardless of what format you come up with.

So my criteria for a message format tends to lean towards those formats that are easiest/fastest/cheapest to train. "." over "+" is a great example...it's fairly standard that keying a period is the "1" button.  Now imagine being in a room with 20 trainees, all with different model phones, and trying to teach each of them how to key in a plus sign on their particular phone...some models of which you've probably never used yourself.

I'd have to say an ordered set of fields has some potential pitfalls, in terms of training materials.  Not impossible, but you have to place a lot of care in how you explain the message format.  It's better if the format can be clear from literal examples alone.  If your instructions contain something to the effect of:

Messages should follow the format:

(ID)   "MUAC"  (Height) (Weight)  (Oedema, y/n)

Even if you supplement the above with examples, people can be pretty literal; it's guaranteed you'll have people texting in with parenthesis, quotation marks, and probably the placeholder words themselves.

I'd lean towards more self-describing message formats where examples alone do the trick (and parameters can come in any order), maybe:

Patient 1234.  MUAC. Height 98. Weight 41. Oedema.

--dm

--

You received this message because you are subscribed to the Google Groups "rapidsms" group.
To post to this group, send email to rapi...@googlegroups.com.
To unsubscribe from this group, send email to rapidsms+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rapidsms?hl=en.

[Kevin Samuel]

I'll go with Nic.

But when it's about usability, your best bet is to test it. Take a bunch
of people, and make them type it, so you see what errors they make.

Usually parsing is not such a big deal, so you can code something, they
try it at a training session, then change it according to the results.

>> (follows paper well, but a bit longer to type switching between nums& letters).

>>
>> or perhaps just a dedicated message for malnutrition
>> MUAC +1234 10.1 98 41.5 Y
>>

>> What are peoples experience of the pro's& cons of the various approaches?

[Tobias McNulty]

On Tue, Aug 24, 2010 at 10:47 AM, David McCann <david.a...@gmail.com> wrote:

Even if you supplement the above with examples, people can be pretty literal; it's guaranteed you'll have people texting in with parenthesis, quotation marks, and probably the placeholder words themselves.

Exactly. It seems that literal transcription of training materials is more the standard and less the exception.

 

I'd lean towards more self-describing message formats where examples alone do the trick (and parameters can come in any order), maybe:

Patient 1234.  MUAC. Height 98. Weight 41. Oedema.

I like this.  An alternative solution, if the parameters are ordered, is to make the placeholder words & punctuation optional, like so:

NUT [patient] 1234[.] [muac] 10.1[.] [height] 98[.] [weight] 41[.] oedema

Of course, your training materials would look more like:

NUT patient 1234. muac 10.1. height 98. weight 41. oedema

(with or without the periods).  So, you can train users in the full syntax, but allow more advanced users to drop the keywords.

Cheers

Tobias

--
Tobias McNulty, Managing Partner

Caktus Consulting Group, LLC
http://www.caktusgroup.com

[Jonathan Jackson]

1)  How often your users are going to be interacting with their system, how long they have to learn (is this a single campaign or the same person using the system everyday for a year), and how you plan to do the re-training of new users would dictate whether to go for something that makes cognitive sense or is easy to extend / type.

2)  We deployed a 10 field system in India to measure reverse osmosis systems that had format like:  

"a7 b2 c3 d4.4 e10 gtest123"

where a,b,c,d, e, f, and g were column headers on a paper form (and here we skipped f).  Just one more option.  Obviously the fact that column g is a string would be very confused to try to write if the row also started with a g in the cell.

In general, I think if they are using the system everyday, most users can learn most formats at varying learning curves.  If you are using it for a short campaign, some percentage of your users who aren't SMS savvy will not be able to use any format, and those who are SMS savvy will.  So, if you have anything even close to this complicated, you have a tough time deploying it for a campaign if you actually want all your data to be correct.

-J

[kieran sharpey - schafer]

> Its not merely a learning thing. It takes around 8 key presses to get
> a '+' to show up no matter how many times youve done it before. On
> most handsets (like nokias) '.' takes only one long press.

Yup good point. This is definitely the case on the 'Zain' 'TNM'
produced handsets that seem very common here.

> I'd lean towards more self-describing message formats where examples alone
> do the trick (and parameters can come in any order), maybe:
> Patient 1234. MUAC. Height 98. Weight 41. Oedema.

Great I'm liking, but how have people got around the problem of
decimal? e.g. for infants we need to know our 3.1 from our 3.9kgs. Do
you think parsing can get around that (number.number) and that users
won't get to confused?

Has anyone tried '.' as the control delimiter?

Cheers, Kieran

[Matt Berg]

Lots of thoughts here.

First, I agree we don't know if + is the best delimiter or not but is has been working relatively well for us.  I'll have our team take a look at the . though.  The nice thing about the + is it is provide a very nice visual delimitation in forms

The advantage of ID +FORM param1 param +FORM2 param1 param2

that we use in childcount is that it allows us to pass multiple forms in a single sms operating on the initial id (or multiple ids) passed.  We can then structure very short and easy to use SMS forms and combine them together to gather more complicated data.  It's also great because it allows great flexibility to a non-tech person creating forms including the ability to put multiple forms, in any order, on a single line of paper. (see our consultation log: http://dl.dropbox.com/u/910925/ChildCount%2B%20Public%20Docs/ChildCount_Forms_v2_1/ChildCount_Form_C_Consultation_v2-1_EN.pdf)

I agree with John if the users use the system frequently then they can learn most formats quite well.  So for forms that are used frequently, I think the emphasis should be placed on efficiency and not verbosity (better for things used less frequently).  For this reason, I prefere having small forms where the end user is required to know the syntax including use of representaive code.  We provide a cheat sheet or paper form to help with that.  Typing the parameter name before every value strikes me as simply too much typing.  

There are other cases where I think it makes sense to just put the keyword first.  This has also worked well for us.

The key for me is keeping it simple, flexible and when possible short with the amount of shorthand you get away with linked to the frequency of use.  For infrequent reports (weekly, monthly) you probably want to use paper to help build the sms messages anyways.

Matt 

Cheers, Kieran

--
You received this message because you are subscribed to the Google Groups "rapidsms" group.
To post to this group, send email to rapi...@googlegroups.com.
To unsubscribe from this group, send email to rapidsms+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rapidsms?hl=en.

--
ICT Director
Millennium Villages Project
Columbia University, NYC
M: +1.646.463.1273
Skype: mlberg

[merrick]

On Aug 24, 11:23 am, Matt Berg <mlb...@gmail.com> wrote:
> Lots of thoughts here.
>
> First, I agree we don't know if + is the best delimiter or not but is has
> been working relatively well for us.  I'll have our team take a look at the
> . though.  The nice thing about the + is it is provide a very nice
> visual delimitation in forms
>
> The advantage of ID +FORM param1 param +FORM2 param1 param2
>
> that we use in childcount is that it allows us to pass multiple forms in a
> single sms operating on the initial id (or multiple ids) passed.  We can
> then structure very short and easy to use SMS forms and combine them
> together to gather more complicated data.  It's also great because it allows
> great flexibility to a non-tech person creating forms including the ability
> to put multiple forms, in any order, on a single line of paper. (see our

> consultation log:http://dl.dropbox.com/u/910925/ChildCount%2B%20Public%20Docs/ChildCou...
> )


Regarding end user usability, wouldn't the server be able to use any
delimiter to parse a message contain multiple forms? If + is
difficult to type why not use an easier delimiter.

On our last project we did more call and response and minimized the
information per message to make the system easier to use. It raises
the cost but makes things easier for the end user.

[David McCann]

The question is, does it really raise the cost of a project?  What I'd love to see is an actual cost comparison between trainingless, simple-question-simple-answer data gathering, versus a large scale rollout with trainings on complex message formats...

--

[Nicolas di Tada]

A few things to contribute:

1) Our approach to decide which delimiter or command indicators to use is to buy a few used phones from a local market and then measure the keystrokes distances for each symbol. Then get the average and sort them by lowest. I'm attaching a sample. '+' doesn't rank very well, and '.' ranked first.

, ? ! and even # are better options, at least for these kind of phones.

2) We've built a lean tool called SeenTags (seentags.googlecode.com). It's basically a forgiving generic parser that we are using with good results and that we are constantly improving. It can be used as either a library or as a webservice so there's no need to deploy it or integrate it. You post your commands and you get the best guess for a parsing. A user can correct, through a simple interface, messages that were not parsed correctly, and the system learns from that. The nice thing is that you don't need to define your syntax upfront. For an agile design, it works nice, because you can start testing different approaches with users and even let them design their own syntax.

Nicolas

[Tim Akinbo]

Nicholas,

Seentags is really an awesome project. It would really be great if there were a python port of the app - it will be a great addition to RapidSMS. Thanks for sharing.

Tim Akinbo

[Jeff Wishnie]

>
>
> Regarding end user usability, wouldn't the server be able to use any
> delimiter to parse a message contain multiple forms? If + is
> difficult to type why not use an easier delimiter.

On the Tostan roll-out a year ago Rowena and I coded the system to handle a variety of delimiters including: + . # and a string of letters like ABC. In the end I think '.' was easiest for people to enter on a standard feature-phone, but people were generally comfortable with all (except the letter-string which was not popular).

Rowena, what did you find later in the project? People stick with '.'?

- Jeff

[ChristopherFabian UNICEF]

after using a nokia 1100 in a stressful environment and having to work
*really* hard to get to the + symbol, i would have traded nice visual
delimiting for less sore fingers in a heartbeat. with total
agnosticism to what it is, i would recommend that anyone 1) try to
pick a character/symbol which is easy to get to (a few presses) in T9
and non- on a basic nokia(or whatever) and 2) pull out a timer and
time people using whatever 3 models of phones are most easily
available to fill out some forms with +, ., etc... write down the time
for each, multiply by # of fields expected... capture this somewhere

i'm first in queue for an "+=bad" tshirt
c

On Aug 24, 11:23 am, Matt Berg <mlb...@gmail.com> wrote:

> Lots of thoughts here.
>
> First, I agree we don't know if + is the best delimiter or not but is has
> been working relatively well for us.  I'll have our team take a look at the
> . though.  The nice thing about the + is it is provide a very nice
> visual delimitation in forms
>
> The advantage of ID +FORM param1 param +FORM2 param1 param2
>
> that we use in childcount is that it allows us to pass multiple forms in a
> single sms operating on the initial id (or multiple ids) passed.  We can
> then structure very short and easy to use SMS forms and combine them
> together to gather more complicated data.  It's also great because it allows
> great flexibility to a non-tech person creating forms including the ability
> to put multiple forms, in any order, on a single line of paper. (see our

> consultation log:http://dl.dropbox.com/u/910925/ChildCount%2B%20Public%20Docs/ChildCou...
> )

> > rapidsms+u...@googlegroups.com<rapidsms%2Bunsubscribe@googlegroups.c om>
> > .