how much training?
39 views
Skip to first unread message
mworkman
Nov 1, 2012, 8:47:06 AM11/1/12
to openxda...@googlegroups.com
Hi again.
For my evaluation study, I've done 2-2,5 hrs training in a hands on workshop, 4 people at a time using mforms 1.3. Subsequent to this they have each performed 4 mock interviews with other participants with the questionnaire being based on their current paper forms. However the error rates for the captured data can be over 10%. As a control, they also complete mock surveys of the same questionnaire using paper forms and this error rate is approx 1%, so understanding the form is not a problem. Can any help with how much training is generally given for first time users of openXdata?
Tx
Mike
For my evaluation study, I've done 2-2,5 hrs training in a hands on workshop, 4 people at a time using mforms 1.3. Subsequent to this they have each performed 4 mock interviews with other participants with the questionnaire being based on their current paper forms. However the error rates for the captured data can be over 10%. As a control, they also complete mock surveys of the same questionnaire using paper forms and this error rate is approx 1%, so understanding the form is not a problem. Can any help with how much training is generally given for first time users of openXdata?
Tx
Mike
Dagmar Timler
Nov 1, 2012, 8:51:39 AM11/1/12
to openxda...@googlegroups.com
Hi Mike
I just asked one of my colleagues who is involved in training and he says 1.5-2hrs in a group of 10 is normal for us.
I am curious that you are seeing such high error rates in your mobile forms. Are these errors that should be caught by validation logic in the forms or are users just selecting the wrong answers?
Dagmar
--
You received this message because you are subscribed to the Google Groups "openXdata Users" group.
To post to this group, send email to openxda...@googlegroups.com.
To unsubscribe from this group, send email to openxdata-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/openxdata-users/-/ZUyc0eHubgMJ.
For more options, visit https://groups.google.com/groups/opt_out.
mworkman
Nov 2, 2012, 6:25:00 AM11/2/12
to openxda...@googlegroups.com
Thanks Dagmar.
I avoided validation logic at first because my focus is on measuring the accuracy with which the users can capture, and validation logic can only prevent the gross errors. I think there are various issues at play here:
1) Repeating group is a headache. The empty screen, then having to click the New(game) key for the first row. And then at the end they keep hitting select to go the the second row, instead of going to options. Then they sometimes hit New, even though there are more options. So its not so much that they actually enter the wrong answer, they either overwrite the first set of items with the second set, or they create a third set when there are only two.
2) My issue with how to select a residential location from a list of 148. I created a hierarchy of district, town/area and farm. But the participants do not accurately know in which district the farms are - in the field the responses are usually based on the town to which the farmer takes them on Saturdays, which is not necessarily closest. For example, many farmers go into Worcester as its the biggest centre, even though it is 60 km further. So they leave the answer out as the dynamic list doesn't contain their desired answer. I've now updated the documentation with a district indicator, but it now means the users have to do a look up on a paper form before they can capture the data, but this defeats one of my objects of having a cell phone provide them with all the information. So I really don't know the answer to this.
3) less of an issue is dates and times.
a) I've tried to set mforms to use year first as that is what they are used to, but v1.3 seems to ignore this. So I've had dates captured as 20/12/1126 - this can be trapped by validation logic
b) dates with the - and times with the : - sometimes the users try to insert the - or : going round in circles trying to find them on the keyboard, and finally leaving the answer out.
c) times are captured as 24hour clock, but the answer displays on the question list as12 hr AM/PM - so they have twice captured the time as 3:00 instead of 15:00
4) In general, the continual changing of the soft key labels causes confusion, many back steps and sometimes the wrong selection.
5) The question I am really trying to answer is whether these participants who are not computer literate, many of whom do not have school leaving certificate (grade 12, standard 10, matric), can use a mobile phone to capture data accurately. Some of the terminology is 'PC' centric e.g. 'Upload Data' required a fair deal of explanation, whereas 'Send Results' would have been more meaningful to them.
So I'll wait and see how things pan out, and hopefully they become more skillful as they progress. It may mean I have to treat the current round as an extended training and extend the study to a further round.
Thanks again
Mike
I avoided validation logic at first because my focus is on measuring the accuracy with which the users can capture, and validation logic can only prevent the gross errors. I think there are various issues at play here:
1) Repeating group is a headache. The empty screen, then having to click the New(game) key for the first row. And then at the end they keep hitting select to go the the second row, instead of going to options. Then they sometimes hit New, even though there are more options. So its not so much that they actually enter the wrong answer, they either overwrite the first set of items with the second set, or they create a third set when there are only two.
2) My issue with how to select a residential location from a list of 148. I created a hierarchy of district, town/area and farm. But the participants do not accurately know in which district the farms are - in the field the responses are usually based on the town to which the farmer takes them on Saturdays, which is not necessarily closest. For example, many farmers go into Worcester as its the biggest centre, even though it is 60 km further. So they leave the answer out as the dynamic list doesn't contain their desired answer. I've now updated the documentation with a district indicator, but it now means the users have to do a look up on a paper form before they can capture the data, but this defeats one of my objects of having a cell phone provide them with all the information. So I really don't know the answer to this.
3) less of an issue is dates and times.
a) I've tried to set mforms to use year first as that is what they are used to, but v1.3 seems to ignore this. So I've had dates captured as 20/12/1126 - this can be trapped by validation logic
b) dates with the - and times with the : - sometimes the users try to insert the - or : going round in circles trying to find them on the keyboard, and finally leaving the answer out.
c) times are captured as 24hour clock, but the answer displays on the question list as12 hr AM/PM - so they have twice captured the time as 3:00 instead of 15:00
4) In general, the continual changing of the soft key labels causes confusion, many back steps and sometimes the wrong selection.
5) The question I am really trying to answer is whether these participants who are not computer literate, many of whom do not have school leaving certificate (grade 12, standard 10, matric), can use a mobile phone to capture data accurately. Some of the terminology is 'PC' centric e.g. 'Upload Data' required a fair deal of explanation, whereas 'Send Results' would have been more meaningful to them.
So I'll wait and see how things pan out, and hopefully they become more skillful as they progress. It may mean I have to treat the current round as an extended training and extend the study to a further round.
Thanks again
Mike
mworkman
Nov 2, 2012, 6:28:52 AM11/2/12
to openxda...@googlegroups.com
For what its worth, most are using their own phones which are a mix of Nokia C60 and Samsung. For some who have Blackberry, I've provided Nokia C1s. And this could be another cause of error - one of the participants has a touch screen phone, and she was continually trying to swipe her fingure on the C1 screen;)
Dagmar Timler
Nov 14, 2012, 1:51:23 PM11/14/12
to openxda...@googlegroups.com
Hi Mike
I am sorry about the delayed response. I think you definitely need validation logic - and for this reason. It just helps the users, and I think it can be quite powerful.
1. You can add validation logic to check how many repeats are expected. Usually this involves a question like "how many children do you have?". See the sample form for an example
2. Is it possible to break up the list alphabetically? Even if the locations are mainly a certain letter, you can extra characters to differentiate.
3. Great - glad to hear that!
4. Yes the workflow needs attention. Maybe someone else in the community has further input
5. Text could be improved. Upload is rather a technical term.
Perhaps you can consider writing patches for some of the items and make improvements to the mobile client before going live? I hope so!
Thanks
Dagmar
--
You received this message because you are subscribed to the Google Groups "openXdata Users" group.
To post to this group, send email to openxda...@googlegroups.com.
To unsubscribe from this group, send email to openxdata-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/openxdata-users/-/NzFUDLClyNkJ.
Sarah Bird
Nov 14, 2012, 4:09:39 PM11/14/12
to openxda...@googlegroups.com
Just a quick thought. You can just translate the mobile to whatever language you like - this could just be your preferred version of english! See http://doc.openxdata.org/1699
mworkman
Nov 19, 2012, 3:38:16 AM11/19/12
to openxda...@googlegroups.com, sb...@alum.mit.edu
Thanks Sarah, that would work.
mworkman
Nov 19, 2012, 3:59:55 AM11/19/12
to openxda...@googlegroups.com
Hi Dagmar, delayed or not, response is welcome. I'll see how the current round goes, and then see if I have funds to maybe do a 'mark 2' study.
1) I'm afraid I can't see how knowing the number of repeats up front would help things. Also "how many children?" is an easy one to answer, but "how many symptoms do you have or have you had in the past week?" is a bit less obvious.
2) Thanks for this, I looked at alphabetic, but some were just too long - adding extra letters will get over this issue - lets hope for Mark 2.
4) I was taught that for good usability, the desired actions are easy to select, emergency actions are easy to select (e.g. save work if system error), but it doesn't matter if the less desired actions are more difficult to find/reach. With openXdata, the actions that are usually available and are consistently in the same place (RHS) are 'Cancel' or 'Back' which are less desired actions. On repeating groups, Select (to edit) is available, whereas 'New' and OK (which are surely the desired options) are buried in the Options menu. Now having written this, I realise that it looks very similar to my issues with J2ME ;-( OpenXdata is based on openRosa (I think) - is openRosa based on J2ME?
5) Sarah's comment of translation is good - I'll do this in the Mark 2 (if and when).
Regards
Mike
1) I'm afraid I can't see how knowing the number of repeats up front would help things. Also "how many children?" is an easy one to answer, but "how many symptoms do you have or have you had in the past week?" is a bit less obvious.
2) Thanks for this, I looked at alphabetic, but some were just too long - adding extra letters will get over this issue - lets hope for Mark 2.
4) I was taught that for good usability, the desired actions are easy to select, emergency actions are easy to select (e.g. save work if system error), but it doesn't matter if the less desired actions are more difficult to find/reach. With openXdata, the actions that are usually available and are consistently in the same place (RHS) are 'Cancel' or 'Back' which are less desired actions. On repeating groups, Select (to edit) is available, whereas 'New' and OK (which are surely the desired options) are buried in the Options menu. Now having written this, I realise that it looks very similar to my issues with J2ME ;-( OpenXdata is based on openRosa (I think) - is openRosa based on J2ME?
5) Sarah's comment of translation is good - I'll do this in the Mark 2 (if and when).
Regards
Mike
mworkman
Nov 19, 2012, 4:18:08 AM11/19/12
to openxda...@googlegroups.com
DAMN. I've just stumbled on 'OK on Right' and 'General Single Question Edit' - which seem as if they could solve quite a few navigation problems. I really must try for another round to see if it will go better - maybe Santa will be generous;-)
Message has been deleted
Dagmar Timler
Nov 11, 2013, 1:36:12 AM11/11/13
to openxda...@googlegroups.com
To clarify for anyone who reads the archives.
1.5-2 hours training would only be sufficient for computer literate users. For non-computer and phone literate users a minimum of 4 hours training is more appropriate.
For those users, it would be prudent to start with phone basics (from turning on the phone, charging the phone to using phone features like SMS and camera) and then only train them to use their specific forms. Our trainers recommend that you ask users to work in pairs to practice capturing and submitting data - and repeat the exercise a couple times.
Finally, if there are users who are resistant to new technology and processes, it is best to get this out of the way at the beginning of the session otherwise it could affect the quality of the training.
Hope this helps.
Dagmar
Reply all
Reply to author
Forward
0 new messages