API Usability Evaluation using Cognitive Dimensions Questionnaire

[Chamila Wijayarathna]

Hi all,

We are currently working on using cognitive dimensions based questionnaire method [1][2] for evaluating the usability of APIs, specially security APIs. We are planning to evaluate cognitive dimensions and the questionnaire [3] used by Steven Clarke and improve it to use in security API context.

However, we are concerned about the length of the questionnaire and how it will affect the outcome of the questionnaire. We are planning to let participants do a programming task which will make use of an API and then answer the CD questionnaire based on their experience. We assume this whole process will take around 3 hours. We have improved the questionnaire with additional 3 dimensions which made the framework to have 15 dimensions. Each dimension has approximately 4 questions which makes questionnaire 60-70 questions long. We feel that participants will feel frustrated in the middle of answering questionnaire, and the quality of the answers they provide will reduce. However, we are not sure is this is the real case since this methodology is already in use, specially in Microsoft according to Clarke.

I would like to know, if some of you have used a cognitive dimensions based questionnaire for usability evaluation, have you observed any issues related to the length of the questionnaire, does anyone has a feedback on this? Also is there anything that can be done to overcome the issues?

Thank You!

[1]. Blackwell, A.F. and Green, T.R., 2000, April. A Cognitive Dimensions questionnaire optimised for users. In Proceedings of the Twelfth Annual Meeting of the Psychology of Programming Interest Group (pp. 137-152).

[2]. Clarke, S., 2004. Measuring API usability. Doctor Dobbs Journal, 29(5), pp.S1-S5.

[3]. https://groups.google.com/forum/?hl=en#!topic/ppig-discuss/7UtLBsmFbTk

[Brad A. Myers]

I don’t have any answers for you, but it sounds like a really interesting experiment. Please let us know about the outcomes. Also, please post it to www.apiusability.org!

 

Brad A. Myers

Professor

Human-Computer Interaction Institute

School of Computer Science

Carnegie Mellon University

5000 Forbes Avenue

Pittsburgh, PA  15213-3891

(412) 268-5150

FAX: (412) 268-1266

b...@cs.cmu.edu

http://www.cs.cmu.edu/~bam

--
You received this message because you are subscribed to the Google Groups "PPIG Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ppig-discuss...@googlegroups.com.
To post to this group, send email to ppig-d...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.