Disabling answers feedback from the LMS

[Bertrand Marron]

Hello,

I’m planning on adding a feature to disable the feedback that’s shown
to students when they answer a question.
For example, on a simple multiple choice problem with only one
attempt, when a student answers, they will know whether their answer
is right or wrong.

I want to make that disablable, as well as the "in-progress" and
"done" states of the sequence navigation.

Could this be done globally?
I could add an option to every XBlock/XModule, but I’d rather do it
globally and make it a course setting.

I’d like to know what you think about adding this feature to Open edX.

Thank you.

--
Bertrand Marron

[Jane Manning]

Hi Bertrand,

This sounds like a useful feature to me.  FWIW, I'd prefer to see it at a finer-grained level than for the whole course.  For instance, you might have "formative" exercises where you _do_ want to give students feedback right away, but don't want to do so on the exams.  A problem-level setting would seem ideal to me.  Note that there's already a "show_correctness" attribute you can set to "never" in problem XML.  I'm unable to find documentation about that, but you can see it in the code:

https://github.com/edx/edx-platform/search?q=show_correctness

So exposing that in the Settings UI might be a good start, though I suspect may not give you everything you want - e.g. the "Progress" page will presumably still tell students what their score was, which you may want to hide until after a deadline if you're wanting to use this feature to make exams harder to game.

good luck -

Jane

[Emma Brunskill]

I think this would be a great feature, particularly if it was enabled at the problem level, as Jane suggested.

Just FYI, there is a little bit of documentation about the show correctness attribute within instructions to course developers

http://files.edx.org/Getting_Started_with_Studio.pdf

However, in my experience, the show_correctness feature only affects whether the solution is ever shown or not, but the student still gets feedback about whether his/her answer is correct or incorrect. If there are only a few possibilities (like a checkbox) then if there aren't many attempts, it's easy for the student still to figure out the correct answer. I'm personally interested in creating an exam where the students don't get immediate feedback on correctness (to try to get a more static view of their state of learning), and so the feature you describe would be great there. As a fast hack, I'm considering specifying all answers as correct when I create a problem and informing students of this setup, but it's not a great solution.

- Emma

[Yoav Bergner]

Yes! I would like to be able to do this too: have an exam without immediate feedback. Alternate to Emma's solution (scoring everything as correct), I can set attempt number to 0 which makes the check button go away leaving only the save button. I can't use the automatic scoring, since the save event doesn't check the answer, but I can score it later using a script. Setting all answers as correct also removes the automatic scoring functionality, so it's just a  question of what is cleaner.

[Sarina Canelake]

Hi Bertrand et al,

I think this is an interesting feature but is a bit orthogonal to our original goals of "gamification" of education. The immediate feedback is useful for student development and engagement. Games are addicting because they give you quick, immediate rewards and the check/red x attempts to do the same!

I do see the use for the feature but would strongly recommend implementing it on a problem-by-problem level. I think this will also have the effect of making course developers think critically about using the feature and when they are overriding the default, gamification behavior.

I don't have anything technical to add to Jane's very good response - I think you'd have to dig down into the Capa xmodule to figure out the best way to implement this. Extensive testing would also be appreciated, as Capa is a somewhat messy area of the codebase. Note we do have a way of "hiding" the progress page somewhere deep in the codebase; however, this never really worked. Students didn't see the "Progress" tab but could still navigate to the /progress URL and see their progress. So further work would be needed to restrict access fully to the progress page, and likely as well to hide the scores that show up at the top of the problem.

Best,

Sarina

[Vinh Nguyen]

Hi Bertrand,

Have you found a way to implement your feature? I'm interested to use it. 

Thanks

Vinh

[Juan Camilo Montoya]

Hello Vinh, 

you might want to take a look at: https://lagunita.stanford.edu/courses/StanfordOnline/OpenEdX/Demo/courseware/Hidden_Content/Hide_correctness/ 

cheers. :)

[Thieme Hennis]

I think it would be great to be able to disable answer feedback. But in my case maybe on a per-question basis.