Thoughts on detecting cheating

[Brad Miller]

I'm soliciting opinions on the following. Starting this weekend whenever you click the run button for an activecode for any language, The browser will compute the Levenshtein distance from the previous submission. It also keeps track of how long it took to make that change. So we know the edit distance and the number of characters changed per second. This came to us as an idea developed by a high school student to flag possible cheating.

Right now, I'm just storing those values in the database, while we figure out the best way to make use of the information. One idea I have is to put up a "red flag" on the activecode interface. "Whoa there pardner thats a LOT of changes! Copy/paste is not a good way to learn to code." Or maybe this is something that should only be available to the instructor? A simple way for an instructor to see for any given assignment the diffs where a big fast change has occurred? Or maybe something else, I've not surveyed the market so maybe there is already a good interface I can get inspired from....

Note -- the parameters of "big fast change" have yet to be determined, another reason for starting out by collecting the data for a while before acting on it.

Brad

Bradley Miller, PhD
Professor Emeritus Computer Science
Luther College
Founder, Runestone Academy LTD
Blog: A Reputable Journal
Schedule a Meeting

I am sending this email at a time that suits my work schedule.
Please do not feel obliged to reply outside of your working hours.

[Joseph DiMuro]

Don't let the culprits know what you know. Keep 'em guessing. ;-) I'd say, make this only available to the professor. Let the culprits wonder how you figured out that they were copy/pasting.

[Andrew Scholer]

I would want to be in the feedback loop for something like that. My fears re: direct feedback from the tool to student would be twofold:

1) It would turn detection avoidance into a game of "How much can I copy and paste at what rate?" Yes, students can try to play the same game if I am the one who talks to them about the patterns in their submissions. But if they insist on playing that game, it should not be easy.

2) The appropriate response is going to vary. How clear is the evidence? Has there been a pattern in the past? Where are we in the term? A canned message is not going to differentiate based on those factors. I don't want the tool to read the riot act to a student with a borderline case at the very start of the term. But I also don't want it to give gentle nudges to a serial cheater at the midpoint of the term.

Andrew Scholer (he/him/his)

Computer Science Instructor/Program Chair

Chemeketa Community College

503.589.7649 

computerscience.chemeketa.edu/people/andrew-scholer/

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/bb03fc46-9343-4c6b-9e1a-d9094f573c9e%40Spark.

[Rob Beezer]

It's a long story, but I once thought I had a clear case of two students cheating. I had it totally wrong. I'd vote for letting a human interpret the data. But then I had very small classes where I knew the students well in a variety of ways.

Rob

>> *Bradley Miller, PhD*

>> Professor Emeritus Computer Science
>> Luther College

>> Founder, Runestone Academy LTD <https://runestone.academy>
>> Blog: A Reputable Journal <https://www.reputablejournal.com>
>> Schedule a Meeting <https://fantastical.app/bonelake-Gj2i/meet-with-brad>

>>
>> I am sending this email at a time that suits my work schedule.
>> Please do not feel obliged to reply outside of your working hours.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "PreTeXt development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pretext-dev...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pretext-dev/bb03fc46-9343-4c6b-9e1a-d9094f573c9e%40Spark

>> <https://groups.google.com/d/msgid/pretext-dev/bb03fc46-9343-4c6b-9e1a-d9094f573c9e%40Spark?utm_medium=email&utm_source=footer>
>> .
>>
>

[Brad Miller]

I'm definitely not saying that this would be automated in any way other than flagging particular submissions that instructors might want to investigate.  There are plenty of legitimate things students could do that might trigger this. 

That is also why my plan is to collect the data for a month or maybe even a semester, and then do some analysis to see how well it might work, versus what the false positives might be.

Brad

Bradley Miller, PhD
Professor Emeritus Computer Science
Luther College
Founder, Runestone Academy LTD
Blog: A Reputable Journal
Schedule a Meeting

I am sending this email at a time that suits my work schedule.
Please do not feel obliged to reply outside of your working hours.

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/CACm44N_n9Cu4uO7651YuKdWdNJF9h0iph4QVtgtkaMaQuwroDQ%40mail.gmail.com.

[Charilaos Skiadas]

I’m also on the side of “would love to have such feedback as instructor, but would rather not automate decisions from it”.

What I would add is that I think something that goes off the “submission” that is triggered with the run button would be limited, unless I misunderstood the intent. Personally I would rather see something that simply records diffs on the code that the student is editing, maybe snapshotting it every 30-60 seconds or whatever a suitable time is, and records those snapshots/diffs (whenever there is a change of course) for the instructor’s perusal. Maybe this is happening already? I’m kind of new to it. Would it be too resource-intensive on the storage side? It doesn’t have to phone home on every change, perhaps it can still only send the collection of changes on the run button click, but it would send basically multiple snapshots on each run, if that makes sense.

Charilaos Skiadas
Professor in Mathematics and Computer Science
Hanover College

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/4057d648-0f32-4930-8531-dad782bea252%40Spark.