Hello Digital Education Group!
Based on the majority of votes, we are going to hold our next Reading Group meeting on the following date:
Wednesday, February 8, 2023, from 12:00 PM to 1:00 PM ET
Zoom details:
Meeting ID: 873 3155 7745
Passcode: digitaledu
We are going to discuss solving programming assignments in CS courses using AI assistance (
Copilot and
ChatGPT).
We will start our discussion using the following paper:
Title: Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language
Abstract: GitHub Copilot is an artificial intelligence model for automatically generating source code from natural language problem descriptions.
Since June 2022, Copilot has officially been available for free to all
students as a plug-in to development environments like Visual
Studio Code. Prior work exploring OpenAI Codex, the underlying
model that powers Copilot, has shown it performs well on typical
CS1 problems thus raising concerns about the impact it will have
on how introductory programming courses are taught. However,
little is known about the types of problems for which Copilot does
not perform well, or about the natural language interactions that a
student might have with Copilot when resolving errors. We explore
these questions by evaluating the performance of Copilot on a
publicly available dataset of 166 programming problems. We find
that it successfully solves around half of these problems on its very
first attempt, and that it solves 60% of the remaining problems
using only natural language changes to the problem description.
We argue that this type of prompt engineering, which we believe
will become a standard interaction between human and Copilot
when it initially fails, is a potentially useful learning activity that
promotes computational thinking skills, and is likely to change the
nature of code writing skill development.
Hoping to see you all on Wednesday.