Encouraging SRS use in the classroom: Bailey 2013
86 views
Skip to first unread message
Gwern Branwen
Sep 11, 2013, 7:34:16 PM9/11/13
to Mnemosyne mailing list
"Internet-Based Spaced Repetition Learning In and Out of the
Classroom: Increasing Independent Student Use", Bailey 2013
http://www.i-repository.net/contents/asia-u/11300207.pdf
> This article presents the results from the second semester of a two-semester project exploring the incorporation a spaced-repetition system (SRS), flashcard computer program, Anki, and its website, into an English as a Foreign Language (EFL) classroom. The overall goal of the project was to teach Japanese university students to be independent users of mobile technology in order to effectively and efficiently learn English. In the first semester, students were introduced to Anki and shown how to use it. At the end of the semester, results showed that student usage of Anki was very low. From an analysis of the first semester of the project (Bailey & Davey, 2011), two possible problem areas were identified: (1) students did not know how to use Anki and therefore could not use it independently; and (2) students did not understand how Anki could help them learn because they had not experienced correct use of the program. For the second semester, solutions were proposed and implemented. Results show that students were able to use Anki independently and correctly, and comparison of usage data from first and second semester showed an overall increase in student usage. Additionally, some results showed a dramatic increase in Anki use, indicating students were using the program outside of the classroom.
>
> ...The horizontal axis represents students, and the vertical axis represents the number of times a student studied a card, a “repetition.” The average number of repetitions per student was 61. Two completed more than 200, and completed zero repetitions. Since this Anki use occurred outside of the classroom, it was not possible to determine where and how the students studied or whether they studied correctly or not.
>
> ...To address the above problems, three solutions were proposed for the second semester.
> Independent Anki use. First, I would teach the students how to create their own Anki user accounts and flashcards from class materials. Second, I would teach them how to access their Anki accounts and study their flashcards. Third, I would teach them how to add new flashcards to their existing deck of flashcards.
> Correct Anki use modeling. As a group, I would regularly and consistently show them how to study with Anki by teacher-modeled use in class, focusing on accurate self-evaluation of answer and providing correct feedback.
>
> ...Materials
> As part of the class content, a new dialog was introduced each week. Each dialog was approximately six to eight lines long with five additional scenarios to practice the dialog structure. From the dialogs, students created flashcards that had a full sentence or question in Japanese on the front and the English translation on the back. From the accompanying scenarios, additional flashcards with key vocabulary were created in the same format. This resulted in approximately 11 to 18 flashcards per week.
>
> ...Independent Anki use: To create more independent users of Anki, the class met in a computer lab once a week. On the first computer lab day, students learned how to access the Anki website on a desktop computer and how to make their own Anki user accounts. In the second week, students learned how to create a new deck and new flashcards from the previous week’s dialog activity. Also, in the second week, during regular class, students learned how to access their Anki website accounts and study their cards on smartphones. In the third week, students learned how to add new cards to their decks. Throughout the rest of the semester, students added new cards on a weekly basis. Students who finished early on those days were encouraged to study their decks. This created an opportunity for additional practice and instruction of correct Anki use.
> Correct Anki use modeling. To increase students' understanding of correct Anki use, correct use was modeled approximately once a week during the semester. The Anki desktop program with a deck of flashcards made from the class materials was displayed on a widescreen television in class. With full class attention, a flashcard with the Japanese language prompt was shown on the screen. A random student was chosen to produce the English equivalent. The student’s answer was written on the board. Other students were called upon to rate the first student’s answer, based on the four options for feedback. The original student was asked for a self-rating. Finally, I gave a rating and explained the rationale behind that rating. This process was repeated for approximately five to ten minutes
> Student-pair Anki use. To increase exposure to correct use of Anki, students participated in pair work once or twice a week. Students logged into their Anki website accounts on a smartphone and passed the device to their partner. This allowed all students to participate, even if they did not own a smartphone. The partner quizzed the first student on their Anki deck by reading the Japanese language prompt. If the first student’s response was correct, the partner pushed the appropriate feedback button. This reduced the problem of students incorrectly rating their own answers. If the response was incorrect, the partner said the correct answer and the first student repeated it. Then, the partner showed the first student the answer on the smartphone and pushed the “again” (incorrect answer) feedback button.
>
> ...As in the earlier chart, the horizontal axis represents students in the same order, and the vertical axis represents the number of times a student studied a card, a “repetition.” From the chart, it is obvious that the usage in the second semester was greater. With a highest usage of 611 repetitions, and a lowest of 92, the average number of repetitions per student was 246 compared to 61 from the first semester, an increase of over 400 percent.
Sounds like a major time-investment to get any usage. A 4x increase
when hardly anyone was using it the first time around...?
--
gwern
http://www.gwern.net
Classroom: Increasing Independent Student Use", Bailey 2013
http://www.i-repository.net/contents/asia-u/11300207.pdf
> This article presents the results from the second semester of a two-semester project exploring the incorporation a spaced-repetition system (SRS), flashcard computer program, Anki, and its website, into an English as a Foreign Language (EFL) classroom. The overall goal of the project was to teach Japanese university students to be independent users of mobile technology in order to effectively and efficiently learn English. In the first semester, students were introduced to Anki and shown how to use it. At the end of the semester, results showed that student usage of Anki was very low. From an analysis of the first semester of the project (Bailey & Davey, 2011), two possible problem areas were identified: (1) students did not know how to use Anki and therefore could not use it independently; and (2) students did not understand how Anki could help them learn because they had not experienced correct use of the program. For the second semester, solutions were proposed and implemented. Results show that students were able to use Anki independently and correctly, and comparison of usage data from first and second semester showed an overall increase in student usage. Additionally, some results showed a dramatic increase in Anki use, indicating students were using the program outside of the classroom.
>
> ...The horizontal axis represents students, and the vertical axis represents the number of times a student studied a card, a “repetition.” The average number of repetitions per student was 61. Two completed more than 200, and completed zero repetitions. Since this Anki use occurred outside of the classroom, it was not possible to determine where and how the students studied or whether they studied correctly or not.
>
> ...To address the above problems, three solutions were proposed for the second semester.
> Independent Anki use. First, I would teach the students how to create their own Anki user accounts and flashcards from class materials. Second, I would teach them how to access their Anki accounts and study their flashcards. Third, I would teach them how to add new flashcards to their existing deck of flashcards.
> Correct Anki use modeling. As a group, I would regularly and consistently show them how to study with Anki by teacher-modeled use in class, focusing on accurate self-evaluation of answer and providing correct feedback.
>
> ...Materials
> As part of the class content, a new dialog was introduced each week. Each dialog was approximately six to eight lines long with five additional scenarios to practice the dialog structure. From the dialogs, students created flashcards that had a full sentence or question in Japanese on the front and the English translation on the back. From the accompanying scenarios, additional flashcards with key vocabulary were created in the same format. This resulted in approximately 11 to 18 flashcards per week.
>
> ...Independent Anki use: To create more independent users of Anki, the class met in a computer lab once a week. On the first computer lab day, students learned how to access the Anki website on a desktop computer and how to make their own Anki user accounts. In the second week, students learned how to create a new deck and new flashcards from the previous week’s dialog activity. Also, in the second week, during regular class, students learned how to access their Anki website accounts and study their cards on smartphones. In the third week, students learned how to add new cards to their decks. Throughout the rest of the semester, students added new cards on a weekly basis. Students who finished early on those days were encouraged to study their decks. This created an opportunity for additional practice and instruction of correct Anki use.
> Correct Anki use modeling. To increase students' understanding of correct Anki use, correct use was modeled approximately once a week during the semester. The Anki desktop program with a deck of flashcards made from the class materials was displayed on a widescreen television in class. With full class attention, a flashcard with the Japanese language prompt was shown on the screen. A random student was chosen to produce the English equivalent. The student’s answer was written on the board. Other students were called upon to rate the first student’s answer, based on the four options for feedback. The original student was asked for a self-rating. Finally, I gave a rating and explained the rationale behind that rating. This process was repeated for approximately five to ten minutes
> Student-pair Anki use. To increase exposure to correct use of Anki, students participated in pair work once or twice a week. Students logged into their Anki website accounts on a smartphone and passed the device to their partner. This allowed all students to participate, even if they did not own a smartphone. The partner quizzed the first student on their Anki deck by reading the Japanese language prompt. If the first student’s response was correct, the partner pushed the appropriate feedback button. This reduced the problem of students incorrectly rating their own answers. If the response was incorrect, the partner said the correct answer and the first student repeated it. Then, the partner showed the first student the answer on the smartphone and pushed the “again” (incorrect answer) feedback button.
>
> ...As in the earlier chart, the horizontal axis represents students in the same order, and the vertical axis represents the number of times a student studied a card, a “repetition.” From the chart, it is obvious that the usage in the second semester was greater. With a highest usage of 611 repetitions, and a lowest of 92, the average number of repetitions per student was 246 compared to 61 from the first semester, an increase of over 400 percent.
Sounds like a major time-investment to get any usage. A 4x increase
when hardly anyone was using it the first time around...?
--
gwern
http://www.gwern.net
Oisín
Sep 12, 2013, 5:26:51 AM9/12/13
to mnemosyne-...@googlegroups.com
On 12 September 2013 00:34, Gwern Branwen <gw...@gwern.net> wrote:
"Internet-Based Spaced Repetition Learning In and Out of the
Classroom: Increasing Independent Student Use", Bailey 2013
http://www.i-repository.net/contents/asia-u/11300207.pdf
It would have been an excellent opportunity to try to examine the effect of studying with Anki on performance on the end-of-semester exam - I'm surprised he didn't do a proper analysis of that, other than this passing remark:
The three top students of first semester showed improvement in the second semester, but nowhere near as dramatically as the top three students of the second semester. This implies that the changes implemented in the second semester were effective in increasing usage for every student, regardless of students’ motivation or other intangible factors.
Apart from that, I'm a little surprised that there was a poor takeup overall of the tool with students. It's hard to say whether the bulk of the resistance stems from usability issues with Anki (I think the new card browser is a little confusing, for example) or just general apathy towards learning an arbitrary new tool and workflow.
Personally, I was hooked on SRS ever since good old Pauker - once you start adding new vocabulary to the deck, you can trust that you'll never "lose" a word... you might forget it temporarily, but the SRS will catch it and automatically prioritise it. Less worry, waaaaay better efficiency...
Oisín
--
gwern
http://www.gwern.net
--
You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mnemosyne-proj-u...@googlegroups.com.
To post to this group, send email to mnemosyne-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mnemosyne-proj-users/CAMwO0gzmq_vU6pS6MMCBmGOyuyV4zjZy7RvMDdFyw_Hwgd_45A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
mattli...@gmail.com
Sep 12, 2013, 2:02:05 PM9/12/13
to mnemosyne-...@googlegroups.com, gw...@gwern.net
> Sounds like a major time-investment to get any usage. A 4x increase
>
> when hardly anyone was using it the first time around...?
Agreed!
It will be very interesting to think about how Mnemosyne use might be encouraged among learners, as opposed to simply being a tool for those who already are inclined to use it.
In just a few weeks, I'm launching a 6-month study examining how Mnemosyne is used, how the algorithm performs, and what its effects are on knowledge mastery with a reasonably large sample of US medical students.
We're throwing everything at them to encourage usage, these students have a high-stakes licensure exam to study for, and they're all highly motivated learners in the first place - so I'm hopeful that many of them will use it avidly. However, I'm sure some won't!
Gwern Branwen
Sep 12, 2013, 2:20:19 PM9/12/13
to Mnemosyne mailing list
On Thu, Sep 12, 2013 at 2:02 PM, <mattli...@gmail.com> wrote:
> It will be very interesting to think about how Mnemosyne use might be encouraged among learners, as opposed to simply being a tool for those who already are inclined to use it.
One strategy I find interesting is to give up on the customized
> It will be very interesting to think about how Mnemosyne use might be encouraged among learners, as opposed to simply being a tool for those who already are inclined to use it.
per-user timing, and instead use a generic expanding schedule, putting
the questions into something like a weekly quiz. The students don't
have to do anything they wouldn't be doing anyway, so compliance is
automatically 100%...
> In just a few weeks, I'm launching a 6-month study examining how Mnemosyne is used, how the algorithm performs, and what its effects are on knowledge mastery with a reasonably large sample of US medical students.
> We're throwing everything at them to encourage usage, these students have a high-stakes licensure exam to study for, and they're all highly motivated learners in the first place - so I'm hopeful that many of them will use it avidly. However, I'm sure some won't!
all the studies of medical students & doctors already done by Kerfoot
and others.
--
gwern
http://www.gwern.net/Spaced%20repetition
mattli...@gmail.com
Sep 12, 2013, 4:29:14 PM9/12/13
to mnemosyne-...@googlegroups.com, gw...@gwern.net
> One strategy I find interesting is to give up on the customized
>
> per-user timing, and instead use a generic expanding schedule, putting
>
> the questions into something like a weekly quiz. The students don't
>
> have to do anything they wouldn't be doing anyway, so compliance is
>
> automatically 100%...
Certainly practical! It'd be great if students were continuing to retain useful information from courses *after* the course concludes, though, especially since the time cost to maintain mastery continues to decrease with ever-expanding intervals.
In that sense, it seems worthwhile to encourage students to begin using a system that "lives on" after a semester-long course.
All the same, I certainly think that spaced formative testing should be a part of just about any course I can imagine.
> Sounds like an interesting study, although maybe a bit redundant with
>
> all the studies of medical students & doctors already done by Kerfoot
>
> and others.
I'm familiar with that line of research.
A few key differences are that the Kerfoot et al. program uses MCQs - very convenient to score automatically, but known to be significantly less effective than requiring students to *recall* information the way Mnemosyne does (vice simply *recognize* the right answer out of a MCQ menu).
They also use a very coarse algorithm - "if right once, test again in 6 weeks; if right twice, delete the card; if wrong, show again in 3 weeks" (or some minor variant of that). That's probably valuable for certain purposes but it is quite different than what the Mnemosyne algorithm is trying to do. It seems misguided to assume that an item is mastered for eternity if it's answered correctly twice, or that 3 weeks is a good length of time to wait to test something that the learner got wrong.
All interesting empirical questions!
-Matt
mattli...@gmail.com
Sep 12, 2013, 4:42:35 PM9/12/13
to mnemosyne-...@googlegroups.com, gw...@gwern.net
> Sounds like an interesting study, although maybe a bit redundant with
>
> all the studies of medical students & doctors already done by Kerfoot
>
> and others.
Oh, also: The Kerfoot work has gone proprietary (QStream) - which is fine of course! All the same, I'm not expecting that they will be transparent about algorithm refinements, that they will share data, etc.
I'm glad people are selling spaced education products, but I'm also glad there is a vibrant space for doing research that isn't commercially influenced.
-Matt
Reply all
Reply to author
Forward
0 new messages