Re: [mnemosyne-proj-users] The "Correct" Process? OR How to Play Well with an Algorithm

[George Wade]

The SuperMemo site had wide ranging papers that go on forever about your thoughts and more. However: try changing to a system logic that is satisfied with taking useful results as a more educated substitute for being right or wrong.

On 29 Nov 2012, at 15:25, berg...@umn.edu wrote:

> Hello everyone,
>
> So I have been using Mnemosyne for about 1 1/2 months now. I thought it would be good to help with all the homework and studying I get here in college. I have a question for you and your dog that uses Mnemosyne:
>
> Is anyone afraid that they aren't playing nice with the algorithm?
>
> Essentially this: I want to help the algorithm help me. I understand there is a very intelligent algorithm at work here, and I am curious to know if there's a way of using the program that's more right/wrong?
>
> You could try to use it to make pancakes, but that's obviously not what I'm saying.
>
> Let me give you an example instead. I open up Mnemosyne, and I go through my daily repetitions. Along the way I encounter a card that I got mostly right, but not entirely right. Is that a 2 or a 4? or neither?
> After I finish my repetitions, I begin to start memorizing new cards. This is where I start to get paranoid.
> See, the way I study is to go over the 15 or so cards that I repeatedly use, over and over, grading them 1 each time, until I feel comfortable that they are properly ingrained in my head.

Critical logic WILL set you in the direction of being paranoid. "I encounter a card that I got mostly right but not entirely." Change that question from "Is that a 2 or a 4 or neither ?" to "How can I grade that for the most useful timing in subsequent repetitions ?" Where "How can I grade that ?" means for me; not for the world of Mnemosyne and SuperMemo.


> Am I screwing with the system when I do this? Am I supposed to mark them a 2 as soon as I can recite the answer?

The average user is supposed to, yes, and learn to live with the very efficient results.

> If anyone involved/knowledgeable in the algorithm could reply, I would be much in gratitude.

Good luck,

George

[Peter Bienstman]

This is also a useful thread:

https://groups.google.com/forum/?fromgroups=#!topic/mnemosyne-proj-users/uDprmAH7vJs

Citeren berg...@umn.edu:

> Hello everyone,
>
> So I have been using Mnemosyne for about 1 1/2 months now. I thought
> it would be good to help with all the homework and studying I get
> here in college. I have a question for you and your dog that uses
> Mnemosyne:
>
> Is anyone afraid that they aren't playing nice with the algorithm?
>
> Essentially this: I want to help the algorithm help me. I understand
> there is a very intelligent algorithm at work here, and I am curious
> to know if there's a way of using the program that's more right/wrong?
>
> You could try to use it to make pancakes, but that's obviously not
> what I'm saying.
>
> Let me give you an example instead. I open up Mnemosyne, and I go
> through my daily repetitions. Along the way I encounter a card that
> I got mostly right, but not entirely right. Is that a 2 or a 4? or
> neither?
> After I finish my repetitions, I begin to start memorizing new
> cards. This is where I start to get paranoid.
> See, the way I study is to go over the 15 or so cards that I
> repeatedly use, over and over, grading them 1 each time, until I
> feel comfortable that they are properly ingrained in my head.
>

> Am I screwing with the system when I do this? Am I supposed to mark
> them a 2 as soon as I can recite the answer?
>

> If anyone involved/knowledgeable in the algorithm could reply, I
> would be much in gratitude.
>

> --
> You received this message because you are subscribed to the Google
> Groups "mnemosyne-proj-users" group.
> To post to this group, send email to mnemosyne-...@googlegroups.com.
> To unsubscribe from this group, send email to
> mnemosyne-proj-u...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mnemosyne-proj-users/-/NhesNPExZ4QJ.
> For more options, visit https://groups.google.com/groups/opt_out.

[Gnome]

I'm not into the algorithm or the memory research, but just by using mnemosyne you'll after a while will get a feeling on how to grade. Here is my 5 cents:

1. Don't be afraid to use high grades when you know it well,, even for the initial grading. If you constantly grade cards low "just in case", you will likely get a big and boring workload later. If you grade a card too high, you'll easily catch up again in my experience if you forget the card.

2. If you forget parts of a card, it is likely that you will not remember it next time if you grade it 2 and greater. So I almost always grade 2 or less if the card is not 100% right and  the information is important to remember.

3. If there are parts of a card that you keep forgetting, make a new card which hold that information and delete it in the old one.

[Scott Youngman]

When you grade a card, you are evaluating how well you remember the answer right then, NOT trying to guess what the algorithm will or should do with it in the future. 

 

Don't be concerned about giving a card exactly the right grade; if you grade it too high or low, the difference in interval to the next time it is shown will be only a few days anyway, so not a crucial difference. When you see it again, you will then find how well you really know it, and can adjust the grade accordingly. Over time, and if you grade according to your current memory, the algorithm will continue to adjust so it shows each card at just the right interval to prevent forgetting the answer. You may want to Google "spaced repetition" or SRS to understand this better.

 

Here are some ways to think of the grades (I have compiled these from various sources). Note that grades 0-1 are for answers you really can't remember, while grades 2 and above are answers you can remember, even if not clearly. So 1 --> 2 is a dividing point indicating a transition from short to long term memory.

0   A is entirely unknown.

1   A is vague, fuzzy; you can't remember the A.
-------
2   Probably about 2 days until you forget the A.

     You "almost" got it. When you saw the A, you thought "I knew it!"

3   Significant effort to remember the A.
     You got the A, but it was hard and maybe your understanding was a bit off.

     Last interval was too long.

 

4   Some effort to remember the A.
     You got the A, but you found it somewhat challenging, not quite easy.

     Last interval was about right.
     Over time, this should be the most common grade.

 

5   No effort to remember the A; it is easy.

     Last interval was too short.

 

== Effect of grades ==
0-1   Keep asking until I can answer with 2+.
        These cards are not scheduled ("not memorized").
  -------
2   Move card --> into schedule ("memorized").
3   Next interval will be shorter.
4   Next interval will be equal.
5   Next interval will be longer.

[Frank Berg]

On Dec 1, 2012, at 5:51 PM, Scott Youngman <syo...@gmail.com> wrote:

Ah, okay this all makes sense. Thanks!

[Gnome]

2   Move card --> into schedule ("memorized").
3   Next interval will be shorter.
4   Next interval will be equal.
5   Next interval will be longer.

This is true for the grading of a card that is not memorized. But as I have understod it, the interval keeps getting longer when you grade 2 or more when the card is scheudeled, the only way to make the interval shorter is to grade 0 or 1.

[Oisín]

Yes, the interval always increases with passing grades, but by different factors. Lower grades can decrease the "easiness" factor though, if it's greater than 1.3 - this is another factor in deriving future intervals.

From the (old?) website:
"If you select grade 4, the easiness factor is unchanged and the card is scheduled for future repetition based on the previous record of repetitions.

If you select grade 5, the easiness factor is increased and the card is scheduled for future repetition based on the previous record of repetitions.

If you select grade 2 or 3, the easiness factor is decreased and the card is scheduled for future repetition based on the previous record of repetitions.

If you select grade 0 or 1, the easiness factor is unchanged and the card is put back in the stack of unmemorised cards."

BTW regarding the OP's questions on the subject of generally playing nice with the algorithm, I'd suggest not to sweat it too much. It's probably a better use of your time to assume that you'll remember a card tomorrow and risk occasional failures, than to repeatedly grade them 1 and cram them.

A good place to find out lots of well-referenced information about spaced repetition is Gwern's page on the subject: http://www.gwern.net/Spaced%20repetition

Oisín

--
You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group.
To post to this group, send email to mnemosyne-...@googlegroups.com.
To unsubscribe from this group, send email to mnemosyne-proj-u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/mnemosyne-proj-users/-/iCqrCAk2y3YJ.

[Scott Youngman]

Oisin wrote:

"Yes, the interval always increases with passing grades, but by different factors. Lower grades can decrease the "easiness" factor though, if it's greater than 1.3 - this is another factor in deriving future intervals."

 

My question: So does changing the easiness factor + considering previous review history result in grade 2 showing a card more often than in recent repetitions? I can't imagine that a grade 2 (essentially meaning "I don't know this very well") would actually result in me seeing it less often than before.

[Michael Campbell]

No, the next interval is longer than the previous, but the DEGREE of "longer" is smaller with a 2 than 3 (or 4 (or 5))...

--

You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group.
To post to this group, send email to mnemosyne-...@googlegroups.com.
To unsubscribe from this group, send email to mnemosyne-proj-u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/mnemosyne-proj-users/-/I4GBSGZl768J.

[pharmtech]

On Nov 29, 4:25 pm, berg2...@umn.edu wrote:
> See, the way I study is to go over the 15 or so cards that I repeatedly use, over and over, grading them 1 each time, until I feel comfortable that they are properly ingrained in my head.

I've encountered the same dilemma. On the one hand, the algorithm is
the most efficient way to memorize something over a long period of
time, where you can afford to forget, and recover the memory over
periods of days or weeks. But, in the real world, people need to
acquire bullet proof memory for an exam in 6 weeks. Or, they may
memorize numbers which aren't as conducive to helps like mnemonics.

I think SRS, although not as efficient as it could be in a perfect
world, is valuable for learning in less-than-ideal circumstances. You
still get the benefit of prioritization/ordering so you spend your
time on harder material (unlike flat "cramming" of all the material
over and over).

[Michael Campbell]

Yes, I have struggled with this over time as well.  In general, I don't *need* to know something by a certain time, and the default SRS works fine for that, but sometimes I'm using it to try and optimize a time-boxed exercise.  For these times the (old) cramming mode didn't work, because it was flat, as you mention.  I think now with the new enhancements to cramming that I helped to propose some of that can be taken care of; it's like a mini-scheduler.

That said, it was about the same as just setting up a card set of what you need to know, and "learning ahead of schedule" with that one set, caring more about order and frequency than absolute day.

--
You received this message because you are subscribed to the Google Groups "mnemosyne-proj-users" group.
To post to this group, send email to mnemosyne-...@googlegroups.com.
To unsubscribe from this group, send email to mnemosyne-proj-u...@googlegroups.com.

[pharmtech]

On Dec 5, 5:59 am, Michael Campbell <michael.campb...@unixgeek.com>
wrote:

>  I think now with the new enhancements to
> cramming that I helped to propose some of that can be taken care of; it's
> like a mini-scheduler.

Are you referring to Anki's new "study sessions" such as "Review
Ahead?" In some ways I like this. For example, configuring the study
session to not update the card's schedules. Compared to v1.2, that's
really nice. 1.2 would update the schedule, sending due dates further
and further into the future, making "Review Early" not too useful for
"cramming." (I know it was never intended for that. But, its title led
everyone to reach for that as a way to study more frequently than
daily.).

But, I don't know yet how useful it really is. I disable schedule
update, and I see the same cards in the same order over and over
again. It's not really feeding back my scores, reprioritizing and
reordering cards for the next cram session. I can change the ordering
to "random," but then I'm not benefiting from SRS's prioritization of
cards (focusing my energy on cards that need more work).

To me, that seems like the persistent frustration I feel. There's
always a large gap between "perfect world" SRS scheduling and
pragmatic cramming. In 1.2, if you managed to discover the per-day
schedule switch and micro-interval assignment (and, Adam's "Forgetting
Index" plugin to retard the growth of "Young" card intervals) it was a
fairly seamless *continuum* from learning and longer-term retention
testing. If you wanted to review early, start 1.2 and review whatever
was due at that moment. If not enough was coming due, use the "Cheat"
plugin to reschedule a block of cards (the next 3 days) to be due
between now and 3 days.

The "learning mode" and "steps" seems ok. But, it doesn't seem to
dynamically adjust schedules like applying SRS (time, ease factor) to
micro intervals. It seems to create a hard either/or, not a natural
flow between learning versus mastery retention. And, that dividing
line between the two seems arbitrary (steps you pull from thin air
rather than a sliding scale of scheduling).

IMO, 1.2 was almost perfect in this regard. All it needed was to 1)
factor actual "FI" performance into the user's desired "FI" goal, and
2) make the rescheduling tool more of a "window" dragging function.
For example, specify the left and right boundaries of a timeframe and
drag the left boundary nearer to now, causing the affected cards to be
rescheduled *relative* to each other, maintaining their order. Perhaps
a center handle to nudge everything in the window more near-term.

(Rescheduling tools seem to define a window, but "sprinkle" the cards
within that window, disregarding the order/priority which has
developed between cards. For example, I might select a group of cards
and say "reschedule between today and tomorrow." A card due in 3 days
might "sprinkle" into today, and a card due today might "sprinkle"
into tomorrow. I think it should be more like condensing or expanding
the existing schedule into a new timeframe. Preserving order/spacing.
Factoring a set of cards into a new timeframe, not randomly seeding
them into new positions.).

To me, that would make "Review ahead" more useful for cramming. You
could drag the schedule back to near term and review ahead, gaining
the benefit of prior reviews reprioritizing/ordering cards without
them moving further into the future.

[Oisín]

On 5 December 2012 06:37, pharmtech <azfu...@gmail.com> wrote:

On the one hand, the algorithm is
the most efficient way to memorize something over a long period of
time, where you can afford to forget, and recover the memory over
periods of days or weeks. But, in the real world, people need to
acquire bullet proof memory for an exam in 6 weeks.

I'd still expect spaced tests to produce significantly better results over a 6 week period, and of course would allow a much greater amount of information to be learned since it's far more time-efficient than cramming.

The question of what time period is suited towards cramming rather than SRS is certainly interesting, though - i.e. if there's a test next week, is it better to cram now or SRS the material?
Most of the studies I've heard about have shown that cramming is inferior in almost every case, although students who cram tend to spend more hours studying, which is of course a useful thing in itself.

Oisín

[pharmtech]

On Dec 6, 3:21 am, Oisín <denpasho...@gmail.com> wrote:
> I'd still expect spaced tests to produce significantly better results over
> a 6 week period, and of course would allow a much greater amount of
> information to be learned since it's far more time-efficient than cramming.

If you mean longer-term, durable memory, I agree. But, if you mean a
larger vocabulary in 6 weeks, I don't think I'd agree with that. Not
if we're defining SRS by its ideal of once per day, 20 minute session?

From my own experience, I can acquire much more (shorter-term)
information through short repetitive practices. Once per day doesn't
work for me, for that purpose. But, it does work very well (and
efficient) for longer-term retention.

I also look at "time-efficient" a bit differently. You touched on it
when you said that, if someone crams, at least they're studying. To
me, SRS is efficient in the sense of "all things being equal." If we
lived in a perfect world where it was a zero-sum choice between
studying or using my time for something equally rewarding, then
cramming would be wasteful.

But, in the real world, we have changing priorities. We might have an
urgent requirement to study for an exam, and the alternative to not
studying is watching tv. We don't have an equally valuable activity to
spend the time on. It's essentially a perishable commodity. If spent
studying, at least it didn't go to as much waste as watching tv.

I know it's more complicated than that. Our brain has a gestation
period. Relaxing, watching tv is part of the process of letting the
brain do what it does. But, I'm thinking of goals that require an
accomplishment faster than we gestate memories.

To me, that gets to the constant friction between SRS ideology and the
real world. There's no denying the brain's delivery of a memory can't
be rushed, just like a real pregnancy can't be. But, things like
school classes expect us to perform faster than our brain. We only
have 4-8 weeks to master a subject which, to truly master, may take a
year.

For me (with Anki 1.2) cramming worked very well when it employed SRS-
style ordering and spacing. I might review 20 times per day, with
cards coming due in 10 minutes to 10 hours. My performance fed back
into the scheduling, influencing relative order and spacing between
cards.

That's definitely not SRS (corresponding to the forgetting curve,
which is measured in days). But, it was a million times more effective
than typical "cramming" the same material over and over again. It
leveraged the SRS's ability to focus me time on what I needed to see.
My time is finite, and I could spend as much as possible reviewing
only the cards I was having the most trouble acquiring and retaining
(short term).

It seems like it's already accepted that this is a valid tactic. For
example, Anki 2 now uses a more isolated "learning mode" with sub-day
"steps" which transitions (abruptly, IMO) to traditional SRS day-sized
intervals.

It seems like this topic is often complicated by definitions.
Efficient based upon circumstances (alternatives to spending time),
and timeframe (short-term performance or long-term mastery).

It's an interesting topic because I think everyone's describing much
the same thing, just from different perspectives.

[Oisín]

On 6 December 2012 20:26, pharmtech <azfu...@gmail.com> wrote:

For me (with Anki 1.2) cramming worked very well when it employed SRS-
style ordering and spacing. I might review 20 times per day, with
cards coming due in 10 minutes to 10 hours. My performance fed back
into the scheduling, influencing relative order and spacing between
cards.

That's definitely not SRS (corresponding to the forgetting curve,
which is measured in days). But, it was a million times more effective
than typical "cramming" the same material over and over again. It
leveraged the SRS's ability to focus me time on what I needed to see.
My time is finite, and I could spend as much as possible reviewing
only the cards I was having the most trouble acquiring and retaining
(short term).

Yes - an SRS program's ability to schedule intelligently within cramming sessions, even without the actual SRS part, is certainly an improvement over more traditional cramming.

Perhaps even more beneficial is that it forces you to perform active recall, as opposed to the passive recall that might be more common in "manual" cramming. It's very well established that active recall is massively superior to passive learning - it's better to even give people a test without any feedback whatsoever, than to let them re-read their notes!

Of course we probably all agree that spaced repetition is immensely better than massed repetition (cramming) in general - at least over "some" timeframe which is probably more than a few days.

What I'd really like to see is some scientific study that asks the question: what minimum timeframe is needed for spaced repetition to beat massed repetition?
My intuition is that it's pretty short - maybe only a few weeks. If all other things are equal (i.e. test subjects spend exactly the same length of time per day either using an SRS in cramming mode, or in spaced repetition mode), not only do you benefit from the inherent superiority in spaced repetition in terms of long term memory formation, but also from the more pragmatic fact that you have more time to spend on new material, since you're not wasting time by repeating the same "known" facts over and over.

But of course, our hypotheses won't help much... empirical testing is needed :)

[pharmtech]

On Dec 6, 3:16 pm, Oisín <denpasho...@gmail.com> wrote:
> My intuition is that it's pretty short - maybe only a few weeks. If all
> other things are equal (i.e. test subjects spend exactly the same length of
> time per day either using an SRS in cramming mode, or in spaced repetition
> mode),

The problem I've had with spending the same amount of time with SRS
(as cramming) is that I had to add too much new material. If I start
with 20 new cards, as soon as I score each "good" they aren't due
until tomorrow. I can add more new cards, that leads to even more
overload of newly created memories.

I think confidence plays a role in the larger picture. You may be
right that starting with a lot of new material (to stay busy) will
eventually lead to enough failures, retries, etc. that an individual
will spend as much time on SRS as they would cramming. But, I like to
review new material more frequently and gain stronger (immediate)
associations. Without that, if I get too much new material, it feels
bewildering. I feel like there's only so much new stuff I can learn at
once. If it's interlaced with some things I know, it creates a more
positive environment.

You mentioned the desire to see conclusive research proving either
position. I wonder if anyone has determined whether the "forgetting
curve" goes to sub-day spacing. It sounds like Ebbinghaus worked in
units of days and that's just the unit of measurement which has
developed with SRS. I'm sure the brain begins to distance itself from
information within minutes. I doubt there's anything magic about the
"1 day" boundary. (Except for sleeping, when the brain processes
things.). I know if I witness an accident, I'm more likely to remember
key details in the first few minutes compared to later that evening.