scheduler feedback needed
Peter Bienstman
Right now the long term scheduler avoids scheduling related cards (e.g. vice
versa) on the same day.
The near term scheduler (i.e. when you just use grades 0 and 1) works
differently: as soon as you memorised the first card of a vice versa pair, the
next card of the pair is scheduled right away.
Are you happy with this behaviour, or would you prefer that the near term
scheduler also tries to postpone seeing the second card to the next day?
Peter
querido
Ben
--
Ben
Laurent
As mentioned in an other message some time ago, I do not see the point
in showing the opposite card right away, as it does not help me
memorize it any better (seeing the answer to the first card of the
pair already helped me remember it equally both ways), and I find it
confusing as to how I should grade the 2nd card (even though I know
the answer, because I just saw it from the 1st card, I find myself
answering 0 or 1 again because I only know it from seeing it just
before, not from memory).
I hope my explanation makes sense...
David A. Harding
> as soon as you memorise the first card of a vice versa pair, the next
> card of the pair is scheduled right away.
When I memorize a card for the first time, can you send all the other
closely related unmemorized cards to the end of the learning card pool
*for this session only*? For example, say I have four cards entered in
the following order:
1. vice
2. versa
3. three
4. four
When I learn "vice", Mnemosyne re-orders the remaining queue as follows:
1. three
2. four
3. versa
But if I close Mnemosyne, ending the current session, the next time I
start Mnemosyne it reverts to showing unmemorized cards in the order of
addition:
1. versa
2. three
3. four
I also want to add that I dislike the way default Anki hides unmemorized
cards after you learn 20 cards in one day. I prefer that Mnemosyne never
hide any cards from me unless I ask it to hide them.
Thanks,
-Dave
--
David A. Harding Website: http://dtrt.org/
1 (609) 997-0765 Email: da...@dtrt.org
Jabber/XMPP: dhar...@jabber.org
Damien Elmes
> cards after you learn 20 cards in one day. I prefer that Mnemosyne never
> hide any cards from me unless I ask it to hide them.
It stops after 20 cards by default because many people new to SRSes
are excited about their new found study tool, learn many new cards in
the first day or two, and then get bogged down with a mountain of
reviews and give up. You have the option of increasing the daily limit
from the first screen you're presented with, and you also have the
option of clicking 'learn more' if you decide you want to keep
studying.
David A. Harding
> many people new to SRSes [...] get bogged down with a mountain of
> reviews and give up.
I agree that many new SRS users give up and that a mountain of reviews
dispirits even long-time users, but do you have any proof that many new
users give up *because* they had a mountain of reviews?
Damien Elmes
few years of supporting Anki users and my students. Take it as you
will.
As for how useful the default is, I gathered some statistics from a
small portion of the users of AnkiOnline.
count | maxNew
18 0
1 1
1 2
6 3
2 4
17 5
2 6
1 7
3 8
38 10
1 12
2 14
2 15
2320 20
2 22
17 25
1 28
43 30
1 32
1 33
6 35
1 36
39 40
1 43
2 45
38 50
1 52
1 55
10 60
1 62
2 70
1 75
1 80
1 90
1 99
51 100
1 108
1 110
1 115
2 116
1 130
5 150
1 156
1 160
20 200
2 214
2 250
7 300
2 400
20 500
2 600
2 700
1 800
1 842
3 999
12 1000
1 1648
4 2000
2 3000
1 4000
2 5000
2 9999
4 10000
2 20000
6 99999
1 999999
1 9999999
1 999999999
Peter Bienstman
> On Sun, Jun 28, 2009 at 01:11:04PM +0200, Peter Bienstman wrote:
> > as soon as you memorise the first card of a vice versa pair, the next
> > card of the pair is scheduled right away.
>
> When I memorize a card for the first time, can you send all the other
> closely related unmemorized cards to the end of the learning card pool
> *for this session only*? For example, say I have four cards entered in
> the following order:
>
> 1. vice
> 2. versa
> 3. three
> 4. four
>
> When I learn "vice", Mnemosyne re-orders the remaining queue as follows:
>
> 1. three
> 2. four
> 3. versa
>
> But if I close Mnemosyne, ending the current session, the next time I
> start Mnemosyne it reverts to showing unmemorized cards in the order of
> addition:
>
> 1. versa
> 2. three
> 3. four
OK, and how exactly do you want to change that behaviour? It looks to me like
what you are describing is exactly "send all the other closely related
unmemorized cards to the end of the learning card pool *for this session
only*".
Would you be happy if 'versa' was only shown for the first time the next day?
Also note that as soon as you start giving a grade to a card, it will become
randomly intermixed with the other cards.
Peter
David A. Harding
> [In a sample of 2750 users, 2320 (84%) have the default setting.]
Do you include new and abandoned accounts in your statistics?
Thanks,
David A. Harding
| On Sunday 28 June 2009 07:05:10 pm David A. Harding wrote:
| | When I memorize a card for the first time, can you send all the other
| | closely related unmemorized cards to the end of the learning card pool
| | *for this session only*? For example, say I have four cards entered in
| | the following order:
|
| other closely related unmemorized cards to the end of the learning
| card pool *for this session only*".
As indicated in the quote above, the example was an example of the
desired behavior. I'm glad it looked exactly like my one-sentence
description. :-)
> Would you be happy if 'versa' was only shown for the first time the
> next day?
I don't want Mnemosyne to hide any cards from me unless I tell it to
hide them. If I run out of other unmemorized cards, I want to see
"versa". If you do make Mnemosyne hide some cards by default, I'd
appreciate if you add an option to show them.
Peter Bienstman
> On Mon, Jun 29, 2009 at 09:29:32AM +0200, Peter Bienstman wrote:
> | On Sunday 28 June 2009 07:05:10 pm David A. Harding wrote:
> | | When I memorize a card for the first time, can you send all the other
> | | closely related unmemorized cards to the end of the learning card pool
> | | *for this session only*? For example, say I have four cards entered in
> | | the following order:
> |
> | It looks to me like what you are describing is exactly "send all the
> | other closely related unmemorized cards to the end of the learning
> | card pool *for this session only*".
>
> As indicated in the quote above, the example was an example of the
> desired behavior. I'm glad it looked exactly like my one-sentence
> description. :-)
Strangely enough, it is also the current Mnemosyne behaviour :-) But as I
mentioned before, you will never be able to observe this cleanly, as grade 0
and 1 cards are always randomised.
> > Would you be happy if 'versa' was only shown for the first time the
> > next day?
>
> I don't want Mnemosyne to hide any cards from me unless I tell it to
> hide them. If I run out of other unmemorized cards, I want to see
> "versa".
That would indeed be the plan: if the only cards that are unmemorised are
actually the related cards that were postponed to tomorrow, show them now.
Peter
Peter Bienstman
> On Mon, Jun 29, 2009 at 04:03:09PM +0900, Damien Elmes wrote:
> > [In a sample of 2750 users, 2320 (84%) have the default setting.]
>
> Do you include new and abandoned accounts in your statistics?
Also, many people don't bother to change the default. Which brings me to my
next question: which values of 'grade 0 cards to hold in your hand' do people
use?
Peter
David A. Harding
> Strangely enough, it is also the current Mnemosyne behaviour :-)
Whoops! I knew it sounded like a good idea. :)
> That would indeed be the plan: if the only cards that are unmemorised
> are actually the related cards that were postponed to tomorrow, show
> them now.
Sounds awesome. That has my vote.
Thanks, Peter.
David A. Harding
> which values of 'grade 0 cards to hold in your hand' do people use?
I sometimes adjust it to suit the subject, but I use 15 the most often.
Damien Elmes
upper limit on the number of abandoned accounts. I made no attempt at
accounting for fresh accounts, as it would take more time.
Meishu
Same as David.
On Jun 29, 5:12 pm, "David A. Harding" <d...@dtrt.org> wrote:
> On Mon, Jun 29, 2009 at 11:02:57AM +0200, Peter Bienstman wrote:
> > which values of 'grade 0 cards to hold in your hand' do people use?
>
> I sometimes adjust it to suit the subject, but I use 15 the most often.
>
> -Dave
> --
> David A. Harding Website: http://dtrt.org/
> Jabber/XMPP: dhard...@jabber.org
Francisco Fiuza Jr
Oisin Mac Fhearai
On 29 Jun 2009, at 07:36, "David A. Harding" <da...@dtrt.org> wrote:
>
> On Mon, Jun 29, 2009 at 02:12:43PM +0900, Damien Elmes wrote:
>> many people new to SRSes [...] get bogged down with a mountain of
>> reviews and give up.
>
> I agree that many new SRS users give up and that a mountain of reviews
> dispirits even long-time users, but do you have any proof that many
> new
> users give up *because* they had a mountain of reviews?
>
only doing 50-100 of the ~200 reviews scheduled each day. When it got
to about 1500 due cards, I finally lost any remaining motivation and
stopped studying. That was about 4 months ago, and I've been using
srses for over 4 years.
So I would say that there is a strong chance that others, especially
newbies, have fallen into the same trap.
Even at 12 new cards per day, the reviews creep up to well over 100 a
day within a few months, and if answering a card takes 20 seconds (eg
scribbling Chinese chars on a tablet), you're talking 30-60 min
sessions daily. If you take a week off, that's four hours of backlog.
I'd recommend a clear limit for new cards daily as Anki does, tweaked
by the user conservatively, so they don't get proportionally bogged
down 6 months later and quit. When I return, I'll probably only take
on 5 or 6 new cards a day to avoid burnout again.
Peter Bienstman
> I can't speak for others, but I fell behind over a couple of weeks,
> only doing 50-100 of the ~200 reviews scheduled each day. When it got
> to about 1500 due cards, I finally lost any remaining motivation and
> stopped studying. That was about 4 months ago, and I've been using
> srses for over 4 years.
> So I would say that there is a strong chance that others, especially
> newbies, have fallen into the same trap.
>
> Even at 12 new cards per day, the reviews creep up to well over 100 a
> day within a few months, and if answering a card takes 20 seconds (eg
> scribbling Chinese chars on a tablet), you're talking 30-60 min
> sessions daily. If you take a week off, that's four hours of backlog.
>
> I'd recommend a clear limit for new cards daily as Anki does, tweaked
> by the user conservatively, so they don't get proportionally bogged
> down 6 months later and quit. When I return, I'll probably only take
> on 5 or 6 new cards a day to avoid burnout again.
Interesting observation, thanks!
It's easy to add a warning + explanation when you reach e.g. 10 new learned
cards. I'd prefer this more gentle approach as opposed to a hard limit where
you forbid people to go on.
On a more personal note, I've been using (the predecessor of) Mnemosyne since
2003, learning roughly 5 cards a day. I'm now at 175 scheduled cards a day on
average, and 8500 cards in my database. There is still plenty of stuff I want
to learn, and if I keep up what I think is this steady, gently pace, I could
be at 350 reps daily in 5 more years...
This very long term aspect is definitely something that needs thinking about,
either by making a more thorough analysis of the logs and tweaking the
algorithm, or by pruning the cards in my database.
Peter
Ben
show some possible limits on what memory can achieve in the long term.
But having to review 175 cards a day seems rough to me. I guess it's
all about how difficult the things are that you memorize.
Here's the way I was looking at it, as summarized by Sherlock Holmes
in "A Study in Scarlet":
"You see," he explained, "I consider that a man's brain originally
is like a little empty attic, and you have to stock it with such
furniture as you choose. A fool takes in all the lumber of every
sort that he comes across, so that the knowledge which might be
useful to him gets crowded out, or at best is jumbled up with a lot
of other things so that he has a difficulty in laying his hands
upon it. Now the skilful workman is very careful indeed as to what
he takes into his brain-attic. He will have nothing but the tools
which may help him in doing his work, but of these he has a large
assortment, and all in the most perfect order. It is a mistake to
think that that little room has elastic walls and can distend to
any extent. Depend upon it there comes a time when for every
addition of knowledge you forget something that you knew before. It
is of the highest importance, therefore, not to have useless facts
elbowing out the useful ones."
At first I thought Holmes was being too pessimistic, but there is a
sense in which he's right---you can only remember so many "things",
even with spaced repetition. After that you spend too much time
reviewing cards every day.
Now, if I only wanted to review, say, 50 cards every day, then I was
hoping that I could still have 10K cards in my deck. This corresponds
to a 0.5% (=50/10K) "review rate". But Peter has a review rate of
175/8500 = 2%, which seems pretty high to me. In order to determine
how big my practical mental attic is, it seems useful to know whether
achieveable review rates are more like 2% or 0.2%.
I realize it's probably all about how hard the things are (e.g. most
people have a vocabulary of 10000+ words, and they manage to remember
them without any software at all) but perhaps in practice users'
long-term review rates cluster in a narrow band.
--
Ben
----------------- Original message -----------------
From: Peter Bienstman <Peter.B...@ugent.be>
To: mnemosyne-...@googlegroups.com
Patrick Kenny
against a hard limit, as for several months I was added 50-100 cards a
day. I have been using Mnemosyne for about eighteen months and have
over 27,000 cards in my database now, but I am averaging only between
200-250 reviews per day (assuming I don't add any new cards).
Perhaps the reason why I have so few reviews (given the size of my
collection) is that nearly all of my cards are very easy and only take a
few seconds to review each, and thus I miss very few. In any case, I
would be hesitant to change the standard algorithm; if something were to
be changed, perhaps a way to more naturally postpone and spread out a
pile of overdue cards would be best.
Cheers,
Patrick
Peter Bienstman
> Thanks for sharing---to me those statistics are very interesting and
> show some possible limits on what memory can achieve in the long term.
> But having to review 175 cards a day seems rough to me. I guess it's
> all about how difficult the things are that you memorize.
>
> Here's the way I was looking at it, as summarized by Sherlock Holmes
> in "A Study in Scarlet":
Nice quote, thanks!
> I realize it's probably all about how hard the things are (e.g. most
> people have a vocabulary of 10000+ words, and they manage to remember
> them without any software at all) but perhaps in practice users'
> long-term review rates cluster in a narrow band.
The answers to these questions lie locked in the data we've collected. Once
2.0 is out the door, I hope to have more time to look at those then.
Cheers,
Peter
--
------------------------------------------------
Peter Bienstman
Ghent University, Dept. of Information Technology
Sint-Pietersnieuwstraat 41, B-9000 Gent, Belgium
tel: +32 9 264 34 46, fax: +32 9 264 35 93
WWW: http://photonics.intec.UGent.be
email: Peter.B...@UGent.be
------------------------------------------------
Peter Bienstman
> Perhaps the reason why I have so few reviews (given the size of my
> collection) is that nearly all of my cards are very easy and only take a
> few seconds to review each, and thus I miss very few. In any case, I
> would be hesitant to change the standard algorithm;
That is definitely not the idea here.
> if something were to
> be changed, perhaps a way to more naturally postpone and spread out a
> pile of overdue cards would be best.
That is in fact already there: Mnemosyne presents you with the most urgent
cards first and takes the lateness in the the review into account when updating
the intervals. The only thing which is not there is 'cheating' with the
scheduler counter to limit it to a certain number of cards. But this is purely
cosmetic aspect and does not require a change to the scheduler itself.
Cheers,
Peter
Gwern Branwen
>
> Now, if I only wanted to review, say, 50 cards every day, then I was
> hoping that I could still have 10K cards in my deck. This corresponds
> to a 0.5% (=50/10K) "review rate". But Peter has a review rate of
> 175/8500 = 2%, which seems pretty high to me. In order to determine
> how big my practical mental attic is, it seems useful to know whether
> achieveable review rates are more like 2% or 0.2%.
FWIW, I'm at 1.8% myself. But I'm a little unclear here; I thought the
idea of spaced repetition was that the review rate would decrease over
time. So wouldn't the question really be 'how long would it take to
hit 0.2%?' and not 'whether'?
--
gwern
Peter Bienstman
I will hit any low number, *provided* you don't add new cards in the
meantime...
Peter
Francisco Fiuza Jr
I really sucks when you have a week of backlog.
Francisco Fiuza Jr
Francisco Fiuza Jr
For example, if I set this value to 80 cards, it will show me only those cards even though there should be around 200 for this day.
If you feel you can see more cards, pressing the button 'view more late cards' would get the next 80 cards.
It would be very usefull for me. Early this year when I went on vacation, I had about 1400 scheduled cards. It took me like 3 weeks to review them, and at the end I had about 350 grade 1 cards. After a session of 80 cards, I would rather go through those that I forgot giving a grade 1, than to review all the cards and watch the pile of grade 1 cards increase dramatically.
This is probably not the way it was designed, but I think it will have a good impact on the motivation. It's like: "Oh well, I have 1500 cards to review, but I won't need to sit for over 3 weeks to keep on my studies."
Regards,
Frank
David A. Harding
> What do you guys think about limiting the number of cards to review
> each day?
I think you're better off setting a time limit instead of a card limit.
For example, I use Mnemosyne in 30 minute increments. If I still have
reviews left after 30 minutes, I congratulate myself on getting started
and happily move on to another task.
Ben
me that there is some minimum limit on how seldomly you can review
something and still remember it. For instance, for a specific card,
here are two sequences of optimal times between review:
A: 1mo, 3mo, 6mo, 1yr, 2yr, 3yr, 5yr, 7yr, 10yr ...
B: 1mo, 3mo, 6mo, 1yr, 1yr, 1yr, 1yr, 1yr, 1yr ...
I would think that pattern B is more plausible than A---eventually
memory would peak. In fact, as a person reaches middle age and
beyond, wouldn't their memory deteriorate?
I'm wondering about this because one reason I use mnemosyne is because
I'm 30 now and I'd like to remember a lot of the stuff I remember
"naturally" now when I'm 60. I'd assume just retaining what I have
now is going to be harder and harder as I age.
--
Ben
----------------- Original message -----------------
From: Peter Bienstman <Peter.B...@ugent.be>
To: mnemosyne-...@googlegroups.com
Peter Bienstman
cards first, and takes into account how late you are when scheduling. So every
time you do X cards, you can be sure that you are doing the most important
ones.
I'm not sure how useful it would be to have your future schedule read 80 cards
every day for several weeks into the future, as then you'd have no way of
knowing how much behind you really are...
Still, this could be implemented in a plugin.
Cheers,
Peter
Peter Bienstman
> Are you sure about this? Unscientifically, it seems more plausible to
> me that there is some minimum limit on how seldomly you can review
> something and still remember it. For instance, for a specific card,
> here are two sequences of optimal times between review:
>
> A: 1mo, 3mo, 6mo, 1yr, 2yr, 3yr, 5yr, 7yr, 10yr ...
> B: 1mo, 3mo, 6mo, 1yr, 1yr, 1yr, 1yr, 1yr, 1yr ...
Right now the algorithm does behave like A. Whether that is the best way on a
30y time span, nobody knows, but that's one of the things I hope to find out by
collecting stats. I'm in it for the long run :-)
Peter
Meishu
I I may repeat what was said by "Supermemo"'s people, they emphasize
the following: reviewing is not learning.
Even if one accumulates 1000 cards to be reviewd, not just 200, and
they have truly learned them first, the reviewing procedure wouldn't
be that long nor would it be painful. I somehow suspect that newbies
to the program abandon it not because of the large amount of cards to
review, but rather due to the large amount AND the fact that they
don't remember anything. This is more due to misuse of the core
requirements of SRS and not so much due to the quantity. If you
follow the core requirement than the quantity is only secondary and
quite personal.
Ben
the card right? Assuming the ideal sequence is as in B, then by
definition when mnemosyne asks you about the card after a period
longer than a year, you won't do well. So won't the actual interval
bounce around the ideal interval? For instance, suppose the ideal
interval is always 1 year. I thought Mnemosyne would behave like
this:
Mnemosyne Internal Grade User
(in years) Gives Card
.5 5
1 4
1.3 2
1.1 3
.9 4
1.1 3
1.0 4
1.2 3
... ...
The numbers are just illustrations, but I thought Mnemosyne would
shorten the interval if you gave the card less than a 4, and would
lengthen it if you gave the card a 4 or 5. So if the ideal interval
is always 1 yr, won't Mnemosyne will approximate that, at least to
some degree?
--
Ben
----------------- Original message -----------------
From: Peter Bienstman <Peter.B...@ugent.be>
To: mnemosyne-...@googlegroups.com
Peter Bienstman
> But won't the algorithm will only behave like A if you keep getting
> the card right?
Yes, good catch, I was assuming that you made no errors.
> Assuming the ideal sequence is as in B, then by
> definition when mnemosyne asks you about the card after a period
> longer than a year, you won't do well. So won't the actual interval
> bounce around the ideal interval? For instance, suppose the ideal
> interval is always 1 year. I thought Mnemosyne would behave like
> this:
>
> Mnemosyne Internal Grade User
> (in years) Gives Card
> .5 5
> 1 4
> 1.3 2
> 1.1 3
> .9 4
> 1.1 3
> 1.0 4
> 1.2 3
> ... ...
>
> The numbers are just illustrations, but I thought Mnemosyne would
> shorten the interval if you gave the card less than a 4, and would
> lengthen it if you gave the card a 4 or 5. So if the ideal interval
> is always 1 yr, won't Mnemosyne will approximate that, at least to
> some degree?
Actually, in your example all the grades are 'pass' grades. And even if you
grade a card 2, the interval will still lengthen a bit, but not as much as
with higher grades.
Cheers,
Peter
Oisín
I think you're underestimating how much variation there is in the
difficulty in learning and remembering the information in each card,
between different subjects. When I studied German briefly before going
on a short holiday, I was able to cover about 80 new cards a day in
about 45 minutes, without feeling burdened - my reviews took about 3-5
seconds per card and were very easy.
However, to learn Chinese characters/words/grammar takes me ~17
seconds per card, because I speak the word and write the character
before checking the answer.
There is an atomic, indivisible amount of work required for cards of
various topics which cannot be broken down much further, which impacts
the difficulty of learning _and_ reviewing cards in an unavoidable
way. Granted, I could separate the pronounciation from the written
form, but that wouldn't have any real benefit (the speaking part takes
two or three seconds - writing the characters is the bottleneck).
However, I certainly agree with you in the presumption that a lot of
newbies to SRS systems make mistakes in how they learn the material
(e.g. assuming that the brain is a perfectly functioning lossless
database where true learning and comprehension will automatically
happen just by reviewing cards - we see this attitude when people ask
for pre-built decks to download... it's wishful thinking that we could
almost upload knowledge into our brains like in the Matrix), and how
they construct their cards (e.g. one card for every conjugation of a
verb in a particular tense, when it would be much better to break it
into single conjugations per card).
Oisín
Francisco Fiuza Jr
> I'm not sure how useful it would be to have your future schedule read 80 cards
> every day for several weeks into the future, as then you'd have no way of
> knowing how much behind you really are...
Scheduled: 80; late cards: 220; unlearned: 30
You can know how behind you are.
My point is, for example, let's say I review 80 cards, and I forget 8 of them giving a grade of 1. I would rather go through those 8 cards I forgot than to keep reviewing the other ones.
That's because if I have 2000 to review, at the end I'll have about 200 grade 1 cards.
Regards,
Frank
Peter Bienstman
> My point is, for example, let's say I review 80 cards, and I forget 8 of
> them giving a grade of 1. I would rather go through those 8 cards I forgot
> than to keep reviewing the other ones.
My personal opinion is that if you have such a big backlog, then it's better
to do the scheduled cards first so as to avoid forgetting any more cards.
Still, the behaviour you describe could be implemented by a custom scheduler
plugin.
Cheers,
Peter