Account Options

  1. Sign in
The old Google Groups will be going away soon.
Switch to the new Google Groups.
Google Groups Home
« Groups Home
One patch, two plugins and development comments (smart.fm, Chinese)
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  16 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Robert Siemer  
View profile  
 More options Feb 8 2011, 2:30 pm
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Tue, 8 Feb 2011 11:30:34 -0800 (PST)
Local: Tues, Feb 8 2011 2:30 pm
Subject: One patch, two plugins and development comments (smart.fm, Chinese)
Hi!

Content:
1) Intro  (short)
2) Patch, plugins code review  (medium)
3) Development comments, “Why not ...?” questions  (long)

1) Intro

I’m creating a huge sophisticated deck for Chinese learning: 1 new
model, about 12 cards per fact (listening, speaking, reading and
writing for a word/character and 1-3 example sentence each word/
character) and over 3800 facts.

Originally I came across Anki while searching for a flash card program
for Debian (I use Ubuntu now). From the Anki plugins I found out about
smart.fm (was a language learning website). I fell in love with their
Flash application (Adobe Flash here) and their Chinese learning
content. Their app does similar things as Anki and they had that
nicely prepared content! So I used it for a year.

Finally, smart.fm is about to convert into a pay site and the content
got neglected (temporarily during the transition I assume). Because of
that and other reasons, I decided to hack features that made smart.fm
superior into Anki and get the content converted.

2) Patch, plugins code review

The three files lie here: http://backsla.sh/anki/
The plugins are small and easy. To support them I had to adapt Anki.
That is why I explain the plugins purposes’ first and than the patch,
chunk by chunk:

--- CardHistory.py

This is to support dynamic cards that change depending on your study
performance. The cards served to the screen originally got two things:
the card’s template and the fact (static). The plugin passes the
current card object down to the pystache renderer to support this:

{{#cardHistory}} Card seen: {{reps}} times <span data-
interval='{{interval}}' /> {{/cardHistory}}

Pystache already supports this, I just wrap the card object into
something more dictionary like. The object is then hooked in via the
(modified) “prepareFields” hook.

I use Javascript to leverage the repetition and study interval. My
cards all have a picture as image association/remembering aid, which I
don’t want to end up as requirement for my brain to retrieve the
answer. If the card got seen several times or is already mature, then
hide the image. It is revealed again together with the answer. I also
use this with additional textual aids in the question.

smart.fm had a version of this, but too limited: sometimes I ended up
knowing the answer from the picture alone. And I used a sheet of paper
to tap that part of the screen.

--- ScriptsAndStyles.py

With these dynamic cards I ended up with some repetitive code in the
card templates. To avoid that tedious copy&paste I hooked into:
high-level “addStyles” for my CSS style sheet,
low-level “drawQuestion” for my js code, because there is no higher
level hook - and
a new “preRenderFormatQA” for some common fields which I need to get
replaced by pystache.
“formatQA” comes too late.

--- anki-1.2.5.patch

That align-code in htlmQuestion is a really ugly hack and hard to
avoid. Why not style sheets? In any case, the UI could have a “left,
right, center, template driven” option, with “template driven” doing
what align=False is doing.

The other lines are copied to let formatQA() run fine. I can’t use the
cache, otherwise my dynamic flash cards won’t work... I see two issues
here, see 3)  Note that I call formatQA with an additional keyword
argument.

It is hard to mix default styling with personal ones. My personal
styling defines the colors, so I got rid of the color defaults. See 3)
as well.

No need for runHook() to discourage hooks with keyword parameter. The
opposite should be true

In the last chunk I allow formatQA to pass arbitrary keyword arguments
through to the hooks. The “prepareFields” hook signature changes as
well, but plugins should be immune to things like that. Mine are.

Damien, I kindly request you to consider inclusion of the easy parts
and discussion for the others. :-)

3) Development comments and “Why not ...?” questions

Why is the Anki Debian package not directly pushed into Debian
unstable (and maybe Ubuntu directly as well)?

The field styling thing: Why are the class names so cryptic??
Otherwise I could use:
<span class='default-modelname-fieldname  my-own-thing-
overwrites'>{{{fieldname}}}</span>

The htmlQuestion issues: a)  the card cache itself. Except for latex
runs, there is not much work to do: why a complicated cache?? b)
formatQA() is really complicated to call. The reason for both is
probably a missing nice ORM. SQLAlchemy can do more than leveraged by
Anki I think. This would also help with the search (it could perfectly
go directly to the fields and card templates and tags, etc.)

Study mode: Anki has only 90% of a study mode. There is an easy fix,
but let’s review the problems first:
i) the user falsely thinks the card is know and creates wrong
associations with studied content “Ah, I thought [knwon] A1 belongs to
[known] Q1, but it seems that [similar new] Q2 is the right
association...<click Answer>” Bang! It is A2 instead. Please rewire A1
to Q1 again...
ii) the user falsely thinks a card is new and clicks <Answer> without
thinking. Bang! With thinking he may have got it. Learning opportunity
gone.
iii) the user correctly recognizes an unknown card as new and thinks:
“I just behave as if I failed” and after dozens of new cards
eventually thinks “I failed again” instead.
iv) this card’s question and answer may not be enough to learn the
fact, especially if the fact has fields not show on this card.

All four are counterproductive and depressing in a learning
environment. To remedy this I suggest to display different buttons
when the card is shown the first time. Like this:
   This card is new.
   <Study item. [1]> <I know the answer anyway! [2-5]>
The shortcuts are just in case you didn’t notice the new buttons,
maybe others are better. Whatever you click, you get the answer. If
you click on “I know the answer anyway” you get the normal evaluation
buttons. On “study item” you get an “Okay.” which works like “Again.”
This fixes i), ii) and iii).

A further improvement: on “study item” you get a study template which
might present the fact in a special way (In my case all 8-16 fields
together, without the need to go to the fact editor)... This improves
iv).

smart.fm had a study mode, and I was about to implement one via
dynamic cards, but I think my proposal is easy (three extra buttons +
simplest logic) and very general. What do you think?

I don’t use the registerPlugin(name, updateId), because it allows for
a plugin to carry different names when it is enabled or disabled. To
be consistent I have to track my own file name in the plugin, which
could be made into a default. Actually, until registerPlugin() serves
a decent purpose, only default parameters should be used to allow for
a literal registerPlugin() without arguments.

Cheers,
Robert


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Damien Elmes  
View profile  
 More options Feb 9 2011, 4:29 am
From: Damien Elmes <reso...@ichi2.net>
Date: Wed, 9 Feb 2011 18:29:12 +0900
Local: Wed, Feb 9 2011 4:29 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

> I use Javascript to leverage the repetition and study interval. My
> cards all have a picture as image association/remembering aid, which I
> don’t want to end up as requirement for my brain to retrieve the
> answer. If the card got seen several times or is already mature, then
> hide the image. It is revealed again together with the answer. I also
> use this with additional textual aids in the question.

Expanding the options available in javascript including control over
audio playing is on the voting site. Please add your requirements and
vote:

http://anki.uservoice.com/forums/63835-general/suggestions/913221-add...

> That align-code in htlmQuestion is a really ugly hack and hard to
> avoid. Why not style sheets?

Legacy. WebKit wasn't always available in Qt. It's mentioned in the
comments of the code.

> In any case, the UI could have a “left,
> right, center, template driven” option, with “template driven” doing
> what align=False is doing.

You're the first person to complain about it. Until I see a real
demand, I suggest you override it in a plugin.

> It is hard to mix default styling with personal ones. My personal
> styling defines the colors, so I got rid of the color defaults. See 3)
> as well.

If you want to define the styling in the template, use {{{ to disable
field styling and do it manually.

> No need for runHook() to discourage hooks with keyword parameter. The
> opposite should be true

Yes, that's a reasonable change.

> Why is the Anki Debian package not directly pushed into Debian
> unstable (and maybe Ubuntu directly as well)?

Because there is an existing maintainer, and I'm quite busy as it is.

> The field styling thing: Why are the class names so cryptic??

- they're less verbose than the format you propose, saving space
- it avoids issues with special characters in the field names like '

> Otherwise I could use:
> <span class='default-modelname-fieldname  my-own-thing-
> overwrites'>{{{fieldname}}}</span>

Can't you use !important in the outer div?

> The htmlQuestion issues: a)  the card cache itself. Except for latex
> runs, there is not much work to do: why a complicated cache??

The cache speeds up the browser, mobile clients and the online
interface. On a fast desktop it doesn't save a great deal, but the
cache is already there for the previous mentioned reasons, and not
using it is a waste. If you don't want it, override it.

> formatQA() is really complicated to call. The reason for both is
> probably a missing nice ORM. SQLAlchemy can do more than leveraged by
> Anki I think. This would also help with the search (it could perfectly
> go directly to the fields and card templates and tags, etc.)

SQLAlchemy's ORM is much slower than direct SQL. formatQA gets called
in bulk for things like imports and bulk browser updates.

> i) the user falsely thinks the card is know and creates wrong
> associations with studied content “Ah, I thought [knwon] A1 belongs to
> [known] Q1, but it seems that [similar new] Q2 is the right
> association...<click Answer>” Bang! It is A2 instead. Please rewire A1
> to Q1 again...
> ii) the user falsely thinks a card is new and clicks <Answer> without
> thinking. Bang! With thinking he may have got it. Learning opportunity
> gone.

Users have the option of separating reviews and new cards if they
want, and Anki used to display new cards last. It mixes them by
default these days because many people said they prefer the variety.

> iii) the user correctly recognizes an unknown card as new and thinks:
> “I just behave as if I failed” and after dozens of new cards
> eventually thinks “I failed again” instead.

I'm having trouble understanding what you're trying to say. Note that
Anki does not say 'fail' anywhere in the study interface; the wording
is deliberately neutral.

> iv) this card’s question and answer may not be enough to learn the
> fact, especially if the fact has fields not show on this card.

If users are following the minimum information principle
(http://www.supermemo.com/articles/20rules.htm), cards should not
depend on each other.

> environment. To remedy this I suggest to display different buttons
> when the card is shown the first time. Like this:
>   This card is new.
>   <Study item. [1]> <I know the answer anyway! [2-5]>
> The shortcuts are just in case you didn’t notice the new buttons,

There are enough people complaining than 4 buttons are 2 too many. I
doubt adding more buttons is a good idea. For an alternative approach
to 'learn mode', please see the cram discussion.

> I don’t use the registerPlugin(name, updateId), because it allows for
> a plugin to carry different names when it is enabled or disabled. To
> be consistent I have to track my own file name in the plugin, which
> could be made into a default. Actually, until registerPlugin() serves
> a decent purpose, only default parameters should be used to allow for
> a literal registerPlugin() without arguments.

Yes, it doesn't serve much use at the moment, and the problem with
disabling you mention is valid.

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Siemer  
View profile  
 More options Feb 9 2011, 2:06 pm
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Wed, 9 Feb 2011 11:06:15 -0800 (PST)
Local: Wed, Feb 9 2011 2:06 pm
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

> Expanding the options available in javascript including control over
> audio playing is on the voting site. Please add your requirements and
> vote:

> http://anki.uservoice.com/forums/63835-general/suggestions/913221-add...

I put a comment there and voted. -- The way I did _that_ part it is
not the way to go. It was just a clean and quick surgery before Anki
gets HTML5 with JS access to internal structures... ;-)

> > In any case, the UI could have a “left,
> > right, center, template driven” option, with “template driven” doing
> > what align=False is doing.

> You're the first person to complain about it. Until I see a real
> demand, I suggest you override it in a plugin.

Ah... I just realized that thanks to the align parameter itself I can
replace the function easily with itself, setting align=False.

> > No need for runHook() to discourage hooks with keyword parameter. The
> > opposite should be true

> Yes, that's a reasonable change.

Perfect!

> > Why is the Anki Debian package not directly pushed into Debian
> > unstable (and maybe Ubuntu directly as well)?

> Because there is an existing maintainer, and I'm quite busy as it is.

Why is the maintainer not pushing?

> > The field styling thing: Why are the class names so cryptic??

> - they're less verbose than the format you propose, saving space
> - it avoids issues with special characters in the field names like '

> > Otherwise I could use:
> > <span class='default-modelname-fieldname  my-own-thing-
> > overwrites'>{{{fieldname}}}</span>

> Can't you use !important in the outer div?

I forgot that. I’ll test how !important works. I can see that you
consider these class names Anki-private and want to make sure they are
unique, but I don’t buy that byte saving... ;-)

> > The htmlQuestion issues: a)  the card cache itself. Except for latex
> > runs, there is not much work to do: why a complicated cache??

> The cache speeds up the browser, mobile clients and the online
> interface.

Any mobile client capable of showing a CSS styled HTML card at all is
certainly able to exchange some fields with a value. I’m not sure
whether the far future of Anki lies in a webapp, written in Python/Qt
transformed via Pyjamas into HTML/JS for full offline/online
functionality or more a desktop app which arbitrates between a native
UI, sqlite and a strong Webkit component. In both cases the web
browser/Webkit will handle string replacements with a smile. The more
data access JS gets within Webkit, the more the question arises “Why
does the card [template] not fill out itself?” Mobile clients and
online interface won’t get faster or save any space maintaining a lame
card cache.

The card browser needs an ORM overhaul if it can’t access fact fields
+ card template in a swift, and works faster with a facts * template
data multiplication overhead. The card browser should be a fact
browser anyway, with one of these little |> triangles at the left of
each fact to fold out it’s cards. -- I understand that Anki is how it
is and my own projects advance like molasses uphill in January;
development is a huge effort. But I don’t agree that _this_ kind of
cache is any good or pointing the way to the future.

> > formatQA() is really complicated to call. The reason for both is
> > probably a missing nice ORM. SQLAlchemy can do more than leveraged by
> > Anki I think. This would also help with the search (it could perfectly
> > go directly to the fields and card templates and tags, etc.)

> SQLAlchemy's ORM is much slower than direct SQL.

It probably is, but that should not affect Anki at all.

> formatQA gets called in bulk for things like imports and bulk browser updates.

For the cache, I assume...

> > iii) the user correctly recognizes an unknown card as new and thinks:
> > “I just behave as if I failed” and after dozens of new cards
> > eventually thinks “I failed again” instead.

> I'm having trouble understanding what you're trying to say. Note that
> Anki does not say 'fail' anywhere in the study interface; the wording
> is deliberately neutral.

The UI feels like “I failed”. The UI should make me feel: “This is
new, learn something!”

> > iv) this card’s question and answer may not be enough to learn the
> > fact, especially if the fact has fields not show on this card.

> If users are following the minimum information principle
> (http://www.supermemo.com/articles/20rules.htm), cards should not
> depend on each other.

I read that page, and I reread it again. You mean this here:

http://www.supermemo.com/articles/20rules.htm#minimum%20information%2...

The word “depend” does not pop up there. The very example there shows
how to split up one QA into several. This is exactly what I do. And
these cards _have_ an obvious relationship: they belong to the same
subject (in the example: the Dead Sea). And it makes perfect sense to
get to know a subject as a whole. This actually follows the higher
ranked rule “Learn before you memorize: build an overall picture of
the learned knowledge.”

http://www.supermemo.com/articles/20rules.htm#Learn%20before%20you%20...

> > environment. To remedy this I suggest to display different buttons
> > when the card is shown the first time. Like this:
> >   This card is new.
> >   <Study item. [1]> <I know the answer anyway! [2-4]>
> > The shortcuts are just in case you didn’t notice the new buttons,

> There are enough people complaining than 4 buttons are 2 too many. I
> doubt adding more buttons is a good idea.

People don’t complain because the _number_ of buttons is to high, they
complain because they don’t know what to do. The choice between four
closely related things is complicated. Deciding between “Study item”
and “Answer anyway” is straight forward. And after that, the decision
gets _really_ easy: “continue” (instead of the complicated four
buttons).

Cheers and thank you for writing Anki!
Robert


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Damien Elmes  
View profile  
 More options Feb 9 2011, 3:15 pm
From: Damien Elmes <reso...@ichi2.net>
Date: Thu, 10 Feb 2011 05:15:23 +0900
Local: Wed, Feb 9 2011 3:15 pm
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

>> Because there is an existing maintainer, and I'm quite busy as it is.

> Why is the maintainer not pushing?

Perhaps they are busy. Last time someone offered to take over, they
said they'd get back to making releases.

> Any mobile client capable of showing a CSS styled HTML card at all is
> certainly able to exchange some fields with a value.

I did rather extensive profiling on a mobile device when I changed the
Anki deck format to SQLite. I don't care what you think it is
"certainly able to"; this was a data-driven decision.

>> I'm having trouble understanding what you're trying to say. Note that
>> Anki does not say 'fail' anywhere in the study interface; the wording
>> is deliberately neutral.

> The UI feels like “I failed”. The UI should make me feel: “This is
> new, learn something!”

Some people have brought this up before. It doesn't seem to be an
issue for everyone, but if I want to increase the user friendliness
factor you're right, I will probably have to improve this. As
mentioned earlier, see the cramming thread for the direction this is
going in.

> The word “depend” does not pop up there. The very example there shows
> how to split up one QA into several. This is exactly what I do. And
> these cards _have_ an obvious relationship: they belong to the same
> subject (in the example: the Dead Sea). And it makes perfect sense to
> get to know a subject as a whole. This actually follows the higher
> ranked rule “Learn before you memorize: build an overall picture of
> the learned knowledge.”

> http://www.supermemo.com/articles/20rules.htm#Learn%20before%20you%20...

I think what they're stressing there is to study outside of the system
first, by doing things like reading textbooks. There is similar advice
in the Anki manual.
http://ankisrs.net/docs/manual.html#_downloading_material

But yes, if users just download a list of material, a preview mode may
make initial answers easier.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Siemer  
View profile  
 More options Feb 18 2011, 4:04 pm
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Fri, 18 Feb 2011 13:04:57 -0800 (PST)
Local: Fri, Feb 18 2011 4:04 pm
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)
Some number:

From Chinese vocabulary, I generated a 1.5MB CSV file. It takes 0.9
seconds to bzip it down to 377kB.

Anki took about 15min to import, and made a 91MB deck out of it (25kB
per fact). On a freshly started Anki, it takes 15 seconds to open that
deck (already checked and optimized). Anki feels sluggish and eats up
more memory (138MB resident) than any of my 21 tabs in Chrome (which
runs already for days).  :-(

On top, it eats up more CPU cycles doing nothing (showing the very
first card), ranking first with “top”, overtaking a concurrently
running rsync doing backup (network limited, ~50kB/s)!!

I havn’t done any investigation yet, but my suspect is that “QA
cache”. -- As the deck goes over the Ankiweb limits (a sync would take
hours anyway), and wouldn’t run on any mobile I know, it would be nice
to have a disable option.

For the CPU it needs in the background, I have no clue.  :-(

Regards

On Feb 9, 5:29 pm, Damien Elmes <reso...@ichi2.net> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Damien Elmes  
View profile  
 More options Feb 18 2011, 4:53 pm
From: Damien Elmes <reso...@ichi2.net>
Date: Sat, 19 Feb 2011 06:53:13 +0900
Local: Fri, Feb 18 2011 4:53 pm
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

> Anki took about 15min to import, and made a 91MB deck out of it (25kB
> per fact). On a freshly started Anki, it takes 15 seconds to open that
> deck (already checked and optimized).

In recent profiling I found two causes for slow deck loading: the
spacedCardCount() in status.py and the updateStudyStats() call in
main.py not using an index on time.  With those two fixed startup
should be quite fast even on a cold cache & decks in the 100s of
megabytes. Comment them out and if you still experience slowdown,
please instrument the code and locate the source of the problem.

> Anki feels sluggish and eats up
> more memory (138MB resident)

The default db page cache is 80MB. If you're actually running low on
memory then you can change this with the default_page_cache pragma on
the deck, but most people want to leave it as it is as it means the
disk needs to be hit less.

> On top, it eats up more CPU cycles doing nothing (showing the very
> first card), ranking first with “top”, overtaking a concurrently
> running rsync doing backup (network limited, ~50kB/s)!!

I see zero CPU usage here when idle. The Mac builds are known to use
the CPU due to toolkit issues, so perhaps the issue is with your Qt
installation or X11 environment.

> I havn’t done any investigation yet, but my suspect is that “QA
> cache”. -- As the deck goes over the Ankiweb limits (a sync would take
> hours anyway), and wouldn’t run on any mobile I know, it would be nice

I know of one person who uses the iPhone client on a 450MB deck. The
desktop memory usage is due to the larger page cache and all the GUI &
python libraries, not some fundamental problem with the database.

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Siemer  
View profile  
 More options Feb 19 2011, 2:35 am
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Fri, 18 Feb 2011 23:35:37 -0800 (PST)
Local: Sat, Feb 19 2011 2:35 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)
1.5MB of data plus 100 Bytes for 34,500 cards is below 5MB.

86MB of cache and indices are going to speed up nothing. 5MB can
easily be kept in memory, even on a mobile (I talk about a huge deck
here!). Any search operation can go linearly over the data and ends up
faster. I actually do this here all the time (I still do consistency
checks and corrections with the data); goes like through butter (using
Python and native lists).

On Feb 19, 5:53 am, Damien Elmes <reso...@ichi2.net> wrote:

> > Anki took about 15min to import, and made a 91MB deck out of it (25kB
> > per fact). On a freshly started Anki, it takes 15 seconds to open that
> > deck (already checked and optimized).

> In recent profiling I found two causes for slow deck loading: the
> spacedCardCount() in status.py and the updateStudyStats() call in
> main.py not using an index on time.  With those two fixed startup
> should be quite fast even on a cold cache & decks in the 100s of
> megabytes. Comment them out and if you still experience slowdown,
> please instrument the code and locate the source of the problem.

The data strategy is seriously flawed I would say.

> > On top, it eats up more CPU cycles doing nothing (showing the very
> > first card), ranking first with “top”, overtaking a concurrently
> > running rsync doing backup (network limited, ~50kB/s)!!

> I see zero CPU usage here when idle. The Mac builds are known to use
> the CPU due to toolkit issues, so perhaps the issue is with your Qt
> installation or X11 environment.

Maybe. I use Ubuntu Maverick 10.10.

> > I havn’t done any investigation yet, but my suspect is that “QA
> > cache”. -- As the deck goes over the Ankiweb limits (a sync would take
> > hours anyway), and wouldn’t run on any mobile I know, it would be nice

> I know of one person who uses the iPhone client on a 450MB deck. The
> desktop memory usage is due to the larger page cache and all the GUI &
> python libraries, not some fundamental problem with the database.

This user will be very enlightened to find out that his deck is only
worth 25MB.

Regards,
Robert


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Damien Elmes  
View profile  
 More options Feb 19 2011, 2:56 am
From: Damien Elmes <reso...@ichi2.net>
Date: Sat, 19 Feb 2011 16:56:49 +0900
Local: Sat, Feb 19 2011 2:56 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

> This user will be very enlightened to find out that his deck is only
> worth 25MB.

The deck has a revision history of about 500,000 entries. The facts
and revision history alone run into the hundreds of megabytes when
stored in text format.

Anki stored the decks in a serialized format a long time ago. Deck
load and save times were painfully slow as the deck size grew. The
current design is the result of incremental improvements over the last
4 and a bit years, and it's pretty frustrating to have you come in
preaching how you think things should be without a full understanding
of the issues involved. If you think you can do it better, please go
and write your own code and stop wasting my time.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Siemer  
View profile  
 More options Feb 20 2011, 1:24 am
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Sat, 19 Feb 2011 22:24:19 -0800 (PST)
Local: Sun, Feb 20 2011 1:24 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)
It is not my intention to get you pissed or waste your time.

I evaluated the “market” and selected Anki (for the second time) as my
tool to study Chinese. And I’m very happy to have it! But yes, I wish
it would suit me even better.

On Feb 19, 3:56 pm, Damien Elmes <reso...@ichi2.net> wrote:

> If you think you can do it better, please go and write your own code and stop wasting my time.

The thing is, you can even do it better yourself. So I’m here to point
at the bad parts. It’s not to badmouth your baby. I want it to get
better and I hope you prefer when I spend time to discuss and improve
the existing Anki instead of writing another SRS from scratch and
fail. No?

I spend over 1h to write a post (e.g. this one), to make them short
and concise. Maybe my rewrites make them sound too formal and harsh...
Also not my intention.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Siemer  
View profile  
 More options Feb 20 2011, 2:18 am
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Sat, 19 Feb 2011 23:18:18 -0800 (PST)
Local: Sun, Feb 20 2011 2:18 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)
On Feb 19, 3:56 pm, Damien Elmes <reso...@ichi2.net> wrote:

> Anki stored the decks in a serialized format a long time ago. Deck
> load and save times were painfully slow as the deck size grew.

Ah... You think I’m against sqlite. That is not the case. Sqlite is
fine!
(Just for the record: non-db-formats don’t have to be slow. I dump my
huge Chinese content via cPickle module in 1.2 seconds on a slow
EeePC. Quick!)

> The current design is the result of incremental improvements over the last
> 4 and a bit years, and it's pretty frustrating to have you come in
> preaching how you think things should be without a full understanding
> of the issues involved.

Don’t blame me for not knowing the history of Anki inside-out. And be
sure that I don’t want to preach or frustrate you.

You remember my 91MB deck? I emptied the following tables: *Deleted
and cardTags, and set question/answer in cards to '' and wush, down to
34MB. Then I dropped all indices. Result: 13MB. I see room for more
cuts, but at least something reasonable!

Afterwards I found out that the *Deleted tables look like a “this is
deleted” column. - Why is that extra? ...so probably some facts and
cards reappeared, no?

The cardTags had 120.000 entries. -- I don’t even use tags. At a
glance that looked like doubled template tags. Just a
sum(length(value)) count showed that it got 5MB in the value column.

> > This user will be very enlightened to find out that his deck is only
> > worth 25MB.

> The deck has a revision history of about 500,000 entries. The facts
> and revision history alone run into the hundreds of megabytes when
> stored in text format.

I’m not sure what you mean with “text” format. But hundreds of
megabytes of history is overkill. Last years review stats should have
one db row per day maximum. Maybe some history statistics in the cards
table as well, but an infinite collection of per card+review history
doesn’t look like a feature. Who wants to carry and sync >10MB decks
knowing that 80% is history?

Robert


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Siemer  
View profile  
 More options Feb 20 2011, 2:45 am
From: Robert Siemer <Robert.Siemer-a...@backsla.sh>
Date: Sat, 19 Feb 2011 23:45:03 -0800 (PST)
Local: Sun, Feb 20 2011 2:45 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)
And yes, my deck still runs in the patched Anki and loads in 1 second.

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
TenNinjas  
View profile  
 More options Feb 20 2011, 5:17 am
From: TenNinjas <tennin...@tenninjas.ca>
Date: Sun, 20 Feb 2011 04:17:07 -0600
Local: Sun, Feb 20 2011 5:17 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

On Sat, 19 Feb 2011 23:45:03 -0800 (PST), Robert Siemer wrote:
> And

yes, my deck still runs in the patched Anki and loads in 1 second.

Two
questions:

1) Does your desk still work in un-patched Anki? If not; why not?
(I understand your features may not be available, but from what I've read
there is no reason for your deck not to be reverse portable apart from those
features.)

2) Is all statistic information still available for your deck? Is
the information still correct?

--
* TenNinjas *


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Damien Elmes  
View profile  
 More options Feb 20 2011, 5:25 am
From: Damien Elmes <reso...@ichi2.net>
Date: Sun, 20 Feb 2011 19:25:05 +0900
Local: Sun, Feb 20 2011 5:25 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

> (Just for the record: non-db-formats don’t have to be slow. I dump my
> huge Chinese content via cPickle module in 1.2 seconds on a slow
> EeePC. Quick!)

cPickle is what Anki used to use, and what Mnemosyne 1.0 uses. Aside
from the fact that it is not fast to read or write as deck sizes grow,
it is a massive security risk. People could craft shared decks that
reformat your computer.

> Maybe some history statistics in the cards
> table as well, but an infinite collection of per card+review history
> doesn’t look like a feature.

The revision logs hold valuable information that could be data mined
in the future to improve the algorithm. Deleting them or collating
them so they are no longer associated with individual cards anymore
would throw all that data away.

There's plenty that could be improved in Anki, and if you check the
archives you'll see that I'm pretty open to constructive criticism.
But your criticism is not constructive - you march in here with a
limited understanding of the issues involved and make bold statements
about perceived design problems with zero attempt to inquire or
understand why things are the way they are, then pass off my
frustration as being due to oversensitivity on my part, and not a
problem with your tone. I don't want to waste any more time on this.
Please do not post here again.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
TenNinjas  
View profile  
 More options Feb 20 2011, 5:27 am
From: TenNinjas <tennin...@tenninjas.ca>
Date: Sun, 20 Feb 2011 04:27:17 -0600
Local: Sun, Feb 20 2011 5:27 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

On Sun, 20 Feb 2011 04:17:07 -0600, TenNinjas wrote:
> On Sat, 19 Feb

2011 23:45:03 -0800 (PST), Robert Siemer wrote:

>> And yes, my deck still

runs in the patched Anki and loads in 1 second.

> Two questions:

> 1)

Does your desk still work in un-patched Anki? If not; why not? (I understand
your features may not be available, but from what I've read there is no reason
for your deck not to be reverse portable apart from those features.)

> 2)

Is all statistic information still available for your deck? Is the information
still correct?

And a third... if your intention is to make this available to
others, how does it work for mobile situations? While not all mobile platforms
are well maintained, anything you plan to share should (in my thinking at
least) work on AnkiMobile and Ankidroid - if not with all features, at least
gracefully degraded and still effective.

--
* TenNinjas *


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Damien Elmes  
View profile  
 More options Feb 20 2011, 5:38 am
From: Damien Elmes <reso...@ichi2.net>
Date: Sun, 20 Feb 2011 19:38:08 +0900
Local: Sun, Feb 20 2011 5:38 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)
TenNinjas: Robert stated above that he has no desire to run the decks
on mobile devices or the website. By chopping out those parts of the
deck he's also disabled the ability to search for tags, use selective
study, sort by question or answer in the browser, used cached LaTeX
files, and a bunch of other stuff. Without the indices various
operations will become heavily dependent on the size of the deck, and
as more cards are added things will get slower and slower - especially
when the deck is not cached in RAM.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
TenNinjas  
View profile  
 More options Feb 20 2011, 10:19 am
From: TenNinjas <tennin...@tenninjas.ca>
Date: Sun, 20 Feb 2011 09:19:55 -0600
Local: Sun, Feb 20 2011 10:19 am
Subject: Re: One patch, two plugins and development comments (smart.fm, Chinese)

So, in summary, what he did removes large portion of Anki's
functionality.

Thanks, I wanted that to be clear before we see people
blindly purging data they need.

On Sun, 20 Feb 2011 19:38:08 +0900, Damien

Elmes wrote:
> TenNinjas: Robert stated above that he has no desire to run
the decks
> on mobile devices or the website. By chopping out those parts of
the
> deck he's also disabled the ability to search for tags, use selective

study, sort by question or answer in the browser, used cached LaTeX
> files,

and a bunch of other stuff. Without the indices various
> operations will

become heavily dependent on the size of the deck, and
> as more cards are

added things will get slower and slower - especially
> when the deck is not

cached in RAM.

--
* TenNinjas *


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »