Re: mongo index

[Jim Garrison]

On 06/16/12 01:39, Laurent Savaete wrote:
> 2012/6/16 Jim Garrison <garr...@wikiotics.org>:
>> my plan is something like
>>
>> 0. index on author as well
>> 1. continuous/auto update
>> 2. move continuous update to be a celery job, possibly
>> 3. make various current queries (such as the front page tag cloud
>> ones)
>> more efficient using saved map-reduce operations in mongodb
>>
>> what are you thinking of? do you have a project that will complement
>> these things?
> sounds good.
> I'm thinking of building more stuff on the "user" side. I've made a
> search_by_tag() function already (basically put your search page code
> into a reusable function) in the 5s widget code.
> I'm sure I'll come up with more functions to build while putting
> together the lesson search page.
> Also, high on my priority list is building this
> <<<lessons_tagged_with(taglist)>>> macro, so we can embed stuff
> wherever. I don't think that's very hard to get to, and it would help
> Ian with what he's doing.
>
> Continuous update will be awesome too!
>

[Bringing ductus-developers onto this thread.]

With both the macro and the function I'd be worried about unnecessary
function/API proliferation. For now lessons_tagged_with() seems like a
good idea, but tomorrow will we need wikipages_tagged_with() or
lessons_tagged_with_and_authored_by() or
flashcard_lessons_tagged_with_and_authored_by() ?

I'd rather see a generic search function that takes a variety of keyword
arguments, and something similar for the macro.

The reason I haven't made such user-friendly functions yet is because I
think it would be a good idea to see what types of queries we are making
(and figure out ways we should be optimizing such queries) before we
nail down an API. That is, I think anything we come up with now will be
short sighted in some way and lead to it being cruft later. (And this
would work against the goal of making something "reusable".)

I don't mind being a bit aggressive with the internal function, as there
are no consequences for getting it wrong (we can always refactor!). But
with the macro, it would be very nice to get the API right the first
time, since we are kind of committed to compatibility once we start
putting something with a given syntax into wiki pages. For this reason
it probably makes sense to slowly phase in different options (just as we
are doing for /special/search, which currently only supports tag search).

Let me know what you think.

[Ian Sullivan]

On 06/16/2012 12:00 PM, Jim Garrison wrote:
> The reason I haven't made such user-friendly functions yet is because I
> think it would be a good idea to see what types of queries we are making
> (and figure out ways we should be optimizing such queries) before we
> nail down an API. That is, I think anything we come up with now will be
> short sighted in some way and lead to it being cruft later. (And this
> would work against the goal of making something "reusable".)
>
> I don't mind being a bit aggressive with the internal function, as there
> are no consequences for getting it wrong (we can always refactor!). But
> with the macro, it would be very nice to get the API right the first
> time, since we are kind of committed to compatibility once we start
> putting something with a given syntax into wiki pages. For this reason
> it probably makes sense to slowly phase in different options (just as we
> are doing for /special/search, which currently only supports tag search).
>
> Let me know what you think.

While it changes user behavior more than refactoring code, we are only
locked in to macros or other public APIs to the extent that we have to
update pages and retrain users. Now, while we are still light on pages
and personally know basically all users, we are flexible enough to make
those changes without much difficulty. Whatever we deploy to the site is
going to need some iterations and probably get refactored down the line
as our content and community grow.

-Ian

[Laurent Savaete]

> With both the macro and the function I'd be worried about unnecessary
> function/API proliferation. For now lessons_tagged_with() seems like a
> good idea, but tomorrow will we need wikipages_tagged_with() or
> lessons_tagged_with_and_authored_by() or
> flashcard_lessons_tagged_with_and_authored_by() ?
>
> I'd rather see a generic search function that takes a variety of keyword
> arguments, and something similar for the macro.

I wrote the code today, and I realised this as I did.

> The reason I haven't made such user-friendly functions yet is because I
> think it would be a good idea to see what types of queries we are making
> (and figure out ways we should be optimizing such queries) before we
> nail down an API. That is, I think anything we come up with now will be
> short sighted in some way and lead to it being cruft later.  (And this
> would work against the goal of making something "reusable".)
>
> I don't mind being a bit aggressive with the internal function, as there
> are no consequences for getting it wrong (we can always refactor!).  But
> with the macro, it would be very nice to get the API right the first
> time, since we are kind of committed to compatibility once we start
> putting something with a given syntax into wiki pages.  For this reason

fair enough, although there's always the bot --replacewikitext
solution (granted, it won't teach users the new trick, though there
are solutions to that)

> it probably makes sense to slowly phase in different options (just as we
> are doing for /special/search, which currently only supports tag search).
>
> Let me know what you think.

I think they are valid comments, but when would we consider that our
api is mature enough that we can roll it out?
My opinion is that we need this now (clearly we are blocked now), and
as Ian said, the number of editors is minimal.

I coded the macro as:
<<PagesByTags tags=tag1,tag2>>
I'm happy to rename it
<<PageList>> or anything, but I think we should get that out as soon as we can.

[Jim Garrison]

On 06/16/12 09:40, Laurent Savaete wrote:
> I coded the macro as:
> <<PagesByTags tags=tag1,tag2>>
> I'm happy to rename it
> <<PageList>> or anything, but I think we should get that out as soon as we can.

Yeah, I think PageList or PageSearchListing makes more sense. The
emphasis in naming should be on search rather than on what we are
currently searching for, since we will eventually want to search on a
lot of things.