Capabilities of Full Text Search with ForestDB

[Brendan Duddridge]

Hi,

I'm noticing that when I want to search now that I've switched to using ForestDB that I can't seem to do a prefixed search.

For example, I have a value in my document of "xbox". I want to be able to find it if I just type in "xb". 

According to the documentation:

NOTE: When using ForestDB storage, Couchbase Lite implements full-text search itself instead of using SQLite, and doesn't support the fancy search syntax. Instead, the search terms are individual words, with implicit "AND" conjunctions.

Is using * for a prefixed query considered "fancy"?

Does this mean that I'll have to emit "x, xb, xbo, and xbox" in my map block in order to facilitate prefixed searches?

Thanks,

Brendan 

[Jens Alfke]

On Jan 13, 2016, at 4:16 PM, Brendan Duddridge <bren...@gmail.com> wrote:

Is using * for a prefixed query considered "fancy"?

Yes, anything with metacharacters or reserved words is “fancy” in this context. The implementation for ForestDB just looks up words. The one special thing it does is ‘stemming’, which means that it tries to remove English suffixes before indexing and querying — that way “big” and “bigger” will match, and “walk” and “walking”. (For both SQLite3 and ForestDB we’re using the “Snowball” stemmer, with sources taken from the sqlite3-unicodesn library. It supports about a dozen languages.)

I could add prefix matching, but not for 1.2. Please file an issue. Mostly it seems pretty easy to implement, except for possible weird interactions with stemming. (Hm, might be a fun Summer Of Code project for someone…)

Does this mean that I'll have to emit "x, xb, xbo, and xbox" in my map block in order to facilitate prefixed searches?

Ugh, I reluctantly say “yes” because I can’t think of any better workaround :/

Although really all you need is “x xb xbo xbox”, since commas and noise-words like “and” are ignored.*

—Jens

* Actually that’s another special thing. Two special things! Amongst our special things are such diverse elements as…

[Brendan Duddridge]

Hi Jens,

Ok thanks. I need to think about what I want to do now. Adding all those split words would probably balloon the index a great deal. So I may be forced to go back to SQLite for now. Just need to figure out the darned encryption issue I was having with that. But that's another story.