Does GenericTypeahead always return the results with the highest score?

[yuan]

Hi:

I have a couple of questions about GenericTypeahead after looked at
the code of cleo.

(1) Looks like GenericTypeahead return all of matched elements. Is
this correct?
If it is true, what if a key(term) are matched with lots of
elements? For example, 1 million. It will comsumes lots of memory and
increase latence.

(2) In the indexDataStore, looks like elements are not sorted by
score. Also, the seach() function has a paramter timeoutMillis.
What if the function search() reutrn earlier because of
timeoutMillis, can GenericTypeahead guarntee return elemetns with
the highest score?


Thanks a lot for the great project.

Yuan

[Jingwei]

Hi,

Please see my reply below.

On Apr 9, 9:01 pm, yuan <huapengy...@gmail.com> wrote:
> Hi:
>
> I have a couple of questions about GenericTypeahead  after looked at
> the code of cleo.
>
> (1) Looks like GenericTypeahead return all of matched elements. Is
> this correct?
>     If it is true, what if a key(term) are matched with lots of
> elements? For example, 1 million.  It will comsumes lots of memory and
> increase latence.

This really depends on the type of Collector you are using. There are
several different types Collector you can use to return over 1 million
elements or top 10 elements. Choose the collector that suits your
needs.

>
> (2) In the indexDataStore, looks like elements are not sorted by
> score.  Also, the seach() function has a paramter timeoutMillis.
> What if the function search() reutrn earlier because of
> timeoutMillis,  can  GenericTypeahead  guarntee return elemetns with
> the highest score?

For typeahead, as a user is typing more characters, the search scope
is narrowing down quickly. The timeoutMillis is for early termination
to guarantee response time but not full coverage. If your program
really need to find the elements with the highest score, you should
NOT consider using timeout.

[yuan]

Thanks Jingwei:

We need return the highest score, we also expect low latency. But if
the inverted list is long, it may take long to process the entire
list.
I am not sure whether there are some ways to contorl the maximum lengh
of the inverted list. I see you mentioned cleo partition in the
document. Looks like the partition is based on elementID rather than
prefix?

If the partition is based on elelmentID, the maximum length of the
invered list may not be very long. But a single query will touch all
of partitions. In a distributed system, it means it will touch all of
nodes. I am understanding correctly?

Thanks.

Yuan

> > Yuan- Hide quoted text -
>
> - Show quoted text -

[Jingwei]

Partitions will help since requests are evaluated in parallel.
Partition is based on elementID. You may want to experiment with your
data sets to see how many partitions you can hold on one box in order
to have acceptable performance.

Best

Jingwei

[Altynai]

Hi Yuan.

Recently,I'm also working with cleo,but I faced a lot of problems,especially dealing with the big data,If you have time,I would like to ask you some questions.My email address is Altyn...@gamil.com

Thanks!

[Jingwei]

Would you mind giving some details about your data? How many elements are you talking about? How did you partition your data? etc.

Thanks.

Jingwei