Considering RS - Question on: Search index size / performance / maintenance

[Roy N]

Hi Everyone,  

Apologies if this has been discussed elsewhere, I have searched the web, the issues and this list but couldnt find a lot of information...

I'm investigating whether RediSearch could be a good fit for our product.  We have a SaaS multitenant application, and have many different "object types", each with different fields (nothing too fancy here).

My questions are: is there any best practice or example on how to index these items into RS?  My initial idea is to create an index for each tenant+object_type, so seomthing like: 1:users (for users in tenant 1), 235:products (for product information from tenant 235) and so on... 

(In our case we never do "cross-tenant" searches).  Is this a sane way to structure things?

Furthermore, we have a reasonably sized main DB (just over 1TB), and so we are quite worried about storage as well as resilience of this infrastructure.  Currently we have an elasticsearch cluster with around 2TB of data...  I see very little mention of the database size, etc in Redisearch documents / blogposts... Is this size something Redisearch can acoomodate easily? (With growth +50%/year)?  

I assume all this  comes down to the underlying Redis servers?  In this case how would sharding work?  Would I take care of that (sharding), say on a per-tenant way? or can I make use of something within Redis or Redisearch itself? 

(We would be using Redisearch and its Redis as a dedicated instance, so wont be colocated on another redis instance... this would be part of a Search microservice)...

With this in mind, what would be the appropriate way to keep backups, etc of this?  (We dont have a lot of experience using redis as a persistance datasource - although we use it as a cache / db "intermediary" quite a lot).

Finally, I'm also interested in Autosuggest, but from the examples I see, how would I keep the index size in check? Is there a way or a best practice to "trim" these autusuggest indices? 

Thanks for all help - we have a hard time keeping elasticsearch up and performing well, which would seem to be one of RS big wins, but I'm not sure we are the right fit?

[Michael Masouras]

Hi Roy,

On Mon, Dec 9, 2019 at 7:34 AM Roy N <rna...@neoassist.com> wrote:

Hi Everyone,  

Apologies if this has been discussed elsewhere, I have searched the web, the issues and this list but couldnt find a lot of information...

I'm investigating whether RediSearch could be a good fit for our product.  We have a SaaS multitenant application, and have many different "object types", each with different fields (nothing too fancy here).

My questions are: is there any best practice or example on how to index these items into RS?  My initial idea is to create an index for each tenant+object_type, so seomthing like: 1:users (for users in tenant 1), 235:products (for product information from tenant 235) and so on... 

(In our case we never do "cross-tenant" searches).  Is this a sane way to structure things?

Yes, that's a possible way to do it. The alternative would be to have one index per type of object, and specify a field "tenant" as a TAG data type and include it in the search query you issue. 

Furthermore, we have a reasonably sized main DB (just over 1TB), and so we are quite worried about storage as well as resilience of this infrastructure.  Currently we have an elasticsearch cluster with around 2TB of data...  I see very little mention of the database size, etc in Redisearch documents / blogposts... Is this size something Redisearch can acoomodate easily? (With growth +50%/year)?   

I assume all this  comes down to the underlying Redis servers?  In this case how would sharding work?  Would I take care of that (sharding), say on a per-tenant way? or can I make use of something within Redis or Redisearch itself? 

(We would be using Redisearch and its Redis as a dedicated instance, so wont be colocated on another redis instance... this would be part of a Search microservice)...

Redis Cluster  is the canonical way of scaling this. I am not sure how RS will do with that data size. It works really well for us, but our data-size is ~5G. 

 

With this in mind, what would be the appropriate way to keep backups, etc of this?  (We dont have a lot of experience using redis as a persistance datasource - although we use it as a cache / db "intermediary" quite a lot).

Redis has two ways to persist, AOF and RDB.  You can backup those files as frequently as you wish.

Finally, I'm also interested in Autosuggest, but from the examples I see, how would I keep the index size in check? Is there a way or a best practice to "trim" these autusuggest indices? 

Thanks for all help - we have a hard time keeping elasticsearch up and performing well, which would seem to be one of RS big wins, but I'm not sure we are the right fit?

I have had a great experience using RS but our data size is much smaller. Reach out privately if you would like more info. 

Michael

 

 Sugestões ou reclamações? Mande-nos um e-mail para ouvi...@neoassist.com

IMPORTANTE: Esta mensagem destina-se exclusivamente a seu destinatário e possui conteúdo confidencial. 
Caso a tenha recebido por engano, por favor, elimine-a imediatamente e notifique seu remetente. 
Qualquer disseminação ou reprodução do seu conteúdo é expressamente proibida e, portanto, ilegal. 
IMPORTANT: This document is intended for the named addressed only and it is confidential. 
If for any reason you have received this document in error, please destroy it and advise the sender 
immediately. The reproduction of this content is explicitly prohibited and, therefore, illegal.

--
You received this message because you are subscribed to the Google Groups "RediSearch Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redisearch+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redisearch/0aaa4119-4546-4875-a420-d6611dd1fd13%40googlegroups.com.