[GSOC] Regarding "Elastic Search/Solr backend" project

[Sandeep Sukhani]

Hi Rick,

I am Sandeep Sukhani from India. I am interested to work on "Update Bookie to permit an Elastic Search/Solr backend for full text indexing of content" project. I have little experience in solr. I have worked on a live project which had solr for indexing and searching of files. Learning Elastic Search wouldn't be hard though.

I have good experience in python and javascript.

I have been working on the project. I wanted to share my findings and see if I am going on the right path. Here is what I feel:

1. Much of the changes are required to be done in models and bcelery package for backend.

2. There should or may not be any changes required to be done on javascript side, unless if needed to change the response format.

Besides I have a doubt. Ideas page says that user should be able to configure bookie to talk to an Elastic Search or Solr bakcend. Does this mean that backend would have provision for both and user can configure any of it?

If so, would user be able to switch to another indexing functionality?

I am still trying to make things more clear. With your guidance it would be done the right way as expected. Please correct me if any of my interpretations are wrong.

Regards,

Sandeep Sukhani

[Richard Harding]

On Sat, 08 Mar 2014, Sandeep Sukhani wrote:

> Hi Rick,
>
> I am Sandeep Sukhani from India. I am interested to work on "Update Bookie
> to permit an Elastic Search/Solr backend for full text indexing of content"
> project. I have little experience in solr. I have worked on a live project
> which had solr for indexing and searching of files. Learning Elastic Search
> wouldn't be hard though.
> I have good experience in python and javascript.
>
> I have been working on the project. I wanted to share my findings and see
> if I am going on the right path. Here is what I feel:
>
> 1. Much of the changes are required to be done in models and
> bcelery package for backend.

Yes, the models/fulltext.py contains the curent whoosh search
implementation. The celery jobs actually take a change in the models and
update the fulltext index with the new content.

> 2. There should or may not be any changes required to be done on javascript
> side, unless if needed to change the response format.

No, not really. We might do some tweaks to the search UI to permit us to
make more interesting searches down the road, but there's nothing that
jumps out at the moment.

> Besides I have a doubt. Ideas page says that user should be able to
> configure bookie to talk to an Elastic Search or Solr bakcend. Does this
> mean that backend would have provision for both and user can configure any
> of it?
> If so, would user be able to switch to another indexing functionality?

Yes! Right now there's a whoosh fulltext index that's pure python and works
locally. It's great for development so that it doesn't require you to bring
up and configure solr or elasticsearch. The idea would be to make the
fulltext search code swappable like the database side is. You would
configure which index to use in your .ini and the makefile would provide
helpers to update/rebuild the index.


--

Rick Harding
@mitechie
http://blog.mitechie.com
http://lococast.net

[Sandeep Sukhani]

Thank you so much for reverting back. Your response would be really helpful for exploring things further.