Lightweight Alternative Search Solutions for SilverStripe?

[Gordon Anderson]

hi

I've never been particularly happy with the search functionality of SilverStripe which motivated me to write an extensive module for Elasticsearch, https://github.com/gordonbanderson/silverstripe-elastica

I am pitching to a potential client, and whilst they sound excited by the possibility of advanced search the above offers, the hosting requirements are an issue.  Like SOLR, Elasticsearch is a Java server that requires *lots* of RAM, ideally in the region of 1GB plus.  And their webserver hosting cost will be less than 3 USD per month...  Whilst I realise likes of a Digital Ocean VPS instance could be used I'm curious to know if any viable lightweight solutions exist.

I used to use the SilverStripe sphinx module,

https://github.com/silverstripe/silverstripe-sphinx/commits/master , where the daemon was actually small enough to escape the Dreamhost robot.  However this module seems all but abandoned :(  A quick search suggests there is no SilverStripe Xapian module, this would be a good candidate for relatively static brochureware sites.

One other idea I had was to use a Postgres instance as the search server (assuming the site is using MySQL on cheap hosting).  Whilst this may seem a bit of a hair-brained idea, the process is similar to that of Elasticsearch.  In an afterWrite hook instead of posting data for indexing to Elasticsearch, one would save data in a remote Postgresql instance.  Searching would ultimately be an SQL query using textsearch, see http://www.postgresql.org/docs/9.3/static/textsearch.html

Lastly I have tried the SilverStripe Lucene module, see https://github.com/gordonbanderson/silverstripe-lucene - I ran into memory issues with this module as it has to load the entire index for each search request.  There eventually becomes a point where third party hosting cannot cope and the developer cannot increase the PHP memory allocation.

Anyhow curious to know what options people are using

Cheers

Gordon

[Corey Sewell]

While probably not the answer your looking for, we went through a similar issue.

We ended up using SOLR with multiple cores on a dedicated VPS. Solr with multiple cores allows one instance to index multiple sites individually. We now have several Silverstripe, EpiServer (C# .Net CMS) and a few bespoke applications all using the one SOLR insurance.

While it was a headache to setup initially we  created shell scripts to automate adding/removing cores depending on the system that will use it.
It also cost us in the beginning but were quickly able to get enough clients utilizing it make it viable and now generates a small profit.

Just food for thought

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to silverstripe-d...@googlegroups.com.
To post to this group, send email to silverst...@googlegroups.com.
Visit this group at https://groups.google.com/group/silverstripe-dev.
For more options, visit https://groups.google.com/d/optout.

[Gordon Anderson]

On Tue, Feb 16, 2016 at 1:45 PM, Corey Sewell <co...@sewell.net.nz> wrote:

While probably not the answer your looking for, we went through a similar issue.

We ended up using SOLR with multiple cores on a dedicated VPS. Solr with multiple cores allows one instance to index multiple sites individually. We now have several Silverstripe, EpiServer (C# .Net CMS) and a few bespoke applications all using the one SOLR insurance.

While it was a headache to setup initially we  created shell scripts to automate adding/removing cores depending on the system that will use it.
It also cost us in the beginning but were quickly able to get enough clients utilizing it make it viable and now generates a small profit.

Just food for thought

Thanks for your feedback.  I was considering this as a possible option with Elasticearch, though I need to check the config options to attempt and isolate indexing from searching as far as possible.

Cheers

Gordon

[off...@netwerkstatt.at]

I was happy using https://github.com/g4b0/silverstripe-searchable-dataobjects for single language sites, also enhanced it a bit for e.g. showing a relevant snippet of the results with search highlighting.

 

Not as advanced as solr/elastic but fine for smaller sites.

 

On a current project we plan to utilise google using https://github.com/dnadesign/silverstripe-googlesitesearch.

Pricing for this personal search: https://www.google.com/work/search/products/gss.html

 

Cheers, Werner

[Roman]

I've also used Google Custom Search (https://cse.google.com/cse/) for several projects. It's really affordable and easy to set up.
Didn't know about the module, so thanks for that ;-)

Best regards - Roman

[Nicolaas Thiemen Francken - Sunny Side Up]

We have also used google search https://github.com/sunnysideup/silverstripe-googlecustomsearch ... not so great because you can not put in any of the common sense that you can put into a custom search engine (e.g. always put products first or whatever), but one marvellous thing you will find is that you are more aware of how Google indexes your site, including any issues you may have (double-ups, etc...).

Nicolaas Thiemen Francken
  www.sunnysideup.co.nz

  phone: +64221697577

[Cam Findlay]

We've been using a very basic version of the PHP Lucene port on our docs sites. Granted we could spend more time to fine tune it, however it seems pretty lightweight and robust option when just waiting some simple search functionality. 

Nice to see there is a module floating around for Lucene, seeing as it's a bit old perhaps we could upgrade it for the 3.x version of SilverStripe and get it up to the module standard as a small project.

Google CSE is good too however have found it a bit "accept how it looks and acts" and you're relying on a third party for that (whereas Lucene only requires PHP which you should have, you're running SilverStripe CMS after all!).

We run Elastic search on the addons site. However are considering replacing this at some point in the future with Solr or some other option. This is because we have a number of good implementations of Solr in house, we know it well, and the addons site spiders A LOT of modules  and versions of modules as DataObjects so needs something a little more heavy duty.

In sum I'm a big +1 for a more robust Lucene search implementation that works for 3.x and 4.x going forward.

[Darren]

Lastly I have tried the SilverStripe Lucene module, see https://github.com/gordonbanderson/silverstripe-lucene - I ran into memory issues with this module as it has to load the entire index for each search request.  There eventually becomes a point where third party hosting cannot cope and the developer cannot increase the PHP memory allocation.

Hi Gordon

I wrote that plugin originally - you've found some forks of an older version that was hosted on google code, but I have a version that I haven't touched for several years living on github also:

https://github.com/DarrenInwood/silverstripe-lucene

This has some stuff that I wrote after the forked versions you're using, so that it can use Java Lucene.  I actually can't remember how it works, and I never wrote any docs for using it, but IIRC it used the PHP Java Bridge (http://php-java-bridge.sourceforge.net/pjb/) to communicate with a locally-spawned java Lucene process, which should be more lightweight than a fullblown Solr server.

It was written originally due to speed concerns of the Zend Lucene branch.  It is much, much faster; however I'm not sure what sort of memory footprint it uses, I no longer have access to any instances of it in use in the wild!

Good luck   :-)

--

Darren

[Florian Thoma]

I am using another fork of the lucene plugin an quite a few sites, https://github.com/Instagraeme/silverstripe-lucene. But I have fixed some issues on https://github.com/xini/silverstripe-lucene that haven't been merged in yet. I think that's the most current version of the plugin (apart from some helpfullrobot changes).

[Gordon Anderson]

Thanks Florian, I will take a look.  That module probably needs some testing love as well :)

I was searching (oops that's almost a pun) for alternatives to a Java information retrieval server and came across Bleve, http://www.blevesearch.com/ - it's written in Go.  There is a REST interface for it, similar to Elasticsearch and SOLR, see https://github.com/blevesearch/bleve-explorer.  Has anyone tried it?  If so I'd be curious it's memory footprint.  I also like the Bleve server status screenshots.

Cheers

Gordon