Search Working Group update

[Chris Davenport]

This an update following the inaugural meeting of the Search Working Group which took place at J and Beyond 2012 recently and which was attended by a group of people sharing a common interest in improving Smart Search in Joomla.  Of course, not everyone with an interest in the subject will have been able to be present in Germany, so please read this summary of our discussions and reply, either on list or to me privately, if you are able to help out in any way.

Given that we will likely want to merge new features for the 3.0 release sometime during July, there isn't going to be time to develop any radical new features for Smart Search in time for Joomla 3.0.  Consequently, it was generally agreed that the focus of our efforts in the short term should be on fixing as many outstanding issues in the current Smart Search as possible.

I have listed the current known issues on the http://docs.joomla.org/Search_Working_Group page together with links to tracker items where appropriate.  Several of these issues have already been fixed and just need a couple of people to test so we can get the fixes committed (these are flagged as "Testers required" in the list).  If this can be done within the next couple of days these issues can be cleared for the 2.5.5 release.  I think that most of the other listed short-term issues can be resolved in time for the 2.5.6 release.

So, what do we need?

Firstly, we have an immediate requirement for people to test the patches that have already been submitted so we can get them committed for the 2.5.5 release.  We also need coders to work on fixes for the other identified tracker items.

Secondly, we need language experts to come forward and help us to identify and hopefully fix outstanding short-term language issues.  We specifically need people who are very familiar with the syntax and grammar of a language, combined with sufficient PHP coding experience to be able to provide fixes specific to that language.  For example, we need someone with knowledge of Japanese to provide a fix for tracker item 27511, most likely along the lines of the fix already present for Chinese.  Going forward these language experts will also be needed for our medium-term goal of developing methods in the language package that will be helpful for search.  For example, one important medium-term goal will be to add PHP stemmers to support as many non-English languages as possible using the recently added JLanguageStemmer class.

Thirdly, we need someone with MSSQL experience and access to an MSSQL platform so we can clear all the outstanding MSSQL issues.  People with experience in and access to other database platforms should also test Smart Search and provide feedback and fixes where appropriate.

We are aware of problems with indexing large sites and we want to raise the bar on where it becomes impractical to use Smart Search because of disk usage, memory requirements or time limits.  It should be remembered that Smart Search is a pure PHP search engine and it is likely that a standalone search engine, such as Solr, will prove to be a better fit on the very largest sites.  However, it should be feasible to use Smart Search on most large sites if we can fix just a small number of currently known issues.  These are not major issues and the fixes should be relatively straightfoward; we just need people to code the fixes and test them on both regular and large sites.

For the time being this dev cms mailing list will be the primary channel of communication.  If the volume of traffic becomes such that it might be deemed to be a nuisance by those not interested in search, then we will most likely start a separate mailing list.

Finally, a big thank you to everyone who came along to our meeting in Germany and who provided feedback and insights on the work done so far.

Chris.

--
Chris Davenport
Joomla Leadership Team - Production Working Group
Joomla Documentation Coordinator

[bobotche]

I notice that it's very difficult to interact with the smart search to
get for example current search results. It would better to create a
gate
throught plugins by which developpers will be able to interact with
this search engine .
Example:
In com_finder/views/search/view.html.php
$dispatcher = JDispatcher::getInstance();
JPluginHelper::importPlugin('content');
$dispatcher->trigger('onFinderIndexerResultsEnabled', array ($results,
$params, $total, $query));

In plugins/content/foo.php
public function onFinderIndexerResultsEnabled($results, $params,
$total, $query){
// do something with args
}

bobotche

On May 31, 11:38 pm, Chris Davenport <chris.davenp...@joomla.org>
wrote:

> This an update following the inaugural meeting of the Search Working Group
> which took place at J and Beyond 2012 recently and which was attended by a
> group of people sharing a common interest in improving Smart Search in
> Joomla.  Of course, not everyone with an interest in the subject will have
> been able to be present in Germany, so please read this summary of our
> discussions and reply, either on list or to me privately, if you are able
> to help out in any way.
>
> Given that we will likely want to merge new features for the 3.0 release
> sometime during July, there isn't going to be time to develop any radical
> new features for Smart Search in time for Joomla 3.0.  Consequently, it was
> generally agreed that the focus of our efforts in the short term should be
> on fixing as many outstanding issues in the current Smart Search as
> possible.
>

> I have listed the current known issues on thehttp://docs.joomla.org/Search_Working_Grouppage together with links to

[Rouven Weßling]

Could you give an example use case to modify the results of the search?

Best regards
Rouven

> --
> You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
> To post to this group, send an email to joomla-...@googlegroups.com.
> To unsubscribe from this group, send email to joomla-dev-cm...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/joomla-dev-cms?hl=en-GB.
>

[bobotche]

Hi Rouven,
In fact there is no reason to modify the search results,
but many reasons to simply use the search results .
Example:
a slider that will display slides according to the search results.
thanks,
bobotche

> >> I have listed the current known issues on thehttp://docs.joomla.org/Search_Working_Grouppagetogether with links to

[Chris Davenport]

Sounds like you need a search API.  Smart Search does not have a proper API at present, but this is something we want to work on during the 3.x cycle.

However, for the most common use-case where you want to retrieve taxonomies for use as search facets it's not particularly difficult to achieve by using a bit of SQL.  Take a look at the mod_finder module for an example of how it's done.

Chris.

[bobotche]

OK, solved
bobotche

On Jun 6, 12:41 am, Chris Davenport <chris.davenp...@joomla.org>
wrote:

> Sounds like you need a search API.  Smart Search does not have a proper API
> at present, but this is something we want to work on during the 3.x cycle.
>
> However, for the most common use-case where you want to retrieve taxonomies
> for use as search facets it's not particularly difficult to achieve by
> using a bit of SQL.  Take a look at the mod_finder module for an example of
> how it's done.
>
> Chris.
>