Query, Backend and Elasticsearch (#925, #888)

[Honza Král]

Hi,

Sorry I took so long before reacting, we are currently ramping up for
1.0 release of elasticsearch.

I have spent some time poking around Haystack internals and I'd like
to discuss some changes we could make to make things a little simpler
for the backends moving forwards. Most specifically this leads with
the separation of SearchQuery and Backend.

I'd like to propose to remove the SearchQuery from the individual
backends - just use the BaseSearchQuery (renaming it to SearchQuery)
and move all backend-specific stuff to the Backend class itself. That
can be greatly simplified in terms of interface where it could then
just accept SearchQuery in a lot of it's APIs, thus eliminating need
for stuff like

https://github.com/toastdriven/django-haystack/blob/master/haystack/backends/elasticsearch_backend.py#L864-L900

The freeing up of the interface of the Engine would also mean that for
elasticsearch we could finally move away from using the lucene query
syntax and to more structured queries that will be able to optimize
better (maybe even used cached filters in some cases) and generally
produce better results - for example a match query on a field will use
the field's analyzer so users can potentially have different analyzers
per field (multi lingual, special analyzers for stuff like url paths
etc).

Another benefit this should bring would be the ability to create
completely custom queries and having them still wrapped in haystack
queryset mechanism. So something like .raw() functionality. This would
simple produce an instance of SearchQuery in the queryset that will
have .raw attribute/method/whatever that the backend will pick up and
know not to do any other modifications to the query on top of doing
pagination and serializing the results.

This will be a backwards incompatible change for people with their own
backends but I do believe it's worth it.


What do you think?

Thanks!

Honza Král
E-Mail: honza...@gmail.com
Phone: +420 606 678585

[Chris Adams]

This seems like an interesting idea — I've definitely felt the same way a few times. After a casual look at the code it seems like it'd also be good just for reducing the number of places we're passing around a ton of keyword arguments in method calls.

Thinking about how a release might work, this sounds like a Haystack 3.0 feature since we'd be breaking compatibility for custom backend implementations. I'm taking a closer look at #925 now to see whether we can do a little simplification without breaking 2.x compatibility and that'll probably translate into more concrete for this proposal.

Chris