Could anyone provide some hints on how they have implemented search functionality on their web site? Did you do it django or use another middleware (eg, Lucene)? If you did it django, did you bypass the ORM and use direct SQL to take advantage of full text search?
I'm currently trying to build the functionality directly in django's ORM as a cascade of functions that would retrieve matches in decreasing relevance, eg:
class Article(meta.Model): title = meta.CharField() description = meta.TextField() keywords = meta.CharField()
> Could anyone provide some hints on how they have implemented search > functionality on their web site? Did you do it django or use another > middleware (eg, Lucene)? If you did it django, did you bypass the ORM > and use direct SQL to take advantage of full text search?
> Could anyone provide some hints on how they have implemented search > functionality on their web site? Did you do it django or use another > middleware (eg, Lucene)? If you did it django, did you bypass the ORM > and use direct SQL to take advantage of full text search?
Hey Kevin,
At World Online, the search engine (lawrence.com/search, ljworld.com/search) uses swish-e (http://swish-e.org/) to index files.
We made a small script that reads a custom Django setting (FULL_TEXT_INDEXING) to find out which models need to be indexed. The indexer runs on a regular basis; for each to-be-indexed model, it grabs all items in the system using get_list() (with optional limiting kwargs designated in FULL_TEXT_INDEXING), renders each result in a template according to its content type, and indexes the resulting rendered template.
Then, when somebody does a search on the site, we use swish-e's Python bindings to retrieve the IDs of the objects that match the search criteria. Then we use Django's get_in_bulk() to retrieve the actual objects. get_in_bulk() takes a list of IDs and returns a dictionary of {id: object}. (Trivia: get_in_bulk() was created to solve this exact problem.)
Hope that helps! It would be pretty cool to open-source this mini search framework and pop it in django/contrib, but that would be up to Jacob to decide.
Adrian
-- Adrian Holovaty holovaty.com | djangoproject.com | chicagocrime.org
On Dec 29, 2005, at 9:31 AM, Adrian Holovaty wrote:
> At World Online, the search engine (lawrence.com/search, > ljworld.com/search) uses swish-e (http://swish-e.org/) to index files.
[snip]
> Hope that helps! It would be pretty cool to open-source this mini > search framework and pop it in django/contrib, but that would be up to > Jacob to decide.
I'd love to open it up; I don't see any reason that we'd need to keep it hidden. I'll put it on my todo list :)
Doesn't Swish-e pose an incompatibility for licensing? Everything Django has been BSD up to this point and I would hate to see anything alter this. Isn't swish-e gpl?
> On Dec 29, 2005, at 9:31 AM, Adrian Holovaty wrote:
>> At World Online, the search engine (lawrence.com/search, >> ljworld.com/search) uses swish-e (http://swish-e.org/) to index files.
> [snip]
>> Hope that helps! It would be pretty cool to open-source this mini >> search framework and pop it in django/contrib, but that would be up to >> Jacob to decide.
> I'd love to open it up; I don't see any reason that we'd need to keep > it hidden. I'll put it on my todo list :)
On 12/29/05, David Pratt <fairwi...@eastlink.ca> wrote:
> Doesn't Swish-e pose an incompatibility for licensing? Everything Django > has been BSD up to this point and I would hate to see anything alter > this. Isn't swish-e gpl?
Swish-e grants a special exemption from automatically GPL'ing linked programs which use the libswish-e API interface:
> On 12/29/05, David Pratt <fairwi...@eastlink.ca> wrote: >> Doesn't Swish-e pose an incompatibility for licensing? Everything >> Django >> has been BSD up to this point and I would hate to see anything alter >> this. Isn't swish-e gpl?
> Swish-e grants a special exemption from automatically GPL'ing linked > programs which use the libswish-e API interface:
Plus, this would be a contributed app -- or possible something distributed separately from Django -- that requires you to install Swish-E separately. I don't see this being a core part of Django.