I use django-haystack 2.4.1 and Solr 4.9.1.
Unfortunately, I'm in the situation in which my search_index.py kept growing and growing, and so did my data, and now doing a full indexing every day takes up to 18 hours! I'm indexing about 200k documents.
The reason is that I have A LOT of "prepare_" functions, and they hit the database.
I have lots of "prepare_" functions for two reasons:
- I store numerical data in the index, because I need advanced searches like "find all objects for which property FOO is greater between the integer values X and Y"
- I'm probably abusing the search index here, but I'm keeping things like "number of likes, number of comments, number of views" so that I can display them in search results (I should be updating a database column instead?)
The code is here if you want, but it ain't pretty:
Can you suggest a strategy to improve this situation?
Thanks in advance!