Customer Sort Order for Document Search results

[Dev Mukherjee]

Hi,

Our application uses the Document Search API, we index a number of Products (used in a shopping cart catalogue). Users are allowed to search the Products using various attributes and document search API works a treat.

We want users to be able to save their search results as a list; again not an issue, we can simply save the search conditions and rerun the Document Search and return the results.

What we'd like to add is the ability to manually sort the search results (and add any new results down the bottom of the list). This is proving to be quite difficult (or at least we're hitting too many road blocks in our thinking).

Our initial thoughts have been storing the order of the list in NDB and then be able to pass a list of doc_ids to fetch from the document store or storing a combination of the search list id and the order in the document (at the risk of hitting the max document size).

We try not to hit the NDB until the use requires full details of the Products, and satisfy all search queries via the document search api.

Has anyone attempted anything similar; maintaining manually sortable lists of results from the document search api?

Is this possible when the document search api is being used? Would appreciate any thinking / pointers anyone out there can share.

Thanks for your time.

Devraj

[Anastasios Hatzis]

Hi Devraj,

I've built my Search API based features, so I can at least show the search results with-out hitting the NDB API. It works for many things, but for example, repeated properties it doesn't. Also for key-property types it is difficult, when the information you use in the search document has been updated after indexing. I've figured a way to use non-stale thumbnails of referenced entities in search results, but it isn't something I would recommend, because in some use-cases it can have downsides regarding client-side caching.

Regarding your questions:

Storing results in NDB: I don't think it is necessary to store search results in NDB, other than just individually selected product keys for a "My favorites" or bookmark feature. It should be sufficient to store the user's search query itself, including the custom sorting (if the sorting really adds benefit to the user). When the user want's to run the search, just use the saved search query on the Search API to retrieve the search documents and the cursor in HTTP query parameter, so the user can navigate to next page of search results.

However, if you want users to share search queries with other users, or search engines to index your search pages, you need to pass all relevant search filters and sorting as HTTP query parameters. Then in NDB, store a structured property or local structured property with the HTTP query parameters and values as your "saved search query", rather than the search query string you send to Search API. This will make it easier to update your search index later with-out breaking saved search queries.

New entries: You would need to store the creation datetime of each product in NDB. For example as date or datetime property with the "auto-add-now" option. Also export this property as date field into your search documents, so you can query for them and have the value in search results.

Next, add an auto-add-now "lastRun" timestamp to your "saved search query" in NDB. Now, if you run a search query, you can compare the search result date field with the property in your "saved search query", and emphasize new items in the UI.

Alternatively, you could add new items in the results below, whatever the current sorting is, by running a second query filtering additionally by the timestamp, and this result is added as a separate block. Maybe this gets too expensive and it should be only done, if the user explicitly requests to show all new items for the saved search query.

However, you have to decide when to update the "lastRun" timestamp in NDB, so the new items are not shown anymore as new items at the next visit. Maybe with a task a few minutes / hours after the user did run a saved search.

I hope this helps a little.

Anastasios