Search index refresh interval?

[viraj.par...@edmodo.com]

Hello,

We have been trying to push a few pieces of content to the Learning Registry using EasyPublish to begin with and have observed some weird behavior. Need help understand that.

I have a couple of resources which were uploaded using the EasyPublish tool and now have been deleted using EasyPublish also. However, the widget on the main site still returns them in the search results. One such example is the resource with URL - https://spotlight.edmodo.com/product/solving-quadratic-equations,379035/.

Can you please shed some light on how and when items get indexed, when they start showing up in the search widget and when they get removed from the results (after deletion from EasyPublish)?

Please let us know if you need more information.

Thanks,

Best regards,

Viraj

[joe hobson]

hey Viraj,

There were issues discovered last week with the search server index, where the indexing process hung and had to be restarted. I had the service go back and reindex the last year's worth of records to verify that everything published recently was showing properly. The index is back to indexing new records every 5 minutes.

From what I can see in the code, it doesn't appear that the LR-Data service was designed with a process for checking and removing previously indexed records. In the past I know that we have periodically done refreshes by dumping the existing index and reindexing everything completely. It's not the greatest method, but over time it's less processor intensive than doing a check of each record to verify that they are still active. Unfortunately the LR doesn't offer a service broadcasting a feed of just deleted record notifications, so there isn't a simple answer to the problem. 

Keep in mind that this is not necessarily a "problem" or even something that's isolated to the search service. Once a record goes into the LR it gets consumed by services all over the place, and how each of those chooses to deal with removing the record after the fact is left to their discretion. You'll find that your example resource is still available on the ISLE OER system as well -- just because it's no longer in the LR, doesn't necessarily mean it's not a valid resource for educators.

I'd be happy to hear suggestions on how we can improve the search index. Given the limitations and the impact on server performance, how often does the community think the index should be refreshed?

Thanks for your feedback. I hope the information was helpful. ... .joe

-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:

joe hobson

   Navigation North Learning

--
You received this message because you are subscribed to the Google Groups "Learning Registry Developers List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to learningreg-d...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.