search not working on new installation - even after enabling open edX Search
Vinayak Bhat
- I installed open edX master on 12.04 ubuntu native installation in aws.
- after that i followed http://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/latest/configuration/edx_search.html to enable search. I installed edx-search using option -2 mentioned in the steps. the edx-search cloned into ../edx-platform/edx-search. I have updated the mentioned flags in common.py of cms and lms.
- after the flags were enabled I do get reindex button in studio. but when I click reindex, there is an error message in in below message bar. also search does not work on lms. if I search with string "demo" it does not return anything -- I have the demoX course in the courses and it shows up in the lms homescreen.
- after looking at the logs and also trying equivalent command from manage.py sudo -u www-data /edx/bin/python.edxapp ./manage.py cms --settings aws reindex_course --all I see the following error:
2016-08-09 19:49:20,589 WARNING 23852 [py.warnings] importlib.py:9 - /edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/require/helpers.py:4: RemovedInDjango19Warning: django.utils.importlib will be removed in Django 1.9.
from django.utils.importlib import import_module
2016-08-09 19:49:26,428 INFO 23852 [dd.dogapi] dog_stats_api.py:66 - Initializing dog api to use statsd: localhost, 8125
Re-indexing all courses might be a time consuming operation. Do you want to continue? [y/N] y
2016-08-09 19:49:38,254 INFO 23852 [urllib3.connectionpool] connectionpool.py:214 - Starting new HTTP connection (1): localhost
2016-08-09 19:49:38,342 INFO 23852 [elasticsearch] base.py:63 - HEAD http://localhost:9200/courseware_index [status:200 request:0.088s]
No handlers could be found for logger "elasticsearch.trace"
2016-08-09 19:49:38,425 INFO 23852 [urllib3.connectionpool] connectionpool.py:214 - Starting new HTTP connection (1): localhost
2016-08-09 19:49:38,436 INFO 23852 [elasticsearch] base.py:63 - HEAD http://localhost:9200/courseware_index [status:200 request:0.011s]
2016-08-09 19:49:38,551 WARNING 23852 [elasticsearch] base.py:82 - GET /courseware_index/_mapping/course_info [status:404 request:0.037s]
2016-08-09 19:49:38,552 ERROR 23852 [search.elastic] elastic.py:385 - error while indexing -
Traceback (most recent call last):
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 363, in index
self._check_mappings(doc_type, source)
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 338, in _check_mappings
for field, value in body.items()
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 339, in <dictcomp>
if (field not in exclude_fields) and (field not in self._get_mappings(doc_type).get('properties', {}))
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 254, in _get_mappings
doc_type=doc_type,
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 313, in get_mapping
'_mapping', doc_type), params=params)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 93, in perform_request
self._raise_error(response.status, raw_data)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
NotFoundError: TransportError(404, u'{"_index":"courseware_index","_type":"_mapping","_id":"course_info","exists":false}')
2016-08-09 19:49:38,568 ERROR 23852 [edx.modulestore] courseware_index.py:635 - Course discovery indexing error encountered, course discovery index may be out of date course-v1:edX+DemoX+Demo_Course
Traceback (most recent call last):
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 631, in index_about_information
searcher.index(cls.DISCOVERY_DOCUMENT_TYPE, [course_info])
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 363, in index
self._check_mappings(doc_type, source)
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 338, in _check_mappings
for field, value in body.items()
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 339, in <dictcomp>
if (field not in exclude_fields) and (field not in self._get_mappings(doc_type).get('properties', {}))
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 254, in _get_mappings
doc_type=doc_type,
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 313, in get_mapping
'_mapping', doc_type), params=params)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 93, in perform_request
self._raise_error(response.status, raw_data)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
NotFoundError: TransportError(404, u'{"_index":"courseware_index","_type":"_mapping","_id":"course_info","exists":false}')
2016-08-09 19:49:38,572 ERROR 23852 [edx.modulestore] courseware_index.py:267 - Indexing error encountered, courseware index may be out of date course-v1:edX+DemoX+Demo_Course - NotFoundError(404, u'{"_index":"courseware_index","_type":"_mapping","_id":"course_info","exists":false}', {u'_type': u'_mapping', u'_id': u'course_info', u'exists': False, u'_index': u'courseware_index'})
Traceback (most recent call last):
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 255, in index
cls.supplemental_index_information(modulestore, structure)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 403, in supplemental_index_information
CourseAboutSearchIndexer.index_about_information(modulestore, structure)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 631, in index_about_information
searcher.index(cls.DISCOVERY_DOCUMENT_TYPE, [course_info])
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 363, in index
self._check_mappings(doc_type, source)
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 338, in _check_mappings
for field, value in body.items()
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 339, in <dictcomp>
if (field not in exclude_fields) and (field not in self._get_mappings(doc_type).get('properties', {}))
File "/edx/app/edxapp/edx-platform/edx-search/search/elastic.py", line 254, in _get_mappings
doc_type=doc_type,
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 313, in get_mapping
'_mapping', doc_type), params=params)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 93, in perform_request
self._raise_error(response.status, raw_data)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
NotFoundError: TransportError(404, u'{"_index":"courseware_index","_type":"_mapping","_id":"course_info","exists":false}')
Traceback (most recent call last):
File "./manage.py", line 116, in <module>
execute_from_command_line([sys.argv[0]] + django_args)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
utility.execute()
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
self.execute(*args, **cmd_options)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
output = self.handle(*args, **options)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/management/commands/reindex_course.py", line 115, in handle
CoursewareSearchIndexer.do_course_reindex(store, course_key)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 375, in do_course_reindex
return cls._do_reindex(modulestore, course_key)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 282, in _do_reindex
indexed_count = cls.index(modulestore, structure_key)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 272, in index
raise SearchIndexingError('Error(s) present during indexing', error_list)
contentstore.courseware_index.SearchIndexingError: Error(s) present during indexing