elasticsearch 5.2.0에서 _reindex 시 "java.lang.RuntimeException: Use mecab-ko-dic 2.0.0 or higher." 라는 메시지와 함께 인덱싱에 실패하는 이유...

[성유경]

안녕하세요~

Elasticsearch 5.2.0 버전에서 mecab-ko-dic 2.0.1 버전을 이용한 형태소분석기를 사용중인데요,

(그리고 mecab-0.996-ko-0.9.2와 mecab-java-0.996 사용 중)

그동안 한글 형태소 분석 검색에는 문제가 없었는데

elasticsearch의 _reindex를 하다가 아래와 같은 에러가 발생하면서 reindex가 중단되어 문의드립니다.

소스를 열어보니 Pos.java에서 품사태그를 분리하는 과정 중에 품사태그가 예측한대로 안들어와 ArrayIndexOutOfBoundException이 발생한 것 같은데요,

1. 입력된 값의 형태가 어떻기 때문인지

2. 만약에 입력된 값의 형태가 잘못되어 있다면 그게 메시지대로 mecab-ko-dic의 버전 문제인지

알고 싶습니다.

테스트할 수 있는 방법이라도 알려주시면 감사하겠습니다!!

failed to execute bulk item (index) index {[new_dmap][crawl_doc][81d8d664a28d38c50486384c771fd1f5], source[{"comment_count":0,"depth2_seq":12,"doc_title":"제1회 수학일기 공모전 우수작 선정결과 (2016년 상반기)","item_seq":392,"item_nm":"A4","depth3_seq":409,"doc_url":"http://cafe.naver.com/seoncam/322","customer_seq":null,"doc_content":"제1회 수학일기 공모전 우수작 선정결과 (2016년 상반기) 발표합니다.   접수결과, 총 25개 작품이 접수되었고, 평가결과, 제출학년이 다양하였으며, 학년에 따른 적용 내용 또한 다양하였음. 이번 공모전에서는 당초 대상 1명, 우수상 1명을 우수상 3명으로 변경하여 시상하며, 장려상을 7명으로 시상합니다.   우수상 3명 : A고 K채원, J중 H가현, J중 N가림 장려상 7명 : H중 O민주, H중 J성호, C초 S지현, H초 J건희, A초 Y채이, C초 S지희, M초 J미래   축하합니다.   수학일기 공모전 하반기에 또 만나요.   세종시 첫마을 한솔중앞 (SF) 세종프렌토 교육문의 044-999-1230   -------------------------- ※ 수학일기 공모전 참고하기   형식 : A4용지 1장 분량의 재미있는 삽화와 스토리. 스토리 전개에 쓰인 교과서 관련 내용 정리. 특별한 형식과 틀은 없으며 웹툰도 가능. 자신이 직접 작성한 것이어야 함. 심사기준 : 진지하거나 재미있거나 얼렁뚱땅, 황당무계한 이야기 모두 괜찮음. 실생활에 관련된 나의 실제 사례 내용도 좋음. 단, 전체적인 스토리가 자연스럽고, 재치 있게 전개되었는지와 수학 정리 내용을 잘 반영한 이야기인지가 중요하겠습니다. 다양한 예시들은 수학일기 공모게시판 (또는 테마별 시리즈 게시판)을 참고하세요.","depth3_nm":"네이버","depth1_nm":"포털","depth2_nm":"카페","doc_datetime":"2016-05-25 00:00:00","like_count":0,"item_grp_nm":"아우디","pub_time":"161021","doc_id":"81d8d664a28d38c50486384c771fd1f5","share_count":0,"item_grp_seq":16,"depth1_seq":2,"doc_writer":"hifivey","locations":null,"customer_id":"innocean","view_count":0,"dislike_count":0}]}

java.lang.RuntimeException: Use mecab-ko-dic 2.0.0 or higher.

at org.bitbucket.eunjeon.mecab_ko_lucene_analyzer.Pos.<init>(Pos.java:108) ~[?:?]

at org.bitbucket.eunjeon.mecab_ko_lucene_analyzer.TokenGenerator.getAnalyzedPoses(TokenGenerator.java:91) ~[?:?]

at org.bitbucket.eunjeon.mecab_ko_lucene_analyzer.Eojeol.append(Eojeol.java:44) ~[?:?]

at org.bitbucket.eunjeon.mecab_ko_lucene_analyzer.TokenGenerator.getNextEojeolTokens(TokenGenerator.java:119) ~[?:?]

at org.bitbucket.eunjeon.mecab_ko_lucene_analyzer.MeCabKoTokenizer.incrementToken(MeCabKoTokenizer.java:120) ~[?:?]

at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:716) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1575) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1320) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:558) ~[elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:520) ~[elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:409) ~[elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:556) ~[elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:546) ~[elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:213) ~[elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:348) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.bulk.TransportShardBulkAction.index(TransportShardBulkAction.java:155) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.bulk.TransportShardBulkAction.handleItem(TransportShardBulkAction.java:134) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.bulk.TransportShardBulkAction.onPrimaryShard(TransportShardBulkAction.java:120) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.bulk.TransportShardBulkAction.onPrimaryShard(TransportShardBulkAction.java:73) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:76) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:49) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:914) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:884) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:327) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:262) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:864) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:861) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:142) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1652) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:873) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:92) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:279) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:258) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:250) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:237) [x-pack-5.2.0.jar:5.2.0]

at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:298) [x-pack-5.2.0.jar:5.2.0]

at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:610) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) [elasticsearch-5.2.0.jar:5.2.0]

at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.2.0.jar:5.2.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]