Wazuh on Elasticsearch 6.0

[Sumesh MS]

Hello Guys

I have upgraded my elasticsearch version from 5.6 to 6.0 . After than indexes are not creating.

The error shows :

[2017-11-20T10:49:33,749][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"wazuh-alerts-2017.11.20", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x2cf20e78>], :response=>{"index"=>{"_index"=>"wazuh-alerts-2017.11.20", "_type"=>"doc", "_id"=>"aJ1n2F8Bpg-aCG3eVpLC", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [wazuh-alerts-2017.11.20] as the final mapping would have more than 1 type: [wazuh-*, doc]"}}}}

It seems more like an elasticsearch issue where ES supports only one type

https://discuss.elastic.co/t/6-0-0alpha-packetbeat-multiple-types/87220

Is anyone  facing similar issues?

Regards
Sumesh




 

[Sumesh MS]

Following with the issue

The logoutput is giving when I have updates the ES6 index templates ( version 6 ) from  https://github.com/wazuh/wazuh/tree/3.0/extensions/elasticsearch

[2017-11-22T09:24:34,701][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"wazuh-alerts-2.1.1-2017.11.22", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x2f141934>], :response=>{"index"=>{"_index"=>"wazuh-alerts-2.1.1-2017.11.22", "_type"=>"doc", "_id"=>"Olpm4l8B8FIdclYYMrZp", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"[data] is defined as a field in mapping [doc] but this name is already used for an object in other types"}}}}

Regards
Sumesh

[Javier Castro]

Hi Sumesh,

did you already have an Elasticsearch installation with Wazuh data? it's not as simple to transition because in version 3.0 our alerts mapping has changed (now user defined fields, along with some others, go into a structure called data) and you will probably have to reindex your data.

Also the whole configuration for the Elastic Stack has changed to reflect that so you are probably including something wrong at some point in the data flow, for example, Logstash no longer uses a template in its configuration.

Regards.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/566378c1-7a26-4de5-a659-7e6dba05edc7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Sumesh MS]

Hi Javier

You are right, it was a mismatch issue with the existing data.

"Logstash no longer uses a template in its configuration." -  Does it mean indext templates are no long valid or required?

Regards
Sumesh MS

[Javier Castro]

Hi Sumesh,

we are still using templates, but instead of trying to insert them everywhere in the Elastic ecosystem (which ends up being more a source of problems than a solution), we've decided to install them where they truly belong: Elasticsearch.

That way we are more certain about why do some errors happen: we look at one place, not several.

Regards.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e7383d08-1a76-4806-bfde-72285e370732%40googlegroups.com.