Bug with Suricata and Logstash?

[Peter Keenan]

I noticed this in Beta 2 when I switched from snort to suricata IDS engine:
-------
[2017-12-29T17:15:44,224][DEBUG][logstash.filters.grok ] Running grok filter {:event=>2017-12-29T13:45:12.738Z 172.17.0.1 SURICATA UDP packet too small}
[2017-12-29T17:15:44,256][DEBUG][logstash.filters.grok ] Event now: {:event=>2017-12-29T13:45:12.738Z 172.17.0.1 SURICATA UDP packet too small}
[2017-12-29T17:15:44,293][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"undefined method `>' for nil:NilClass", "backtrace"=>["(eval):782689:in `initialize'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):782687:in `initialize'", "org/jruby/RubyProc.java:281:in `call'", "(eval):782739:in `initialize'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):782729:in `initialize'", "org/jruby/RubyProc.java:281:in `call'", "(eval):54352:in `filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:398:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:379:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in `start_workers'"]}
[2017-12-29T17:15:44,382][INFO ][logstash.inputs.tcp ] Automatically switching from json to json_lines codec {:plugin=>"tcp"}
[2017-12-29T17:15:44,386][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@charset = "UTF-8"
[2017-12-29T17:15:44,387][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@id = "json_lines_56f87b97-1127-4ead-abe7-c4df6768185e"
[2017-12-29T17:15:44,387][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@enable_metric = true
[2017-12-29T17:15:44,387][DEBUG][logstash.codecs.jsonlines] config LogStash::Codecs::JSONLines/@delimiter = "\n"
[2017-12-29T17:15:44,469][DEBUG][io.netty.util.internal.logging.InternalLoggerFactory] Using SLF4J as the default logging framework
[2017-12-29T17:15:44,483][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NoMethodError: undefined method `>' for nil:NilClass>, :backtrace=>["(eval):782689:in `initialize'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):782687:in `initialize'", "org/jruby/RubyProc.java:281:in `call'", "(eval):782739:in `initialize'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):782729:in `initialize'", "org/jruby/RubyProc.java:281:in `call'", "(eval):54352:in `filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:398:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:379:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in `start_workers'"]}
[2017-12-29T17:15:44,485][DEBUG][io.netty.channel.MultithreadEventLoopGroup] -Dio.netty.eventLoopThreads: 8
---------------------------------------------

I upgraded to beta 3 and am getting the same result, anyone else seeing that?

[Doug Burks]

Hi Peter,

Yes, this is a known issue and it should be resolved in our next release:
https://github.com/Security-Onion-Solutions/security-onion/issues/1179

In the meantime, you could try replacing your
/etc/logstash/conf.d/1033_preprocess_snort.conf with the newer version
here:
https://github.com/dougburks/elastic-test/blob/master/configfiles/1033_preprocess_snort.conf

Then restart Logstash as follows:
sudo docker restart so-logstash

> --
> Follow Security Onion on Twitter!
> https://twitter.com/securityonion
> ---
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks

[Peter Keenan]

On Friday, December 29, 2017 at 4:00:29 PM UTC-5, Doug Burks wrote:
> Hi Peter,
>
> Yes, this is a known issue and it should be resolved in our next release:
> https://github.com/Security-Onion-Solutions/security-onion/issues/1179
>
> In the meantime, you could try replacing your
> /etc/logstash/conf.d/1033_preprocess_snort.conf with the newer version
> here:
> https://github.com/dougburks/elastic-test/blob/master/configfiles/1033_preprocess_snort.conf
>
> Then restart Logstash as follows:
> sudo docker restart so-logstash
>

Thanks Doug, I replaced the file and restarted, so far so good!