my new toy (learning + ES)

[bui]

https://github.com/buixor/nxapi

Still some pending stuff before we can do a proper release, but it's already working & used.

Feedback is more than welcome !

[Didier C.]

I'll definitely have a look at it when I will be able to solve ENOTIME.

Cheers,

--
You received this message because you are subscribed to the Google Groups "naxsi-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to naxsi-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[mex]

i just discovered the elastic-search-thingie, will definetely habve a look

into this.

nice work!

[bui]

You should as well have a look @ kibana (this definitely is one of the reasons I started tooking at naxsi + ES),

it just a mind-blowing stuff to do sexy dashboards :)

[mex]

we kicked splunk last year and are using kibana instead, hahaha

[mex]

hi guys,

one question regarding naxsi / elasticsearch:

we're running a centralized logserver, using rsyslog to transfer

logs, logstash to push them into elasticsearch and kibana

for displaying and searching; a nice intro in how to do it

might be found here:

http://edgeofsanity.net/article/2012/06/17/central-logging-with-open-source-software.html

my question: would it make sense to create a logstash-input-filter

for naxsi-logs, similar to 

http://ddpbsd.blogspot.nl/2011/10/3woo-you-got-your-ossec-in-my-logstash_26.html

to get naxsi-alerts from distribited installations into a centralized

log-server? or am i missing something?

you know. i'm always a fan of getting infos from various sources

into a centralized database for better analysis :)

regards, 

mex

 

[in...@ecsystems.nl]

Op zaterdag 15 maart 2014 10:52:20 UTC+1 schreef Harry Tuttle:

you know. i'm always a fan of getting infos from various sources

into a centralized database for better analysis :)

I would be for that as well but not tied in with naxsi but tied in with nginx (http://nginx.org/en/docs/http/ngx_http_log_module.html) or Lua (maybe agentzh can comment here).

[craiga...@gmail.com]

Hi Mex,

Did you get anywhere with this setup? 

I have just gone down the same route on centralised log server with ELK, but I used logstash-forwarder to transport the logs instead on rsyslog, I have all the data being pushed into kibana, I just need to understand better how to make it more usable and to parse the messages.

Any pointers would be awesome, happy to share what I have done also (although I don't think it is much above a standard setup)

Craig

[bui]

Hi !

If you happen to create a logstash "plugin" for naxsi, please let me know :)

On my side, we use rsyslog + nxtool to inject data into ES and visualize it with kibana, works as well !

cheers,

[mex]

Hi Craig

yes, we're using a log-forwarder too (log-courier on this side) since
we had some issues with logstash-forwarder
https://github.com/driskell/log-courier but found it more convenient
(rsyslog cannot glob() )

once the filters are stable i'd create a pattern/naxsi - plugin

please note: the filter isnt able (atm) to detect multiple events
in one line, but i'm working on it; it catches the first evnet only

but the dasboards are nice :DDDD

i'm writing an extensive article on how to setup this thingie, but it
will take some more
days until this is finished; please find the dashboard an a screenshot attached


my filters:


if [type] == "ngx_err" {

grok {
# naxsi 1 - normal event
match => { "message" => "%{DATA:request_date}
%{DATA:request_date} %{DATA:err_level} %{DATA:pid_1}: %{DATA:pid_2}
NAXSI_FMT: ip=%{IP:PEER_IP}&server=%{HOST:Host}&uri=%{URIPATH:URI}&learning=%{DATA:learning}&vers=%{DATA:naxsi_version}&total_processed=%{NONNEGINT:processed}&total_blocked=%{NONNEGINT:blocked}&block=%{NONNEGINT:block}&cscore0=%{DATA:cscore}&score0=%{DATA:score}&zone0=%{DATA:zone}&id0=%{NONNEGINT:sid}&var_name0=%{DATA:var_name}"
}
add_tag => [ "t_naxsi_event"]
add_field => [ "naxsi_sensor", "%{host}" ]
remove_tag => ["_grokparsefailure"]
}

grok {
# naxsi 2 - build-in events
match => { "message" => "%{DATA:request_date}
%{DATA:request_date} %{DATA:err_level} %{DATA:pid_1}: %{DATA:pid_2}
NAXSI_FMT: ip=%{IP:PEER_IP}&server=%{HOST:Host}&uri=%{URIPATH:URI}&learning=%{DATA:learning}&vers=%{DATA:naxsi_version}&total_processed=%{NONNEGINT:processed}&total_blocked=%{NONNEGINT:blocked}&block=%{NONNEGINT:block}&zone0=%{DATA:zone}&id0=%{NONNEGINT:sid}&var_name0=%{DATA:var_name}"
}
add_tag => [ "t_naxsi_event"]
remove_tag => ["_grokparsefailure"]
add_field => [ "naxsi_sensor", "%{host}" ]
}

mutate {
remove_field => [ "syslog_program", "deliverer", "err_level",
"pid_1", "pid_2", "proccessed", "block", "blocked" ]
join => ["request_date", " "]

}

date {
match => ["request_date", "yyyy/MM/dd HH:mm:ss",
"dd/MMM/YYYY:HH:mm:ss Z", "ISO8601"]
target => "@timestamp"