Released GeoIP plugin to work together with ElasticSearch + Kibana v3

[y.ken.studio]

I've released a plugin fluent-plugin-geoip to use MaxMind GeoIP.
http://rubygems.org/gems/fluent-plugin-geoip

It is adding information about geographical location from IP address
which has included GeoLite City database for cost free version to use.

# What is ElasticSearch?

ElasticSearch is a database based Apache Lucene like Solr.
http://www.elasticsearch.org/

# What is kibana v3?

Kibana is a browser based analytics and search interface for ElasticSearch.
http://three.kibana.org/

# Quick Guide

Let's store apache access log with geolocation data to ElasticSearch.
After that, you can download kibana-v3 and add map panel for your dashboard.

<source>
type tail
path /var/log/httpd/access.log
pos_file /var/log/td-agent/httpd-access.log.pos
tag apache.access
format apache2
</source>

<match apache.access>
type geoip

# buffering time (default: 60s)
flush_interval 1s

# tag settings
add_tag_prefix es.

# specify geoip lookup field (default: host)
geoip_lookup_key host

# specify adding field and name
enable_key_country_code geoip_country
</match>

<match es.apache.access>
type copy
<store>
type stdout
</store>
<store>
type elasticsearch
host localhost
port 9200
type_name apache
include_tag_key true
tag_key @log_name
logstash_format true
flush_interval 10s
</store>
</match>

For more details, Let's check it out on the respository README.
https://github.com/y-ken/fluent-plugin-geoip

Thank you.

--
Kentaro Yoshida
twitter: https://twitter.com/yoshi_ken
GitHub: https://github.com/y-ken

[Masahiro Nakagawa]

Hi ken,

This is great!

Some companies have already used Fluentd + ElasticSearch + Kibana in production.

geoip plugin helps their log analytics :)

Thanks,

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Kiyoto Tamura]

Hi Ken,

Thanks a lot for this contribution! Do you think it's okay for us to port this to docs.fluentd.org? It would be great if you can translate yourself, but if not, I will find someone in the community (possibly myself) to do it. Of course, the translated article will be attributed to you.

Let me know what you think!