Logstash - database causing crash
90 views
Skip to first unread message
Petr Řeřicha
Sep 18, 2018, 2:08:38 PM9/18/18
to security-onion
Hello everybody,
I'd like to ask how can I use database for geoip filter. I've created new config file for filtering logs, which worked pretty well I guess. Then I tried to redirect to another database GeoLiteCity.dat in /etc/logstash/ writing this in my filter database => "/etc/logstash/GeoLiteCity.dat and it showed me error in Logstash's file telling me that path is wrong or the file could not be read. I even tried the other database format .mmdb and it's not working for me either. Logstash is running on Docker image and /usr/share/logstash folders are missing. But it still works, I suppose Docker uses some symlinks or something (maybe this files /usr/share/GeoIP/*). My actual question would be, where should I put my .dat file to make it read that geoip database and is it possible to edit MaxMind's database somehow (.dat). Thanks for your help :)
I'd like to ask how can I use database for geoip filter. I've created new config file for filtering logs, which worked pretty well I guess. Then I tried to redirect to another database GeoLiteCity.dat in /etc/logstash/ writing this in my filter database => "/etc/logstash/GeoLiteCity.dat and it showed me error in Logstash's file telling me that path is wrong or the file could not be read. I even tried the other database format .mmdb and it's not working for me either. Logstash is running on Docker image and /usr/share/logstash folders are missing. But it still works, I suppose Docker uses some symlinks or something (maybe this files /usr/share/GeoIP/*). My actual question would be, where should I put my .dat file to make it read that geoip database and is it possible to edit MaxMind's database somehow (.dat). Thanks for your help :)
Wes Lambert
Sep 20, 2018, 12:23:15 PM9/20/18
to securit...@googlegroups.com
Hi Petr,
Is there any reason, you don't want to use the existing source_ip and destination_ip fields (which are already using geoip), or the default GeoLiteCity2 DB?
Thanks,
Wes
On Tue, Sep 18, 2018 at 2:08 PM Petr Řeřicha <petr.r...@gmail.com> wrote:
Hello everybody,
I'd like to ask how can I use database for geoip filter. I've created new config file for filtering logs, which worked pretty well I guess. Then I tried to redirect to another database GeoLiteCity.dat in /etc/logstash/ writing this in my filter database => "/etc/logstash/GeoLiteCity.dat and it showed me error in Logstash's file telling me that path is wrong or the file could not be read. I even tried the other database format .mmdb and it's not working for me either. Logstash is running on Docker image and /usr/share/logstash folders are missing. But it still works, I suppose Docker uses some symlinks or something (maybe this files /usr/share/GeoIP/*). My actual question would be, where should I put my .dat file to make it read that geoip database and is it possible to edit MaxMind's database somehow (.dat). Thanks for your help :)
--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.
Petr Řeřicha
Sep 21, 2018, 11:45:33 AM9/21/18
to security-onion
Dne čtvrtek 20. září 2018 11:23:15 UTC-5 Wes napsal(a):
Hello Wess,
I'm using those fields, but I'd like to edit geoip database because it is very inaccurate. I've already found folders and even database files in Docker image and now I'm trying to figure out how to use another database which is already edited by me. Do you know where to find defined path to the database which is being used? Even if I redirect path in my config file to Docker's image database which is already used, it won't work and Logstash crashes while starting.
Thanks
I'm using those fields, but I'd like to edit geoip database because it is very inaccurate. I've already found folders and even database files in Docker image and now I'm trying to figure out how to use another database which is already edited by me. Do you know where to find defined path to the database which is being used? Even if I redirect path in my config file to Docker's image database which is already used, it won't work and Logstash crashes while starting.
Thanks
Wes Lambert
Sep 24, 2018, 8:46:56 AM9/24/18
to securit...@googlegroups.com
You should be able to override the database path by explicitly declaring it in the config:
Thanks,
Wes
Petr Řeřicha
Sep 24, 2018, 5:24:31 PM9/24/18
to security-onion
Dne pondělí 24. září 2018 7:46:56 UTC-5 Wes napsal(a):
Yes, I should be able to do that, but it seems to cause the problem. If I use .dat or even newer one .mmdb it says that path is wrong or file cannot be opened or something like that. Logstash is one of the containers in docker so it might cause problem too.
Thanks for your patience,
Petr
Thanks for your patience,
Petr
Wes Lambert
Sep 25, 2018, 10:56:43 AM9/25/18
to securit...@googlegroups.com
You need to use a volume to mount the path and then refer to the path that is relative to the containers view:
So, using LOGSTASH_OPTIONS in /etc/nsm/securityonion.conf, if you do something like:
--volume /etc/logstash/mynewgoeipdatabase.mmdb:/mynewgeoipdatabase.mmdb:ro
Then you would refer to it in the config like:
geoip {
source => "source_ip"
database => "/mynewgeoipdatabase.mmdb"
}If you need further assistance, it may behoove you to post on the Elastic forum(s) as well.
Thanks,
Wes
Reply all
Reply to author
Forward
0 new messages