Need help setting up filters for logstash for Mongo 3.0

[Mongo-user1]

Did anyone set up the filters for logstash to parse logs for mongo 3? Format of logs are quite different between Mongo 2 and 3.

[Wan Bachtiar]

Did anyone set up the filters for logstash to parse logs for mongo 3?

Hi,

It’s been a while since you posted the question, have you found a solution for this ?

I ran a quick test for this on latest Logstash currently v2.3.1 and latest MongoDB currently v3.2.5, and found that logstash already has support for MongoDB log v3+ format.

An example conf file:

input {
    file {
        path => "/path/to/mongodb.log"
    }
}
filter {
    grok {
        match => [ "message", "%{MONGO3_LOG}"]
    }
}

Should be able to capture common log fields such as timestamp, severity, components, etc. For example:

{
       "message" => [
        [0] "2016-04-20T16:02:34.328+1000 I COMMAND  [conn3] command test.$cmd command: isMaster { isMaster: 1.0 } keyUpdates:0 writeConflicts:0 numYields:0 reslen:163 locks:{} protocol:op_command 0ms",
        [1] "command test.$cmd command: isMaster { isMaster: 1.0 } keyUpdates:0 writeConflicts:0 numYields:0 reslen:163 locks:{} protocol:op_command 0ms"
    ],
      "@version" => "1",
    "@timestamp" => "2016-04-20T06:02:35.213Z",
          "path" => "/path/to/mongodb.log",
          "host" => "hostname01",
     "timestamp" => "2016-04-20T16:02:34.328+1000",
      "severity" => "I",
     "component" => "COMMAND",
       "context" => "conn3"
}

For more patterns spec see logstash-patterns-core: mongodb_spec.rb

If you have further questions on logstash, you may get faster responses by posting a question on logstash forum. See also Logstash Help.

Best regards,

Wan.

​