Postfix log consolidation..

[Thomas Johnson]

I'm very new to fluentd, and just starting to play around with it.

I'd like to use it to store information generated by postfix.  One email can generate as few as a dozen lines of logging data, or as many as several hundred.  I'd like to do something to consolidate or coalesce the logging data so that I don't have to store every individual line of logging data in a database.

Has anyone done a postfix plugin for fluentd?  Or have any recommendations or suggestions on the best way to approach this?

Thanks-

[kiyoto]

Hi Thomas,

asaik, I don't think there is a plugin specific to postfix per se (here is the list of all rubygems-registered plugins: http://fluentd.org/plugin/), but some users have used fluentd to parse Postfix mail logs.

For example, these blog articles (albeit written in Japanese) explains how he uses fluentd to parse postfix maillogs and store them in redis:

part 1: http://inokara.hateblo.jp/entry/2013/07/11/071017
part 2: http://inokara.hateblo.jp/entry/2013/07/13/222120

In particular, he uses the in_tail plugin to parse maillogs (I am copy-and-pasting the config file snippet from part 1 here)

<source>
  type tail
  format /^(?<date>[^ ]+) (?<host>[^ ]+) (?<process>[^:]+): (?<message>((?<key>[^ :]+)[ :])? ?((to|from)=<(?<address>[^>]+)>)?.*)$/
  path /var/log/mail.log
  tag redis.maillog
  pos_file /tmp/fluent.pos
</source>

Let me know if you have further questions.

Kiyoto