How to conditionally add fields to record?

[Rich Megginson]

I want to add fields to a record only if the corresponding incoming record contains the field with a non-empty value.

For example, in rsyslog I can do something like this:

    if strlen($!_BOOT_ID) > 0 then {
        set $.systemd!t!BOOT_ID = $!_BOOT_ID;
    }

To convert this:

{

...

"_BOOT_ID": "foo",

}

to this:

                    "systemd": {
                        "t": {
                            "BOOT_ID": "foo",

I tried doing this in fluentd:

<filter something**>
  type record_transformer
  enable_ruby
  <record>
    dummy ${((record['systemd'] ||= {})['t'] ||= {})['BOOT_ID'] = record['_BOOT_ID'] if record['_BOOT_ID']}
  </record>

  remove_keys dummy

</filter>

But this doesn't work - it seems that the ${ ... } is evaulated in some context in which the record is a copy?

[Naotoshi Seo]

record-transformer does not support nested keys yet, so I think it is impossible to do with record-transformer plugin.

Writing your own filter plugin should be nice. It is easy, I will show you.

Create filter_some.rb at somewhere like /tmp/plugin as:

```

module Fluent

  class SomeFilter < Filter

    Plugin.register_filter('some', self)

    def filter(tag, time, record)

      new_record = record.dup

      new_record['systemd'] ||= {}

      new_record['systemd']['t'] ||= {}

      new_record['systemd']['t']['BOOT_ID'] ||= record['_BOOT_ID']

      new_record

    end

  end

end

```

Run your fluentd with -p option as

bundle exec fluentd -c example.conf -p plugin

I tried with following example.conf

<source>

  type dummy

  tag something.foo

  auto_increment_key _BOOT_ID

  dummy {"message":"foo"}

</source>

<filter something.**>

  type some

</filter>

<match **>

  type stdout

</match>

I got

2016-08-10 12:42:00 +0900 something.foo: {"message":"foo","_BOOT_ID":0,"systemd":{"t":{"BOOT_ID":0}}}

2016-08-10 12:42:01 +0900 something.foo: {"message":"foo","_BOOT_ID":1,"systemd":{"t":{"BOOT_ID":1}}}

2016-08-10 12:42:02 +0900 something.foo: {"message":"foo","_BOOT_ID":2,"systemd":{"t":{"BOOT_ID":2}}}

2016-08-10 12:42:03 +0900 something.foo: {"message":"foo","_BOOT_ID":3,"systemd":{"t":{"BOOT_ID":3}}}

Regards,

Naotoshi a.k.a. sonots

[Mr. Fiber]

Or use record-modifier filter.

<source>
  @type dummy

  tag something.foo
  auto_increment_key _BOOT_ID
  dummy { "message":"foo"}
</source>

<filter something**>
  @type record_modifier
  remove_keys dummy

  <record>
    dummy ${((record['systemd'] ||= {})['t'] ||= {})['BOOT_ID'] = record['_BOOT_ID'] if record['_BOOT_ID']}
  </record>

</filter>

<match something**>
  @type stdout
</match>

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.