Renaming Json keys

[Sumit Gandhi]

Morning everyone,

I started working on fluentd recently. I want to rename the json keys. I am using the record transformer plugin provided by fluentd. I am able to rename the key but it doesn't remove the original key from the json. Sharing the config snippet -

<filter asa>

  @type record_transformer

  <record>

    server_name "server-asa"

    message_num ${message_number}

  </record>

  remove_keys message_number

</filter>

Is there any directive that can rename the key as well as remove the original one.

Thanks for any help.

Sumit

[Mr. Fiber]

>  it doesn't remove the original key from the json

What does this sentence mean?

message_number still exist in record?

I tried your configuration and it seems work as expected.

- log

2016-10-31 18:19:42 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type forward
  </source>
  <filter asa>
    @type record_transformer
    remove_keys "message_number"

    <record>
      server_name server-asa
      message_num ${message_number}
    </record>

  </filter>
  <match asa>
    @type stdout
  </match>
</ROOT>
2016-10-31 18:19:42 +0900 [info]: listening fluent socket on 0.0.0.0:24224
2016-10-31 18:21:15.000000000 +0900 asa: {"server_name":"server-asa","message_num":"1"}

- test command

echo '{"message_number":"1"}' | fluent-cat asa

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sumit Gandhi]

Hi,

Sorry for the misunderstanding. Yes, the field "message_number" does get removed because of "remove_keys". But if you see, in this case to rename a field(message_number to message_num), I have to first copy it's(message_number) value to message_num and then remove message_number. Can we do both the operations with a single directive. In logstash there is a rename_field option. Is there a similar one in fluentd?

Thanks in advance for any help.

Regards,

Sumit 

[Mr. Fiber]

One plugin exists for renaming keys.

https://github.com/shunwen/fluent-plugin-rename-key

But this is now output plugin.

Implementing filter version is better for data pipeline...

--

[Hec]

For some reason removing a key makes the value of the copied one null

    <filter raw.systemd.*>

      @type record_transformer

      @id remove_systemd_fields

      remove_keys message

      <record>

        log ${record["message"]}

      </record>

    </filter>

log comes out as null. 

Any ideas?

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.