How to get value from JSON in Fluentd?

48 views
Skip to first unread message

Pietro Zanella

unread,
Feb 20, 2018, 11:13:31 AM2/20/18
to Fluentd Google Group
Hi,
I have the follow JSON:
  
{
   
"activityTimestamp" : "2016-05-13T14:14:57.387Z",

   
"activityType" : "TEST",

   
"description" : "message
",
   "geoIP" : {
     
"anonymizerStatus" : "private",

     
"countryCode" : "GB",

     
"ipAddress" : "127.0.0.1",

   
},

   
"householdId" : "000000",

   
"ipAddress" : "127.0.0.1",
   
...
}

I need to output the timestamp, description and countryCode; can I access to countryCode using Fluentd?
How can I do? Have you any idea?

My <match> tag is:
<match tag.output>
  @type file
  path /home/out/test_
  buffer_path /home/out/buffer
  format csv
  delimiter |
  fields activityTimestamp,description,countryCode
  ...
</match>

Help me!
Thanks in advance!




Mr. Fiber

unread,
Feb 23, 2018, 8:32:07 AM2/23/18
to Fluentd Google Group
Currently, csv formatter doesn't support nested field access.
Applying flatten-hash filter before output is one way: https://github.com/kazegusuri/fluent-plugin-flatten-hash#filter


Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Pietro Zanella

unread,
Feb 26, 2018, 4:02:53 AM2/26/18
to flu...@googlegroups.com
Thanks a lot Masahiro!
I found another solution to get countryCode inside complex json object:

    <filter tag.kafka>
       @type record_transformer
       enable_ruby yes
       auto_typecast yes
       <record>
            country ${record ["geoIP"]["countryCode"]}
       </record>
       ...
    </filter>

Thanks for the support!

--
You received this message because you are subscribed to a topic in the Google Groups "Fluentd Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fluentd/KUOOyx7R4qs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fluentd+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages