Re: td-agent fails to launch, can't find plugin json-schema-filter after install

[Mr. Fiber]

`json_schema_filter` is filter, not output plugin.

Use <filter>, not <match>.

http://docs.fluentd.org/articles/config-file

Masahiro

On Mon, Oct 17, 2016 at 11:46 AM, James T. <kcp...@gmail.com> wrote:

Hello! Has anyone had any luck trying to install the json-schema-filter plugin and getting it to work? I've tried installing the plugin with the td-agent-gem install method and the regular gem install method, but when I try to launch td-agent with config that has the json-schema specified, the td-agent fails to start. I get:

Starting td-agent: 2016-10-16 19:40:43 -0700 [error]: fluent/supervisor.rb:373:rescue in main_process: config error file="/etc/td-agent/td-agent.conf" error="Unknown output plugin 'json_schema'. Run 'gem search -rd fluent-plugin' to find plugins"

When I do the gem search above, the filter comes back in the list of available gems:

fluent-plugin-json-schema-filter (0.0.3)

    Author: Anthony Johnson

    Homepage: https://github.com/ansoni/fluent-plugin-json-schema-filter

    Fluentd Filter plugin to validate incoming records against a json

    schema

My config file that contains the 'match' for the json-schema plugin is as follows (copied from the github repo):

<match **>

  @type json_schema

  mode enrich

  schema_file /home/testuser/schemas/test.json

  enrich_valid valid

  enrich_invalid invalid

  add_validation_error true # results in a validation-error key being added to the record

</match>

Any help would be really appreciated! Thank you!

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James T.]

Thank you for the response! I tried to do that and changed the config to:

<filter **>

@type json_schema

mode enrich

schema_file /home/testuser/schemas/test.json

enrich_valid valid

enrich_invalid invalid

add_validation_error true # results in a validation-error key being added to the record

</filter>

 <match **>

@type file

path /var/log/fluent/filtered-input.log

</match> 

That works and td-agent starts, however it looks like the filter is not getting applied as specified. I then have a 'match' to test everything after the filter, and dump it all to a file, but nothing is getting to the log. If I comment out the <filter> section, data starts going to the log again. I presume its a configuration issue with the 'filter' piece, but it looks to be correct as seen above? Do not know what is missing...?

Thank you!

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.

[Mr. Fiber]

I don't have a experience of this plugin, so I can't reply to the behaviour.

If you debug the situation but no luck,

asking it to plugin author with configuration and actual dataset is better.

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.