Confluent & Apache NiFi

[Cerulean Blue]

I'm curious of any of the Confluent folks has worked with Apache NiFi? I'm in need of an NiFi processor that writes to Kafka utilizing the scheme registry.

Has anyone worked with this?

Thanks.

Jeff
Sent from my iPhone

[Ewen Cheslack-Postava]

I think the current support for Kafka in NiFi, via the GetKafka and PutKafka processors, currently only support Strings, which the processor internally converts to byte[] encoded as UTF8. It doesn't look like it's currently possible to set the serializers, which would be required to integrate easily with Confluent's schema registry using the existing tools in Confluent Platform.

In fact, it looks like PutKafka is still using the old producer API, so the new producers serializer provided with Confluent Platform wouldn't be trivial to integrate.

However, the existing processor might be the best place to start if you wanted to enable this support. If the processor can be generalized to support different types of data and serializers with a couple of additional settings for specifying the serializer, it might not be that big a modification to integrate with the schema registry just by setting the serializers and the URL to the schema registry.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/3C33DD0A-1B88-4FBD-832F-CE09A09580DE%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen