Testing Kafka streams using Avro

[Martin Maier]

Hi there,

in our system, we develop a lot of Kafka streams and now finally started using Quarkus for this.

In my experience, Kafka streams are mostly developed fast, but the integration is often a bit tricky. So we found a best practice for us to develop them driven by integration tests.

In https://quarkus.io/guides/kafka#starting-kafka-in-a-test-resource it's mentioned how it could be done, but in our case we use Avro for de-/serialization and so we need a schema registry for our integration tests.

We ended up developing a small library that simplifies writing integration tests for this setup.

https://github.com/Ippen-Digital/quarkus-kafka-avro-integration-testing

So my question to this community is if there are more teams with this use case and if there is a need to extend this library for your needs.

We are really consuming a lot from this community, but want to give something back. Maybe this could be an opportunity?

Greetings

[Martin Maier]

Things coming in my mind were:

- also provide client functionality for non-Avro setups? But maybe the complexity then isn´t that big as people will need an extra library for this?

- add examples for other common use cases like directly using Kafka streams without connectors, joining kafka streams, using compacted topics or maybe just other more complex scenarios.