Kafka brokers with different bootstrap servers.

[Anderson Soares]

Hi All,

I've got a scenario where I need to have two Knative brokers however each will be pointing to different bootstrap servers.

kubectl get brokers

broker01   http://kafka-broker-ingress.knative-eventing.svc.cluster.local/default/broker01   8d    True    

broker02   http://kafka-broker-ingress.knative-eventing.svc.cluster.local/default/broker02   11m   True

My bootstrap servers:

cluster01-bootstrap.kafka:9092

cluster02-bootstrap.kafka:9092

The point here is that both brokers use the same kafka-broker-config configmap.

I know so far that I can have multiple bootstrap servers by just separating with comma.

bootstrap.servers: "cluste01-bootstrap.kafka:9092, cluster02-bootstrap.kafka:9092".

However what about if I want that each broker use a different bootstrap server?

All I found here: https://knative.dev/docs/eventing/broker/kafka-broker/

Thanks in advance.

[Pierangelo Di Pilato]

Hi Anderson,

On Tue, Nov 16, 2021 at 6:04 PM Anderson Soares
<soaresan...@gmail.com> wrote:
>
> Hi All,
>
> I've got a scenario where I need to have two Knative brokers however each will be pointing to different bootstrap servers.
>
> kubectl get brokers
> broker01 http://kafka-broker-ingress.knative-eventing.svc.cluster.local/default/broker01 8d True
> broker02 http://kafka-broker-ingress.knative-eventing.svc.cluster.local/default/broker02 11m True
>
> My bootstrap servers:
> cluster01-bootstrap.kafka:9092
> cluster02-bootstrap.kafka:9092
>
> The point here is that both brokers use the same kafka-broker-config configmap.
>
> I know so far that I can have multiple bootstrap servers by just separating with comma.
> bootstrap.servers: "cluste01-bootstrap.kafka:9092, cluster02-bootstrap.kafka:9092".
>
> However what about if I want that each broker use a different bootstrap server?
>

each broker can reference a different ConfigMap that has the same
fields of `kafka-broker-config` but it doesn't have to be
`kafka-broker-config`.

# ... Some Broker fields omitted
spec:
config:
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config-cluster2
namespace: knative-eventing

is a valid `config` for the Kafka Broker, then in
`kafka-broker-config-cluster2` you will set:

bootstrap.servers: "cluster02-bootstrap.kafka:9092"

> All I found here: https://knative.dev/docs/eventing/broker/kafka-broker/
>
>
> Thanks in advance.
>

If this wasn't clear from the documentation, feel free to contribute!

> --
> You received this message because you are subscribed to the Google Groups "Knative Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to knative-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/knative-users/c8fab11f-6927-4165-b20c-6717e10c95c8n%40googlegroups.com.

--

Pierangelo Di Pilato

Software Engineer

Red Hat, Inc

https://www.redhat.com/