K8s and external consumer access to Kafka

[mszd...@gmail.com]

Does anyone have a YAML configuration to allow  a consumer connection to Kafka from outside the k8s cluster?

I have tried using NodePort  for this in the Kafka YAML file  but Kafka throws errors:

Yml:

apiVersion: v1
kind: Service
metadata:
  name: kafka
  namespace: voltha
spec:
  type: NodePort
  selector:
    app: kafka
  ports:
  - protocol: TCP
    port: 9092
    targetPort: 9092

    nodePort: 30092

......

Error creating broker listeners from 'PLAINTEXT://:tcp://10.104.210.190:9092': No security protocol defined for listener PLAINTEXT://:TCP

I have not yet tried any changes to nginx or zookeeper yaml configurations

thanks,

m

[David Bainbridge]

I am attempting to understand what you are attempting to accomplish. Presumably, if you have VOLTHA running under k8s, you can get a list of services offered similar to the following:

$ kubectl get --all-namespaces svc

NAMESPACE     NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                        AGE

default       etcd-restore-operator       ClusterIP   10.233.21.100   <none>        19999/TCP                                                      32m

default       kubernetes                  ClusterIP   10.233.0.1      <none>        443/TCP                                                        40m

default       voltha-kafka                ClusterIP   10.233.7.46     <none>        9092/TCP                                                       32m

default       voltha-kafka-headless       ClusterIP   None            <none>        9092/TCP                                                       32m

default       voltha-zookeeper            ClusterIP   10.233.59.49    <none>        2181/TCP                                                       32m

default       voltha-zookeeper-headless   ClusterIP   None            <none>        2888/TCP,3888/TCP                                              32m

kube-system   kube-dns                    ClusterIP   10.233.0.3      <none>        53/UDP,53/TCP                                                  39m

kube-system   kubernetes-dashboard        ClusterIP   10.233.18.229   <none>        443/TCP                                                        39m

kube-system   tiller-deploy               ClusterIP   10.233.4.237    <none>        44134/TCP                                                      39m

voltha        default-http-backend        ClusterIP   10.233.27.120   <none>        80/TCP                                                         32m

voltha        freeradius                  ClusterIP   None            <none>        1812/UDP,1813/UDP,18120/TCP                                    32m

voltha        grafana                     ClusterIP   None            <none>        8883/TCP,2003/TCP,2004/TCP,8126/TCP,8125/TCP                   32m

voltha        ingress-nginx               NodePort    10.233.34.209   <none>        80:30080/TCP,443:30443/TCP                                     32m

voltha        netconf                     ClusterIP   None            <none>        830/TCP                                                        32m

voltha        vcli                        NodePort    10.233.46.189   <none>        5022:30110/TCP                                                 32m

voltha        vcore                       ClusterIP   None            <none>        8880/TCP,18880/TCP,50556/TCP                                   32m

voltha        voltha                      NodePort    10.233.29.67    <none>        8882:30125/TCP,8001:30881/TCP,8443:31413/TCP,50555:31431/TCP   32m

In this listing, you can see that kafka is exposed on 10.233.7.46:9092. When you are on a node in the cluster are you able to connect as a kafka client on that IP and port?

--
You received this message because you are subscribed to the Google Groups "VOLTHA Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to voltha-discus...@opencord.org.
To post to this group, send email to voltha-...@opencord.org.
Visit this group at https://groups.google.com/a/opencord.org/group/voltha-discuss/.
To view this discussion on the web visit https://groups.google.com/a/opencord.org/d/msgid/voltha-discuss/309012df-0567-4a7c-8ea8-0284a3869ff6%40opencord.org.
For more options, visit https://groups.google.com/a/opencord.org/d/optout.

[David Bainbridge]

As an added note, I found https://github.com/kubernetes/charts/blob/master/incubator/kafka/README.md. It describes how to expose the standard Kafka chart outside of k8s. I played with it a very little and couldn't get it working, where I could get an SSH tunnel working. But I suspect somewhere in this README is the "correct" way to expose Kafa correctly outside of k8s.