Can't connect to dockerized kafka container in OS X

[İnanç Gümüş]

Hi.

I'd been having problems launching Kafka (more about this in this recent thread: https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/confluent-platform/DRd_XfeW0C8).

Now, I launched Kafka, however, I can't connect it:

# the kafka container uses this port
$ telnet localhost 29092
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host

PS: I'm trying this docker-compose.yml: 

https://github.com/confluentinc/cp-docker-images/blob/v3.2.1/examples/kafka-single-node/docker-compose.yml

This is my altered docker-compose.yml file:

I tried to expose the ports, however, I think, in network host mode, it can't be done. 

---
version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    network_mode: host
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
    extra_hosts:
      - "moby:127.0.0.1"

  kafka:
    image: confluentinc/cp-kafka:latest
    network_mode: host
    ports:
      - "29092:29092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: localhost:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092
    extra_hosts:
      - "moby:127.0.0.1"

[Michael Noll]

If you are running Docker on Mac OS, then (slightly simplified) you must telnet to the IP address of your Docker Machine, which is the Linux VM in which your actual containers run.

Think:  your Mac laptop -> Docker Machine (Linux VM) -> Docker containers.

If you configure your containers to listen on "localhost:29092" (for example), then Docker will make them listen on localhost:29092 in your Linux VM, not on your Mac laptop!

What you would need to do is sth like:

    # You should have already done this step, which creates a Docker Machine (VM) named "confluent"

    $ docker-machine create --driver virtualbox --virtualbox-memory 6000 confluent

    # Next, configure your terminal window to attach it to your new Docker Machine.

    $ eval $(docker-machine env confluent)

    # Now you can talk to e.g. the broker, but you must telnet to the IP of the Docker Machine.

    # Typically, the Docker Machine IP is sth like 192.168.99.100.

    $ telnet `docker-machine ip confluent` 29092

Note: When talking to a broker, even if you can successfully telnet to its port, this isn't a proper test that you can successfully write messages to it.  If telnet works but producing messages to the broker's endpoint does not, you must configure the broker setting `advertised.listeners` appropriately.  This is not an issue of Kafka, but a networking issue caused by how Docker works on Macs (with its awkward requirement of running your containers inside a Linux VM).

Hope this helps,

Michael

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/3fbd04e4-9177-45e7-86e1-1131853f99d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Michael G. Noll

Product Manager | Confluent

+1 650 453 5860 | @miguno

Follow us: Twitter | Blog