How to debug a connector using docker images?

[Miguel Silvestre]

Hi all,

I'm developing a connector to kafka connector. 

I have the following docker-compose in order to get things running:

version: '3'

services:

  zookeeper:

    image: confluentinc/cp-zookeeper:5.5.3

    environment:

      ZOOKEEPER_CLIENT_PORT: 2181

      zk_id: '1'

    ports:

      - '2181:2181'

  kafka:

    hostname: kafka

    image: confluentinc/cp-enterprise-kafka:5.5.3

    links:

      - zookeeper

    ports:

      - '9092:9092'

    environment:

      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'

      KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://:9092'

      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1

      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1

  schema-registry:

    image: confluentinc/cp-schema-registry:5.5.3

    links:

      - kafka

      - zookeeper

    ports:

      - '8081:8081'

    environment:

      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: 'zookeeper:2181'

      SCHEMA_REGISTRY_HOST_NAME: schema-registry

  control-center:

    image: confluentinc/cp-enterprise-control-center:5.5.3

    depends_on:

      - zookeeper

      - kafka

      - schema-registry

    ports:

      - '9021:9021'

    environment:

      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'kafka:9092'

      CONTROL_CENTER_ZOOKEEPER_CONNECT: 'zookeeper:2181'

      CONTROL_CENTER_CONNECT_CLUSTER: 'connect:8083'

      CONTROL_CENTER_REPLICATION_FACTOR: 1

      CONTROL_CENTER_CONFLUENT_CONTROLCENTER_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081'

      CONTROL_CENTER_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081'

  connect:

    image: confluentinc/cp-kafka-connect:5.5.3

    hostname: connect

    container_name: connect

    depends_on:

      - zookeeper

      - kafka

    ports:

      - 8083:8083

      - 5005:5005

    environment:

      CONNECT_BOOTSTRAP_SERVERS: 'kafka:9092'

      CONNECT_REST_PORT: 8083

      CONNECT_REST_ADVERTISED_HOST_NAME: 'connect'

      CONNECT_GROUP_ID: compose-connect-group

      CONNECT_CONFIG_STORAGE_TOPIC: connect-configs

      CONNECT_OFFSET_STORAGE_TOPIC: connect-offsets

      CONNECT_STATUS_STORAGE_TOPIC: connect-status

      CONNECT_KEY_CONVERTER_SCHEMAS_ENABLE: 'false'

      CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE: 'false'

      CONNECT_INTERNAL_KEY_CONVERTER: 'org.apache.kafka.connect.json.JsonConverter'

      CONNECT_INTERNAL_VALUE_CONVERTER: 'org.apache.kafka.connect.json.JsonConverter'

      CONNECT_KEY_CONVERTER: 'org.apache.kafka.connect.converters.ByteArrayConverter'

      CONNECT_VALUE_CONVERTER: 'org.apache.kafka.connect.converters.ByteArrayConverter'

      CONNECT_LOG4J_ROOT_LOGLEVEL: 'INFO'

      CONNECT_LOG4J_LOGGERS: 'org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR'

      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: '1'

      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: '1'

      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: '1'

      CONNECT_PLUGIN_PATH: '/usr/share/java'

      CONNECT_DEBUG: y

      DEBUG_SUSPEND_FLAG: y

    volumes:

      - ./target/kafka-connect-gitlab-0.0.1-SNAPSHOT-development/share/java/kafka-connect-gitlab:/usr/share/java/kafka-connect-gitlab/

  gitlab:

    image: gitlab/gitlab-ce:13.6.5-ce.0

    environment:

      GITLAB_OMNIBUS_CONFIG: |

        gitlab_rails['initial_root_password'] = "password"

        gitlab_rails['initial_shared_runners_registration_token'] = "token"

    ports:

      - 8929:80

But when I try to remotely debug it, my debugger can not connect to port 5005. So I guess there's something missing?

There's other thing. When I install the connector on kafka-connect the task is always UNASSIGNED. Why is that?

Thank you!

[Miguel Silvestre]

I can debug this using the confluent distribution of connect:

export CONNECT_DEBUG=y
export DEBUG_SUSPEND_FLAG=y

./confluent local services connect start

This way I can attach the debugger to port 5005. Only when using the docker image I can't. Isn't it available for docker images?

Thanks.

--

Miguel Silvestre

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/66685824-4ce1-41a8-b13f-58539783ebb3n%40googlegroups.com.

[Miguel Silvestre]

No one?

It is not possible?

--

Miguel Silvestre