Help Needed: AS400 IBMi CDC Connector: Connectivity Challenge with Debezium
Charan Subramanya
I am reaching out to seek some guidance on a connectivity challenge we are facing while setting up CDC for an AS400 IBMi source system using Debezium.
Here is our situation —
We were initially working with a DB2 connector setup, however, upon further discovery we identified that the actual source system is AS400 IBMi. We then reworked our configuration and set up the AS400 IBMi connector accordingly.
When we attempted to deploy the connector, we encountered request timeout errors — the connector is unable to establish a direct connection to the AS400 IBMi database. After discussing with the database team, we found out that their environment has a middleware tool acting as an intermediary layer between the application and the AS400 IBMi database. All the required certificates are stored within this middleware tool and all database connectivity is routed through it. There is no direct access to the AS400 IBMi database from outside this layer.
Given this setup, we are facing the following challenge:
- The Debezium connector is unable to bypass this middleware layer to connect directly to the database
- We are unsure if there is a supported way to route the connector through the middleware or extract the certificates and configure them directly in the connector
We would love to hear from the community on:
1. Has anyone faced a similar setup where AS400 IBMi access is restricted via a middleware layer?
2. Is there a recommended approach or workaround to establish connectivity in such a scenario?
3. Are there any alternative CDC approaches that could work in this kind of restricted environment?
Any guidance, pointers, or shared experiences would be extremely helpful.
Thank you in advance for your time and support!
Chris Cranford
On behalf of a colleague:
Collect the certificates from the middleware, using openssl:
https://stackoverflow.com/questions/7885785/using-openssl-to-get-the-certificate-from-a-server
However, the big question is whether or not does this middleware offer access to the journals. If that information isn't currently available, then are you able to add that capability yourself? If so, the connector uses a limited set of calls, so the middleware simply acts as a translation layer for collecting table structures and data. But if that isn't possible, then this would be where the middleware becomes the limiting factor.
The other option is to place the CDC connector alongside your middlware and have appropriate security to protect it.
Hope that helps.
-cc
Thank you in advance for your time and support! --
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/08cd86b9-e41c-4fdd-a8fe-e564b261a49en%40googlegroups.com.
Martin Sillence
Charan Subramanya
Let me update on the issue with the Debezium AS400/IBM i connector.
ENVIRONMENT
- Debezium Connector: io.debezium.connector.db2as400.As400RpcConnector
- Confluent Kafka Connect: 7.9.6 (self-managed, standalone mode)
- Java: OpenJDK 21 (Red Hat build 21.0.11+10-LTS)
- OS: Red Hat Linux
- Target: Confluent Cloud Kafka
- IBM i (AS400) Host: *.*.*.*, Port: 992 (SSL only, non-SSL ports blocked)
PROBLEM:
When I register the connector via the Kafka Connect REST API (POST /connectors), the request times out after ~90 seconds and returns:
HTTP 500 - Request timed out
Looking at the Connect worker logs, I can see the following error repeating multiple times before the final timeout:
java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms
This fires 3 times and then Connect gives up with:
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Request timed out
WHAT WE HAVE TRIED
1. Added --add-opens JVM flags for Java 21 module restrictions:
--add-opens java.base/java.util=ALL-UNNAMED
--add-opens java.base/java.lang=ALL-UNNAMED
--add-opens java.base/java.io=ALL-UNNAMED
--add-opens java.base/sun.net.util=ALL-UNNAMED
--add-opens java.base/java.net=ALL-UNNAMED
2. Extracted SSL certificate from PFX file provided by the AS400 team:
openssl pkcs12 -legacy -in as400.pfx -nokeys -out as400.crt -passin file:/tmp/pfxpass.txt
3. Imported certificate into JVM truststore:
keytool -importcert -file as400.crt -alias as400-ibmi \
-keystore /opt/confluent-7.9.6/etc/kafka/as400_truststore.jks \
-storepass changeit -noprompt
4. Confirmed network connectivity:
- telnet to 172.17.18.30 port 992 succeeds
- SSL handshake appears to succeed (no certificate error in logs)
CONNECTOR CONFIGURATION
{
"name": "ace-db2-ibmi-source",
"config": {
"name": "ace-db2-ibmi-source",
"connector.class": "io.debezium.connector.db2as400.As400RpcConnector",
"tasks.max": "1",
"topic.prefix": "db2_",
"database.hostname": "0.0.0.0",
"database.port": "992",
"database.user": "AFLITESTQL",
"database.password": "******",
"database.schema": "LD7DTA",
"secure": "true",
"ssl.truststore.location": "/opt/confluent-7.9.6/etc/kafka/as400_truststore.jks",
"ssl.truststore.password": "******
"ssl.truststore.type": "JKS",
"table.include.list": "LD7DTA.CHDRPF,LD7DTA.ACMVPF",
"snapshot.mode": "initial",
"signal.data.collection": "ASNCDC.DEBEZIUM_SIGNAL",
"producer.override.bootstrap.servers": "<confluent-cloud-bootstrap>:9092",
"producer.override.security.protocol": "SASL_SSL",
"producer.override.sasl.mechanism": "PLAIN",
"producer.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username='<API_KEY>' password='<API_SECRET>';",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": "true",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "true",
"topic.creation.enable": "true",
"topic.creation.default.replication.factor": "1",
"topic.creation.default.partitions": "3",
"errors.log.enable": "true",
"errors.log.include.messages": "true",
"errors.tolerance": "none"
}
}
OBSERVATIONS
1. Other Debezium connectors (PostgreSQL) are working fine on the same Kafka Connect worker — so the issue is specific to the AS400 connector.
2. The AS400 team has confirmed that non-SSL ports are blocked. Only port 992 (SSL) is accessible.
QUESTIONS
1. Is the "idle timeout expired: 30000ms" error typically an SSL certificate trust issue, or does it indicate the IBM i host servers are not running/responding?
2. Is there a specific way to configure the SSL truststore for the Debezium AS400 connector that is different from the standard JVM truststore approach?
3. Are there any known issues with the As400RpcConnector on Java 21 with SSL on port 992?
4. Is the "signal.data.collection" property mandatory for initial snapshot? Could it be causing the timeout if the signal table does not exist on the AS400?
5. Are there any additional connector properties specific to SSL configuration for this connector that we may be missing?
Any guidance or pointers from the community would be greatly appreciated.
Charan Subramanya
Charan Subramanya
Hi Team,
I wanted to follow up regarding the issue we reported with the Debezium AS400 / IBM i connector timing out while attempting to establish connectivity over SSL (port 992).
We have already validated the following from our side:
- SSL certificate imported successfully into the JVM truststore
- Network connectivity to port 992 is working
- Other Debezium connectors are functioning correctly on the same Kafka Connect worker
- Issue appears specific to the AS400 connector during initialization
However, we are still seeing repeated:
java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms
followed by:
HTTP 500 - Request timed out
Could you please help us with any additional guidance on:
- Whether this behavior indicates an IBM i host service issue vs SSL/truststore issue
- Any SSL-specific properties required for As400RpcConnector
- Known compatibility considerations with Java 21
- Any IBM i services/jobs that must be enabled for journal access over SSL
Please let us know if any additional logs or diagnostics are required from our side.
Thanks & Regards
Charan Subramanya
Charan Subramanya
Hi Team,
I am trying to configure the io.debezium.connector.db2as400.As400RpcConnector connector in Kafka Connect for CDC from IBM i / AS400.
The connector plugin loads successfully and appears correctly in /connector-plugins, but connector validation and creation hang indefinitely and eventually return HTTP 500 timeout errors from Kafka Connect REST API.
Environment details:
Kafka Connect: Confluent Platform 7.9.6
Connector: io.debezium.connector.db2as400.As400RpcConnector
Kafka target: Confluent Cloud
Confluent Cloud connectivity is working (other connectors are publishing successfully)
IBM i host is reachable
Only open IBM i port is 992
Ports 446, 8470, 8471 are blocked/unavailable
Connector configuration:
"name": "ace-db2-ibmi-source",
"config": {
"name": "ace-db2-ibmi-source",
"connector.class": "io.debezium.connector.db2as400.As400RpcConnector",
"tasks.max": "1",
"topic.prefix": "db2_",
"database.hostname": "0.0.0.0",
"database.port": "992",
"database.user": "AFLITESTQL",
"database.password": "******",
"database.schema": "LD7DTA",
"secure": "true",
"ssl.truststore.location": "/opt/confluent-7.9.6/etc/kafka/as400_truststore.jks",
"ssl.truststore.type": "JKS",
"table.include.list": "LD7DTA.CHDRPF,LD7DTA.ACMVPF",
"snapshot.mode": "initial",
"signal.data.collection": "ASNCDC.DEBEZIUM_SIGNAL",
"producer.override.bootstrap.servers": "<confluent-cloud-bootstrap>:9092",
"producer.override.security.protocol": "SASL_SSL",
"producer.override.sasl.mechanism": "PLAIN",
"producer.override.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username='<API_KEY>' password='<API_SECRET>';",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": "true",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "true",
"topic.creation.enable": "true",
"topic.creation.default.replication.factor": "1",
"topic.creation.default.partitions": "3",
"errors.log.enable": "true",
"errors.log.include.messages": "true",
"errors.tolerance": "none"
}
}
Observed behavior:
TCP connectivity to <host>:992 succeeds
Connector plugin loads correctly
Validation hangs during:
"The worker is currently performing multi-property validation"Eventually Kafka Connect REST API returns HTTP 500 timeout
Connector never gets registered
No explicit stack trace or validation error is returned
Additional troubleshooting performed:
Reduced config to minimal properties only
Removed:
table.include.list
signal.data.collection
Confluent Cloud producer override configs
Added SSL bypass properties:
database.ssl.validate.server.certificate=false
ibmi.verify.server.certificate=false
ssl.validation=false
as400.ssl.validate=false
Verified plugin loading through /connector-plugins
Verified other connectors can publish successfully to Confluent Cloud
Current suspicion:
The connector may be hanging during IBM i SSL/TLS or protocol negotiation on port 992.
Questions:
Is port 992 officially supported/recommended for As400RpcConnector?
Does the connector expect IBM i database host server / DRDA protocol rather than Telnet SSL on 992?
Are there known TLS/JT400 compatibility issues with older IBM i systems?
Does the connector require any additional IBM i services or ports besides 992?
Is there a way to enable connector-level debug logging specifically for IBM i connection negotiation?
Are there known requirements for truststore format/certificates with this connector?
Any guidance would be greatly appreciated.
Thanks,