debezium server for non kafka deploy using sql server

[Nattapat Saenharn]

Hi,

I just started using debezium server for my project and I did some reseach about CDC for SQL server without Kafka. I'm also using Nats.io Jetsteam for sink but I found error when start debezium

I found the same error with this link (https://issues.redhat.com/browse/DBZ-6093?jql=project%20%3D%20DBZ%20AND%20text%20~%20%22503%22) and I tried to solve it by using the suggested solution in the link but it can't be solved. 

Below is the error from debezium server
```
{
  "timestamp": "2023-06-28T06:33:29.394Z",
  "sequence": 162,
  "loggerClassName": "org.slf4j.impl.Slf4jLogger",
  "loggerName": "io.debezium.server.ConnectorLifecycle",
  "level": "ERROR",
  "message": "Connector completed: success = 'false', message = 'Stopping connector after error in the application's handler method: java.io.IOException: Error Publishing: 503 No Responders Available For Request', error = 'io.debezium.DebeziumException: java.io.IOException: Error Publishing: 503 No Responders Available For Request'",
  "threadName": "pool-7-thread-1",
  "threadId": 28,
  "mdc": {
   
  },
  "ndc": "",
  "hostName": "2df8491342e9",
  "processName": "io.debezium.server.Main",
  "processId": 1,
  "exception": {
    "refId": 1,
    "exceptionType": "io.debezium.DebeziumException",
    "message": "java.io.IOException: Error Publishing: 503 No Responders Available For Request",
    "frames": [
      {
        "class": "io.debezium.server.nats.jetstream.NatsJetStreamChangeConsumer",
        "method": "handleBatch",
        "line": 139
      },
      {
        "class": "io.debezium.embedded.ConvertingEngineBuilder",
        "method": "lambda$notifying$2",
        "line": 101
      },
      {
        "class": "io.debezium.embedded.EmbeddedEngine",
        "method": "run",
        "line": 912
      },
      {
        "class": "io.debezium.embedded.ConvertingEngineBuilder$2",
        "method": "run",
        "line": 229
      },
      {
        "class": "io.debezium.server.DebeziumServer",
        "method": "lambda$start$1",
        "line": 170
      },
      {
        "class": "java.util.concurrent.ThreadPoolExecutor",
        "method": "runWorker",
        "line": 1128
      },
      {
        "class": "java.util.concurrent.ThreadPoolExecutor$Worker",
        "method": "run",
        "line": 628
      },
      {
        "class": "java.lang.Thread",
        "method": "run",
        "line": 829
      }
    ],
    "causedBy": {
      "exception": {
        "refId": 2,
        "exceptionType": "java.io.IOException",
        "message": "Error Publishing: 503 No Responders Available For Request",
        "frames": [
          {
            "class": "io.nats.client.impl.NatsJetStream",
            "method": "processPublishResponse",
            "line": 153
          },
          {
            "class": "io.nats.client.impl.NatsJetStream",
            "method": "publishSyncInternal",
            "line": 123
          },
          {
            "class": "io.nats.client.impl.NatsJetStream",
            "method": "publish",
            "line": 44
          },
          {
            "class": "io.debezium.server.nats.jetstream.NatsJetStreamChangeConsumer",
            "method": "handleBatch",
            "line": 136
          },
          {
            "class": "io.debezium.embedded.ConvertingEngineBuilder",
            "method": "lambda$notifying$2",
            "line": 101
          },
          {
            "class": "io.debezium.embedded.EmbeddedEngine",
            "method": "run",
            "line": 912
          },
          {
            "class": "io.debezium.embedded.ConvertingEngineBuilder$2",
            "method": "run",
            "line": 229
          },
          {
            "class": "io.debezium.server.DebeziumServer",
            "method": "lambda$start$1",
            "line": 170
          },
          {
            "class": "java.util.concurrent.ThreadPoolExecutor",
            "method": "runWorker",
            "line": 1128
          },
          {
            "class": "java.util.concurrent.ThreadPoolExecutor$Worker",
            "method": "run",
            "line": 628
          },
          {
            "class": "java.lang.Thread",
            "method": "run",
            "line": 829
          }
        ]
      }
    }
  }
}
```

Below is the application.properties

```
debezium.source.connector.class=io.debezium.connector.sqlserver.SqlServerConnector
debezium.source.offset.storage.file.filename=data/offsets.dat

debezium.source.database.hostname=mssql
debezium.source.database.port=1433
debezium.source.database.user=sa
debezium.source.database.password=somepassword!
debezium.source.database.names=testDB
debezium.source.database.encrypt=false
debezium.source.topic.prefix=testDB

debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory
debezium.source.schema.history.internal.file.filename=data/schemahistory.dat

debezium.sink.type=nats-jetstream
debezium.sink.nats-jetstream.url=nats://nats:4222
debezium.sink.nats-jetstream.create-stream=true
debezium.sink.nats-jetstream.subjects=testDB.*.*
```

and below is docker-compose.yml file

```

mssql:
    image: mcr.microsoft.com/mssql/server:2019-latest
    ports:
      - 1433:1433
    environment:
      - ACCEPT_EULA=Y
      - MSSQL_PID=Standard
      - MSSQL_SA_PASSWORD=somepassword!
      - MSSQL_AGENT_ENABLED=true

  nats:
    image: docker.io/nats:2.9
    command:
      - "--debug"
      - "--http_port=8222"
      - "--js"
    ports:
      - 4222:4222
      - 8222:8222

  debezium:
    image: docker.io/debezium/server:2.3.0.Final
    volumes:
      - ./cli/application.properties:/debezium/conf/application.properties
      - ./.docker_volumes/debezium:/debezium/data

```

If anyone knows how to solve this issue, I would appreciate your help.

Thank you

[Nick Chomey]

Like many others, I just fought with this for a while. I figured out that yuo need to do something like 

debezium.sink.nats-jetstream.subjects=testDB,testDB.*.*

or testDB,testDB.>

you have to specify just "testDB" for the DDL schema statements. But then the prefix.db.table needs the .*.* or .> (which matches 1 or more subtopics)

Its very bizarre, and not documented whatsoever. I hope this helps.

[Chris Cranford]

Hi Nick -

Would you be willing to submit a PR to make the format a bit more clear in the docs?

-cc

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/f70eaeaf-e6b3-4dff-987b-9a5f34b02e03n%40googlegroups.com.

[Nick Chomey]

Done!  Clarify nats-jetstream subject documentation by nickchomey · Pull Request #6664 · debezium/debezium

You received this message because you are subscribed to a topic in the Google Groups "debezium" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/debezium/6uaGNiYHB2o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/f96542c5-99e3-4741-b548-3c9876c338db%40gmail.com.