mongodb data-index & management-console docker

[Tamás Sipos]

Hi Kogito experts,

I’m try to create a docker compose stack and somehow the managment console - data index connection not working. Pretty sure I missing some config somewhere.

So tried from source  data-index-service-mongodb with management-console in dev mode and works like a charm.

Kogito version: 1.2.0.Final

Mongo, kafka, dataindex works fine, no error whatsoever.

But the managment console ui  have some difficulties to grab the data.

http://localhost:8085/ProcessInstances/

Request URL:

http://localhost:8085/ProcessInstances/"http://localhost:8180"/graphql

In Dev mode the same url look like:

Request URL:

http://localhost:8180/graphql

data-index-service-mongodb conf:

# MongoDB
kogito.persistence.type=mongodb
quarkus.mongodb.connection-string=mongodb://localhost:27017
quarkus.mongodb.database=kogito
quarkus.mongodb.health.enabled=true
quarkus.mongodb.metrics.enabled=true

kafka-servers=localhost:9093

quarkus.kafka.bootstrap-servers=${kafka-servers}
mp.messaging.incoming.kogito-processinstances-events.bootstrap.servers=${kafka-servers}
mp.messaging.incoming.kogito-processdomain-events.bootstrap.servers=${kafka-servers}
mp.messaging.incoming.kogito-usertaskinstances-events.bootstrap.servers=${kafka-servers}
mp.messaging.incoming.kogito-usertaskdomain-events.bootstrap.servers=${kafka-servers}
mp.messaging.incoming.kogito-jobs-events.bootstrap.servers=${kafka-servers}

management-console conf:

quarkus.http.cors=true
quarkus.http.port=8085
quarkus.oidc.enabled=true
quarkus.oidc.tenant-enabled=false
quarkus.http.auth.permission.roles1.paths=/*
quarkus.http.auth.permission.roles1.policy=permit

Also not working when data index runs from docker image but  management console in dev mode. Wired localhost issue net::ERR_EMPTY_RESPONSE

version: '3.7'

services:

  mongodb:

    image: mongo:4.4-bionic

    container_name: mongodb

    ports:

      - 27017:27017

    networks:

      my-net:

        aliases:

          - mongodb

  zookeeper:

    image: 'bitnami/zookeeper:latest'

    ports:

      - '2181:2181'

    environment:

      - ALLOW_ANONYMOUS_LOGIN=yes

    networks:

      my-net:

        aliases:

          - zookeeper

  kafka:

    image: 'bitnami/kafka:2.7.0'

    ports:

      - '9092:9092'

      - '9093:9093'

    environment:

      - KAFKA_BROKER_ID=1

      - KAFKA_LISTENERS=CLIENT://:9092,EXTERNAL://:9093

      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT

      - KAFKA_ADVERTISED_LISTENERS=CLIENT://kafka:9092,EXTERNAL://localhost:9093

      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181

      - ALLOW_PLAINTEXT_LISTENER=yes

      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT

    depends_on:

      - zookeeper

    networks:

      my-net:

        aliases:

          - kafka

  dataindex:

    image: quay.io/kiegroup/kogito-data-index-mongodb:1.2

    container_name: dataindex

    environment:

#      - QUARKUS_HTTP_CORS=false

      - QUARKUS_MONGODB_CONNECTION_STRING=mongodb://mongodb:27017

      - QUARKUS_KAFKA_BOOTSTRAP_SERVERS=kafka:9092

      - MP_MESSAGING_INCOMING_KOGITO_PROCESSINSTANCES_EVENTS_BOOTSTRAP_SERVERS=kafka:9092

      - MP_MESSAGING_INCOMING_KOGITO_PROCESSDOMAIN_EVENTS_BOOTSTRAP_SERVERS=kafka:9092

      - MP_MESSAGING_INCOMING_KOGITO_USERTASKINSTANCES_EVENTS_BOOTSTRAP_SERVERS=kafka:9092

      - MP_MESSAGING_INCOMING_KOGITO_USERTASKDOMAIN_EVENTS_BOOTSTRAP_SERVERS=kafka:9092

      - MP_MESSAGING_INCOMING_KOGITO_JOBS_EVENTS_BOOTSTRAP_SERVERS=kafka:9092

    ports:

      - 8180:8180

    networks:

      my-net:

        aliases:

          - dataindex

  managementconsole:

    image: quay.io/kiegroup/kogito-management-console:1.2

    container_name: managementconsole

    environment:

      - KOGITO_DATAINDEX_HTTP_URL="http://localhost:8180"

    ports:

      - 8085:8080

    networks:

      my-net:

        aliases:

          - managementconsole

networks:

  my-net:

    name: my-net

    driver: bridge

Best regards,

Tamás

[Cristiano Nicolai]

Hi Tamás,

Two simple changes to fix that:

Data Index port mapping should be:

ports:
      - 8180:8080

Management Console env, removing "

environment:
      - KOGITO_DATAINDEX_HTTP_URL=http://localhost:8180

--
You received this message because you are subscribed to the Google Groups "Kogito development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kogito-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/4fbc521c-af96-4594-b1cf-70028cabd64dn%40googlegroups.com.

--

Best regards,

Cristiano Nicolai

[Tamás Sipos]

Ahh.... Thanks :)

This config confused me

https://github.com/kiegroup/kogito-apps/blob/master/data-index/data-index-service/data-index-service-common/src/main/resources/META-INF/microprofile-config.properties#L11

Found a bad config in dataindex, maybe they fixed it already

2021-03-04 09:28:49,192 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.kafka.bootstrap-servers" was provided; it will be ignored; verify that the dependency extension for this configuration is set or you did not make a typo

https://quarkus.io/version/1.11/guides/kafka#configuring-the-kafka-connector

Having some problem in the Process Details UI, maybe my test process the issue.
When a process is Completed it is fine. 

When Active getting a javascript error:

TypeError: nodes.forEach is not a function

    at createNodeDropDown (ProcessDetailsNodeTrigger.tsx:81)

    at ProcessDetailsNodeTrigger_ProcessDetailsNodeTrigger (ProcessDetailsNodeTrigger.tsx:150)

the new improved  docker-compose.yml

version: '3.7'

services:

  mongodb:

    image: mongo:4.4-bionic

    container_name: mongodb

    ports:

      - 27017:27017

    networks:

      my-net:

        aliases:

          - mongodb

  zookeeper:

    image: 'bitnami/zookeeper:latest'

    container_name: zookeeper

    ports:

      - '2181:2181'

    environment:

      - ALLOW_ANONYMOUS_LOGIN=yes

    networks:

      my-net:

        aliases:

          - zookeeper

  kafka:

    image: 'bitnami/kafka:2.7.0'

    container_name: kafka

    ports:

      - '9092:9092'

      - '9093:9093'

    environment:

      - KAFKA_BROKER_ID=1

      - KAFKA_LISTENERS=CLIENT://:9092,EXTERNAL://:9093

      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT

      - KAFKA_ADVERTISED_LISTENERS=CLIENT://kafka:9092,EXTERNAL://localhost:9093

      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181

      - ALLOW_PLAINTEXT_LISTENER=yes

      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT

    depends_on:

      - zookeeper

    networks:

      my-net:

        aliases:

          - kafka

  dataindex:

    image: quay.io/kiegroup/kogito-data-index-mongodb:1.2

    container_name: dataindex

    environment:

      - QUARKUS_MONGODB_CONNECTION_STRING=mongodb://mongodb:27017

      - KAFKA_BOOTSTRAP_SERVERS=kafka:9092

    ports:

      - 8180:8080

    networks:

      my-net:

        aliases:

          - dataindex

  managementconsole:

    image: quay.io/kiegroup/kogito-management-console:1.2

    container_name: managementconsole

    environment:

      - KOGITO_DATAINDEX_HTTP_URL=http://localhost:8180

    ports:

      - 8085:8080

    networks:

      my-net:

        aliases:

          - managementconsole

networks:

  my-net:

    name: my-net

    driver: bridge

[Cristiano Nicolai]

Sounds like an issue on Management Console, are you using the process management addon in your Kogito runtime?

If so, could you share the result of the following REST call:

/management/processes/${processId}/nodes

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/17a1a64a-2984-48ce-93f6-0ea90a617387n%40googlegroups.com.

[Tamás Sipos]

Hi,

Thank you the fast response. 

in the documentation mention a kogito.service.url configuration 

https://docs.jboss.org/kogito/release/1.3.0/html_single/#proc-management-console-using_kogito-developing-process-services

Kogito runtime application.properties :o)

Ohhh

https://github.com/kiegroup/kogito-runtimes/blob/master/kogito-codegen-modules/kogito-codegen-api/src/main/java/org/kie/kogito/codegen/api/context/KogitoBuildContext.java#L32

cereate application.properties

kogito.dataindex.http.url=http://localhost:8180
kogito.service.url=http://localhost:8080

And working !!!

Any plans for quarkus yaml config support ?

The problem in Management Console UI side the url was wrong:

http://localhost:8085/Process/null/management/processes/ReportProcess/nodes

The correct one is:

http://localhost:8080/management/processes/ReportProcess/nodes

The Kogito runtime send the "source url" value for the data index, and the default value is null?

This should be mandatory or the Management Console handle the empty value

I don't understand how know the data-index this url?

From  kafka events ?

empty config:

  "source": "/ReportProcess",

  "type": "ProcessInstanceEvent",

with kogito.service.url=http://localhost:8080

  "source": "http://localhost:8080/ReportProcess",

  "type": "ProcessInstanceEvent",

[Cristiano Nicolai]

Correct, kogito.service.url does the tricky, it is pushed into the messages so data index stores it and thats how Management Console knows where to call, imagine that you could have multiple runtimes in different urls sharing the same data index and management console.

Regarding the issue on Management Console, there seems to be a bug already fixed: https://issues.redhat.com/browse/KOGITO-3684, although it says fix on 1.2 I believe is not there, should land in 1.3

As for YAML, if you're using Quarkus you can try https://quarkus.io/guides/config-reference#yaml, although I'm not sure it will work out of the fox with the Kogito specific configurations.

To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/6824fc72-cf27-4319-b788-cf2dcbaf5e0cn%40googlegroups.com.

[Nieves Miras Chueca]

Hi,

Absolute agree with you about the Kogito Management console have to handle correctly this situation. I have created https://issues.redhat.com/browse/KOGITO-4596 track of the issue.

Thanks for catching this!

Best Regards,

Neus Miras

> *http://localhost:8085/Process/null <http://localhost:8085/Process/null>*
> /management/processes/ReportProcess/nodes

>>>>> So tried from source <https://github.com/kiegroup/kogito-apps> data-index-service-mongodb

>>>>> <https://groups.google.com/d/msgid/kogito-development/4fbc521c-af96-4594-b1cf-70028cabd64dn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .

>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Cristiano Nicolai
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kogito development mailing list" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kogito-developm...@googlegroups.com.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/kogito-development/17a1a64a-2984-48ce-93f6-0ea90a617387n%40googlegroups.com

>>> <https://groups.google.com/d/msgid/kogito-development/17a1a64a-2984-48ce-93f6-0ea90a617387n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .

>>>
>>
>>
>> --
>> Best regards,
>>
>> Cristiano Nicolai
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Kogito development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kogito-developm...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/kogito-development/6824fc72-cf27-4319-b788-cf2dcbaf5e0cn%40googlegroups.com
> <https://groups.google.com/d/msgid/kogito-development/6824fc72-cf27-4319-b788-cf2dcbaf5e0cn%40googlegroups.com?utm_medium=email&utm_source=footer>
> .

>


--
Best regards,

Cristiano Nicolai

--
You received this message because you are subscribed to the Google Groups "Kogito development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kogito-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/CAOVo6cRTJHPE1Xta0x0jfTVrG4DYdfd4wnJ4oqEP7aZnCNk%2B9A%40mail.gmail.com.