dotCMS docker not able to connect to Postgresql-14

178 views
Skip to first unread message

Kachina Vazhuthi Ilavaluthi

unread,
Jan 25, 2023, 8:40:48 AM1/25/23
to dotCMS User Group
Recently, We installed dotCMS in docker containers. The dotCMS runs on a docker container and connects to a postgres db on the host machine. After a OS restart couple of days back, the dotCMS is not connecting to the Postgres DB. We are getting the below connection error. Any help would be appreciated. thanks.

removing Logger :com.dotcms.dotpubsub.PostgresPubSubImpl due to EXPIRED
20:14:55.305 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
21:14:55.350 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
22:14:55.402 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
23:14:55.431 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
00:14:55.475 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
01:14:55.525 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
02:14:55.552 INFO reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
03:08:24.479 WARN dotpubsub.PostgresPubSubImpl - Connection Error: Timeout starting connection
com.impossibl.postgres.jdbc.PGSQLSimpleException: Connection Error: Timeout starting connection
at com.impossibl.postgres.jdbc.ErrorUtils.makeSQLException(ErrorUtils.java:138) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:326) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:279) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
at com.impossibl.postgres.jdbc.PGDriver.connect(PGDriver.java:104) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
at java.sql.DriverManager.getConnection(DriverManager.java:251) ~[java.sql:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.getConnection(PostgresPubSubImpl.java:221) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:150) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:58) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.restart(PostgresPubSubImpl.java:186) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:156) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:58) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.restart(PostgresPubSubImpl.java:186) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:156) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:58) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.restart(PostgresPubSubImpl.java:186) ~[dotcms_22.03.1_999999.jar:?]
at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:156) ~[dotcms_22.03.1_999999.jar:?]  


Will Ezell

unread,
Jan 25, 2023, 9:05:40 AM1/25/23
to dot...@googlegroups.com
Are there enough PG server connections available to dotCMS?  

SELECT * FROM pg_stat_activity;



--
http://dotcms.com - Open Source Java Content Management
---
You received this message because you are subscribed to the Google Groups "dotCMS User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dotcms+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dotcms/38d9bb86-eb3c-443c-9189-0835468d6cffn%40googlegroups.com.


--



382 NE 191st St #92150
Miami, Florida 33179-3899
Main: 
305-900-2001 | Direct: 978.294.9429

Kachina Vazhuthi Ilavaluthi

unread,
Feb 8, 2023, 5:26:06 AM2/8/23
to dotCMS User Group
The issue is recurring now in another instance. Below is the error I see in the postgres log.

cat /var/lib/pgsql/14/data/log/postgresql-Wed.log
2023-02-08 09:53:40.569 UTC [2250] FATAL: canceling authentication due to timeout
2023-02-08 09:54:41.604 UTC [2327] FATAL: canceling authentication due to timeout
2023-02-08 09:55:43.619 UTC [2395] FATAL: canceling authentication due to timeout   

Will Ezell

unread,
Feb 8, 2023, 12:23:25 PM2/8/23
to dot...@googlegroups.com
This is the Pub/Sub connection failing. We try to create the url automatically, but it seems like this is not working for some reason.  You can try setting the correct Postgres Pub/Sub JDBC URL manually. To do this, set an environmental variable DOT_POSTGRES_PUBSUB_JDBC_URL to the full url for their connection, including username and password and any url params that you need set - format is
 
jdbc:pgsql://{username}:{password}@{hostname}/{db_name}
 
Also the username and password need to be URLEncoded (not the whole url) if they contain special chars.

Kachina Vazhuthi Ilavaluthi

unread,
Feb 9, 2023, 7:22:31 AM2/9/23
to dotCMS User Group
Will Ezel,  
Thanks for your timely responses. I appreciate it. 
I tried your suggestion. below is my docker-compose.yml. The connection to the Database is failing even after adding the DOT_POSTGRES_PUBSUB_JDBC_URL.

=====================================================
dotcms:
image: dotcms/dotcms:22.03.4_lts
environment:
CMS_JAVA_OPTS: '-Xmx1g '
LANG: 'C.UTF-8'
TZ: 'UTC'
DOT_POSTGRES_PUBSUB_JDBC_URL: "jdbc:pgsql://postgres:post...@10.222.13.46:5432/postgres"
#DB_BASE_URL: "jdbc:postgresql://10.222.13.46:5432/postgres"
#DB_USERNAME: 'postgres'
#DB_PASSWORD: 'postgres'
DOT_ES_AUTH_BASIC_PASSWORD: 'admin'
DOT_ES_ENDPOINTS: 'https://opensearch:9200'
DOT_INITIAL_ADMIN_PASSWORD: 'admin'            
=====================================================
ERROR
dotcms_1      | 04:25:41.035  WARN  dotpubsub.PostgresPubSubImpl - Restarting PGNotificationListener in 1000 ms to retry postgres pub/sub connection
dotcms_1      | 04:25:43.038  INFO  dotpubsub.PostgresPubSubImpl - Starting PostgresPubSub. Have servers:0
dotcms_1      | 04:25:43.038  INFO  dotpubsub.PostgresPubSubImpl - PGNotificationListener connecting to pub/sub...
dotcms_1      | 04:26:43.049  WARN  dotpubsub.PostgresPubSubImpl - Connection Error: Timeout starting connection
dotcms_1      | com.impossibl.postgres.jdbc.PGSQLSimpleException: Connection Error: Timeout starting connection
dotcms_1      |         at com.impossibl.postgres.jdbc.ErrorUtils.makeSQLException(ErrorUtils.java:138) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:326) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:279) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.PGDriver.connect(PGDriver.java:104) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
dotcms_1      |         at java.sql.DriverManager.getConnection(DriverManager.java:251) ~[java.sql:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.getConnection(PostgresPubSubImpl.java:226) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:155) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:59) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.restart(PostgresPubSubImpl.java:191) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:161) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:59) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.restart(PostgresPubSubImpl.java:191) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:161) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:59) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.QueuingPubSubWrapper.start(QueuingPubSubWrapper.java:74) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.<init>(OSGIUtil.java:146) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.<init>(OSGIUtil.java:139) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil$OSGIUtilHolder.<clinit>(OSGIUtil.java:136) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.getInstance(OSGIUtil.java:132) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb.loadStarterSite_aroundBody6(DotCMSInitDb.java:98) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb$AjcClosure7.run(DotCMSInitDb.java:1) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
dotcms_1      |         at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb.loadStarterSite(DotCMSInitDb.java:82) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb.lambda$0(DotCMSInitDb.java:45) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at io.vavr.control.Try.run(Try.java:118) ~[vavr-0.10.3.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb.InitializeDb_aroundBody2(DotCMSInitDb.java:45) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb$AjcClosure3.run(DotCMSInitDb.java:1) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
dotcms_1      |         at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.db.DotCMSInitDb.InitializeDb(DotCMSInitDb.java:37) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.startup.runalways.Task00004LoadStarter.executeUpgrade(Task00004LoadStarter.java:14) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.startup.StartupTasksExecutor.executeStartUpTasks(StartupTasksExecutor.java:133) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.liferay.portal.servlet.MainServlet.init_aroundBody0(MainServlet.java:121) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.liferay.portal.servlet.MainServlet$AjcClosure1.run(MainServlet.java:1) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
dotcms_1      |         at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.liferay.portal.servlet.MainServlet.init(MainServlet.java:100) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1164) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1117) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1010) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5264) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698) ~[catalina.jar:9.0.60]
dotcms_1 

Will Ezell

unread,
Feb 9, 2023, 10:28:36 AM2/9/23
to dot...@googlegroups.com
You need to set both the DB_XXXX properties for the main postgres connection AND set the DOT_POSTGRES_PUBSUB_JDBC_URL. All of these should be set:

DOT_POSTGRES_PUBSUB_JDBC_URL
DB_BASE_URL
DB_USERNAME
DB_PASSWORD

Kachina Vazhuthi Ilavaluthi

unread,
Feb 13, 2023, 3:34:22 AM2/13/23
to dotCMS User Group
Will,

I tried your suggestion, but it didn't help. Still the postgres connection is failing. Below is the error log and the docker compose file used.
I tested the postgres, it is accepting connections, and I'm able to connect to it from a db client with the same credentials used in the docker compose configuration. I'm not able to proceed further, your support will be greatly appreciated. 

Thanks
Kachi.

[1] 76539
[root@10-222-13-46 dxp-service]# Creating network "dxpservice_opensearch-net" with the default driver
Creating volume "dxpservice_cms-shared" with default driver
Creating dxpservice_opensearch_1 ... done
Creating dxpservice_opensearch_1 ...
Creating dxpservice_dotcms_1     ... done
Attaching to dxpservice_opensearch_1, dxpservice_dotcms_1
opensearch_1  | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch_1  | OpenSearch Security Demo Installer
opensearch_1  |  ** Warning: Do not use on production or public reachable systems **
opensearch_1  | Basedir: /usr/share/opensearch
opensearch_1  | OpenSearch install type: rpm/deb on NAME="Amazon Linux"
opensearch_1  | OpenSearch config dir: /usr/share/opensearch/config
opensearch_1  | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch_1  | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch_1  | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch_1  | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch_1  | Detected OpenSearch Version: x-content-1.3.6
opensearch_1  | Detected OpenSearch Security Version: 1.3.6.0
opensearch_1  |
opensearch_1  | ### Success
opensearch_1  | ### Execute this script now on all your nodes and then start all nodes
opensearch_1  | ### OpenSearch Security will be automatically initialized.
opensearch_1  | ### If you like to change the runtime configuration
opensearch_1  | ### change the files in ../securityconfig and execute:
opensearch_1  | "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/plugins/opensearch-security/securityconfig" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv
opensearch_1  | ### or run ./securityadmin_demo.sh
opensearch_1  | ### To use the Security Plugin ConfigurationGUI
opensearch_1  | ### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
opensearch_1  | ### (Ignore the SSL certificate warning because we installed self-signed demo certificates)
opensearch_1  | Enabling OpenSearch Security Plugin
dotcms_1      | Using default starter
dotcms_1      |
dotcms_1      | Starting dotCMS ...
dotcms_1      | -------------------
dotcms_1      |
dotcms_1      | NOTE: Picked up JDK_JAVA_OPTIONS:  --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
dotcms_1      | 13-Feb-2023 08:20:05.947 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name:   Apache Tomcat/9.0.60
dotcms_1      | 13-Feb-2023 08:20:05.949 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Mar 9 2022 14:52:25 UTC
dotcms_1      | 13-Feb-2023 08:20:05.949 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.60.0
dotcms_1      | 13-Feb-2023 08:20:05.949 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
dotcms_1      | 13-Feb-2023 08:20:05.949 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            3.10.0-1160.76.1.el7.x86_64
dotcms_1      | 13-Feb-2023 08:20:05.949 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
dotcms_1      | 13-Feb-2023 08:20:05.950 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /java
dotcms_1      | 13-Feb-2023 08:20:05.950 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           11.0.16+8-LTS
dotcms_1      | 13-Feb-2023 08:20:05.950 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Amazon.com Inc.
dotcms_1      | 13-Feb-2023 08:20:05.950 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /srv/dotserver/tomcat-9.0.60
dotcms_1      | 13-Feb-2023 08:20:05.950 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /srv/dotserver/tomcat-9.0.60
dotcms_1      | 13-Feb-2023 08:20:05.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
dotcms_1      | 13-Feb-2023 08:20:05.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
dotcms_1      | 13-Feb-2023 08:20:05.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED
dotcms_1      | 13-Feb-2023 08:20:05.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
dotcms_1      | 13-Feb-2023 08:20:05.977 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
dotcms_1      | 13-Feb-2023 08:20:05.977 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/srv/dotserver/tomcat-9.0.60/conf/logging.properties
dotcms_1      | 13-Feb-2023 08:20:05.981 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
dotcms_1      | 13-Feb-2023 08:20:05.981 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xverify:none
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dfile.encoding=UTF8
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dpdfbox.fontcache=/data/local/dotsecure
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dlog4j2.formatMsgNoLookups=true
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.library.path=/usr/lib/x86_64-linux-gnu/
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseShenandoahGC
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UnlockExperimentalVMOptions
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:ShenandoahUncommitDelay=1000
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:ShenandoahGuaranteedGCInterval=10000
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -javaagent:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/lib/byte-buddy-agent-1.9.0.jar
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx1G
dotcms_1      | 13-Feb-2023 08:20:05.982 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx1g
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.tomcat.util.digester.PROPERTY_SOURCE=org.apache.tomcat.util.digester.EnvironmentPropertySource
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/srv/dotserver/tomcat-9.0.60
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/srv/dotserver/tomcat-9.0.60
dotcms_1      | 13-Feb-2023 08:20:05.983 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/srv/dotserver/tomcat-9.0.60/temp
dotcms_1      | 13-Feb-2023 08:20:05.989 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent An older version [1.2.23] of the Apache Tomcat Native library is installed, while Tomcat recommends a minimum version of [1.2.30]
dotcms_1      | 13-Feb-2023 08:20:05.989 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded Apache Tomcat Native library [1.2.23] using APR version [1.6.5].
dotcms_1      | 13-Feb-2023 08:20:05.989 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true], UDS [false].
dotcms_1      | 13-Feb-2023 08:20:05.989 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
dotcms_1      | 13-Feb-2023 08:20:05.992 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.1f  31 Mar 2020]
dotcms_1      | 13-Feb-2023 08:20:06.708 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio2-8080"]
dotcms_1      | 13-Feb-2023 08:20:06.716 WARNING [main] org.apache.tomcat.util.net.Nio2Endpoint.bind The NIO2 connector requires an exclusive executor to operate properly on shutdown
dotcms_1      | 13-Feb-2023 08:20:06.774 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio2-8081"]
dotcms_1      | 13-Feb-2023 08:20:06.774 WARNING [main] org.apache.tomcat.util.net.Nio2Endpoint.bind The NIO2 connector requires an exclusive executor to operate properly on shutdown
dotcms_1      | 13-Feb-2023 08:20:06.775 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio2-8082"]
dotcms_1      | 13-Feb-2023 08:20:06.775 WARNING [main] org.apache.tomcat.util.net.Nio2Endpoint.bind The NIO2 connector requires an exclusive executor to operate properly on shutdown
dotcms_1      | 13-Feb-2023 08:20:06.779 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-openssl-nio2-8443"]
dotcms_1      | 13-Feb-2023 08:20:06.780 WARNING [main] org.apache.tomcat.util.net.Nio2Endpoint.bind The NIO2 connector requires an exclusive executor to operate properly on shutdown
dotcms_1      | 13-Feb-2023 08:20:06.964 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [1420] milliseconds
dotcms_1      | 13-Feb-2023 08:20:07.144 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
dotcms_1      | 13-Feb-2023 08:20:07.144 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.60]
dotcms_1      | 13-Feb-2023 08:20:07.170 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/srv/dotserver/tomcat-9.0.60/webapps/ROOT]
dotcms_1      | 13-Feb-2023 08:20:09.867 INFO [main] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
dotcms_1      | INFO  2023-02-13 08:20:11,486 [main] com.dotcms.util.AsciiArt:():                                                                                    
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():                                                                                    
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():                                                                                    
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():            OOOO                            7777777   7777       7777     77777777  
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():            OOOO                 OO       777777777  77777      77777    77777777  
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():            OOOO                OOO      77777       777777     77777   7777        
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():      OOOOOOOOOO   OOOOOOOO   OOOOOOOOO 7777         7777777   777777   7777        
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():     OOOO  OOOOO  OOOO  OOOO    OOOO    7777         7777777  7777777    777777    
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():    OOOO    OOOO  OOO    OOOO   OOOO    7777         77777777 777 7777     777777  
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():    OOOO    OOOO OOOO    OOOO   OOOO    7777         777  777 777 7777        7777  
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():    OOOO    OOOO  OOO    OOOO   OOOO    77777        777  777777  7777         7777
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():     OOOO   OOOO  OOOO   OOO    OOOO     77777       777   7777   7777        7777  
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():      OOOOOOOOOO   OOOOOOOO      OOOOO    777777777  777   7777   7777  777777777  
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():                                                                                    
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():                                                          Content Management System
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():                                                                                    
dotcms_1      | INFO  2023-02-13 08:20:11,499 [main] com.dotcms.util.AsciiArt:():                                                                                    
opensearch_1  | [2023-02-13T08:20:12,458][INFO ][o.o.n.Node               ] [d6cdc34291bc] version[1.3.6], pid[102], build[tar/cbf74db21db3eb4d79c43caeafc23eec592bf697/2022-10-04T20:26:16.847170Z], OS[Linux/3.10.0-1160.76.1.el7.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/11.0.16/11.0.16+8]
opensearch_1  | [2023-02-13T08:20:12,459][INFO ][o.o.n.Node               ] [d6cdc34291bc] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true]
opensearch_1  | [2023-02-13T08:20:12,460][INFO ][o.o.n.Node               ] [d6cdc34291bc] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-8683348036654618694, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xmx1G, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
dotcms_1      | INFO  2023-02-13 08:20:12,509 [main] com.dotmarketing.util.Config:(): Initializing properties reader.
dotcms_1      | INFO  2023-02-13 08:20:12,509 [main] com.dotmarketing.util.Config:(): Loading dotCMS [dotmarketing-config.properties] Properties...
dotcms_1      | INFO  2023-02-13 08:20:12,650 [main] com.dotmarketing.util.Config:(): dotCMS Properties [dotmarketing-config.properties] Loaded
dotcms_1      | INFO  2023-02-13 08:20:12,699 [main] com.dotmarketing.util.Config:(): Loading dotCMS [dotcms-config-cluster.properties] Properties...
dotcms_1      | INFO  2023-02-13 08:20:12,700 [main] com.dotmarketing.util.Config:(): dotCMS Properties [dotcms-config-cluster.properties] Loaded
dotcms_1      | INFO  2023-02-13 08:20:12,702 [main] com.dotmarketing.util.Config:(): Assigned default refresh: 5 minutes.
dotcms_1      | INFO  2023-02-13 08:20:12,875 [main] com.dotcms.api.web.RequestThreadLocalListener:(): Starting RequestThreadLocalListener
dotcms_1      | 08:20:13.226  INFO  loggers.Log4jUtil - Async Logger enabled: true
dotcms_1      | 08:20:13.898  INFO  business.CacheLocator - loading cache administrator: ChainableCacheAdministratorImpl
dotcms_1      | 08:20:13.988  WARN  lang.System - No Valid License Found : /data/local/dotsecure/license/license.dat
dotcms_1      | 08:20:13.988  INFO  enterprise.LicenseUtil - Setting up default license
dotcms_1      | 08:20:14.012  INFO  caffine.CaffineCache - ***   Cache Config Memory : esquerycache: 10000
dotcms_1      | 08:20:14.012  INFO  caffine.CaffineCache - ***   Cache Config Memory : esquerycountcache: 10000
dotcms_1      | 08:20:14.012  INFO  caffine.CaffineCache - ***   Cache Config Memory : graphqlquerycache: 5000
dotcms_1      | 08:20:14.012  INFO  caffine.CaffineCache - ***   Cache Config Memory : default: 1000
dotcms_1      | 08:20:14.012  INFO  caffine.CaffineCache - ***   Cache Config Memory : adminconfigpool: 100
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : companypool: 10
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : contentletcache: 5000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : csscache: 2500
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : assetmetadatacache: 10000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : identifiercache: 50000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : indiciescache: 8
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : logmappercache: 10
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : navcache: 1000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : parentpermissionablecache: 10000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : permissioncache: 50000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : portletpool: 100
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : publishingendpointcache: 100
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : pushedassetscache: 5000
dotcms_1      | 08:20:14.013  INFO  caffine.CaffineCache - ***   Cache Config Memory : templatecache: 500
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : translatedquerycache: 5000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : versioncache: 50000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : versioninfocache: 50000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : workflowactioncache: 10000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : workflowtaskcache: 10000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : shorty: 25000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : identifier404cache: 5000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : velocitycache: 1000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : vanityurldirectcache: 25000
dotcms_1      | 08:20:14.014  INFO  caffine.CaffineCache - ***   Cache Config Memory : vanityurlsitecache: 5000
dotcms_1      | 08:20:14.122  WARN  hikari.HikariConfig - jdbc/dotCMSPool - leakDetectionThreshold is less than 2000ms or more than maxLifetime, disabling it.
dotcms_1      | 08:20:14.123  INFO  hikari.HikariDataSource - jdbc/dotCMSPool - Starting...
opensearch_1  | [2023-02-13T08:20:14,511][WARN ][stderr                   ] [d6cdc34291bc] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
opensearch_1  | [2023-02-13T08:20:14,512][WARN ][stderr                   ] [d6cdc34291bc] SLF4J: Defaulting to no-operation (NOP) logger implementation
opensearch_1  | [2023-02-13T08:20:14,512][WARN ][stderr                   ] [d6cdc34291bc] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
opensearch_1  | [2023-02-13T08:20:14,545][INFO ][o.o.s.s.t.SSLConfig      ] [d6cdc34291bc] SSL dual mode is disabled
opensearch_1  | [2023-02-13T08:20:14,545][INFO ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] OpenSearch Config path is /usr/share/opensearch/config
dotcms_1      | 08:20:14.741  INFO  hikari.HikariDataSource - jdbc/dotCMSPool - Start completed.
dotcms_1      | 08:20:14.742  INFO  db.DataSourceStrategyProvider - Datasource loaded using custom class com.dotmarketing.db.SystemEnvDataSourceStrategy
dotcms_1      | 08:20:14.914  INFO  servlet.VisitorFilter - VisitorLogger Filter Started
opensearch_1  | [2023-02-13T08:20:15,036][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] JVM supports TLSv1.3
opensearch_1  | [2023-02-13T08:20:15,040][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] Config directory is /usr/share/opensearch/config/, from there the key- and truststore files are resolved relatively
dotcms_1      | 13-Feb-2023 08:20:15.137 INFO [main] org.apache.catalina.core.ApplicationContext.log org.tuckey.web.filters.urlrewrite.UrlRewriteFilter INFO: loaded (conf ok)
dotcms_1      | 08:20:15.451  INFO  util.SystemProperties - Loading file:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/classes/system.properties
dotcms_1      | 08:20:15.451  INFO  util.SystemProperties - Loading file:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/classes/system-ext.properties
dotcms_1      | 08:20:15.464  INFO  util.SystemProperties - Loading file:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/classes/system.properties
dotcms_1      | 08:20:15.464  INFO  util.SystemProperties - Loading file:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/classes/system-ext.properties
opensearch_1  | [2023-02-13T08:20:15,783][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] TLS Transport Client Provider : JDK
opensearch_1  | [2023-02-13T08:20:15,783][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] TLS Transport Server Provider : JDK
opensearch_1  | [2023-02-13T08:20:15,783][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] TLS HTTP Provider             : JDK
opensearch_1  | [2023-02-13T08:20:15,783][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2]
opensearch_1  | [2023-02-13T08:20:15,783][INFO ][o.o.s.s.DefaultSecurityKeyStore] [d6cdc34291bc] Enabled TLS protocols for HTTP layer      : [TLSv1.3, TLSv1.2]
dotcms_1      | 08:20:15.847  INFO  util.PropsLoader - Loading file:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/classes/portal.properties
dotcms_1      | 08:20:15.847  INFO  util.PropsLoader - Loading file:/srv/dotserver/tomcat-9.0.60/webapps/ROOT/WEB-INF/classes/portal-ext.properties
opensearch_1  | [2023-02-13T08:20:16,433][INFO ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] Clustername: elastic-cluster
opensearch_1  | [2023-02-13T08:20:16,452][WARN ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] Directory /usr/share/opensearch/config has insecure file permissions (should be 0700)
opensearch_1  | [2023-02-13T08:20:16,452][WARN ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] File /usr/share/opensearch/config/kirk.pem has insecure file permissions (should be 0600)
opensearch_1  | [2023-02-13T08:20:16,452][WARN ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] File /usr/share/opensearch/config/esnode.pem has insecure file permissions (should be 0600)
opensearch_1  | [2023-02-13T08:20:16,452][WARN ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] File /usr/share/opensearch/config/root-ca.pem has insecure file permissions (should be 0600)
opensearch_1  | [2023-02-13T08:20:16,452][WARN ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] File /usr/share/opensearch/config/esnode-key.pem has insecure file permissions (should be 0600)
opensearch_1  | [2023-02-13T08:20:16,452][WARN ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] File /usr/share/opensearch/config/kirk-key.pem has insecure file permissions (should be 0600)
opensearch_1  | [2023-02-13T08:20:17,333][INFO ][o.o.p.c.PluginSettings   ] [d6cdc34291bc] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600
opensearch_1  | [2023-02-13T08:20:17,880][INFO ][o.o.i.r.ReindexPlugin    ] [d6cdc34291bc] ReindexPlugin reloadSPI called
opensearch_1  | [2023-02-13T08:20:17,880][INFO ][o.o.i.r.ReindexPlugin    ] [d6cdc34291bc] Unable to find any implementation for RemoteReindexExtension
opensearch_1  | [2023-02-13T08:20:17,909][INFO ][o.o.j.JobSchedulerPlugin ] [d6cdc34291bc] Loaded scheduler extension: opendistro_anomaly_detector, index: .opendistro-anomaly-detector-jobs
opensearch_1  | [2023-02-13T08:20:17,956][INFO ][o.o.j.JobSchedulerPlugin ] [d6cdc34291bc] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions
opensearch_1  | [2023-02-13T08:20:17,962][INFO ][o.o.j.JobSchedulerPlugin ] [d6cdc34291bc] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config
opensearch_1  | [2023-02-13T08:20:17,965][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [aggs-matrix-stats]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [analysis-common]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [geo]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [ingest-common]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [ingest-geoip]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [ingest-user-agent]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [lang-expression]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [lang-mustache]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [lang-painless]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [mapper-extras]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [opensearch-dashboards]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [parent-join]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [percolator]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [rank-eval]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [reindex]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [repository-url]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded module [transport-netty4]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-alerting]
opensearch_1  | [2023-02-13T08:20:17,966][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-anomaly-detection]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-asynchronous-search]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-cross-cluster-replication]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-index-management]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-job-scheduler]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-knn]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-ml]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-observability]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-performance-analyzer]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-reports-scheduler]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-security]
opensearch_1  | [2023-02-13T08:20:17,967][INFO ][o.o.p.PluginsService     ] [d6cdc34291bc] loaded plugin [opensearch-sql]
opensearch_1  | [2023-02-13T08:20:17,998][INFO ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml
opensearch_1  | [2023-02-13T08:20:18,020][DEPRECATION][o.o.d.c.s.Settings       ] [d6cdc34291bc] [node.max_local_storage_nodes] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version.
opensearch_1  | [2023-02-13T08:20:18,041][INFO ][o.o.e.NodeEnvironment    ] [d6cdc34291bc] using [1] data paths, mounts [[/usr/share/opensearch/data (/dev/mapper/rootvg-rootvol)]], net usable_space [1.3gb], net total_space [9.7gb], types [xfs]
opensearch_1  | [2023-02-13T08:20:18,042][INFO ][o.o.e.NodeEnvironment    ] [d6cdc34291bc] heap size [1gb], compressed ordinary object pointers [true]
opensearch_1  | [2023-02-13T08:20:18,099][INFO ][o.o.n.Node               ] [d6cdc34291bc] node name [d6cdc34291bc], node ID [6ClgGWGERe6hFZeUtqD41w], cluster name [elastic-cluster], roles [master, remote_cluster_client, data, ingest]
dotcms_1      | 13-Feb-2023 08:20:19.197 WARNING [main] org.glassfish.jersey.internal.Errors.logErrors The following warnings have been detected: WARNING: The (sub)resource method findVersions in com.dotcms.rest.api.v1.content.ContentVersionResource contains empty path annotation.
dotcms_1      | WARNING: The (sub)resource method getJvmInfo in com.dotcms.rest.api.v1.maintenance.JVMInfoResource contains empty path annotation.
dotcms_1      | WARNING: A HTTP GET method, public javax.ws.rs.core.Response com.dotcms.rest.elasticsearch.ESContentResourcePortlet.search(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.String,java.lang.String,boolean,boolean) throws com.dotmarketing.exception.DotDataException,com.dotmarketing.exception.DotSecurityException, should not consume any entity.
dotcms_1      | WARNING: The (sub)resource method listAvailableApps in com.dotcms.rest.api.v1.apps.AppsResource contains empty path annotation.
dotcms_1      | WARNING: The (sub)resource method createApp in com.dotcms.rest.api.v1.apps.AppsResource contains empty path annotation.
dotcms_1      | WARNING: The (sub)resource method deleteIndividualAppSecret in com.dotcms.rest.api.v1.apps.AppsResource contains empty path annotation.
dotcms_1      | WARNING: A HTTP GET method, public javax.ws.rs.core.Response com.dotcms.rest.api.v1.vtl.VTLResource.get(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,javax.ws.rs.core.UriInfo,java.lang.String,java.util.Map), should not consume any entity.
dotcms_1      | WARNING: A HTTP GET method, public javax.ws.rs.core.Response com.dotcms.rest.api.v1.vtl.VTLResource.get(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,javax.ws.rs.core.UriInfo,java.lang.String,java.lang.String,java.util.Map), should not consume any entity.
dotcms_1      | WARNING: A HTTP GET method, public javax.ws.rs.core.Response com.dotcms.rest.api.v1.vtl.VTLResource.dynamicGet(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,javax.ws.rs.core.UriInfo,java.lang.String,java.lang.String), should not consume any entity.
dotcms_1      | WARNING: A HTTP GET method, public javax.ws.rs.core.Response com.dotcms.rest.api.v1.vtl.VTLResource.dynamicGet(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,javax.ws.rs.core.UriInfo,java.lang.String), should not consume any entity.
dotcms_1      | WARNING: The (sub)resource method statusCheck in com.dotcms.rest.api.v1.system.monitor.MonitorResource contains empty path annotation.
dotcms_1      | WARNING: The (sub)resource method issueApiToken in com.dotcms.rest.api.v1.authentication.ApiTokenResource contains empty path annotation.
dotcms_1      | WARNING: The (sub)resource method getIndexStatus in com.dotcms.rest.api.v1.index.ESIndexResource contains empty path annotation.
dotcms_1      |
dotcms_1      | 08:20:20.142  INFO  tiles.TilesPlugin - Tiles definition factory loaded for module ''.
dotcms_1      | 08:20:20.162  INFO  validator.ValidatorPlugIn - Loading validation rules file from '/WEB-INF/validator-rules.xml'
dotcms_1      | 08:20:20.162  INFO  validator.ValidatorPlugIn - Loading validation rules file from '/WEB-INF/validation.xml'
dotcms_1      | 08:20:20.618  INFO  util.RestHighLevelClientProvider - disabling String.intern() in Jackson
dotcms_1      | 08:20:20.620  WARN  util.DotRestHighLevelClientProvider - Elastic RestHighLevelClient will be initialized without certificates
dotcms_1      | 08:20:20.625  INFO  util.DotRestHighLevelClientProvider - Initializing Elastic RestHighLevelClient using Basic authentication
dotcms_1      | 08:20:20.629  INFO  util.DotRestHighLevelClientProvider - Initializing Elastic RestHighLevelClient using endpoints [
dotcms_1      | 08:20:20.629  INFO  util.DotRestHighLevelClientProvider -   - https://opensearch:9200
dotcms_1      | 08:20:20.641  INFO  util.DotRestHighLevelClientProvider - ]
dotcms_1      | 08:20:20.641  INFO  util.DotRestHighLevelClientProvider - Initializing Elastic RestHighLevelClient using a secured https connection
dotcms_1      | 08:20:21.360  INFO  util.DotRestHighLevelClientProvider - RestHighLevelClientProvider com.dotcms.content.elasticsearch.util.DotRestHighLevelClientProvider loaded successfully
dotcms_1      | 08:20:21.495  ERROR util.DotRestHighLevelClientProvider$1 - [host=https://opensearch:9200]
dotcms_1      | 08:20:21.496  ERROR business.ESIndexAPI - Elasticsearch Attempt #1 : Connection refused
opensearch_1  | [2023-02-13T08:20:22,714][WARN ][o.o.s.c.Salt             ] [d6cdc34291bc] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
opensearch_1  | [2023-02-13T08:20:22,738][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Message routing enabled: true
opensearch_1  | [2023-02-13T08:20:22,789][INFO ][o.o.s.f.SecurityFilter   ] [d6cdc34291bc] <NONE> indices are made immutable.
opensearch_1  | [2023-02-13T08:20:23,049][INFO ][o.o.a.b.ADCircuitBreakerService] [d6cdc34291bc] Registered memory breaker.
opensearch_1  | [2023-02-13T08:20:23,340][INFO ][o.o.m.c.b.MLCircuitBreakerService] [d6cdc34291bc] Registered ML memory breaker.
opensearch_1  | [2023-02-13T08:20:23,800][INFO ][o.o.t.NettyAllocator     ] [d6cdc34291bc] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={opensearch.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]
opensearch_1  | [2023-02-13T08:20:23,874][INFO ][o.o.d.DiscoveryModule    ] [d6cdc34291bc] using discovery type [single-node] and seed hosts providers [settings]
opensearch_1  | [2023-02-13T08:20:24,276][WARN ][o.o.g.DanglingIndicesState] [d6cdc34291bc] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
opensearch_1  | [2023-02-13T08:20:24,748][INFO ][o.o.p.h.c.PerformanceAnalyzerConfigAction] [d6cdc34291bc] PerformanceAnalyzer Enabled: false
opensearch_1  | [2023-02-13T08:20:24,771][INFO ][o.o.n.Node               ] [d6cdc34291bc] initialized
opensearch_1  | [2023-02-13T08:20:24,771][INFO ][o.o.n.Node               ] [d6cdc34291bc] starting ...
opensearch_1  | [2023-02-13T08:20:24,868][INFO ][o.o.t.TransportService   ] [d6cdc34291bc] publish_address {192.168.144.2:9300}, bound_addresses {0.0.0.0:9300}
opensearch_1  | [2023-02-13T08:20:25,005][WARN ][o.o.b.BootstrapChecks    ] [d6cdc34291bc] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
opensearch_1  | [2023-02-13T08:20:25,014][INFO ][o.o.c.c.Coordinator      ] [d6cdc34291bc] setting initial configuration to VotingConfiguration{6ClgGWGERe6hFZeUtqD41w}
opensearch_1  | [2023-02-13T08:20:25,138][INFO ][o.o.c.s.MasterService    ] [d6cdc34291bc] elected-as-master ([1] nodes joined)[{d6cdc34291bc}{6ClgGWGERe6hFZeUtqD41w}{Zu281M_sRxGtqd7UaH2o1g}{192.168.144.2}{192.168.144.2:9300}{dimr}{shard_indexing_pressure_enabled=true} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 1, version: 1, delta: master node changed {previous [], current [{d6cdc34291bc}{6ClgGWGERe6hFZeUtqD41w}{Zu281M_sRxGtqd7UaH2o1g}{192.168.144.2}{192.168.144.2:9300}{dimr}{shard_indexing_pressure_enabled=true}]}
opensearch_1  | [2023-02-13T08:20:25,173][INFO ][o.o.c.c.CoordinationState] [d6cdc34291bc] cluster UUID set to [-dDjJm99SQmvWRbDc_PKAA]
opensearch_1  | [2023-02-13T08:20:25,195][INFO ][o.o.c.s.ClusterApplierService] [d6cdc34291bc] master node changed {previous [], current [{d6cdc34291bc}{6ClgGWGERe6hFZeUtqD41w}{Zu281M_sRxGtqd7UaH2o1g}{192.168.144.2}{192.168.144.2:9300}{dimr}{shard_indexing_pressure_enabled=true}]}, term: 1, version: 1, reason: Publication{term=1, version=1}
opensearch_1  | [2023-02-13T08:20:25,203][INFO ][o.o.a.c.ADClusterEventListener] [d6cdc34291bc] Cluster is not recovered yet.
opensearch_1  | [2023-02-13T08:20:25,210][INFO ][o.o.i.i.ManagedIndexCoordinator] [d6cdc34291bc] Cache master node onMaster time: 1676276425210
opensearch_1  | [2023-02-13T08:20:25,214][WARN ][o.o.p.c.s.h.ConfigOverridesClusterSettingHandler] [d6cdc34291bc] Config override setting update called with empty string. Ignoring.
opensearch_1  | [2023-02-13T08:20:25,231][INFO ][o.o.h.AbstractHttpServerTransport] [d6cdc34291bc] publish_address {192.168.144.2:9200}, bound_addresses {0.0.0.0:9200}
opensearch_1  | [2023-02-13T08:20:25,231][INFO ][o.o.n.Node               ] [d6cdc34291bc] started
opensearch_1  | [2023-02-13T08:20:25,231][INFO ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] Node started
opensearch_1  | [2023-02-13T08:20:25,231][INFO ][o.o.s.c.ConfigurationRepository] [d6cdc34291bc] Will attempt to create index .opendistro_security and default configs if they are absent
opensearch_1  | [2023-02-13T08:20:25,233][INFO ][o.o.s.OpenSearchSecurityPlugin] [d6cdc34291bc] 0 OpenSearch Security modules loaded so far: []
opensearch_1  | [2023-02-13T08:20:25,238][INFO ][o.o.s.c.ConfigurationRepository] [d6cdc34291bc] Background init thread started. Install default config?: true
opensearch_1  | [2023-02-13T08:20:25,252][INFO ][o.o.a.c.HashRing         ] [d6cdc34291bc] Node added: [6ClgGWGERe6hFZeUtqD41w]
opensearch_1  | [2023-02-13T08:20:25,258][INFO ][o.o.a.c.HashRing         ] [d6cdc34291bc] Add data node to AD version hash ring: 6ClgGWGERe6hFZeUtqD41w
opensearch_1  | [2023-02-13T08:20:25,259][INFO ][o.o.a.c.HashRing         ] [d6cdc34291bc] All nodes with known AD version: {6ClgGWGERe6hFZeUtqD41w=ADNodeInfo{version=1.3.6, isEligibleDataNode=true}}
opensearch_1  | [2023-02-13T08:20:25,259][INFO ][o.o.a.c.HashRing         ] [d6cdc34291bc] Rebuild AD hash ring for realtime AD with cooldown, nodeChangeEvents size 0
opensearch_1  | [2023-02-13T08:20:25,259][INFO ][o.o.a.c.HashRing         ] [d6cdc34291bc] Build AD version hash ring successfully
opensearch_1  | [2023-02-13T08:20:25,260][INFO ][o.o.a.c.ADDataMigrator   ] [d6cdc34291bc] Start migrating AD data
opensearch_1  | [2023-02-13T08:20:25,260][INFO ][o.o.a.c.ADDataMigrator   ] [d6cdc34291bc] AD job index doesn't exist, no need to migrate
opensearch_1  | [2023-02-13T08:20:25,260][INFO ][o.o.a.c.ADClusterEventListener] [d6cdc34291bc] Init AD version hash ring successfully
opensearch_1  | [2023-02-13T08:20:25,334][INFO ][o.o.g.GatewayService     ] [d6cdc34291bc] recovered [0] indices into cluster_state
opensearch_1  | [2023-02-13T08:20:25,495][INFO ][o.o.c.m.MetadataCreateIndexService] [d6cdc34291bc] [.opendistro_security] creating index, cause [api], templates [], shards [1]/[1]
opensearch_1  | [2023-02-13T08:20:25,503][INFO ][o.o.c.r.a.AllocationService] [d6cdc34291bc] updating number_of_replicas to [0] for indices [.opendistro_security]
opensearch_1  | [2023-02-13T08:20:25,711][INFO ][o.o.c.r.a.AllocationService] [d6cdc34291bc] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.opendistro_security][0]]]).
opensearch_1  | [2023-02-13T08:20:25,743][INFO ][o.o.s.c.ConfigurationRepository] [d6cdc34291bc] Index .opendistro_security created?: true
opensearch_1  | [2023-02-13T08:20:25,743][INFO ][o.o.s.c.ConfigurationRepository] [d6cdc34291bc] Node started, try to initialize it. Wait for at least yellow cluster state....
opensearch_1  | [2023-02-13T08:20:25,747][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'config' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:25,869][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] create_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:25,957][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'config' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:25,957][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'roles' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:25,972][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,035][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'roles' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,035][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'rolesmapping' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles_mapping.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:26,051][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,094][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'rolesmapping' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,094][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'internalusers' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/internal_users.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:26,112][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,149][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'internalusers' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,149][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'actiongroups' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/action_groups.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:26,169][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,209][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'actiongroups' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,209][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'tenants' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/tenants.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:26,223][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,256][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'tenants' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,257][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'nodesdn' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
opensearch_1  | [2023-02-13T08:20:26,267][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,308][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'nodesdn' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,308][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'whitelist' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
opensearch_1  | [2023-02-13T08:20:26,319][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,357][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'whitelist' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,357][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Will update 'audit' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/audit.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
opensearch_1  | [2023-02-13T08:20:26,387][INFO ][o.o.c.m.MetadataMappingService] [d6cdc34291bc] [.opendistro_security/49HB2KL5TSma_XDpCVWPRQ] update_mapping [_doc]
opensearch_1  | [2023-02-13T08:20:26,430][INFO ][o.o.s.s.ConfigHelper     ] [d6cdc34291bc] Doc with id 'audit' and version 2 is updated in .opendistro_security index.
opensearch_1  | [2023-02-13T08:20:26,855][INFO ][stdout                   ] [d6cdc34291bc] [FINE] No subscribers registered for event class org.opensearch.security.securityconf.DynamicConfigFactory$NodesDnModelImpl
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][stdout                   ] [d6cdc34291bc] [FINE] No subscribers registered for event class org.greenrobot.eventbus.NoSubscriberEvent
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing on REST API is enabled.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from REST API auditing.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing on Transport API is enabled.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from Transport API auditing.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing of request body is enabled.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Bulk requests resolution is disabled during request auditing.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Index resolution is enabled during request auditing.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Sensitive headers auditing is enabled.
opensearch_1  | [2023-02-13T08:20:26,856][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing requests from kibanaserver users is disabled.
opensearch_1  | [2023-02-13T08:20:26,858][WARN ][o.o.s.a.r.AuditMessageRouter] [d6cdc34291bc] No endpoint configured for categories [BAD_HEADERS, FAILED_LOGIN, MISSING_PRIVILEGES, GRANTED_PRIVILEGES, OPENDISTRO_SECURITY_INDEX_ATTEMPT, SSL_EXCEPTION, AUTHENTICATED, INDEX_EVENT, COMPLIANCE_DOC_READ, COMPLIANCE_DOC_WRITE, COMPLIANCE_EXTERNAL_CONFIG, COMPLIANCE_INTERNAL_CONFIG_READ, COMPLIANCE_INTERNAL_CONFIG_WRITE], using default endpoint
opensearch_1  | [2023-02-13T08:20:26,858][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing of external configuration is disabled.
opensearch_1  | [2023-02-13T08:20:26,858][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing of internal configuration is enabled.
opensearch_1  | [2023-02-13T08:20:26,858][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing only metadata information for read request is enabled.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing will watch {} for read requests.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing read operation requests from kibanaserver users is disabled.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing only metadata information for write request is enabled.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing diffs for write requests is disabled.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing write operation requests from kibanaserver users is disabled.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Auditing will watch <NONE> for write requests.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] .opendistro_security is used as internal security index.
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.a.i.AuditLogImpl   ] [d6cdc34291bc] Internal index used for posting audit logs is null
opensearch_1  | [2023-02-13T08:20:26,870][INFO ][o.o.s.c.ConfigurationRepository] [d6cdc34291bc] Hot-reloading of audit configuration is enabled
opensearch_1  | [2023-02-13T08:20:26,871][INFO ][o.o.s.c.ConfigurationRepository] [d6cdc34291bc] Node 'd6cdc34291bc' initialized
opensearch_1  | WARNING: An illegal reflective access operation has occurred
opensearch_1  | WARNING: Illegal reflective access by org.opensearch.security.support.Base64Helper$DescriptorNameSetter (file:/usr/share/opensearch/plugins/opensearch-security/opensearch-security-1.3.6.0.jar) to field java.io.ObjectStreamClass.name
opensearch_1  | WARNING: Please consider reporting this to the maintainers of org.opensearch.security.support.Base64Helper$DescriptorNameSetter
opensearch_1  | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
opensearch_1  | WARNING: All illegal access operations will be denied in a future release
dotcms_1      | 08:20:27.573  INFO  startup.StartupTasksExecutor - Running Startup Tasks
dotcms_1      | 08:20:27.597  INFO  cfg.Environment - Hibernate 2.1.8
dotcms_1      | 08:20:27.598  INFO  cfg.Environment - hibernate.properties not found
dotcms_1      | 08:20:27.599  INFO  cfg.Environment - using CGLIB reflection optimizer
dotcms_1      | 08:20:27.599  INFO  cfg.Environment - using JDK 1.4 java.sql.Timestamp handling
dotcms_1      | 08:20:27.600  INFO  cfg.Configuration - configuring from resource: /hibernate.cfg.xml
dotcms_1      | 08:20:27.600  INFO  cfg.Configuration - Configuration resource: /hibernate.cfg.xml
dotcms_1      | 08:20:27.625  INFO  cfg.Configuration - Configured SessionFactory: null
dotcms_1      | 08:20:27.626  INFO  cfg.Configuration - Mapping resource: META-INF/portal-hbm.xml
dotcms_1      | 08:20:27.649  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.AddressHBM -> Address
dotcms_1      | 08:20:27.673  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.CompanyHBM -> Company
dotcms_1      | 08:20:27.675  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.ImageHBM -> Image
dotcms_1      | 08:20:27.676  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.PasswordTrackerHBM -> PasswordTracker
dotcms_1      | 08:20:27.676  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.UserHBM -> User_
dotcms_1      | 08:20:27.678  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.UserTrackerHBM -> UserTracker
dotcms_1      | 08:20:27.679  INFO  cfg.Binder - Mapping class: com.liferay.portal.ejb.UserTrackerPathHBM -> UserTrackerPath
dotcms_1      | 08:20:27.679  INFO  cfg.Binder - Mapping class: com.liferay.portlet.admin.ejb.AdminConfigHBM -> AdminConfig
dotcms_1      | 08:20:27.680  INFO  cfg.Configuration - Mapping resource: com/dotmarketing/beans/DotCMSSeq.hbm.xml
dotcms_1      | 08:20:27.690  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.Inode -> inode
dotcms_1      | 08:20:27.700  INFO  cfg.Binder - Mapping joined-subclass: com.dotmarketing.portlets.containers.model.Container -> dot_containers
dotcms_1      | 08:20:27.700  INFO  cfg.Binder - Mapping joined-subclass: com.dotmarketing.portlets.folders.model.Folder -> folder
dotcms_1      | 08:20:27.701  INFO  cfg.Binder - Mapping joined-subclass: com.dotmarketing.portlets.links.model.Link -> links
dotcms_1      | 08:20:27.701  INFO  cfg.Binder - Mapping joined-subclass: com.dotmarketing.portlets.structure.model.Field -> field
dotcms_1      | 08:20:27.702  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.workflows.model.WorkflowTask -> workflow_task
dotcms_1      | 08:20:27.703  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.workflows.model.WorkflowComment -> workflow_comment
dotcms_1      | 08:20:27.703  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.workflows.model.WorkflowHistory -> workflow_history
dotcms_1      | 08:20:27.706  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.workflows.model.WorkFlowTaskFiles -> workflowtask_files
dotcms_1      | 08:20:27.706  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.Permission -> permission
dotcms_1      | 08:20:27.706  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.PermissionReference -> permission_reference
dotcms_1      | 08:20:27.707  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.Tree -> tree
dotcms_1      | 08:20:27.765  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.MultiTree -> multi_tree
dotcms_1      | 08:20:27.779  INFO  util.ReflectHelper - reflection optimizer disabled for: com.dotmarketing.beans.MultiTree, BulkBeanException: Index 1 out of bounds for length 0 (property setParent1)
dotcms_1      | 08:20:27.781  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.Clickstream -> clickstream
dotcms_1      | 08:20:27.788  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.ClickstreamRequest -> clickstream_request
dotcms_1      | 08:20:27.788  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.Clickstream404 -> clickstream_404
dotcms_1      | 08:20:27.789  INFO  cfg.Binder - Mapping class: com.dotmarketing.tag.model.Tag -> tag
dotcms_1      | 08:20:27.789  INFO  cfg.Binder - Mapping class: com.dotmarketing.tag.model.TagInode -> tag_inode
dotcms_1      | 08:20:27.795  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.ChallengeQuestion -> challenge_question
dotcms_1      | 08:20:27.795  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.UsersToDelete -> users_to_delete
dotcms_1      | 08:20:27.796  INFO  cfg.Binder - Mapping class: com.dotmarketing.plugin.model.Plugin -> plugin
dotcms_1      | 08:20:27.796  INFO  cfg.Binder - Mapping class: com.dotmarketing.business.Role -> cms_role
dotcms_1      | 08:20:27.796  INFO  cfg.Binder - Mapping class: com.dotmarketing.business.UsersRoles -> users_cms_roles
dotcms_1      | 08:20:27.800  INFO  cfg.Binder - Mapping class: com.dotmarketing.business.LayoutsRoles -> layouts_cms_roles
dotcms_1      | 08:20:27.800  INFO  cfg.Binder - Mapping class: com.dotmarketing.business.Layout -> cms_layout
dotcms_1      | 08:20:27.800  INFO  cfg.Binder - Mapping class: com.dotmarketing.business.PortletsLayouts -> cms_layouts_portlets
dotcms_1      | 08:20:27.801  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.hostvariable.model.HostVariable -> host_variable
dotcms_1      | 08:20:27.801  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.FixAudit -> fixes_audit
dotcms_1      | 08:20:27.802  INFO  cfg.Binder - Mapping class: com.dotmarketing.beans.ContainerStructure -> container_structures
dotcms_1      | 08:20:27.802  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.templates.model.TemplateVersionInfo -> template_version_info
dotcms_1      | 08:20:27.803  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.containers.model.ContainerVersionInfo -> container_version_info
dotcms_1      | 08:20:27.803  INFO  cfg.Binder - Mapping class: com.dotmarketing.portlets.links.model.LinkVersionInfo -> link_version_info
dotcms_1      | 08:20:27.803  INFO  cfg.Configuration - Mapping resource: com/dotmarketing/beans/DotCMSSeq_NOSQLGEN.hbm.xml
dotcms_1      | 08:20:27.809  INFO  cfg.Binder - Mapping class: com.dotmarketing.plugin.model.PluginProperty -> plugin_property
dotcms_1      | 08:20:27.812  INFO  cfg.Configuration - processing one-to-many association mappings
dotcms_1      | 08:20:27.812  INFO  cfg.Binder - Mapping collection: com.dotmarketing.beans.Clickstream.clickstreamRequests -> clickstream_request
dotcms_1      | 08:20:27.812  INFO  cfg.Configuration - processing one-to-one association property references
dotcms_1      | 08:20:27.812  INFO  cfg.Configuration - processing foreign key constraints
dotcms_1      | 08:20:27.818  INFO  dialect.Dialect - Using dialect: com.dotcms.repackage.net.sf.hibernate.dialect.PostgreSQLDialect
dotcms_1      | 08:20:27.820  INFO  cfg.SettingsFactory - Use outer join fetching: true
dotcms_1      | 08:20:27.835  WARN  connection.UserSuppliedConnectionProvider - No connection properties specified - the user must supply JDBC connections
dotcms_1      | 08:20:27.837  INFO  transaction.TransactionManagerLookupFactory - No TransactionManagerLookup configured (in JTA environment, use of process level read-write cache is not recommended)
dotcms_1      | 08:20:27.837  INFO  cfg.SettingsFactory - Use scrollable result sets: true
dotcms_1      | 08:20:27.837  INFO  cfg.SettingsFactory - Use JDBC3 getGeneratedKeys(): false
dotcms_1      | 08:20:27.837  INFO  cfg.SettingsFactory - Optimize cache for minimal puts: false
dotcms_1      | 08:20:27.837  INFO  cfg.SettingsFactory - Query language substitutions: {}
dotcms_1      | 08:20:27.837  INFO  cfg.SettingsFactory - cache provider: com.dotmarketing.db.NoCacheProvider
dotcms_1      | 08:20:27.838  INFO  cfg.Configuration - instantiating and configuring caches
dotcms_1      | 08:20:27.845  INFO  impl.SessionFactoryImpl - building session factory
dotcms_1      | 08:20:27.875  INFO  util.ReflectHelper - reflection optimizer disabled for: com.dotmarketing.beans.MultiTree, BulkBeanException: Index 1 out of bounds for length 0 (property setTreeOrder)
dotcms_1      | 08:20:28.176  INFO  impl.SessionFactoryObjectFactory - Not binding factory to JNDI, no JNDI name configured
dotcms_1      | 08:20:28.188  INFO  startup.StartupTasksExecutor - Not Running Startup Tasks: Task00001LoadSchema
dotcms_1      | 08:20:28.190  INFO  startup.StartupTasksExecutor - Not Running Startup Tasks: Task00002LoadClusterLicenses
dotcms_1      | 08:20:28.190  INFO  startup.StartupTasksExecutor - Running Startup Tasks : Task00003CreateSystemRoles
dotcms_1      | 08:20:28.205  INFO  startup.StartupTasksExecutor - Not Running Startup Tasks: Task00004LoadStarter
dotcms_1      | 08:20:28.207  INFO  startup.StartupTasksExecutor - Not Running Startup Tasks: Task00005LoadFixassets
dotcms_1      | 08:20:28.221  INFO  startup.StartupTasksExecutor - Running Startup Tasks : Task00006CreateSystemLayout
dotcms_1      | 08:20:28.243  INFO  caffine.CaffineCache - ***   Building Cache : dotcmslayoutcache, size:1000,Concurrency:32
dotcms_1      | 08:20:28.293  INFO  startup.StartupTasksExecutor - Not Running Startup Tasks: Task00007RemoveSitesearchQuartzJob
dotcms_1      | 08:20:28.294  INFO  startup.StartupTasksExecutor - Running Startup Tasks : Task00040CheckAnonymousUser
dotcms_1      | 08:20:28.529  INFO  apps.SecretsKeyStoreHelper - KeyStore loaded successfully after `1` tries.
dotcms_1      | 08:20:28.529  INFO  startup.StartupTasksExecutor - Not Running Startup Tasks: Task00050LoadAppsSecrets
dotcms_1      | 08:20:28.529  INFO  startup.StartupTasksExecutor - Finishing startup tasks.
dotcms_1      | 08:20:28.529  INFO  startup.StartupTasksExecutor - ---
dotcms_1      | 08:20:28.529  INFO  startup.StartupTasksExecutor -
dotcms_1      | 08:20:28.529  INFO  startup.StartupTasksExecutor - Running Upgrade Tasks
dotcms_1      | 08:20:28.529  INFO  startup.StartupTasksExecutor - Database version: 220202
dotcms_1      | 08:20:28.617  INFO  reindex.ReindexThread - --- ReindexThread Running (every 60000 millis)
dotcms_1      | 08:20:28.617  INFO  caffine.CaffineCache - ***   Building Cache : system_group, size:1000,Concurrency:32
dotcms_1      | 08:20:28.625  INFO  startup.StartupTasksExecutor - Finishing upgrade tasks.
dotcms_1      | 08:20:28.625  INFO  startup.StartupTasksExecutor - Running Backported Tasks
dotcms_1      | 08:20:28.627  INFO  reindex.ReindexThread - ---  ReindexThread is starting, background indexing will begin
dotcms_1      | 08:20:28.634  INFO  startup.StartupTasksExecutor - Not Running Backported Tasks: Task220401CreateClusterLockTable
dotcms_1      | 08:20:28.638  INFO  startup.StartupTasksExecutor - Not Running Backported Tasks: Task220402UpdateDateTimezones
dotcms_1      | 08:20:28.641  INFO  startup.StartupTasksExecutor - Running Backported Tasks : Task220912UpdateCorrectShowOnMenuProperty
dotcms_1      | 08:20:28.645  INFO  runonce.Task220912UpdateCorrectShowOnMenuProperty - Updating value of 'Show On Menu' field in all contents of type 'c541abb1-69b3-4bc5-8430-5e09e5239cc8'
dotcms_1      | 08:20:28.647  INFO  runonce.Task220912UpdateCorrectShowOnMenuProperty - Updating value of 'Show On Menu' field in all contents of type '33888b6f-7a8e-4069-b1b6-5c1aa9d0a48d'
dotcms_1      | 08:20:28.653  INFO  startup.StartupTasksExecutor - Finishing Backported tasks.
dotcms_1      | 08:20:28.660  INFO  caffine.CaffineCache - ***   Building Cache : indiciescache, size:8,Concurrency:32
dotcms_1      | 08:20:28.689  INFO  servlet.MainServlet - SuppressPropertiesBeanIntrospector enabled for legacy Struts applications
dotcms_1      | 08:20:28.696  INFO  caffine.CaffineCache - ***   Building Cache : companypool, size:10,Concurrency:32
dotcms_1      | 08:20:28.702  INFO  caffine.CaffineCache - ***   Building Cache : portletcache, size:1000,Concurrency:32
dotcms_1      | 08:20:28.773  INFO  caffine.CaffineCache - ***   Building Cache : userdotcmscache, size:1000,Concurrency:32
dotcms_1      | 08:20:28.773  INFO  caffine.CaffineCache - ***   Building Cache : useremaildotcmscache, size:1000,Concurrency:32
dotcms_1      | 08:20:28.792  INFO  config.DotInitializationService - Initializing DotCMS services
dotcms_1      | 08:20:28.793  INFO  config.DotInitializationService - Initializing :com.dotcms.api.system.event.SystemEventProcessorFactoryInitializer
dotcms_1      | 08:20:28.794  INFO  config.DotInitializationService - Initializing Done:com.dotcms.api.system.event.SystemEventProcessorFactoryInitializer, duration:0 seconds
dotcms_1      | 08:20:28.794  INFO  config.DotInitializationService - Initializing :com.dotcms.api.system.event.PayloadVerifierFactoryInitializer
dotcms_1      | 08:20:28.797  INFO  config.DotInitializationService - Initializing Done:com.dotcms.api.system.event.PayloadVerifierFactoryInitializer, duration:0 seconds
dotcms_1      | 08:20:28.797  INFO  config.DotInitializationService - Initializing :com.dotcms.system.event.local.business.LocalSystemEventSubscribersInitializer
dotcms_1      | 08:20:28.808  INFO  caffine.CaffineCache - ***   Building Cache : dotcmsrolecache, size:1000,Concurrency:32
dotcms_1      | 08:20:28.808  INFO  caffine.CaffineCache - ***   Building Cache : dotcmsrolekeycache, size:1000,Concurrency:32
dotcms_1      | 08:20:28.850  INFO  caffine.CaffineCache - ***   Building Cache : contentletcache, size:5000,Concurrency:32
dotcms_1      | 08:20:29.062  INFO  caffine.CaffineCache - ***   Building Cache : contenttypecache, size:1000,Concurrency:32
dotcms_1      | 08:20:29.153  INFO  caffine.CaffineCache - ***   Building Cache : foldercache, size:1000,Concurrency:32
dotcms_1      | 08:20:29.156  INFO  caffine.CaffineCache - ***   Building Cache : identifiercache, size:50000,Concurrency:32
dotcms_1      | 08:20:29.156  INFO  caffine.CaffineCache - ***   Building Cache : identifier404cache, size:5000,Concurrency:32
dotcms_1      | 08:20:29.173  INFO  business.FolderAPIImpl - Subscribing the folder listener: com.dotmarketing.portlets.folders.business.ApplicationContainerFolderListener, to the folder: com.dotmarketing.portlets.folders.model.Folder@344eaf7e[name=containers,sortOrder=0,showOnMenu=false,hostId=8a7d5e23-da1e-420a-b4f0-471e7da8ea2d,title=containers,filesMasks=,defaultFileType=33888b6f-7a8e-4069-b1b6-5c1aa9d0a48d,modDate=2021-03-11 18:19:08.28,iDate=2021-03-11 18:19:08.276,type=folder,owner=,inode=d6315999-4131-4e02-99d9-920a0bf2a4f8,identifier=ce965ea9-bdba-42c2-b42d-a01e1a79ddbd]
dotcms_1      | 08:20:29.179  INFO  business.FolderAPIImpl - Subscribing the folder listener: com.dotmarketing.portlets.folders.business.ApplicationTemplateFolderListener, to the folder: com.dotmarketing.portlets.folders.model.Folder@594793d1[name=templates,sortOrder=0,showOnMenu=false,hostId=8a7d5e23-da1e-420a-b4f0-471e7da8ea2d,title=templates,filesMasks=,defaultFileType=33888b6f-7a8e-4069-b1b6-5c1aa9d0a48d,modDate=2021-04-29 16:55:14.215,iDate=2021-04-29 16:55:14.197,type=folder,owner=dotcms.org.1,inode=f7f9708b-1530-4707-a514-7b15aa316a9c,identifier=01f7c900-af59-4b95-aad5-5c919742e863]
dotcms_1      | 08:20:29.198  INFO  config.DotInitializationService - Initializing Done:com.dotcms.system.event.local.business.LocalSystemEventSubscribersInitializer, duration:0 seconds
dotcms_1      | 08:20:29.198  INFO  config.DotInitializationService - Initializing :com.dotmarketing.portlets.personas.business.PersonaAPIImpl
dotcms_1      | 08:20:29.200  INFO  config.DotInitializationService - Initializing Done:com.dotmarketing.portlets.personas.business.PersonaAPIImpl, duration:0 seconds
dotcms_1      | 08:20:29.200  INFO  config.DotInitializationService - Initializing :com.dotcms.rendering.velocity.events.ExceptionHandlersInitializer
dotcms_1      | 08:20:29.201  INFO  config.DotInitializationService - Initializing Done:com.dotcms.rendering.velocity.events.ExceptionHandlersInitializer, duration:0 seconds
dotcms_1      | 08:20:29.201  INFO  config.DotInitializationService - Initializing :com.dotcms.publishing.PushPublishFiltersInitializer
dotcms_1      | 08:20:29.211  INFO  publishing.PushPublishFiltersInitializer -  Loading PushPublish Filter:  ContentOnly.yml
dotcms_1      | 08:20:29.265  INFO  publishing.PushPublishFiltersInitializer - FilterDescriptor{key=ContentOnly.yml, title=Content and Relationships, default=false, roles=DOTCMS_BACK_END_USER, filters{{excludeQuery=, excludeClasses=[ContentType, Template, Containers, Folder, Host, Links, Workflow, Language, Rule, User, OSGI], dependencies=true, excludeDependencyQuery=, excludeDependencyClasses=[ContentType, Template, Containers, Folder, Host, Links, Workflow, Language, Rule], forcePush=false, relationships=true}}}
dotcms_1      | 08:20:29.266  INFO  publishing.PushPublishFiltersInitializer -  Loading PushPublish Filter:  ForcePush.yml
dotcms_1      | 08:20:29.267  INFO  publishing.PushPublishFiltersInitializer - FilterDescriptor{key=ForcePush.yml, title=Force Push Everything, default=false, roles=DOTCMS_BACK_END_USER, filters{{excludeQuery=, excludeClasses=[], dependencies=true, excludeDependencyQuery=, excludeDependencyClasses=[], forcePush=true, relationships=true}}}
dotcms_1      | 08:20:29.267  INFO  publishing.PushPublishFiltersInitializer -  Loading PushPublish Filter:  Intelligent.yml
dotcms_1      | 08:20:29.268  INFO  publishing.PushPublishFiltersInitializer - FilterDescriptor{key=Intelligent.yml, title=Everything and Dependencies, default=true, roles=DOTCMS_BACK_END_USER, filters{{excludeQuery=, excludeClasses=[], dependencies=true, excludeDependencyQuery=, excludeDependencyClasses=[], forcePush=false, relationships=true}}}
dotcms_1      | 08:20:29.268  INFO  publishing.PushPublishFiltersInitializer -  Loading PushPublish Filter:  ShallowPush.yml
dotcms_1      | 08:20:29.268  INFO  publishing.PushPublishFiltersInitializer - FilterDescriptor{key=ShallowPush.yml, title=Only Selected Items, default=false, roles=DOTCMS_BACK_END_USER, filters{{excludeQuery=, excludeClasses=[], dependencies=false, excludeDependencyQuery=, excludeDependencyClasses=[], forcePush=false, relationships=false}}}
dotcms_1      | 08:20:29.269  INFO  publishing.PushPublishFiltersInitializer -  Loading PushPublish Filter:  WebContentOnly.yml
dotcms_1      | 08:20:29.270  INFO  publishing.PushPublishFiltersInitializer - FilterDescriptor{key=WebContentOnly.yml, title=Content, Assets and Pages, default=false, roles=DOTCMS_BACK_END_USER, filters{{excludeQuery=, excludeClasses=[Host, Workflow, ContentType, User, OSGI, Language], dependencies=true, excludeDependencyQuery=, excludeDependencyClasses=[Host, Workflow, ContentType, Language], forcePush=false, relationships=true}}}
dotcms_1      | 08:20:29.270  INFO  config.DotInitializationService - Initializing Done:com.dotcms.publishing.PushPublishFiltersInitializer, duration:0 seconds
dotcms_1      | 08:20:29.270  INFO  config.DotInitializationService - Initializing :com.dotmarketing.business.ThemeAPIImpl
dotcms_1      | 08:20:29.280  INFO  caffine.CaffineCache - ***   Building Cache : languagecacheimpl, size:1000,Concurrency:32
dotcms_1      | 08:20:29.297  INFO  caffine.CaffineCache - ***   Building Cache : hostcache, size:1000,Concurrency:32
dotcms_1      | 08:20:29.299  INFO  config.DotInitializationService - Initializing Done:com.dotmarketing.business.ThemeAPIImpl, duration:0 seconds
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger - InitServlet: Setting Default Timezone: Coordinated Universal Time
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Initializing dotCMS
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Using database: PostgreSQL
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Using dialect : com.dotcms.repackage.net.sf.hibernate.dialect.PostgreSQLDialect
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Company Name  : dotcms.org
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    License       : COMMUNITY EDITION
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Licensed to   : dotCMS Community License
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Server id     : 0b6c0b4a
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    License id    : COMMUNITY EDITION
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -    Expires       : never
dotcms_1      | 08:20:29.312  INFO  servlets.StartupLogger -
dotcms_1      | 08:20:29.312  INFO  servlets.InitServlet -
dotcms_1      | 08:20:29.312  INFO  servlets.InitServlet -
dotcms_1      | 08:20:29.327  INFO  business.VanityUrlAPIImpl - Populating Vanity URLs for Site 'default'
dotcms_1      | 08:20:29.335  INFO  caffine.CaffineCache - ***   Building Cache : vanityurlsitecache, size:5000,Concurrency:32
dotcms_1      | 08:20:29.335  INFO  business.VanityUrlAPIImpl - Populating Vanity URLs for Site 'System Host'
dotcms_1      | 08:20:29.373  INFO  impl.StdSchedulerFactory - Using default implementation for ThreadExecutor
dotcms_1      | 08:20:29.374  INFO  simpl.SimpleThreadPool - Job execution threads will use class loader of thread: main
dotcms_1      | 08:20:29.387  INFO  core.SchedulerSignalerImpl - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
dotcms_1      | 08:20:29.389  INFO  core.QuartzScheduler - Quartz Scheduler v.1.8.6 created.
dotcms_1      | 08:20:29.391  INFO  quartz.DotJobStore - Using db table-based data access locking (synchronization).
dotcms_1      | 08:20:29.393  INFO  quartz.DotJobStore - JobStoreCMT initialized.
dotcms_1      | 08:20:29.394  INFO  core.QuartzScheduler - Scheduler meta-data: Quartz Scheduler (v1.8.6) 'dotCMSQuartz' with instanceId 'NON_CLUSTERED'
dotcms_1      |   Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
dotcms_1      |   NOT STARTED.
dotcms_1      |   Currently in standby mode.
dotcms_1      |   Number of jobs executed: 0
dotcms_1      |   Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
dotcms_1      |   Using job-store 'com.dotmarketing.quartz.DotJobStore' - which supports persistence. and is clustered.
dotcms_1      |
dotcms_1      | 08:20:29.394  INFO  impl.StdSchedulerFactory - Quartz scheduler 'dotCMSQuartz' initialized from an externally provided properties instance.
dotcms_1      | 08:20:29.394  INFO  impl.StdSchedulerFactory - Quartz scheduler version: 1.8.6
dotcms_1      | 08:20:29.394  INFO  init.DotInitScheduler - Automatic Content Reindexation Cron Thread schedule disabled on this server
dotcms_1      | 08:20:29.394  INFO  init.DotInitScheduler - Deleting ContentReindexerJob Job
dotcms_1      | 08:20:29.400  INFO  init.DotInitScheduler - Users To Delete Cron Thread schedule disabled on this server
dotcms_1      | 08:20:29.400  INFO  init.DotInitScheduler - Deleting UsersToDeleteJob Job
dotcms_1      | 08:20:29.473  INFO  init.DotInitScheduler - LinkCheckerJob Cron Job schedule disabled on this server
dotcms_1      | 08:20:29.541  INFO  quartz.DotJobStore - ClusterManager: detected 1 failed or restarted instances.
dotcms_1      | 08:20:29.541  INFO  quartz.DotJobStore - ClusterManager: Scanning for instance "8ea56e2f-2c4b-4624-87e0-84a13bd2640f"'s failed in-progress jobs.
dotcms_1      | 08:20:29.544  INFO  quartz.DotJobStore - ClusterManager: ......Freed 1 acquired trigger(s).
dotcms_1      | 08:20:29.546  INFO  core.QuartzScheduler - Scheduler dotCMSQuartz_$_NON_CLUSTERED started.
dotcms_1      | 08:20:29.590  INFO  runtime.ConcurrentParserPoolImpl - Initializing Velocity Parser Pool - min: 20 max:100 @ main
dotcms_1      | 08:20:29.603  INFO  caffine.CaffineCache - ***   Building Cache : velocitycache, size:1000,Concurrency:32
dotcms_1      | 08:20:29.638  INFO  caffine.CaffineCache - ***   Building Cache : velocitymacrocache, size:1000,Concurrency:32
dotcms_1      | 08:20:29.671  INFO  dotpubsub.PostgresPubSubImpl - Starting PostgresPubSub. Have servers:0
dotcms_1      | 08:20:29.671  INFO  dotpubsub.PostgresPubSubImpl - PGNotificationListener connecting to pub/sub...
dotcms_1      | 08:20:30.086  WARN  struts.MultiMessageResources - Duplicate resource property definition (key=was ==> is now): Move=Move ==> Move
dotcms_1      | 08:20:35.564  INFO  caffine.CaffineCache - ***   Building Cache : translatedquerycache, size:5000,Concurrency:32
opensearch_1  | [2023-02-13T08:20:35,638][DEPRECATION][o.o.d.c.m.IndexNameExpressionResolver] [d6cdc34291bc] this request accesses system indices: [.opendistro_security], but in a future major version, direct access to system indices will be prevented by default
dotcms_1      | 08:20:35.748  WARN  business.ESContentFactoryImpl - ----------------------------------------------
dotcms_1      | 08:20:35.748  WARN  business.ESContentFactoryImpl - Elasticsearch SEARCH error in index 'cluster_6a5d9b2e71.working_20230209094959'
dotcms_1      | 08:20:35.748  WARN  business.ESContentFactoryImpl - Thread: Thread-10
dotcms_1      | 08:20:35.749  WARN  business.ESContentFactoryImpl - ES Query: {"size":10000,"timeout":"15000ms","query":{"query_string":{"query":"+contenttype:host +working:true -basetype:6 -basetype:3","fields":[],"type":"best_fields","default_operator":"or","max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"auto_generate_synonyms_phrase_query":true,"fuzzy_transpositions":true,"boost":1.0}},"_source":{"includes":["inode","identifier"],"excludes":[]},"sort":[{"moddate":{"order":"desc"}}],"track_total_hits":10000000}
dotcms_1      | 08:20:35.749  WARN  business.ESContentFactoryImpl - Class org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=index_not_found_exception, reason=no such index [cluster_6a5d9b2e71.working_20230209094959]]
dotcms_1      | 08:20:35.749  WARN  business.ESContentFactoryImpl - ----------------------------------------------
opensearch_1  | [2023-02-13T08:20:55,210][INFO ][o.o.c.r.a.DiskThresholdMonitor] [d6cdc34291bc] low disk watermark [85%] exceeded on [6ClgGWGERe6hFZeUtqD41w][d6cdc34291bc][/usr/share/opensearch/data/nodes/0] free: 1.3gb[13.5%], replicas will not be assigned to this node
dotcms_1      | 08:21:00.012  INFO  job.FreeServerFromClusterJob - Server 0b6c0b4a-1697-4d03-9055-2bec7bda89ae with license null was Removed
dotcms_1      | 08:21:00.044  INFO  caffine.CaffineCache - ***   Building Cache : publishingendpointcache, size:100,Concurrency:32
opensearch_1  | [2023-02-13T08:21:25,212][INFO ][o.o.i.i.ManagedIndexCoordinator] [d6cdc34291bc] Performing move cluster state metadata.
opensearch_1  | [2023-02-13T08:21:25,213][INFO ][o.o.i.i.MetadataService  ] [d6cdc34291bc] ISM config index not exist, so we cancel the metadata migration job.
opensearch_1  | [2023-02-13T08:21:25,214][INFO ][o.o.i.i.ManagedIndexCoordinator] [d6cdc34291bc] Performing ISM template migration.
opensearch_1  | [2023-02-13T08:21:25,214][INFO ][o.o.i.i.m.ISMTemplateService] [d6cdc34291bc] Doing ISM template migration 1 time.
opensearch_1  | [2023-02-13T08:21:25,215][INFO ][o.o.i.i.m.ISMTemplateService] [d6cdc34291bc] Use 2023-02-13T07:20:25.210Z as migrating ISM template last_updated_time
opensearch_1  | [2023-02-13T08:21:25,215][INFO ][o.o.i.i.m.ISMTemplateService] [d6cdc34291bc] ISM templates: {}
opensearch_1  | [2023-02-13T08:21:25,216][INFO ][o.o.i.i.m.ISMTemplateService] [d6cdc34291bc] Policies to update: []
opensearch_1  | [2023-02-13T08:21:25,218][INFO ][o.o.i.i.m.ISMTemplateService] [d6cdc34291bc] Failure experienced when migrating ISM Template and update ISM policies: {}
opensearch_1  | [2023-02-13T08:21:25,256][INFO ][o.o.c.s.ClusterSettings  ] [d6cdc34291bc] updating [plugins.index_state_management.template_migration.control] from [0] to [-1]
opensearch_1  | [2023-02-13T08:21:25,263][INFO ][o.o.i.i.m.ISMTemplateService] [d6cdc34291bc] Successfully update template migration setting
dotcms_1      | 08:21:28.736  INFO  reindex.ReindexThread - --- ReindexThread Paused (every 3600000 millis)
dotcms_1      | 08:21:29.996  WARN  dotpubsub.PostgresPubSubImpl - Connection Error: Timeout starting connection

dotcms_1      | com.impossibl.postgres.jdbc.PGSQLSimpleException: Connection Error: Timeout starting connection
dotcms_1      |         at com.impossibl.postgres.jdbc.ErrorUtils.makeSQLException(ErrorUtils.java:138) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:326) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:279) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.PGDriver.connect(PGDriver.java:104) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
dotcms_1      |         at java.sql.DriverManager.getConnection(DriverManager.java:251) ~[java.sql:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.getConnection(PostgresPubSubImpl.java:226) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:155) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:59) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.QueuingPubSubWrapper.start(QueuingPubSubWrapper.java:74) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.<init>(OSGIUtil.java:146) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.<init>(OSGIUtil.java:139) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil$OSGIUtilHolder.<clinit>(OSGIUtil.java:136) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.getInstance(OSGIUtil.java:132) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.servlets.InitServlet.init(InitServlet.java:247) ~[dotcms_22.03.4_999999.jar:?]

dotcms_1      |         at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1164) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1117) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1010) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5264) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1185) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1933) ~[catalina.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
dotcms_1      |         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
dotcms_1      |         at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-util.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118) ~[?:?]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:1095) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:477) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1618) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:319) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:946) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386) ~[catalina.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
dotcms_1      |         at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-util.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) ~[?:?]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:919) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:263) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardService.startInternal(StandardService.java:432) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:927) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.Catalina.start(Catalina.java:772) ~[catalina.jar:9.0.60]
dotcms_1      |         at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
dotcms_1      |         at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
dotcms_1      |         at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
dotcms_1      |         at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
dotcms_1      |         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345) ~[bootstrap.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476) ~[bootstrap.jar:9.0.60]
dotcms_1      | Caused by: java.io.IOException: Timeout starting connection
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.startup(ServerConnectionFactory.java:459) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.connect(ServerConnectionFactory.java:200) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.connect(ServerConnectionFactory.java:143) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.connect(ServerConnectionFactory.java:124) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.system.BasicContext.<init>(BasicContext.java:213) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.PGDirectConnection.<init>(PGDirectConnection.java:235) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:317) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         ... 57 more
dotcms_1      | 08:21:29.996  WARN  dotpubsub.PostgresPubSubImpl - Restarting PGNotificationListener in 0 ms to retry postgres pub/sub connection
dotcms_1      | 08:21:30.999  INFO  dotpubsub.PostgresPubSubImpl - Starting PostgresPubSub. Have servers:0
dotcms_1      | 08:21:30.999  INFO  dotpubsub.PostgresPubSubImpl - PGNotificationListener connecting to pub/sub...
opensearch_1  | [2023-02-13T08:22:25,213][INFO ][o.o.i.i.ManagedIndexCoordinator] [d6cdc34291bc] Cancel background move metadata process.
opensearch_1  | [2023-02-13T08:22:25,213][INFO ][o.o.i.i.ManagedIndexCoordinator] [d6cdc34291bc] Performing move cluster state metadata.
opensearch_1  | [2023-02-13T08:22:25,213][INFO ][o.o.i.i.MetadataService  ] [d6cdc34291bc] Move metadata has finished.
dotcms_1      | 08:22:31.009  WARN  dotpubsub.PostgresPubSubImpl - Connection Error: Timeout starting connection

dotcms_1      | com.impossibl.postgres.jdbc.PGSQLSimpleException: Connection Error: Timeout starting connection
dotcms_1      |         at com.impossibl.postgres.jdbc.ErrorUtils.makeSQLException(ErrorUtils.java:138) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:326) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:279) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.PGDriver.connect(PGDriver.java:104) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
dotcms_1      |         at java.sql.DriverManager.getConnection(DriverManager.java:251) ~[java.sql:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.getConnection(PostgresPubSubImpl.java:226) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:155) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:59) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.restart(PostgresPubSubImpl.java:191) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.setUpPersistantConnection(PostgresPubSubImpl.java:161) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.PostgresPubSubImpl.start(PostgresPubSubImpl.java:59) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotcms.dotpubsub.QueuingPubSubWrapper.start(QueuingPubSubWrapper.java:74) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.<init>(OSGIUtil.java:146) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.<init>(OSGIUtil.java:139) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil$OSGIUtilHolder.<clinit>(OSGIUtil.java:136) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at org.apache.felix.framework.OSGIUtil.getInstance(OSGIUtil.java:132) ~[dotcms_22.03.4_999999.jar:?]
dotcms_1      |         at com.dotmarketing.servlets.InitServlet.init(InitServlet.java:247) ~[dotcms_22.03.4_999999.jar:?]

dotcms_1      |         at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1164) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1117) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1010) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5264) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1185) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1933) ~[catalina.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
dotcms_1      |         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
dotcms_1      |         at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-util.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118) ~[?:?]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:1095) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:477) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1618) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:319) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:946) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1396) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1386) ~[catalina.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
dotcms_1      |         at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-util.jar:9.0.60]
dotcms_1      |         at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) ~[?:?]
dotcms_1      |         at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:919) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:263) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardService.startInternal(StandardService.java:432) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:927) ~[catalina.jar:9.0.60]

dotcms_1      |         at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[catalina.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.Catalina.start(Catalina.java:772) ~[catalina.jar:9.0.60]
dotcms_1      |         at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
dotcms_1      |         at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
dotcms_1      |         at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
dotcms_1      |         at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
dotcms_1      |         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345) ~[bootstrap.jar:9.0.60]
dotcms_1      |         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476) ~[bootstrap.jar:9.0.60]
dotcms_1      | Caused by: java.io.IOException: Timeout starting connection
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.startup(ServerConnectionFactory.java:459) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.connect(ServerConnectionFactory.java:200) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.connect(ServerConnectionFactory.java:143) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.protocol.v30.ServerConnectionFactory.connect(ServerConnectionFactory.java:124) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.system.BasicContext.<init>(BasicContext.java:213) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.PGDirectConnection.<init>(PGDirectConnection.java:235) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         at com.impossibl.postgres.jdbc.ConnectionUtil.createConnection(ConnectionUtil.java:317) ~[pgjdbc-ng-0.8.7.jar:0.8.7]
dotcms_1      |         ... 60 more
dotcms_1      | 08:22:31.009  WARN  dotpubsub.PostgresPubSubImpl - Restarting PGNotificationListener in 1000 ms to retry postgres pub/sub connection
dotcms_1      | 08:22:33.012  INFO  dotpubsub.PostgresPubSubImpl - Starting PostgresPubSub. Have servers:0
dotcms_1      | 08:22:33.012  INFO  dotpubsub.PostgresPubSubImpl - PGNotificationListener connecting to pub/sub...


==========================================================================================

docker-compose.yml configuration:

# Sample Docker Compose file for dotCMS 22.03.4 LTS ##
version: '3.5'

networks:
#db_net:
opensearch-net:

volumes:
cms-shared:
#dbdata:
opensearch-data:

services:
opensearch:
image: opensearchproject/opensearch:1.3.6
environment:
cluster.name=elastic-cluster
- discovery.type=single-node
- data
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xmx1G "
ulimits:
memlock:
soft: -1 # Set memlock to unlimited (no soft or hard limit)
hard: -1
nofile:
soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
hard: 65536
ports:
- 9200:9200
- 9600:9600
volumes:
- opensearch-data:/usr/share/opensearch/data
networks:
- opensearch-net

dotcms:
image: dotcms/dotcms:22.03.4_lts
environment:
CMS_JAVA_OPTS: '-Xmx1g '
LANG: 'C.UTF-8'
TZ: 'UTC'
DOT_POSTGRES_PUBSUB_JDBC_URL: "jdbc:pgsql://postgres:post...@10.222.13.46:5432/postgres"
DB_BASE_URL: "jdbc:postgresql://10.222.13.46:5432/postgres"
DB_USERNAME: 'postgres'
DB_PASSWORD: 'postgres'
DOT_ES_AUTH_BASIC_PASSWORD: 'admin'
DOT_ES_ENDPOINTS: 'https://opensearch:9200'
DOT_INITIAL_ADMIN_PASSWORD: 'admin'
depends_on:
- opensearch
volumes:
- cms-shared:/data/shared
#- {license_local_path}/license.zip:/data/shared/assets/license.zip
networks:
#- db_net
- opensearch-net
ports:
- "8082:8082"
- "8443:8443"

christy jones

unread,
Feb 21, 2023, 9:53:38 AM2/21/23
to dotCMS User Group
We produce CDL Drivers license Ids SSN Passport Birth Certificate Covid-19 card. in this countries United Sates Australia Canada United Kingdom Whatsapp/Calls/Texts:+1 4708395024

Kachina Vazhuthi Ilavaluthi

unread,
Feb 21, 2023, 10:12:36 AM2/21/23
to dotCMS User Group
Will Ezel,

Any suggestions on this isse. The dotcms is failing to connect to the postgres. Now we have this issue on our UAT and PRODCUTION dotCMS environments as well blocking customers. Need your help in resolving this issue quickly. thanks.

Will Ezell

unread,
Feb 28, 2023, 9:27:39 AM2/28/23
to dot...@googlegroups.com
I don't have any further guidance - I would suggest raising a support ticket?

Todd Jacobsen

unread,
Mar 1, 2023, 12:18:20 PM3/1/23
to dotCMS User Group
I have some thoughts about this - it may be a routing/networking issue. You said that the postgres server at 10.222.13.46 is running locally on the docker host - NOT from docker-compose. 

By default, docker containers use the "bridge" Docker network driver which sets up an internal networks managed by Docker, visibly only to the containers. The docker-compose file you shared is using "bridge" mode.

In "bridge" mode, Docker containers cannot normally access services running on the Docker host machine. This is by design. See the docker docs on their available networking drivers 
https://docs.docker.com/network/

As I understand your current setup, I would not expect that dotCMS running in Docker bridge mode could connect to a postgres server (or any server) running on the Docker host. There are caveats because routing can be tricky.

I'm not suggesting you do this, but just FYI the most common way to allow containers to access services running on the Docker host is to use "host" networking mode rather than "bridge" networking mode. There are significant trade-offs to using "host" vs "bridge" so do not make this change unless you fully understand the consequences.

If this was working before the restart, perhaps someone configured some custom routing to allow dotcms to reach postgres. Docker bridge networks make extensive use of iptables in linux. Perhaps someone set up a custom iptables rule to allow this to work. That is just a fairly wild guess though.

HTH,
Todd
Reply all
Reply to author
Forward
0 new messages