Kieserver war scanner thread hangs intermittently

[Richa Singh]

Hi,

We have deployed KieServer.war version 7.73.2 in docker on Azure app service (https://quay.io/repository/kiegroup/kie-server?tab=tags&tag=7.73.0.Final) and have enabled the scanner.

I see the warning messages in Kie Scanner and after sometime the scanner thread appears to hang.   Are there any known issues with scanner? Also I see there are no new updates after version 7.73.0. What will be the path forward to enable us to use kie server?

Failed to define class .module-info in Module "deployment.kie-server.war" from Service Module Loader: java.lang.NoClassDefFoundError: Failed to link /module-info (Module "deployment.kie-server.war" from Service Module Loader): /module-info is not a class because access_flag ACC_MODULE is set 2023-04-13T16:55:26.055660010Z at java.base/java.lang.ClassLoader.defineClass1(Native Method) 2023-04-13T16:55:26.055665411Z at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017) 2023-04-13T16:55:26.055669211Z at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1096) 2023-04-13T16:55:26.055672811Z at org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:424) 2023-04-13T16:55:26.055677111Z at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:555) 2023-04-13T16:55:26.055681411Z at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:339) 2023-04-13T16:55:26.055685211Z at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:126) 2023-04-13T16:55:26.055688711Z at org.jboss.modules.Module.loadModuleClass(Module.java:753) 2023-04-13T16:55:26.055692211Z at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:247) 2023-04-13T16:55:26.055695511Z at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:410) 2023-04-13T16:55:26.055699111Z at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398) 2023-04-13T16:55:26.055702411Z at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:128) 2023-04-13T16:55:26.055716311Z at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:576) 2023-04-13T16:55:26.055719812Z at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:576) 2023-04-13T16:55:26.055723212Z at deployment.kie-server.war//org.drools.reflective.classloader.ProjectClassLoader.internalLoadClass(ProjectClassLoader.java:173) 2023-04-13T16:55:26.055727012Z at deployment.kie-server.war//org.drools.reflective.classloader.ProjectClassLoader.loadClass(ProjectClassLoader.java:146) 2023-04-13T16:55:26.055730812Z at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) 2023-04-13T16:55:26.055733912Z at deployment.kie-server.war//org.kie.server.services.drools.DroolsKieServerExtension.createContainer(DroolsKieServerExtension.java:115) 2023-04-13T16:55:26.055737712Z at deployment.kie-server.war//org.kie.server.services.drools.DroolsKieServerExtension.updateContainer(DroolsKieServerExtension.java:138) 2023-04-13T16:55:26.055740912Z at deployment.kie-server.war//org.kie.server.services.impl.KieServerScannerEventListener.onKieScannerUpdateResultsEvent(KieServerScannerEventListener.java:76) 2023-04-13T16:55:26.055744212Z at deployment.kie-server.war//org.drools.compiler.kie.builder.impl.event.KieScannerEventSupport.lambda$fireKieScannerUpdateResultsEventImpl$1(KieScannerEventSupport.java:38) 2023-04-13T16:55:26.055747812Z at deployment.kie-server.war//org.drools.core.event.AbstractEventSupport.notifyAllListeners(AbstractEventSupport.java:60) 2023-04-13T16:55:26.055751212Z at deployment.kie-server.war//org.drools.compiler.kie.builder.impl.event.KieScannerEventSupport.fireKieScannerUpdateResultsEventImpl(KieScannerEventSupport.java:38) 2023-04-13T16:55:26.055754612Z at deployment.kie-server.war//org.kie.scanner.KieRepositoryScannerImpl.updateKieModule(KieRepositoryScannerImpl.java:207) 2023-04-13T16:55:26.055758012Z at deployment.kie-server.war//org.kie.scanner.KieRepositoryScannerImpl.internalUpdate(KieRepositoryScannerImpl.java:186) 2023-04-13T16:55:26.055761112Z at deployment.kie-server.war//org.kie.scanner.KieRepositoryScannerImpl.internalUpdate(KieRepositoryScannerImpl.java:55) 2023-04-13T16:55:26.055764312Z at deployment.kie-server.war//org.drools.compiler.kie.builder.impl.AbstractKieScanner.scanNow(AbstractKieScanner.java:147) 2023-04-13T16:55:26.055767512Z at deployment.kie-server.war//org.drools.compiler.kie.builder.impl.AbstractKieScanner$ScanTask.run(AbstractKieScanner.java:127) 2023-04-13T16:55:26.055770712Z at java.base/java.util.TimerThread.mainLoop(Timer.java:556) 2023-04-13T16:55:26.055774013Z at java.base/java.util.TimerThread.run(Timer.java:506) 2023-04-13T16:55:26.055777313Z

[Richa Singh]

On further troubleshooting, we found 

1. Scanner pulls artifacts and deploys.

2. After 10 minutes it accesses maven, sees a new artifact version 

3. it tries to pull the artifact but gets  401 response(  Caused by: org.apache.http.client.HttpResponseException: status code: 401, reason phrase: Unauthorized (401)

4.After that the timer thread does not respond. The scanner status for the container says

scanner:

status: started

5. The server needs to be restarted.

The settings we are using is as per:

https://docs.drools.org/7.73.0.Final/drools-docs/html_single/index.html#_ch.kie.server

Any guidance will be highly appreciated

Thanks

Richa