new prod server error

[Andrew Meyer]

Bouncing between setting up new environments I am now getting this error on a new server:

 ...killed.
[2024-05-14T13:35:51,534] ERROR boot.SpringApplication - Application run failed
io.micronaut.context.exceptions.NoSuchBeanException: No bean of type [io.micronaut.context.event.ApplicationEventPublisher<io.micronaut.context.event.StartupEvent>] exists. Make sure the bean is not disabled by bean requirements (enable trace logging for 'io.micronaut.context.condition' to check) and if the bean is enabled then ensure the class is declared a bean and annotation processing is enabled (for Java and Kotlin the 'micronaut-inject-java' dependency should be configured as an annotation processor).
        at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2805) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1617) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:867) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at io.micronaut.context.BeanLocator.getBean(BeanLocator.java:94) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at io.micronaut.context.DefaultBeanContext.publishEvent(DefaultBeanContext.java:1703) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:356) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:194) ~[micronaut-inject-3.8.6.jar!/:3.8.6]
        at grails.boot.GrailsApp.createApplicationContext(GrailsApp.groovy:184) ~[grails-core-5.3.2.jar!/:5.3.2]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:304) [spring-boot-2.7.12.jar!/:2.7.12]
        at grails.boot.GrailsApp.run(GrailsApp.groovy:99) [grails-core-5.3.2.jar!/:5.3.2]
        at grails.boot.GrailsApp.run(GrailsApp.groovy:485) [grails-core-5.3.2.jar!/:5.3.2]
        at grails.boot.GrailsApp.run(GrailsApp.groovy:472) [grails-core-5.3.2.jar!/:5.3.2]
        at grails.boot.GrailsApp$run.call(Unknown Source) [grails-core-5.3.2.jar!/:?]
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:148) [groovy-3.0.9.jar!/:3.0.9]
        at rundeckapp.Application.execRunApp(Application.groovy:92) [classes!/:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107) [groovy-3.0.9.jar!/:3.0.9]
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.invoke(StaticMetaMethodSite.java:44) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:100) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:217) [groovy-3.0.9.jar!/:3.0.9]
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:222) [groovy-3.0.9.jar!/:3.0.9]
        at rundeckapp.Application.main(Application.groovy:81) [classes!/:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) [rundeck-4.15.0-20230725.war:?]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) [rundeck-4.15.0-20230725.war:?]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) [rundeck-4.15.0-20230725.war:?]
        at org.springframework.boot.loader.WarLauncher.main(WarLauncher.java:59) [rundeck-4.15.0-20230725.war:?]

[rac...@rundeck.com]

Hi Andrew,

More context is needed. Is that happening at the moment of login? Could you elaborate?

Thanks!

[Andrew Meyer]

I configured the server the same way as the others then I restarted the application using systemd.

I can provide config...but I'm just as confused at the error.

[Andrew Meyer]

What else can I provide to help you to help me?

[rac...@rundeck.com]

Hi Andrew,

1. Please check the prerequisites (especially the Java version).
2. How did you install the instance? Could you describe the process? Which OS and version are you using?
3. Is that the full service.log output? Is this happening at the moment of launching Rundeck?

Regards.

​

[rac...@rundeck.com]

Also, provide the Rundeck version and the Database. Greetings.

[Andrew Meyer]

Hello, this is happening to me again.  I'm running Rundeck 4.15.x using the RPM install, on RHEL 8 latest, using Oracle 19c for the DB.  Java version is 11.0.18.

The only other thing I did recently is change the UID and GID of the rundeck user.

[rac...@rundeck.com]

Hi Andrew,

Hello, this is happening to me again. I’m running Rundeck 4.15.x using the RPM install, on RHEL 8 latest, using Oracle 19c for the DB. Java version is 11.0.18.

So, is it a random issue then? is the same server of the original post? could you describe how you install/upgrade your instance?

The only other thing I did recently is change the UID and GID of the rundeck user.

Why do you change the rundeck user UID/GID?

What about this question?:

Is that the full service.log output? Is this happening at the moment of launching Rundeck?

I’m pretty sure that you’re facing a Rundeck installation integrity issue. Did you test a fresh install without those (GID/UID) changes?

Regards!

​

[Andrew Meyer]

This is a fresh install.  I have been exporting and importing the jobs, etc to it.

I changed it because the UID/GID conflicted with something else installed in our environment.  We try to keep everything uniform across all environments.

I can give you the most recent...

[2024-09-04T10:06:02,512] ERROR boot.SpringApplication - Application run failed

[2024-09-04T10:06:49,959] ERROR boot.SpringApplication - Application run failed

[rac...@rundeck.com]

Hi,

Similar to this, it seems hard to reproduce due to the permissions changes and some environment customization. Could you double-check the permissions used in your server? (and compare with the working servers) Also, try in a test environment without these changes to determine what is the root cause. Pretty sure that some env changes cause a Rundeck core initialization failure.

Regards.