Brand new installation failed to stop

[Jin Mao]

Gents,

I was playing around with installing and uninstalling Rundeck 4.3.2 and 4.5.0 packages on a Ubuntu 18 server. The purpose was to build automation to deploy the Rundeck service. There is no customization to files under /etc/rundeck.

At a certain point, I am not sure what was changed, Rundeck started failing to start. Even I did 'dpkg -P rundeck && /bin/rm -rf /etc/rundeck /var/lib/rundeck /var/log/rundeck', the problem still exists. 

I made sure that /var/lib/rundeck, /etc/rundeck, /var/log/rundeck are owned by rundeck:rundeck. The disk has enough space. There is no other tenant on the server.

However, Java produces a tremendous amount of exception messages. But I couldn't really understand what it was trying to say.

[2022-09-19T09:59:48,096] WARN  context.AnnotationConfigServletWebServerApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'requestMappingHandlerMapping' defined in org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'grailsInterceptorMappedInterceptor': Cannot create inner bean '(inner bean)#4c5bcc6e' of type [org.grails.plugins.web.interceptors.GrailsInterceptorHandlerInterceptorAdapter] while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name '(inner bean)#4c5bcc6e': Unsatisfied dependency expressed through method 'setInterceptors' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'controllerBaseInterceptor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'uiPluginService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'frameworkService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'executionService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'notificationService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mailService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mailMessageBuilderFactory': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mailMessageContentRenderer': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'groovyPagesTemplateEngine': Cannot resolve reference to bean 'gspTagLibraryLookup' while setting bean property 'tagLibraryLookup'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gspTagLibraryLookup': Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'rundeck.RepositoryTagLib': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'repoClient': Cannot resolve reference to bean 'repositoryManager' while setting bean property 'repositoryManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'repositoryManager': Error setting property values; nested exception is org.springframework.beans.PropertyBatchUpdateException; nested PropertyAccessExceptions (1) are:

PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'repositoryDefinitionListDatasourceUrl' threw exception; nested exception is java.lang.ExceptionInInitializerError

[2022-09-19T09:59:48,171] ERROR boot.SpringApplication - Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'requestMappingHandlerMapping' defined in org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'grailsInterceptorMappedInterceptor': Cannot create inner bean '(inner bean)#4c5bcc6e' of type [org.grails.plugins.web.interceptors.GrailsInterceptorHandlerInterceptorAdapter] while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name '(inner bean)#4c5bcc6e': Unsatisfied dependency expressed through method 'setInterceptors' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'controllerBaseInterceptor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'uiPluginService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'frameworkService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'executionService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'notificationService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mailService': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mailMessageBuilderFactory': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mailMessageContentRenderer': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'groovyPagesTemplateEngine': Cannot resolve reference to bean 'gspTagLibraryLookup' while setting bean property 'tagLibraryLookup'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gspTagLibraryLookup': Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'rundeck.RepositoryTagLib': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'repoClient': Cannot resolve reference to bean 'repositoryManager' while setting bean property 'repositoryManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'repositoryManager': Error setting property values; nested exception is org.springframework.beans.PropertyBatchUpdateException; nested PropertyAccessExceptions (1) are:

PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'repositoryDefinitionListDatasourceUrl' threw exception; nested exception is java.lang.ExceptionInInitializerError

Java is openjdk version "1.8.0_342", same as other working installations. 

The only way to start rundeck was:

java -jar /PATH_TO/rundeck-4.3.2-20220714.war -d /$TMPDIR

I feel directory permission was messed up but not sure what exactly it was. Did anyone experience this issue before or may know a solution?

Thanks!

[Jin Mao]

I knew it is a silly human error and I found it.

During the task, a directory called /tmp/rundeck was manually created by a different user (not rundeck). This directory ownership blocked java from using this directory as a temporary directory. I traced it down by printing the full rundeckd runtime command line and validating each individual configuration. I really wish there was a more informational prompt saying permission denied but there were only mysterious messages about failing to create beans.