StackOverflowException when running cryostat agent in a spring.boot application

[Martin Skarsaune]

Hi

It appears that java.util loggers refer to themselves if I run cryostat agent on startup of a spring petclinic example, hence causing a stack overflow once cryostat attempts to log a HTTP request. Any tips on configuring the agent to not interfere with the log setup of the application? 

Cheers!

Martin

[Andrew Azores]

Hi Martin,

Admittedly, I am no expert on log library configurations and not very
familiar with Spring either. I am looking at the Spring Petclinic
example and trying to work through setting it up with the Cryostat Agent
and have also been dealing with logger incompatibilities, but in my case
it manifests as a classloading conflict at startup rather than a Stack
Overflow.

So far the workaround I have found is a patch like this:

```
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-logging</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
```

added to the petclinic's dependencies. But, if you already have some
running Spring workloads, I don't think you would want to modify the
logging configuration so severely and do a rebuild just to get the
Cryostat Agent running.

In the end the problem is that, for ease of distribution and
consumption, the Cryostat Agent is packaged as a single shaded "fat JAR"
which includes SLF4J and the JDK1.4 provider and a small number of other
dependencies. If I remove the provider from the Agent JAR then my tests
show the Agent failing to pick up any provider and emitting no logs at
all. If I include the provider then we have the classloading conflict
with the petclinic's own dependencies.

I'll keep experimenting and let you know if I come up with anything good.

> Cheers!
>
> Martin
>

Here's the classloading conflict stacktrace in case it's useful/interesting:

```
Oct 03, 2023 1:52:41 PM io.cryostat.agent.Agent lambda$agentmain$2
INFO: Cryostat Agent starting...
Standard Commons Logging discovery in action with spring-jcl: please
remove commons-logging.jar from classpath in order to avoid potential
conflicts
Exception in thread "main" java.lang.reflect.InvocationTargetException
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
Caused by: java.lang.IllegalArgumentException: LoggerFactory is not a
Logback LoggerContext but Logback is on the classpath. Either remove
Logback or the competing implementation (class
org.slf4j.jul.JDK14LoggerFactory loaded from
file:/workspace/BOOT-INF/lib/cryostat-agent-0.3.0-SNAPSHOT.jar). If you
are using WebLogic you will need to add 'org.slf4j' to
prefer-application-packages in WEB-INF/weblogic.xml:
org.slf4j.jul.JDK14LoggerFactory
at org.springframework.util.Assert.instanceCheckFailed(Assert.java:713)
at org.springframework.util.Assert.isInstanceOf(Assert.java:632)
at
org.springframework.boot.logging.logback.LogbackLoggingSystem.getLoggerContext(LogbackLoggingSystem.java:388)
at
org.springframework.boot.logging.logback.LogbackLoggingSystem.beforeInitialize(LogbackLoggingSystem.java:122)
at
org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationStartingEvent(LoggingApplicationListener.java:238)
at
org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:220)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
at
org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136)
at
org.springframework.boot.context.event.EventPublishingRunListener.starting(EventPublishingRunListener.java:75)
at
org.springframework.boot.SpringApplicationRunListeners.lambda$starting$0(SpringApplicationRunListeners.java:54)
at java.base/java.lang.Iterable.forEach(Unknown Source)
at
org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
at
org.springframework.boot.SpringApplicationRunListeners.starting(SpringApplicationRunListeners.java:54)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:304)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
at
org.springframework.samples.petclinic.PetClinicApplication.main(PetClinicApplication.java:34)
... 8 more
```

--
Andrew Azores
Principal Software Engineer, Java Monitoring
Red Hat Canada