CAS 5.0.x newer ldaptive?

[Baron Fujimoto]

I'm working with Oracle to troubleshoot a bug we've encountered with their JDK (1.8u231+) and LDAP errors. According to their analysis, they're claiming that the problem lies with the ldaptive library being used by this old (I know) version of CAS. More specifically that the subsequent JDKs adhere to spec, and the ldaptive library appears to be testing for unspecified behaviour. They are recommending I try a newer version of the ldaptive library which does not appear to have the same code.

I added the following to our pom.xml:

        <dependency>

            <groupId>org.ldaptive</groupId>

            <artifactId>ldaptive</artifactId>

            <version>2.0.1</version>

        </dependency>

When I ran "mvn clean package" I think it looked like it was including the 2.0.1 version of ldaptive in the build. However, it seems like I'm still seeing LDAP problems. When I try to login, it will often result in the errors such as the following being logged:

2021-01-27 12:10:56,974 DEBUG [org.apereo.cas.authentication.LdapAuthenticationHandler] - <Attempting LDAP authentication for baron>

2021-01-27 12:10:56,986 WARN [org.ldaptive.pool.BlockingConnectionPool] - <connection failed check out validation: org.ldaptive.pool.AbstractConnectionPool$DefaultPooledConnectionProxy@4b6106ff>

2021-01-27 12:10:56,989 ERROR [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <LdapAuthenticationHandler: Unexpected LDAP error  (Details: Validation of connection failed)>

Eventually the validation succeeds, then so does the authentication.

How can I verify which version of ldaptive is being used by CAS? I don't think I saw anything indicating this in the logs. If I search for ldaptive in my overlay work directory I find the following:

=====

$ grep -ilr ldaptive .

./target/cas.war

./target/war/work/org.apereo.cas/cas-server-webapp/WEB-INF/classes/log4j2.xml

./target/war/work/org.apereo.cas/cas-server-webapp/WEB-INF/lib/person-directory-impl-1.8.4.jar

./target/war/work/org.apereo.cas/cas-server-webapp/WEB-INF/lib/ldaptive-1.2.0.jar

./target/war/work/org.apereo.cas/cas-server-webapp/WEB-INF/lib/ldaptive-beans-1.2.0.jar

./target/cas/WEB-INF/classes/log4j2.xml

./target/cas/WEB-INF/lib/person-directory-impl-1.8.4.jar

./target/cas/WEB-INF/lib/ldaptive-1.2.0.jar

./target/cas/WEB-INF/lib/ldaptive-beans-1.2.0.jar

./target/cas/WEB-INF/lib/ldaptive-beans-2.0.1.jar

./target/cas/WEB-INF/lib/ldaptive-2.0.1.jar

./pom.xml

./etc/cas/config/log4j2.xml

=====

I see an ldaptive-2.0.1.jar and ldaptive-beans-2.0.1.jar, but also ldaptive-1.2.0.jar and ldaptive-beans-1.2.0.jar. The 1.2.0 versions are always present after the build even if I delete them first, so something must be re-including them. How can I ensure that the new ldaptive is used in place of the old one?

Unrelated, but I'm also seeing the following errors in the build now that weren't present when I originally built this long ago:

Downloading: https://repo.spring.io/libs-snapshot/com/github/duosecurity/duo_client_java/-SNAPSHOT/maven-metadata.xml

[WARNING] Could not transfer metadata com.github.duosecurity:duo_client_java:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

[WARNING] Failure to transfer com.github.duosecurity:duo_client_java:-SNAPSHOT/maven-metadata.xml from https://repo.spring.io/libs-snapshot was cached in the local repository, resolution will not be reattempted until the update interval of spring-libs-snapshots has elapsed or updates are forced. Original error: Could not transfer metadata com.github.duosecurity:duo_client_java:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

Downloading: https://repo.spring.io/libs-snapshot/com/github/duosecurity/duo_client_java/duo-client/-SNAPSHOT/maven-metadata.xml

[WARNING] Could not transfer metadata com.github.duosecurity.duo_client_java:duo-client:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

[WARNING] Failure to transfer com.github.duosecurity.duo_client_java:duo-client:-SNAPSHOT/maven-metadata.xml from https://repo.spring.io/libs-snapshot was cached in the local repository, resolution will not be reattempted until the update interval of spring-libs-snapshots has elapsed or updates are forced. Original error: Could not transfer metadata com.github.duosecurity.duo_client_java:duo-client:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

Downloading: https://repo.spring.io/libs-snapshot/com/github/duosecurity/duo_client_java/duo-example-admin/-SNAPSHOT/maven-metadata.xml

[WARNING] Could not transfer metadata com.github.duosecurity.duo_client_java:duo-example-admin:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

[WARNING] Failure to transfer com.github.duosecurity.duo_client_java:duo-example-admin:-SNAPSHOT/maven-metadata.xml from https://repo.spring.io/libs-snapshot was cached in the local repository, resolution will not be reattempted until the update interval of spring-libs-snapshots has elapsed or updates are forced. Original error: Could not transfer metadata com.github.duosecurity.duo_client_java:duo-example-admin:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

Downloading: https://repo.spring.io/libs-snapshot/com/github/duosecurity/duo_client_java/duo-client-all/-SNAPSHOT/maven-metadata.xml

[WARNING] Could not transfer metadata com.github.duosecurity.duo_client_java:duo-client-all:-SNAPSHOT/maven-metadata.xml from/to spring-libs-snapshots (https://repo.spring.io/libs-snapshot): Not authorized , ReasonPhrase:Unauthorized.

It seems prudent to resolve these build errors as well.

--

Baron Fujimoto <ba...@hawaii.edu> :: UH Information Technology Services
minutas cantorum, minutas balorum, minutas carboratum desendus pantorum 

[Richard Frovarp]

For the Duo jar, you should put them in your own local repo? Or maybe the cache dir? I know that by default it feels like you have to hunt through a  variety of external repos to find that dependency. I run a local Nexus install for us so I only have to hunt down once.

For the LDAP problems, I think on the Shib list they say don't use the JNDI LDAP connector, in particular with JDK 11. But even then, I don't think the 1.8 JNDI provider is that great. So, you may be able to move over to the UnboundID provider and have better success? To determine what version is being used, I would recommend looking at what is in your resulting build artifact. I'm building to a war, and therefore the place for me to look to see what is being placed in the war file. So if the file is present in the lib dir where you are running it, it might be using. I don't remember if there is an authoritative way the JRE loads libs or not. I generally assume that if there are two in the classpath, it is going to randomly load out of one of them. It's probably deterministic in some way, but if you have to ask about load order, you're probably in a space you don't want to be in.

I know that with overlays I've had trouble getting the version I want to be the only one. I don't think the normal Maven rules for dependency versions fully apply for overlays. You're best off putting excludes in to ensure the version you don't want is excluded.

You can also exclude out of the overlay, and I see that I'm doing that. I just don't remember why I'm doing that. I have spring and log4j in those excludes. I'm excluding Duo out of a normal dependency section. I have a different GAV bringing that dependency in.

We're using the UnboundID provider.

---

From: cas-...@apereo.org <cas-...@apereo.org> on behalf of Baron Fujimoto <ba...@hawaii.edu>
Sent: Wednesday, January 27, 2021 19:06
To: CAS Users <cas-...@apereo.org>
Subject: [cas-user] CAS 5.0.x newer ldaptive?

 

--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/CAAjLUL20VkUPAgFORD_Jb-nNaoNK_EiZb3uc_BfN8KF8gSyThg%40mail.gmail.com.

[Jeffrey Miller]

For the Duo dependency, spring updated their use for artifacts in their repo and now libs-release (and probably libs-snapshot) is no longer available publicly

https://spring.io/blog/2020/10/29/notice-of-permissions-changes-to-repo-spring-io-fall-and-winter-2020

[Ray Bon]

Baron,

Check what packages are included in the build with:

mvn dependency:tree

You can <exclude> transitive dependencies for the old library.

But, the overlay, itself, may include an old version of the library, which means that you would have to manually remove the old one from the war. Maybe the exclude can be applied to the overlay too???

Ray

On Wed, 2021-01-27 at 15:06 -1000, Baron Fujimoto wrote:

Notice: This message was sent from outside the University of Victoria email system. Please be cautious with links and sensitive information.

-- 

Ray Bon

Programmer Analyst

Development Services, University Systems

2507218831 | CLE 019 | rb...@uvic.ca

I respectfully acknowledge that my place of work is located within the ancestral, traditional and unceded territory of the Songhees, Esquimalt and WSÁNEĆ Nations.

[Baron Fujimoto]

Hmm, the older ldaptive-1.2.0.jar and ldaptive-beans-1.2.0.jar don't appear in the packages dependency tree, I'm not sure how I would exclude them from the build. If I manually delete them from the resulting cas.war file, then when I start CAS, I encounter the following error/exception:

ERROR [org.springframework.boot.SpringApplication] - <Application startup failed>

java.lang.UnsupportedClassVersionError: org/ldaptive/ConnectionFactory has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 (unable to load class [org.ldaptive.ConnectionFactory])

If my google-fu is right, this suggests that the newer ldaptive jar files included via the overlay pom.xml were compiled with Java 11 rather than the Java 8 we are using. If I wanted to continue down this path, I suppose that means I would need to compile the newer ldaptive jars myself from source using our installed Java 8? And if successful, then manually place them in the resulting cas.war file? Or is there an easier way?

--

- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/8bce280697887a84a31350dec8e78364b7ea07a5.camel%40uvic.ca.

[Richard Frovarp]

You have to do overlay exclusions. Which for me looks like. Obviously close the XML out as appropriate. I have further build plugins.

<build>

<plugins>

<plugin>

<artifactId>maven-war-plugin</artifactId>

<version>2.6</version>

<configuration>

<failOnMissingWebXml>false</failOnMissingWebXml>

<recompressZippedFiles>false</recompressZippedFiles>

<archive>

<compress>false</compress>

<manifestFile>${project.build.directory}/war/work/org.apereo.cas/cas-server-webapp/META-INF/MANIFEST.MF</manifestFile>

</archive>

<overlays>

<overlay>

<groupId>org.apereo.cas</groupId>

<artifactId>cas-server-webapp</artifactId>

<excludes>

<!-- <exclude>WEB-INF/cas.properties</exclude> -->

<exclude>WEB-INF/classes/application*.properties</exclude>

<exclude>WEB-INF/lib/spring*.jar</exclude>

<exclude>WEB-INF/lib/log4j*.jar</exclude>

</excludes>

</overlay>

</overlays>

<warName>cas</warName>

</configuration>

</plugin>

But that isn't going to help you to put ldaptive in there. If you look at the pom for ldaptive 2.0, they are calling for Java 11. Guessing they are using something that is JDK 11 specific. Maybe you can rebuild with 8 and it will work, I don't know. 

I would go with the easier solution of giving the UnboundID provider a try. https://apereo.github.io/cas/5.3.x/installation/Configuration-Properties-Common.html#ldap-connection-settings Bypass the JDK code altogether.

[Baron Fujimoto]

[Forking this thread to separate the ldaptive and maven repo issues]

Per that spring-io advisory, it says, "The /snapshot, /milestone, and /release repositories will remain available, but please fetch our releases from a central repository". The only reference I see in our pom.xml to spring-io is:

    <repositories>

        …

        <repository>

            <id>spring-milestones</id>

            <url>https://repo.spring.io/milestone</url>

        </repository>

    </repositories>

Would updating or supplementing with something else resolve these errors? Unfortunately, my experience with maven and these build strategies is pretty limited to what I've used to successfully build CAS in the past. I'll happily RTFM if someone could kindly point me to the appropriate FM that would explain this or how to pursue the other suggested strategies.

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/5a361cc6-a099-4a45-86c6-d479db055159n%40apereo.org.

[Ray Bon]

Baron,

You are approaching the amount of work required to upgrade cas and java.

Is there a reason why you are not upgrading cas? (rhetorical)

We upgraded to openJDK 11 for our applications (open source and internal) and the process was relatively painless.

One potential trap with compiling ldaptive, it might depend on capabilities at java 11 (first thing that comes to mind is newer syntax and lombok dependencies).

Ray

[Richard Frovarp]

I'm guessing that it just doesn't exist anymore. Browsing from

https://repo.spring.io/snapshot/

They don't have anything for Duo that is obvious. It was a SNAPSHOT dependency, and thus shouldn't have been used in anything released. I see in my pom:

    <dependency>

      <groupId>org.apereo.cas</groupId>

      <artifactId>cas-server-support-duo</artifactId>

      <version>${cas.version}</version>

      <exclusions>

        <exclusion>

          <artifactId>duo_client_java</artifactId>

          <groupId>com.github.duosecurity</groupId>

        </exclusion>

      </exclusions>

    </dependency>

Which removes the snapshot one.

I have 

    <dependency>

      <groupId>com.duosecurity</groupId>

      <artifactId>duo-client</artifactId>

      <version>0.2.1</version>

    </dependency>

However, 0.2.1 isn't in central. I think I grabbed it from their GitHub, compiled, and put it in my own Maven repo. They do have 0.3.0 in Maven Central. No idea if it is compatible or not.

Their source is here:

https://github.com/duosecurity/duo_client_java

But obviously they can't be trusted to tag their releases. The only tag they have is 0.2.2, despite the fact they have 0.3.0 in Central. So you probably need to get git version from them, build, and reference locally. mvn install on a local project will stick it in your local .m2 cache dir, making it available as a dependency.

[Ray Bon]

Baron,

Try removing that repository entry. Maven should default to maven central. If not, you can add it as a repo (get the url, https://repo1.maven.org/maven2/, from this page, https://mvnrepository.com/repos/central).

From this page, https://mvnrepository.com/artifact/org.ldaptive/ldaptive, you might find a version that works and runs on java 8.

Ray

On Thu, 2021-01-28 at 11:19 -1000, Baron Fujimoto wrote:

Notice: This message was sent from outside the University of Victoria email system. Please be cautious with links and sensitive information.

-- 

[Baron Fujimoto]

Since the approach of trying to use an updated ldaptive library was rapidly getting complicated, I tried the suggested UnboundIDProvider[*] with the following in my cas.properties:

cas.authn.ldap[0].providerClass=org.ldaptive.provider.unboundid.UnboundIDProvider

But CAS throws this exception on startup:

ERROR [org.apereo.cas.configuration.support.Beans] - <org.ldaptive.provider.unboundid.UnboundIDProvider>

java.lang.ClassNotFoundException: org.ldaptive.provider.unboundid.UnboundIDProvider

So perhaps I'm missing some other prerequisite? I don't see other mention or discussion of it in the docs though.

[*] I can't seem to find a 5.0.x version of /Configuration-Properties-Common.html#ldap-connection-settings but this property is also listed in the 5.0.x properties <https://apereo.github.io/cas/5.0.x/installation/Configuration-Properties.html>

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/a804a14d7388bf1771f73dc216cee6453d8505eb.camel%40ndsu.edu.

[King, Robert]

I’m not certainas to the 5.0 version, but to use the UnboundIDProvider in 6.x you need to add :

 

compile "com.unboundid:unboundid-ldapsdk:4.0.9"

 

to the Gradle build dependencies.  I would assume it would be the same for Maven.

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/CAAjLUL1Ga0Ecx-hBVG8meY89fRhtY5BRrVqZbXczonVycDDqrA%40mail.gmail.com.

[Richard Frovarp]

Sorry, you don't need it in 5.3 as that one is default. So I didn't see that in my dependencies. You'll need the ldaptive dependency for that. You'll want to match version to the other ldaptive version you have. 1.2 appears to bring in the unboundid dependency:

https://search.maven.org/artifact/org.ldaptive/ldaptive-unboundid/1.2.0/jar

<dependency>

  <groupId>org.ldaptive</groupId>

  <artifactId>ldaptive-unboundid</artifactId>

  <version>1.2.0</version>

</dependency>

[Baron Fujimoto]

Mahalo nui loa! The UnboundID workaround appears to solve the problem so I don't need to further pursue trying to substitute a newer version of the Ldaptive library. I wasn't able to find any documentation on the UnboundIDProvider though (what does it do specifically, and where would I have found that requisite dependency). Did I miss that somewhere? Now I can focus on upgrading to a current version of CAS.

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/7aec64cef858fe75335f241d4129d613f095ee0c.camel%40ndsu.edu.

[Richard Frovarp]

It's hard for me to remember where I first came across the "Always use UnboundID" rule to ldaptive. Don't remember if it was CAS, Shibboleth, or my own direct use of ldaptive.

Under v1 docs, there is a very prominent note about Java 9

http://www.ldaptive.org/v1/

Provider info:

http://www.ldaptive.org/v1/docs/guide/providers.html

which is pretty sparse on the always use UnboundID bit.