CAS 6.6.11 : help needed for cas-server-support-gauth-couchdb debugging

[spfma...@e.mail.fr]

Hi,

 

I am still trying to understand what is wrong with "cas-server-support-gauth-couchdb" (only the first authenticator is recorded in the database, none is working anymore after a restart).

 

As I am not a Java dev (I don't have the skills and don't have the most convenient tools), my idea was to add some logging directives here and there to trace the process, using the latest branch of the application source code (not the overlay one).

 

Can someone confirm I am doing the right way :

- add "import lombok.extern.slf4j.Slf4j;" if missing on the top of the class file

- anotate the class definition with "@Slf4j"

- put stuff like "LOGGER.debug" or "LOGGER.info" as needed

 

VSCode is my tool, and it seems convenient extensions for Java/Maven/Gradle are not able to handle a big project like CAS (language server crashing and restarting all the time, Gradle extensions unable to build a tree of all subprojects without crashing, ...) so I don't mind using the good old manual way instead of wasting time.

 

After modifying the code here and there, I rebuild the whole app with "./gradlew clean build --parallel --configure-on-demand --stacktrace --no-daemon -x checkstyleMain" at the root of the project.

 

And "cas/webapp/cas-server-webapp-jetty$ ../../gradlew bootRun --parallel --configure-on-demand --build-cache --stacktrace --no-daemon -x checkstyleMain" allows me to try it (we use it with Jetty in production).

 

The app is running, I can reproduce the problems but I have the feeling my modifications don't exist  as none of my custom logging messages is displayed.

 

For an example, I added a simple logging flag in this file "support/cas-server-support-gauth-couchdb/src/main/java/org/apereo/cas/couchdb/gauth/credential/GoogleAuthenticatorAccountCouchDbRepository.java" this way :

 

@View(name = "by_username", map = "function(doc) { if(doc.secretKey) { emit(doc.username, doc) } }")

public List<CouchDbGoogleAuthenticatorAccount> findByUsername(final String username) {

LOGGER.debug("[MY_DEBUG_STUFF] findByUsername@GoogleAuthenticatorAccountCouchDbRepository={}", username);

try {

return queryView("by_username", username.trim().toLowerCase());

} catch (final DocumentNotFoundException e) {

LOGGER.trace(e.getMessage(), e);

}

return new ArrayList<>(0);

}

 

as I think it's the one responsible for database lookup, according to the request I have seen coming on database side.

 

But nothing in the logs ... Maybe I am not tagging the right source file ?

 

So why not tweak a known existing log message, it is safer. In "support/cas-server-support-gauth-couchdb/src/main/java/org/apereo/cas/gauth/token/GoogleAuthenticatorCouchDbTokenRepository.java" I changed the message in "cleanInternal" method. The string "Removing tokens older than" is only found in this file, so I think it's spot on.

 

After rebuilding and restarting the application, I still get the original message in my logs.

 

DEBUG [org.apereo.cas.gauth.token.GoogleAuthenticatorCouchDbTokenRepository] - <Removing tokens older than [2023-08-23T17:37:11.946486]

 

Could someone tell me what I am missing or doing wrong ? Of course, I have deleted all Gradle dirs, used a find to delete all ".class" files and rebuild the projects several times but I am stuck.

 

Reagrds

 

 

---

FreeMail powered by mail.fr

[Ray Bon]

Could you use a different storage system?

I do not see the couchdb module in the current development branch. Not sure if it is being removed or if a different module takes on that feature.

Instead of running gradlew in vscode, you can run it from the command line. The 'clean' part of the command will remove all .class files; no need to get rid of gradle directories unless you are changing gradle version (which you should not).

Once you build the project, remove 'clean'; only modified packages will be rebuilt (will be fine for logging, but not for api changes).

It is possible that method is not being called. You could put your logging statement in every method in that class to be sure. Also, use error level logging. Default logging for that class may not show at info or debug. Or add to log4j2.xml:

<AsyncLogger name="org.apereo.cas.couchdb.gauth.credential" level="trace">

If you want a more 'capable' development environment, here are some notes on intellij (I think there is a free version), https://apereo.github.io/cas/development/developer/Build-Process.html#intellij-idea

Ray

On Wed, 2023-08-23 at 17:43 +0200, spfma.tech via CAS Community wrote:

Notice: This message was sent from outside the University of Victoria email system. Please be cautious with links and sensitive information.

Hi,

 

I am still trying to understand what is wrong with "cas-server-support-gauth-couchdb" (only the first authenticator is recorded in the database, none is working anymore after a restart).

 

As I am not a Java dev (I don't have the skills and don't have the most convenient tools), my idea was to add some logging directives here and there to trace the process, using the latest branch of the application source code (not the overlay one).

 

Can someone confirm I am doing the right way :

- add "import lombok.extern.slf4j.Slf4j;" if missing on the top of the class file

- anotate the class definition with "@Slf4j"

- put stuff like "LOGGER.debug" or "LOGGER.info" as needed

 

VSCode is my tool, and it seems convenient extensions for Java/Maven/Gradle are not able to handle a big project like CAS (language server crashing and restarting all the time, Gradle extensions unable to build a tree of all subprojects without crashing, ...) so I don't mind using the good old manual way instead of wasting time.

 

After modifying the code here and there, I rebuild the whole app with "./gradlew clean build --parallel --configure-on-demand --stacktrace --no-daemon -x checkstyleMain" at the root of the project.

 

And "cas/webapp/cas-server-webapp-jetty$ ../../gradlew bootRun --parallel --configure-on-demand --build-cache --stacktrace --no-daemon -x checkstyleMain" allows me to try it (we use it with Jetty in production).

 

The app is running, I can reproduce the problems but I have the feeling my modifications don't exist  as none of my custom logging messages is displayed.

 

For an example, I added a simple logging flag in this file "support/cas-server-support-gauth-couchdb/src/main/java/org/apereo/cas/couchdb/gauth/credential/GoogleAuthenticatorAccountCouchDbRepository.java" this way :

 

@View(name= "by_username", map= "function(doc) { if(doc.secretKey) { emit(doc.username, doc) } }")

public List<CouchDbGoogleAuthenticatorAccount>findByUsername(finalString username) {

LOGGER.debug("[MY_DEBUG_STUFF] findByUsername@GoogleAuthenticatorAccountCouchDbRepository={}", username);

try {

return queryView("by_username",username.trim().toLowerCase());

} catch (finalDocumentNotFoundException e) {

LOGGER.trace(e.getMessage(), e);

}

return newArrayList<>(0);

}

[John]

Were no longer on 6.x series, as in your case, we used couchdb for many items and it is being removed in 7.x so we went ahead and moved on and it looks as though it was removed on the tag for 6.6.10, maybe on accident? because it still exists in the 6.6.x branch. Maybe an ooops by one of the devs and tagged 6.6.10 off wrong branch? Idk..

[spfma...@e.mail.fr]

Hi,

Thanks for your answer.

Is there any reason why it has been removed ?

Regards

--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/2d74742b-5ed9-4ae5-bdea-07fdf52b5bdan%40apereo.org.

[spfma...@e.mail.fr]

Hi,

 

Thanks for your answer.

 

I chose this storage system because my goal is to setup an active/passive pair of servers (with continous db replication on the passive side and automatic seemless failover)  in order to provider high availability.

It was the only supported backend I have found providing an easy way to achieve this goal (no three tier cluster with qorum and/or manual failover with conventional RDBMS).

 

But according you John's answer, I think I will have to change my mind anyway.

 

As my computer does not meet the requirements for serious Java developement, I am working remotely on an beefed up VM with plenty of RAM and CPU cores. And for that, VSCode has a very nice remote session extension, using ssh. Since Java related extensions don't seem to work correctly this way (maybe they work better localy, I don't have enough resources to test it), I am indeed using two shell sessions to run commands : one for building (clean build), and the other one for running (bootRun).

 

I have seen some posts here and there relating unexplainable problems with Gradle, and wiping out all the folders solved them. So I gave a try too !

 

My actual log4j config has a logger defined this way :

 

<Logger name="org.apereo.cas" level="debug">

<appender-ref ref="casFile" />

<appender-ref ref="casConsole" />

<!-- <appender-ref ref="casSyslog" />-->

</Logger>

 

And I am adding "LOGGER. debug" directives here in there. Should it be ok ?

 

I had a look at several IDE, and IDEA free has no remote support unfortunately.  Need to have a look at Eclipse and Netbeans too, but it seems they have the same limitations. So better make a wise choice before investing time and energy in such a complex product.

 

Regards

[John]

The 6.6.10 thing looks to be a developer ooops, so it should have stayed and worked. The removal is probably because it has low usage, maybe based on maven download stats. In any case, we moved a lot after I saw the deprecation for couchdb in v7 to what we could using Rest based ones, that could, ldap for those that could and internal ticket registry is now using the AMQP registry which is easy setup for replication. https://apereo.github.io/cas/development/ticketing/Messaging-AMQP-Ticket-Registry.html

And yes I concur, trying to find a common backend store for all the modules is not easy and having to complicate things just makes it harder to automate and we wanted automation throughout the entire lifecycle. Our prod env is now fully ephemeral, they are re-built automatically weekly, not really because CAS needs the updates but the docker containers themselves, OS/Java/Tomcat have frequent security updates. Are you using or plan on using docker? whats your prod environment going to look like? If docker or kubernetes, an easy, one we almost opted for was just use file based storage in the docker volumes which are stored on our nfs servers, which are also already replicated over multiple sites. Tickets cannot be stored on disk but the new AMQP solves that.

 What are you using for auth backend? is it ldap? the easiest way we have found since our backend auth is AD, is to utilize ldap storage when it is available since of of course AD is already replicated, multi-master.

[Ray Bon]

The paid for version of intellij does not support remote editing either (sigh). Your dev setup sounds fine and you should not have to worry about your local machine since it is only used for editing. I only use intellij for code completion and class/method references. I always build/run on the command line.

Are you creating a log4j2.xml file or adding to the one already in the project, https://github.com/apereo/cas/blob/6.6.x/webapp/cas-server-webapp-resources/src/main/resources/log4j2.xml

When running, the default location for the log config file is /etc/cas/log4j2.xml (at least when using the overlay), so make sure you are editing the correct file. By setting your custom loggers to 'error' or 'fatal', you do not have to edit the log config.

When you say no records are in the database after a restart; are you talking about a cas restart, a couchdb restart, or both?

Is it possible that a cas restart re-initializes the db? (I have not used any cas db functionality, so am unfamiliar with its operation or config.)

Can you check that the records exist in couchdb? 

How are cas tickets being stored?

I would guess that cas finds a record in couchdb by TGT id. If the ticket store is lost on a restart, then cas would have no way of finding anything in the db. (Again, I know nothing of how cas uses databases.)

Ray

On Thu, 2023-08-24 at 09:36 +0200, spfma.tech via CAS Community wrote:

Notice: This message was sent from outside the University of Victoria email system. Please be cautious with links and sensitive information.

Hi,

 

Thanks for your answer.

 https://github.com/apereo/cas/blob/6.6.x/webapp/cas-server-webapp-resources/src/main/resources/log4j2.xml

I chose this storage system because my goal is to setup an active/passive pair of servers (with continous db replication on the passive side and automatic seemless failover)  in order to provider high availability.

It was the only supported backend I have found providing an easy way to achieve this goal (no three tier cluster with qorum and/or manual failover with conventional RDBMS).

 

But according you John's answer, I think I will have to change my mind anyway.

 

As my computer does not meet the requirements for serious Java developement, I am working remotely on an beefed up VM with plenty of RAM and CPU cores. And for that, VSCode has a very nice remote session extension, using ssh. Since Java related extensions don't seem to work correctly this way (maybe they work better localy, I don't have enough resources to test it), I am indeed using two shell sessions to run commands : one for building (clean build), and the other one for running (bootRun).

 

I have seen some posts here and there relating unexplainable problems with Gradle, and wiping out all the folders solved them. So I gave a try too !

 

My actual log4j config has a logger defined this way :

 

<Loggername="org.apereo.cas"level="debug">

<appender-refref="casFile"/>

<appender-refref="casConsole"/>

[spfma...@e.mail.fr]

Hi,

 

I thought the Ultimate edition has it : https://www.jetbrains.com/help/idea/remote-development-starting-page.html

But I will never be offered this tool anyhow !

 

I am using my main production logfile at "/etc/cas/config log4j2.xml", with all levels between "trace" and "debug". And I see plenty of debug messages so I think it's ok.

 

I am now studying the problem with a simple CAS instance built from the sources, with a dummy JSON service and the internal "casuser" account. I just added "cas-server-support-json-service-registry", ""cas-server-support-gauth" and "cas-server-support-gauth-couchdb" and the related "cas.properties" configuration directives :

 

##########################

# MFA (global settings) #

##########################

cas.authn.mfa.triggers.global.global-provider-id: mfa-gauth

#cas.authn.mfa.triggers.global.global-provider-id: mfa-simple

########################

# Google Authenticator #

########################

cas.authn.mfa.gauth.core.multiple-device-registration-enabled: true

cas.authn.mfa.gauth.core.issuer: CAS

cas.authn.mfa.gauth.core.label: OUR_CORP

cas.authn.mfa.gauth.couch-db.create-if-not-exists: true

cas.authn.mfa.gauth.couch-db.db-name: cas_gauth

cas.authn.mfa.gauth.couch-db.password: password

cas.authn.mfa.gauth.couch-db.username: admin

cas.authn.mfa.gauth.couch-db.url: http://localhost:5984

 

CouchDb is running as a local Docker container, with a persistent volume (I had to create the database manually, as in spite of having set "cas.authn.mfa.gauth.couch-db.create-if-not-exists" to true, there are no design documents inside and authenticators registering can not work. There is an older post in this ML about that, I used the informations they provided and it works after manually creating the missing items).

 

When I login for the first time, I am asked to pair a new authenticator and the process is successful. And can login again and again it's ok. 

If I check the database, I have a record related to this authenticator, having a name, and id and user name.

 

If I restart CAS, the database content is still the same of course but the codes provided by the authenticator are not working anymore, as if they were wrong. And I have an error message in the logs :

2023-08-25 11:04:22,487 ERROR [org.apereo.cas.authentication.DefaultAuthenticationManager] - <Authentication has failed. Credentials may be incorrect or CAS cannot find authentication handler that supports [GoogleAuthenticatorTokenCredential(super=OneTimeTokenCredential(token=********), accountId=1692865323865)] of type [GoogleAuthenticatorTokenCredential]. Examine the configuration to ensure a method of authentication is defined and analyze CAS logs at DEBUG level to trace the authentication event.>
2023-08-25 11:04:22,487 ERROR [org.apereo.cas.authentication.DefaultAuthenticationManager] - <[GoogleAuthenticatorAuthenticationHandler]: [Secret cannot be null.]>

 

I still have this record in the databasen with id=1692865323865 in the database, related to the "casuser" and the registered authenticator". The "secretKey" property is still not null.

 

I have set "cas.authn.mfa.gauth.core.multiple-device-registration-enabled" to true, and I am indeed allowed to pair additional authenticators with my accounts. But doing so gives no result, there is still only one record in the database.

If I manually add a forged record corresponding to a second authenticator, it's better, I have a list of authenticators I can choose.

 

So I decided to study the internals a bit further, by adding logging directives here and there.

 

But I have more and more the feeling something is wrong or is beyond my current understanding to say the least.

 

As you suggested, maybe I am looking at the wrong place, expecting to see log messages from methods which are never called in this use case ?

 

There is some gargabe collector removing the old tokens (and it's working flawlessly) logging something like :

2023-08-25 11:01:11,218 DEBUG [org.apereo.cas.gauth.token.GoogleAuthenticatorCouchDbTokenRepository] - <Removing tokens older than [2023-08-25T11:00:41.218288]>

 

After greping the whole source tree, it seems this message is unique and indeed located in "cleanInternal" method from "support/cas-server-support-gauth-couchdb/src/main/java/org/apereo/cas/gauth/token/GoogleAuthenticatorCouchDbTokenRepository.java". Sounds logical.

 

So my idea was to simply alter the message, rebuild the app and see if this simple modification is working.

 

And guess what ... it's not working as expected.

 

After starting from scratch ("gradlew clean" and deletion of "~/.gradle/caches"), I built the app again and  even tried to execute the war from "cas/webapp/cas-server-webapp-jetty/build/libs/cas-server-webapp-jetty-6.6.11-SNAPSHOT.war"

 

It's running but the log message is still displayin the same text as in original source code.

 

Just for science, I tried with "bootRun" and it's still the same.

 

So I used the Fernflower package provided with Idea installation to see the source code from the generated classe.

The compiled class from the code I have modified has the expected content (support/cas-server-support-gauth-couchdb/build/classes/java/main/org/apereo/cas/gauth/token/GoogleAuthenticatorCouchDbTokenRepository.class).

 

And it's been created or modified a couple of minutes after I modified the source code and rebuilt. So far it's normal.

 

So what is used in the webapp ?

 

After exploding "webapp/cas-server-webapp-jetty/build/libs/cas-server-webapp-jetty-6.6.11-SNAPSHOT.war" and its "WEB-INF/lib/cas-server-support-gauth-couchdb-6.6.11-SNAPSHOT.jar" package, I discovered two things :

- the file "org/apereo/cas/gauth/token/GoogleAuthenticatorCouchDbTokenRepository.class" has a modification time set ... two days ago, at a time I am not even at work.

- the content is still the original one, without my modifications.

 

Having cleaned the build tree and destroyed all cache, I have no explanations.

 

So maybe I am right and wrong at the same time, having put debug outputs at the right locations but unable to see them because of another code being executed.

 

So far i am a bit stuck in the middle of the twilight zone.

 

Regards

[John]

Looks like from your config, you don't have a static value set for gauth encryption, each restart without consistent values would generate a new key each time,

you could for dev testing, set below, restart. And also, wipe your couchdb records so you can re-register,

cas.authn.mfa.gauth.crypto.enabled=false

or just take the auto-generated values that get sent to log and then set the below with whats in log,

cas.authn.mfa.gauth.crypto.enabled=true

cas.authn.mfa.gauth.crypto.encryption.key=
cas.authn.mfa.gauth.crypto.encryption.key-size=
cas.authn.mfa.gauth.crypto.signing.key=
cas.authn.mfa.gauth.crypto.signing.key-size=

[Ray Bon]

Thanks for the tip on the ultimate edition.

Cas uses a number of keys for various tasks. If the key is not present in your config, cas will create one on boot. It will be different each time cas starts and, of course, anything persisted with the earlier key will no longer be accessible.

There will be some log messages to let you know:

cas | 2023-08-25 18:52:34,189 WARN [ org.aper.cas.util.ciph.BaseStringCipherExecutor] - <Secret key for encryption is not defined for [Ticket-granting Cookie]; CAS will attempt to auto-generate the encryption key> [main]

cas | 2023-08-25 18:52:34,201 WARN [ org.aper.cas.util.ciph.BaseStringCipherExecutor] - <Generated encryption key [-Tt6GwcfiQu6Sg_gwYlnUzxhRxOJVFbT6gQra7tSwzs] of size [256] for [Ticket-granting Cookie]. The generated key MUST be added to CAS settings:

cas |

cas | cas.tgc.crypto.encryption.key=-Tt6GwcfiQu6Sg_gwYlnUzxhRxOJVFbT6gQra7tSwzs

cas |

cas | > [main]

cas | 2023-08-25 18:52:34,204 WARN [ org.aper.cas.util.ciph.BaseStringCipherExecutor] - <Secret key for signing is not defined for [Ticket-granting Cookie]. CAS will attempt to auto-generate the signing key> [main]

cas | 2023-08-25 18:52:34,204 WARN [ org.aper.cas.util.ciph.BaseStringCipherExecutor] - <Generated signing key [Gc2fjXd0ev0L7r_FnwN1XStSxYgatoeoipld5nGt78KfKM5FBTwWYbsvOox4LDcvCLmP8-4JEdDkmzvpJJ1kWg] of size [512] for [Ticket-granting Cookie]. The generated key MUST be added to CAS settings:

cas |

cas | cas.tgc.crypto.signing.key=Gc2fjXd0ev0L7r_FnwN1XStSxYgatoeoipld5nGt78KfKM5FBTwWYbsvOox4LDcvCLmP8-4JEdDkmzvpJJ1kWg

You can use the key values generated by cas rather than trying to create them yourself.

It looks like the web runner is pulling from the repository instead of using the built files in the project.

There is this gradlew command, publishToMavenLocal, to install locally. (See bottom of https://apereo.github.io/cas/development/developer/Build-Process.html#sample-build-aliases.)

You can also copy the jar file, to which you made code changes, to your ~/.m2/repository/...

Ray

[spfma...@e.mail.fr]

Hi,

 

Thanks for this answer, I was indeed able to fix this part of the problem with it.

 

Regards

--

- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/cc86cf85-9f5c-49cf-8632-20e79437fd20n%40apereo.org.

[spfma...@e.mail.fr]

Hi,

 

Thank you very much for this very valuable answer !

 

I totally forgot to make those keys persistent, after seeing them during the startups and not even being able to notice they were different each time ...

 

And I also purposely ignored 'publishToMavenLocal' in my command line, as I thought I was using Gradle so no link with Maven. What a mistake ... it's a new world I really have to invest some time in as I have not even scratched the surface !

 

So after adding it, all my debug message were available.

 

But as CouchDB backend support seems to be dropped, I will not invest more time in it. I can now use my registered authenticator even after restarting, but adding another one is still not working.

So there is problem, as it's perfectly fine with the same configuration and a JPA backend (MariaDB).

 

Regards