CAS 6.3 Hazelcast ticket registry

[Baron Fujimoto]

I'm working on an upgrade from CAS 5.0.x to CAS 6.3. I seem to be stuck trying to get the Hazelcast ticket registry working.

I have the following defined in our build.gradle:

    implementation "org.apereo.cas:cas-server-support-hazelcast-ticket-registry:${casServerVersion}"

and in cas.properties (mildly redacted):

cas.ticket.registry.hazelcast.cluster.members=cas_node1.example.edu, cas_node2.example.edu
cas.ticket.registry.hazelcast.cluster.instance-name=cas-registry

When CAS is started, the following is logged:

2021-06-14 12:01:35,580 DEBUG [org.apereo.cas.config.HazelcastTicketRegistryConfiguration] - <Creating Hazelcast instance for members [[cas_node1.example.edu, cas_node2.example.edu]]>
2021-06-14 12:01:35,645 DEBUG [org.apereo.cas.hz.HazelcastConfigurationFactory] - <Skipped Hazelcast Multicast configuration since feature is disabled>
2021-06-14 12:01:35,741 WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [4.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.>
2021-06-14 12:01:35,741 WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [4.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.>
2021-06-14 12:01:36,514 WARN [com.hazelcast.cp.CPSubsystem] - <[cas_node1.example.edu]:5701 [dev] [4.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.>
2021-06-14 12:01:38,135 DEBUG [org.apereo.cas.config.HazelcastTicketRegistryConfiguration] - <Created Hazelcast map configuration for [DefaultTicketDefinitionProperties(cascadeRemovals=false, storageName=serviceTicketsCache, storageTimeout=10, storagePassword=null, excludeFromCascade=false)]>
2021-06-14 12:01:38,162 DEBUG [org.apereo.cas.config.HazelcastTicketRegistryConfiguration] - <Created Hazelcast map configuration for [DefaultTicketDefinitionProperties(cascadeRemovals=false, storageName=proxyTicketsCache, storageTimeout=10, storagePassword=null, excludeFromCascade=false)]>
2021-06-14 12:01:38,164 DEBUG [org.apereo.cas.config.HazelcastTicketRegistryConfiguration] - <Created Hazelcast map configuration for [DefaultTicketDefinitionProperties(cascadeRemovals=false, storageName=ticketGrantingTicketsCache, storageTimeout=28800, storagePassword=null, excludeFromCascade=false)]>
2021-06-14 12:01:38,164 DEBUG [org.apereo.cas.config.HazelcastTicketRegistryConfiguration] - <Created Hazelcast map configuration for [DefaultTicketDefinitionProperties(cascadeRemovals=false, storageName=proxyGrantingTicketsCache, storageTimeout=28800, storagePassword=null, excludeFromCascade=false)]>
2021-06-14 12:01:38,165 DEBUG [org.apereo.cas.config.HazelcastTicketRegistryConfiguration] - <Created Hazelcast map configuration for [DefaultTicketDefinitionProperties(cascadeRemovals=false, storageName=transientSessionTicketsCache, storageTimeout=300, storagePassword=null, excludeFromCascade=true)]>
2021-06-14 12:01:38,223 INFO [org.apereo.cas.util.CoreTicketUtils] - <Ticket registry encryption/signing is turned off. This MAY NOT be safe in a clustered production environment. Consider using other choices to handle encryption, signing and verification of ticket registry tickets, and verify the chosen ticket registry does support this behavior.>

I don't understand the hazelcast.cp.CPSubsystem warning, nor what should be done about it.

Similarly, It's not clear to me where the advisory about "Ticket registry encryption/signing is turned off" should be addressed. Explicitly setting "cas.tgc.crypto.enabled=true" appears to have no effect, for example.

I don't see anything else obviously relevant logged. If I test a login using /cas/login, I see the TGT created on cas_node1, but I never see it replicated on cas_node2, unlike in our CAS 5 deployment. AFAICT, the Hazelcast ticket registry is not working or replicating the tickets to other node.

I'm also unsure of the relevance, but when I shut CAS down on a node, I see the following logged:

DEBUG [org.apereo.cas.ticket.registry.HazelcastTicketRegistry] - <Hazelcast instance is not active!>

I don't know if this means that this is just reporting that HazelcastTicketRegistry was shut down normally as part of CAS shutting down, or perhaps an indication that HazelcastTicketRegistry was not running as expected.

Finally, in our CAS 5 instance, we had the following CAS properties set:

cas.ticket.registry.hazelcast.crypto.signing.key=*****
cas.ticket.registry.hazelcast.crypto.encryption.key=*****

But I'm not finding their equivalent in the 6.3 docs. Am I missing them somewhere?

Any assistance or guidance would be appreciated.

[*] Minor log wording "bug" for com.hazelcast.instance.AddressPicker: "aware of that" should probably be "aware that"?
--
Baron Fujimoto <ba...@hawaii.edu> :: UH Information Technology Services
minutas cantorum, minutas balorum, minutas carboratum desendus pantorum

[Ray Bon]

Baron,

These are the setting we use:

cas.ticket.registry.hazelcast.cluster.members=host1,host2

cas.ticket.registry.hazelcast.cluster.instanceName=cas-dev

cas.ticket.registry.hazelcast.crypto.signing.key=blahblah

cas.ticket.registry.hazelcast.crypto.encryption.key=foobar

cas.ticket.registry.hazelcast.crypto.enabled=true

Could it be the space after the ',' in your members list?

The crypto config can be found at one of the two 'here' links under the hazelcast properties, https://apereo.github.io/cas/6.3.x/configuration/Configuration-Properties.html#hazelcast-ticket-registry

I too see the note on the CP subsystem but have not had time to look into it.

Ray

P.S. technically, hazelcast is not a replicated cache but distrubuted

On Mon, 2021-06-14 at 12:57 -1000, Baron Fujimoto wrote:

Notice: This message was sent from outside the University of Victoria email system. Please be cautious with links and sensitive information.

[Baron Fujimoto]

Hmm, I was apparently wrong, about Hazelcast not working originally. Further experimentation seems to indicate otherwise, though I'm still somewhat confused as to what's happening.

Unlike CAS 5.0.x, after I create a TGT on the source node (e.g. node1), I don't see it being logged on the other Hazelcast nodes (node2). However if I then shut down node1 and reload /cas/login, the UI reports "Credentials are rejected/invalid and authentication attempt has failed." above the Duo MFA iframe. But in node2's log, I then see:

DEBUG [org.apereo.cas.ticket.registry.AbstractTicketRegistry] - <Encoded original ticket id [TGT-1-*****IJZpAj5I8s-node1] to [812...265]>

So the TGT from node1 does in fact appear to be available to node2.

If I continue with the Duo authentication, it succeeds. despite the confusing user experience. But here the logs/flow also confuse me further.

ERROR [org.apereo.cas.adaptors.duo.authn.DuoSecurityAuthenticationHandler] - <Transaction has expired. Please check that the system time is correct.>

com.duosecurity.duoweb.DuoWebException: Transaction has expired. Please check that the system time is correct.

INFO [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <[mfa-duo] exception details: [Transaction has expired. Please check that the system time is correct.].>

ERROR [org.apereo.cas.web.flow.resolver.impl.AbstractCasWebflowEventResolver] - <1 errors, 0 successes>

org.apereo.cas.authentication.AuthenticationException: 1 errors, 0 successes

...

DEBUG [org.apereo.cas.web.flow.actions.MultifactorAuthenticationBypassAction] - <Bypass rules determined MFA should execute for user [baron] and provider [mfa-duo]>

DEBUG [org.apereo.cas.web.flow.actions.MultifactorAuthenticationBypassAction] - <Authentication updated to forget any existing bypass for user [baron] for provider [mfa-duo]>

DEBUG [org.apereo.cas.util.http.SimpleHttpClient] - <Response code received from server matched [200].>

DEBUG [org.apereo.cas.adaptors.duo.authn.BasicDuoSecurityAuthenticationService] - <Received Duo ping response [{"response": "pong", "stat": "OK"}]>

2021-06-14 15:36:08,570 DEBUG [org.apereo.cas.adaptors.duo.authn.BaseDuoSecurityAuthenticationService] - <Contacting Duo to inquire about username [baron]>

2021-06-14 15:36:09,103 DEBUG [org.apereo.cas.adaptors.duo.authn.BaseDuoSecurityAuthenticationService] - <Received Duo admin response [{"code": 40301, "message": "Access forbidden", "message_detail": "Wrong integration type for this API.", "stat": "FAIL"}]>

2021-06-14 15:36:09,106 WARN [org.apereo.cas.adaptors.duo.authn.BaseDuoSecurityAuthenticationService] - <Duo returned an Invalid response with message [Access forbidden] and detail [Wrong integration type for this API.] when determining user account. This maybe a configuration error in the admin request and Duo will still be considered available.>

2021-06-14 15:36:09,113 DEBUG [org.apereo.cas.adaptors.duo.authn.BaseDuoSecurityAuthenticationService] - <Fetched and cached duo user account [DuoSecurityUserAccount(status=AUTH, enrollPortalUrl=null, username=baron, message=null)]>

2021-06-14 15:36:09,113 DEBUG [org.apereo.cas.web.flow.actions.CheckWebAuthenticationRequestAction] - <Authenticated request is identified as web-based via type [application/x-www-form-urlencoded]>

[then this gets recorded as a login failure]

DEBUG [org.apereo.cas.web.support.AbstractThrottledSubmissionHandlerInterceptorAdapter] - <Recording submission failure for [/cas/login]>

DEBUG [org.apereo.cas.web.support.AbstractInMemoryThrottledSubmissionHandlerInterceptorAdapter] - <Recording submission failure [10.11.12.13]>

[Then it tries the TGT again from Hazelcast?]

DEBUG [org.apereo.cas.ticket.registry.AbstractTicketRegistry] - <Encoded original ticket id [TGT-1-*****IJZpAj5I8s-node1] to [812...265]>

DEBUG [org.apereo.cas.ticket.registry.HazelcastTicketRegistry] - <Locating map name [ticketGrantingTicketsCache] for ticket definition [DefaultTicketDefinition(implementationClass=class org.apereo.cas.ticket.TicketGrantingTicketImpl, prefix=TGT, properties=DefaultTicketDefinitionProperties(cascadeRemovals=false, storageName=ticketGrantingTicketsCache,,storageTimeout=28800, storagePassword=null, excludeFromCascade=false), order=2147483647)]>

DEBUG [org.apereo.cas.ticket.registry.HazelcastTicketRegistry] - <Located Hazelcast map instance [ticketGrantingTicketsCache]>

WARN [org.apereo.cas.ticket.registry.AbstractTicketRegistry] - <Ticket passed is null and cannot be decoded>

[Then it appears to try for an authentication method based on the Duo MFA?]

DEBUG [org.apereo.cas.web.flow.resolver.impl.AbstractCasWebflowEventResolver] - <Handling authentication transaction for credential [DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Authentication credentials provided for this transaction are [[DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]]>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Candidate/Registered authentication handlers for this transaction are [[org.apereo.cas.authentication.handler.support.HttpBasedServiceCredentialsAuthenticationHandler, org.apereo.cas.authentication.LdapAuthenticationHandler, org.apereo.cas.adaptors.duo.authn.DuoSecurityAuthenticationHandler]]>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Authentication handler resolvers for this transaction are [[org.apereo.cas.authentication.handler.ByCredentialTypeAuthenticationHandlerResolver, org.apereo.cas.authentication.handler.RegisteredServiceAuthenticationHandlerResolver]]>

DEBUG [org.apereo.cas.authentication.AuthenticationHandlerResolver] - <Default authentication handlers used for this transaction are [HttpBasedServiceCredentialsAuthenticationHandler,LdapAuthenticationHandler,mfa-duo]>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Resolved and finalized authentication handlers to carry out this authentication transaction are [[org.apereo.cas.authentication.handler.ByCredentialTypeAuthenticationHandlerResolver, org.apereo.cas.authentication.handler.RegisteredServiceAuthenticationHandlerResolver]]>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Candidate resolved authentication handlers for this transaction are [[...HttpBasedServiceCredentialsAuthenticationHandler, ...LdapAuthenticationHandler, ...DuoSecurityAuthenticationHandler]]>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Attempting to authenticate credential [DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Authentication handler [HttpBasedServiceCredentialsAuthenticationHandler] does not support the credential type [DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]. Trying next...>

2021-06-14 15:38:31,709 DEBUG [org.apereo.cas.authentication.handler.support.AbstractUsernamePasswordAuthenticationHandler] - <Credential is not one of username/password and is not accepted by handler [LdapAuthenticationHandler]>

2021-06-14 15:38:31,709 DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Authentication handler [LdapAuthenticationHandler] does not support the credential type [DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]. Trying next...>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Attempting authentication of [baron] using [mfa-duo]>

DEBUG [org.apereo.cas.adaptors.duo.authn.DuoSecurityAuthenticationHandler] - <Verified Duo authentication for user [baron]>

INFO [org.apereo.cas.adaptors.duo.authn.DuoSecurityAuthenticationHandler] - <Successful Duo authentication for [baron]>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Authentication handler [mfa-duo] successfully authenticated [DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Final principal resolved for this authentication event is [SimplePrincipal(id=baron, attributes={})]>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Authentication policy resolvers for this transaction are [[org.apereo.cas.authentication.policy.RegisteredServiceAuthenticationPolicyResolver]]>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Authentication policy resolvers produced no candidate authentication handler. Using default policies>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Executing authentication policy [AtLeastOneCredentialValidatedAuthenticationPolicy]>

DEBUG [org.apereo.cas.authentication.handler.support.AbstractUsernamePasswordAuthenticationHandler] - <Credential is not one of username/password and is not accepted by handler [LdapAuthenticationHandler]>

DEBUG [org.apereo.cas.authentication.policy.AtLeastOneCredentialValidatedAuthenticationPolicy] - <Authentication policy is satisfied having found at least one authentication transactions>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Authentication policy resolvers for this transaction are [[org.apereo.cas.authentication.policy.RegisteredServiceAuthenticationPolicyResolver]]>

DEBUG [org.apereo.cas.authentication.DefaultAuthenticationEventExecutionPlan] - <Authentication policy resolvers produced no candidate authentication handler. Using default policies>

DEBUG [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Executing authentication policy [AtLeastOneCredentialValidatedAuthenticationPolicy]>

DEBUG [org.apereo.cas.authentication.handler.support.AbstractUsernamePasswordAuthenticationHandler] - <Credential is not one of username/password and is not accepted by handler [LdapAuthenticationHandler]>

DEBUG [org.apereo.cas.authentication.policy.AtLeastOneCredentialValidatedAuthenticationPolicy] - <Authentication policy is satisfied having found at least one authentication transactions>

[Which appears to result in a successful authentication]

INFO [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Authenticated principal [baron] with attributes [{}] via credentials [[DuoSecurityCredential(username=baron, signedDuoResponse=AUTH|YmF...Q==|14...5b, providerId=mfa-duo-1...)]].>

DEBUG [org.apereo.cas.web.flow.resolver.impl.AbstractCasWebflowEventResolver] - <Issuing ticket-granting tickets for service [null]>

DEBUG [org.apereo.cas.web.flow.resolver.impl.AbstractCasWebflowEventResolver] - <Resolved single event [success] via [org.apereo.cas.adaptors.duo.web.flow.DuoSecurityAuthenticationWebflowEventResolver] for this context>

But does this suggest it would have failed if not for the Duo MFA? Since the MFA is opt-in for our users, this is presumably a problem.

The WARNing "Ticket passed is null and cannot be decoded" is troubling and seems like it should not have failed?

--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/e074279013816af0986fbba0860a086a30cdc787.camel%40uvic.ca.

[Ray Bon]

Baron,

The first failed attempt at duo may be the result of cas trying both duo integrations (iFrame and redirect). I do not remember both options being tried but the failure message looks like you have cas set for one configuration and duo set for the other. But since it eventually succeeded, those log messages might indicate a fall through flow.

We found that the redirect flow was sloooowww and there would be a flash of the duo page for non registered users, so we opted to stick with the iFrame.

As far as the debug messages go that talk about failures, that may be related to the UI state (if one exists) when switching servers. You could try completing the log in flow for one application, then stop one server and access a second app. Given the time it takes a user to complete log in, it will be rare for a user to be in the middle of the log in flow and have the server go offline.

Ray