Elytron programmatic login with FORM authentication

[Alex]

Hi,

we are currently migrating from legacy security subsystem to Elytron and have a Struts2 based web application deployed in JBoss EAP 7.3.6 which should support multiple "flavors" of authentication. 

The standard way of logging in should be that a user manually provides credentials in a login form (j_security_check) and clicks the corresponding button. This works well with Elytron in our setup.

The second possibility is, that the GET request to protected content of the web application can contain a custom cookie that contains a JWT token. This cookie is intercepted by a io.undertow.server.HttpHandler which deals with the incoming request in its io.undertow.server.HttpHandler#handleRequest method. This handler is registered by io.undertow.servlet.api.DeploymentInfo#addSecurityWrapper with a DeploymentInfo which is provided by an implementation of io.undertow.servlet.ServletExtension. The ServletExtension is registered as a service provider in META-INF/services/io.undertow.servlet.ServletExtension.

The request handling in our implementation of io.undertow.server.HttpHandler#handleRequest extracts the JWT token from the cookie, pre-validates it and determines the contained username. This username and the token as a password are used as inputs for a call to javax.servlet.http.HttpServletRequest#login.

With the legacy security subsystem, the behavior of the server was, that this call to login triggered the authentication against the configured legacy security domain AND created a session in Undertow so that the HTTP 200 response for the previous GET request contained a Set-Cookie header with a fresh JSESSIONID cookie.

With Elytron, javax.servlet.http.HttpServletRequest#login doesn't do anything, neither an authentication against an Elytron security domain and security realm nor the creation of a session is triggered. The browser simply shows the login form which should get skipped by the described interception process.

I debugged the implementation of javax.servlet.http.HttpServletRequest#login that comes with JBoss. We start in io.undertow.servlet.spec.HttpServletRequestImpl#login which calls "login = sc.login(username, password)". This SecurityContext, when using Elytron, is org.wildfly.elytron.web.undertow.server.SecurityContextImpl. org.wildfly.elytron.web.undertow.server.SecurityContextImpl#login first checks "if (httpAuthenticator == null)". The "httpAuthenticator" is only set in org.wildfly.elytron.web.undertow.server.SecurityContextImpl#authenticate which gets called by a call to javax.servlet.http.HttpServletRequest#authenticate.

This explains, why a plain call to io.undertow.servlet.spec.HttpServletRequestImpl#login was doing nothing. I tried to call javax.servlet.http.HttpServletRequest#authenticate first, to instantiate that httpAuthenticator internally, and then javax.servlet.http.HttpServletRequest#login. This at least finally triggered the authentication and authorization against the configured Elytron security domain and security realm. Authentication/authorization were successful but Undertow still didn't issue a new JSESSIONID cookie and the browser again showed the login form instead of proceeding to the protected resources.

I'm currently out of ideas, how to proceed with this issue und how to achieve the same behavior as with the legacy security subsystem. Why does the Elytron implementation of io.undertow.security.api.SecurityContext behave so differently compared to the one for legacy security (io.undertow.security.impl.SecurityContextImpl)? How am I supposed to log in programatically in a FORM based web application using Elytron with javax.servlet.http.HttpServletRequest#login and/or javax.servlet.http.HttpServletRequest#authenticate?

The relevant JBoss configuration for all this looks like this:

Undertow:

<application-security-domains>

    <application-security-domain name="my_app_security_domain" http-authentication-factory="MyHttpAuthFactory"/>

</application-security-domains>

Elytron:

<security-domains>

    <security-domain name="MySecurityDomain" default-realm="MyCachingRealm" permission-mapper="default-permission-mapper">

        <realm name="MyCachingRealm" role-decoder="FromRolesAttributeDecoder"/>

    </security-domain>

</security-domains>

<security-realms>

    <custom-realm name="MyCustomRealm" module="module name redacted" class-name="class name redacted"/>

    <caching-realm name="MyCachingRealm" realm="MyCustomRealm" maximum-age="300000"/>

    <identity-realm name="local" identity="$local"/>

</security-realms>

<mappers>

    <simple-permission-mapper name="default-permission-mapper" mapping-mode="first">

        <permission-mapping>

            <principal name="anonymous"/>

            <permission-set name="default-permissions"/>

        </permission-mapping>

        <permission-mapping match-all="true">

            <permission-set name="login-permission"/>

            <permission-set name="default-permissions"/>

        </permission-mapping>

    </simple-permission-mapper>

    <constant-realm-mapper name="local" realm-name="local"/>

    <constant-realm-mapper name="MyRealmMapper" realm-name="MyCachingRealm"/>

    <simple-role-decoder name="FromRolesAttributeDecoder" attribute="Roles"/>

</mappers>

<http>

    <http-authentication-factory name="MyHttpAuthFactory" security-domain="MySecurityDomain" http-server-mechanism-factory="global">

        <mechanism-configuration>

            <mechanism mechanism-name="FORM" realm-mapper="MyRealmMapper">

                <mechanism-realm realm-name="MyRealm"/>

            </mechanism>

        </mechanism-configuration>

    </http-authentication-factory>

    <provider-http-server-mechanism-factory name="global"/>

</http>

[Darran Lofthouse]

First of all, is there any way your custom mechanism could be converted to an authentication mechanism?  It sounds like your pattern would be ideally suited to an authentication mechanism which is called before the FORM authentication mechanism.

Other than that is there any chance of a small reproducer I could check the sequences with, it sounds like you have gone quite far in identifying your authentication attempt is happening before we are ready for authentication to occur.  The problem here is I am not sure if we have a suitable point your handler could be interleaved as I assume it also needs to be successful to prevent FORM authentication kicking in.

[Alex]

Thanks Darran for the quick response!

Writing a custom mechanism was the first thing I thought of and this is a task I can look into in the future. Currently, I'm running out of time and I'm forced to make this work with Elytron somehow, so the idea was to migrate the application, as it is, to Elytron.

I have prepared a very basic reproducer for you which is attached. It's an adapted version of your simple-webapp from the elytron-examples in the wildfly-security-incubator, so you can quickly build a WAR with Maven, ready to be deployed to the server. The ZIP file also contains a CLI script with the necessary Elytron configuration and assumes the WildFly/JBoss to be run in domain mode. There is no real token logic or cookie interception in this example because the problem boils down to the fact that javax.servlet.http.HttpServletRequest#login no longer behaves as it did with the legacy security subsystem. The MyDummyTokenHandler contains a couple of comments with further information.

I've also attached log outputs for org.wildfly.security in TRACE mode, so you can see, that the authentication/authorization is actually taking place.

I'm really looking forward to any hints and suggestions!

[Alex]

Hello Darran,

have you already had time to try out my reproducer?

[Darran Lofthouse]

I am just debugging this one at the moment, first of all I think this needs to go back a step and remove the following line from the handler:

request.authenticate(response);

The problem this line will cause is that it triggers the full authentication process so that is why even though you later authenticate a login form is sent back as the authentication mechanism has already been executed and prepared it's response.

The main focus needs to be the error when that line is omited.

[Darran Lofthouse]

I have created this bug report and if testing goes ok I should have a proposed fix later today:

https://issues.redhat.com/browse/ELYWEB-133

Overall I think it does just need a small adjustment to create the HttpAuthenticator on demand for either method without assuming authenticate is called first.

[Alex]

Many thanks for looking at this issue!

Can you confirm, that the login form is skipped in the reproducer when only javax.servlet.http.HttpServletRequest#login is called and the httpAuthenticator is lazily initialized inside org.wildfly.elytron.web.undertow.server.SecurityContextImpl#login(String, String)?

Could we get this fix in JBoss EAP 7.3.7 as well, i.e., backported to elytron-web 1.6.x (current version in EAP 7.3.6 is 1.6.2.Final-redhat-00001)? The fix in elytron-web 1.10 wouldn't help us for the current situation.

[Darran Lofthouse]

I will confirm once I have run some more tests but FYI to request a fix against JBoss EAP you would need to raise a support case through the customer portal, this google group is just for community discussions.

[Darran Lofthouse]

With my fix in place the first page I see from the application is now the "Hello World!" page.

From the TRACE logging there is no mention of FORM authentication anymore:

2021-04-22 15:25:28,616 INFO  [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0010: Deployed "simple-webapp.war" (runtime-name : "simple-webapp.war")

2021-04-22 15:26:00,209 TRACE [org.wildfly.security.http.servlet] (default task-1) Created ServletSecurityContextImpl enableJapi=true, integratedJaspi=true, applicationContext=default-host /simple-webapp

2021-04-22 15:26:00,211 TRACE [org.wildfly.security] (default task-1) Principal assigning: [testuser], pre-realm rewritten: [testuser], realm name: [MyCachingRealm], post-realm rewritten: [testuser], realm rewritten: [testuser]

2021-04-22 15:26:00,212 TRACE [org.wildfly.security] (default task-1) Created wrapper RealmIdentity for 'testuser' and placing in cache.

2021-04-22 15:26:00,214 TRACE [org.wildfly.security] (default task-1) verifyEvidence Credential obtained from identity and cached for principal='testuser'

2021-04-22 15:26:00,214 TRACE [org.wildfly.security] (default task-1) Associating credential for 'testuser' with identity.

2021-04-22 15:26:00,214 TRACE [org.wildfly.security] (default task-1) getAuthorizationIdentity Caching AuthorizationIdentity for principal='testuser'

2021-04-22 15:26:00,215 TRACE [org.wildfly.security] (default task-1) Role mapping: principal [testuser] -> decoded roles [regular_user] -> domain decoded roles [] -> realm mapped roles [regular_user] -> domain mapped roles [regular_user]

2021-04-22 15:26:00,215 TRACE [org.wildfly.security] (default task-1) Authorizing principal testuser.

2021-04-22 15:26:00,215 TRACE [org.wildfly.security] (default task-1) Authorizing against the following attributes: [Roles] => [regular_user]

2021-04-22 15:26:00,215 TRACE [org.wildfly.security] (default task-1) Authorizing against the following runtime attributes: [] => []

2021-04-22 15:26:00,215 TRACE [org.wildfly.security] (default task-1) Permission mapping: identity [testuser] with roles [regular_user] implies ("org.wildfly.security.auth.permission.LoginPermission" "") = true

2021-04-22 15:26:00,215 TRACE [org.wildfly.security] (default task-1) Authorization succeed

2021-04-22 15:26:00,219 TRACE [org.wildfly.security] (default task-1) Caching identity for 'testuser' against session scope.

2021-04-22 15:26:00,219 TRACE [org.wildfly.security] (default task-1) Role mapping: principal [testuser] -> decoded roles [regular_user] -> domain decoded roles [] -> realm mapped roles [regular_user] -> domain mapped roles [regular_user]

I have prepared the fix so it can potentially be backported but as I say to request that you will need to raise a support case via the customer portal so the support team can get it scheduled.

[Alex]

At first glance, this looks good to me, both the log outputs and the fact that the welcome page is displayed after the programmatic login.

What about session creation? I guess, after the first successful programmatic authentication attempt, Undertow issues a new JSESSIONID cookie to the browser. What happens when you refresh that "Hello World!" page? Is there a new authentication attempt with a new resulting JSESSIONID cookie or is request.getUserPrincipal() != null in org.wildfly.security.examples.MyDummyTokenHandler#handleRequest and the JESSIONID from the first authentication attempt is reused?

In case there is only one authentication attempt and a session including the JSESSIONID cookie is created only once, this works as we need it and I'll raise a case with the Red Hat support.

[Darran Lofthouse]

The identity is stored in the HTTP session and the session cookie passed back to the client.

The problem in your reproducer is the HttpHandler has the following code:

        // This always evaluates to null.

        if (request.getUserPrincipal() != null) {

            // Already authenticated.

            return;

        }

The HttpHandler is executing before an existing cached identity is restored, if the HttpHandler allowed the request to proceed without calling login we would reach the point where authenticate() is called on the SecurityContextImpl and at that point any previously cached identity would be restored.

[Alex]

Did I understand you correctly that checking request.getUserPrincipal() for null in the handler is not a good idea and for each request in the handler the call to javax.servlet.http.HttpServletRequest#login is necessary, even after the first successful authentication?

[Darran Lofthouse]

Yes and no ;-)

Yes - calling request.getUserPrincipal() in the handler is not a good idea.

Your handler is executing before have has an opportunity to restore any cached identity so it would always return null.

No - calling HttpServletRequest#login is not necessary on every call, just the ones you receive the token you want to use for authentication.

If you do not call login we will attempt to restore any previously cached identity before the request reaches the servlet.

[Alex]

The token is part of every request, as it is provided in a cookie. With the legacy security subsystem that null check against request.getUserPrincipal() was a viable way to prevent the authentication process from starting over with each request.

What cached identity do you mean? One in the caching security realm because of my particular Elytron configuration I provided with the CLI script or are you talking about some kind of intermediate cache in Undertow?

Meanwhile, I cloned the elytron-web Git repo and checked out the 1.6.2.Final tag, matching the version in JBoss EAP 7.3.6, and adapted org.wildfly.elytron.web.undertow.server.SecurityContextImpl#login(String, String) to instantiate httpAuthenticator the same way org.wildfly.elytron.web.undertow.server.SecurityContextImpl#authenticate does if that field is null. After building and replacing the JAR in the org.wildfly.security.elytron-web.undertow-server module in the server I tried to verify the change with the same reproducer I sent you with calling only javax.servlet.http.HttpServletRequest#login in the handler. The changed code got executed, according to trace log for org.wildfly.security authentication and authorization were successful but the login from still appeared. My plan was to locally verify the fix with the real web app before requesting a backport to JBoss EAP but even the reproducer failed.

Did you change something else in your fix?

[Darran Lofthouse]

I need to check one more thing tomorrow to double check the position your handler is executing.

By cached identity I mean after a successful programmatic authentication via a call to login we automatically associate the resulting identity with the HTTP session so it can be restored on future requests.

Regarding the reproducer, yes I removed the call to authenticate as that triggers the execution of the configured authentication mechanisms and you want them to run after you have had the opportunity to call login not before as it is the mechanisms that trigger the sending of the login form.

[Alex]

For the mentioned local tests with the changed undertow-server artifact, I also removed the call to javax.servlet.http.HttpServletRequest#authenticate from the reproducer. Only javax.servlet.http.HttpServletRequest#login was called but this still wasn't enough to skip the login form. Because you said you were able to access the welcome page of the reproducer through programmatic login after your fix, I concluded that simply instantiating httpAuthenticator in  org.wildfly.elytron.web.undertow.server.SecurityContextImpl#login(String, String), as I did in my local fix, isn't enough. 

Now I'm a little confused. But you're right: let's put it off until tomorrow. :)

[Darran Lofthouse]

Just had another run with the deployment and a debugger.

This block of code will never return an identity as your handler is running before we have restored the cached identity from the session.

        if (request.getUserPrincipal() != null) {

            // Already authenticated.

            return;

        }

The remaining code I have added to the handler is:

        final String AUTHENTICATED = "authenticated";

        HttpSession session = request.getSession(false);

        if (session == null || session.getAttribute(AUTHENTICATED) == null) {

            request.login("testuser", "testpassword");

            session = session == null ? request.getSession() : session;

            session.putValue(AUTHENTICATED, AUTHENTICATED);

        }

This code is using a value in the session to check if the login call should now be skipped.

There is no call to authenticate in the handler as if there is no identity to restore from the HTTP session that would send the login form.

[Alex]

OK, interesting. Thanks for that snippet for the web app side.

What did you change in org.wildfly.elytron.web.undertow.server.SecurityContextImpl#login(String, String)?

[Darran Lofthouse]

I have linked the pull requests with the changes to https://issues.redhat.com/browse/ELYWEB-133

[Alex]

It finally works, both the reproducer and the real web app!

I applied your PR locally to elytron-web 1.6.2, built it and replaced the JAR of the module org.wildfly.security.elytron-web.undertow-server in an instance of JBoss EAP 7.3.6. In addition, I adapted the handlers of the reproducer and the real web app with the logic to manually store the authentication state in the session. The unnecessary check for the principal from the request and any test calls to javax.servlet.http.HttpServletRequest#authenticate are gone.

The test user for the reproducer is successfully authenticated and authorized and the "Hello World" page is shown. In the real web app, containing the cookie interception and token logic, everything seems to work fine, too. After successful authentication by the handler, JSESSIONID stays stable until logout or closing the browser. If the programmatic login fails (e.g., due to failed signature validation), the login form is shown as a fallback and manual login works without any problems.

Many thanks Darran for your help! You really saved me a headache. ;)

I will open a Red Hat support case today with the request to backport your fix from https://issues.redhat.com/browse/ELYWEB-133 to JBoss EAP 7.3.7.

[Madhava Alampally]

Hi Alex, 

I've tried this article and getting the following trace. Can you please confirm where/how you have defined "FromRolesAttributeDecoder"?

If you have GIT url to grab the working sample for custom authentication, that would be great.

Thanks & Regards

Madhava

07:38:03,811 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /simple-webapp/: javax.servlet.ServletException: UT010030: User already logged in
        at io.undertow.servlet.spec.HttpServletRequestImpl.login(HttpServletRequestImpl.java:499)
        at org.wildfly.security.examples.MyDummyTokenHandler.handleRequest(MyDummyTokenHandler.java:44)
        at org.wildfly.security.examples.MyServletExtension.lambda$null$0(MyServletExtension.java:17)
        at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53)
        at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
        at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
        at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59)
        at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
        at org.wildfly.elytron.web.undertow.server.servlet.CleanUpHandler.handleRequest(CleanUpHandler.java:38)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
        at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131)
        at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
        at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79)
        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)
        at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
        at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
        at java.lang.Thread.run(Thread.java:750)