LDAP DN problem

[Adam Harding]

I'm using XNAT with LDAP (Active Directory) authentication (as
described at http://nrg.wikispaces.com/Enhanced+Authentication+Options
), and recently an account became unable to log in. The AD account
works normally in interactive lookups and the myriad applications
using AD for authentication, including two other instances of XNAT on
two other systems configured identically, except for names &
addresses, with the same scripts. In particular, they're using
identical authentication.properties files, using the same account to
bind.

webapps/xnat/logs/xdat.log gets the following error and stack trace
upon attempting to log in with the problem account ("theusername",
here) on the problem system:

<timestamp> [http-8080-9] ERROR
org.nrg.xnat.security.LDAPAuthenticator - theusername:Error retrieving
DN for theusername from server
javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308:
LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data
525, vece]
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3041)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2987)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2789)
at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2703)
at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:293)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:
175)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:
193)
at
com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:
136)
at
com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:
66)
at
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:
667)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:
288)
at javax.naming.InitialContext.init(InitialContext.java:223)
at
javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:
134)
at
org.nrg.xnat.security.LDAPAuthenticator.openContext(LDAPAuthenticator.java:
191)
at
org.nrg.xnat.security.LDAPAuthenticator.verifyLogin(LDAPAuthenticator.java:
410)
at
org.nrg.xnat.security.LDAPAuthenticator.authenticate(LDAPAuthenticator.java:
580)
at
org.nrg.xnat.security.LDAPAuthenticator.authenticate(LDAPAuthenticator.java:
495)
at
org.nrg.xdat.security.Authenticator.Authenticate(Authenticator.java:
60)
at
org.nrg.xdat.turbine.modules.actions.XDATLoginUser.doPerform(XDATLoginUser.java:
89)
at
org.apache.turbine.modules.actions.VelocityAction.doPerform(VelocityAction.java:
46)
at
org.apache.turbine.util.velocity.VelocityActionEvent.perform(VelocityActionEvent.java:
82)
at
org.apache.turbine.modules.actions.VelocityAction.perform(VelocityAction.java:
72)
at org.apache.turbine.modules.ActionLoader.exec(ActionLoader.java:96)
at
org.apache.turbine.modules.pages.DefaultPage.doBuild(DefaultPage.java:
113)
at org.apache.turbine.modules.Page.build(Page.java:53)
at org.apache.turbine.modules.PageLoader.exec(PageLoader.java:98)
at org.apache.turbine.Turbine.doGet(Turbine.java:751)
at org.apache.turbine.Turbine.doPost(Turbine.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
191)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:
465)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:
852)
at org.apache.coyote.http11.Http11Protocol
$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:
489)
at java.lang.Thread.run(Thread.java:619)
<timestamp> [http-8080-9] ERROR
org.nrg.xdat.turbine.modules.actions.XDATLoginUser -
org.nrg.xdat.security.XDATUser$PasswordAuthenticationException:
Invalid Login and/or Password
at
org.nrg.xnat.security.LDAPAuthenticator.authenticate(LDAPAuthenticator.java:
512)
at
org.nrg.xdat.security.Authenticator.Authenticate(Authenticator.java:
60)
at
org.nrg.xdat.turbine.modules.actions.XDATLoginUser.doPerform(XDATLoginUser.java:
89)
at
org.apache.turbine.modules.actions.VelocityAction.doPerform(VelocityAction.java:
46)
at
org.apache.turbine.util.velocity.VelocityActionEvent.perform(VelocityActionEvent.java:
82)
at
org.apache.turbine.modules.actions.VelocityAction.perform(VelocityAction.java:
72)
at org.apache.turbine.modules.ActionLoader.exec(ActionLoader.java:96)
at
org.apache.turbine.modules.pages.DefaultPage.doBuild(DefaultPage.java:
113)
at org.apache.turbine.modules.Page.build(Page.java:53)
at org.apache.turbine.modules.PageLoader.exec(PageLoader.java:98)
at org.apache.turbine.Turbine.doGet(Turbine.java:751)
at org.apache.turbine.Turbine.doPost(Turbine.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
191)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:
465)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:
852)
at org.apache.coyote.http11.Http11Protocol
$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:
489)
at java.lang.Thread.run(Thread.java:619)

The account was recently moved to a different OU, so the DN has
changed. It doesn't look like the DN being submitted gets logged.
Adding "logger.debug("DN:" + dn);" before the try block at
LDAPAuthenticator.java:409, updating the deployment and changing from
ERROR to DEBUG at log4j.properties:66 & 75, relaunching, and showing
attempting a login with the account shows the old DN-- containing the
old OU-- is being used, printing this immediately before printing the
stack trace:
<timestamp> [http-8080-1] DEBUG
org.nrg.xnat.security.LDAPAuthenticator - DN:CN=theusername,OU=old
ou,OU=shown here,DC=iowa,DC=uiowa,DC=edu

Any idea why this isn't refreshing on a particular system? In general,
how should I avoid and fix the problem? I don't know why it would only
happen on one XNAT system vs. the others.

Thanks,
Adam

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To post to this group, send email to xnat_di...@googlegroups.com.
To unsubscribe from this group, send email to xnat_discussi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xnat_discussion?hl=en.

[hans.j....@gmail.com]

Adam,

Does the username have weird characters in it "i.e. [-*&^%$#@!]" or does the
username have database keyword in it "i.e. alter"?

These problems have been discussed before.

Hans

[Adam Harding]

No such oddities with the username. I verified that one of the working
XNAT instances is using a DN with the current OU path (not the old
one).

I think the likely reason it fails on some instances (one here, by
chance) and succeeds on others (two here, by chance) is that the
account happened to log in only just before the OU change on the now-
failing instance and only just after the OU change on those those that
work. They're not "still" working, but working as usual with an
initial account. A subsequent DN change would probably cause the same
problem on the other systems.

The behavior currently implemented, described here:
http://nrg.wikispaces.com/Enhanced+Authentication+Options

seems consistent with this:

1) Attempt to log in with a particular CN
XNAT performs lookup using SEARCHBASE=(&(objectClass=user)(CN=%USER
%)), then creates an account in the database, storing the DN (rather
than just the key used for value lookup, which is the search template;
the CN, in this case). Authentication against LDAP can proceed.
2) Change DN for the user
The OU for the CN changes, so the DN changes.
3) Attempt to log in with the same CN later
XNAT finds a local account which matches the CN, and uses its stored
record of what the DN was at initial lookup to attempt to authenticate
the user. The LDAP server replies "525 user not found" and
authentication against LDAP halts.

It looks like a matter of needing to store only the key used to look
up the value, rather than caching the value itself. Possibilities:

Upon successful authentication, store in the local database's account
representation only the string the user provided, rather than the DN
returned. Do subsequent lookups based on whatever search template is
in effect.

Upon successful authentication, store locally both the string the user
provided and the search template in effect at the time of most recent
authentication. For subsequent login attempts, first try searching
with the stored template, then try the search with the active
template.

I imagine that database changes required by changing this application
behavior would only involve altering the stored stored values.

On May 18, 4:43 pm, "hans.j.john...@gmail.com"

<hans.j.john...@gmail.com> wrote:
> Adam,
>
> Does the username have weird characters in it "i.e. [-*&^%$#@!]" or does the
> username have database keyword in it "i.e. alter"?
>
> These problems have been discussed before.
>
> Hans
>
>
>
> > From: Adam Harding <adam-hard...@uiowa.edu>
> > Reply-To: <xnat_di...@googlegroups.com>
> > Date: Tue, 18 May 2010 14:07:08 -0700 (PDT)
> > To: xnat_discussion <xnat_di...@googlegroups.com>
> > Subject: LDAP DN problem
>
> > I'm using XNAT with LDAP (Active Directory) authentication (as

> > described athttp://nrg.wikispaces.com/Enhanced+Authentication+Options

> For more options, visit this group athttp://groups.google.com/group/xnat_discussion?hl=en.

[Olsen, Timothy]

This seems like a design flaw in the LDAPAuthenticator. We didn't expect the DN to change.

If I remember correctly, caching the DN saved an extra query to the LDAP server on subsequent logins (though I haven't reviewed the code yet). If the DN is unstable, then we should definitely be prepared for new DNs. I think your second proposal would be my preference. This would still allow the caching of the DN, but be prepared for modified DNs.

I'll start sketching out the fix.

Tim

[Olsen, Timothy]

I've refactored the code to handle an updated DN. Unfortunately, I don't have a configured AD server to test against. Previously, I used one of Hans test servers for this, but I don't have access to this. Furthermore, I'm leaving on vacation (Wisconsin here we come) tomorrow afternoon and probably wouldn't get to test this until after I return (June 1st). I've double-checked the code and walked one of our other developers (John Paulett) through it. It should be ready to go. If you can test it for me I would appreciate it. Otherwise it would probably sit untested until June.

If there are issues with the code, John can probably patch it while I'm out (he'll be listening to this thread). Otherwise, it will be a top priority when I return.

You can download the file from here:
http://bitbucket.org/nrg/xdat_release_1_4/raw/d5ea94c466cb/plugin-resources/webapp/xnat/java/org/nrg/xnat/security/LDAPAuthenticator.java

And put it here:

XNAT/plugin-resources/webapp/xnat/java/org/nrg/xnat/security/

You'll need to run the bin/update script, and deploy it to the server. WARNING: if you're previously updated xdat-1.jar is only updated in the webapp dir, then this would overwrite it. Copy the xdat-1.jar you want into the XNAT/plugin-resources/repository/xdat/jars/ before you run the update.

FYI, we are gradually moving all XNAT development over to Mercurial. Its not 'official' yet, but it is what we are using for our development at this point. We are still finalizing a few things before we can make the move official.

One other point, the LDAPAuthenticator class was a quick implementation of authentication support for LDAP. It will remain around and usable, but in addition we are reviewing our support for other authentication schemes. In particular we would like to support Shibboleth. This would be very helpful here at WASH U. It may be of interest to you guys as well, but it is still down the road a bit. Any feedback or requests here would be appreciated.

Tim

-----Original Message-----
From: xnat_di...@googlegroups.com [mailto:xnat_di...@googlegroups.com] On Behalf Of Adam Harding
Sent: Wednesday, May 19, 2010 10:42 AM
To: xnat_discussion
Subject: Re: LDAP DN problem

[Adam Harding]

Thanks for the quick fix. I'll test it using a copy of the database
backing the XNAT instance that shows this problem for a known account.
I should also be able to test non-CN changes to the DN. Is there any
particular logging I should enable or diagnostic output to expect? I
can have a look and add more if useful of course.

> If there are issues with the code, John can probably patch it while I'm out (he'll be listening to this thread).

Thanks. I'll try to filter anything minor even before that, too.

> if you're previously updated xdat-1.jar is only updated in the webapp dir, then this would overwrite it.

Thanks for mentioning this. I've been testing modifications with
direct replacement in the app server and adding them to the source
only when this is complete or when I test with the update script. Good
thing not to assume, of course!

I know the move to Mercurial is pending; are there any procedural
changes? We'll likely continue to build XNAT from source, so if you'd
like help shaking it down I imagine we'll go through it anyway.

> the LDAPAuthenticator class was a quick implementation of authentication support for LDAP.

> In particular we would like to support Shibboleth.

We are *very* interested in exactly the same thing. As chance would
have it, I resumed work this morning on integrating our Shibboleth SP
with the web environment here. We had a test instance federated with
InCommon a while back, but I haven't worked further on it in months.
Unfortunately, we're running XNAT on its own server and fronting
through its own local instance of Apache, so we may end up having to
do some reverse-proxy trickery or run another SP. The former could be
problematic due to XNAT's/DicomServer's standard operation, and we'd
like to avoid burning through SPs and endpoint maintenance on a per-
application basis.

I don't think we were planning on making XNAT our first Shibbolized
app, but I think we'd be very interested in this feature. I haven't
looked into open source authenticator classes in a while, but I agree
that a more general implementation is wise. You can probably get by
using Shibboleth for authn only (as with the LDAP authenticator), but
you might want to think about using (optionally?) IdP-relased
attributes maybe as markers.

We can follow up on this later, say after your vacation!

Thanks,
Adam

On May 20, 12:44 pm, "Olsen, Timothy" <t...@npg.wustl.edu> wrote:
> I've refactored the code to handle an updated DN.  Unfortunately, I don't have a configured AD server to test against.  Previously, I used one of Hans test servers for this, but I don't have access to this.  Furthermore, I'm leaving on vacation (Wisconsin here we come) tomorrow afternoon and probably wouldn't get to test this until after I return (June 1st).  I've double-checked the code and walked one of our other developers (John Paulett) through it.  It should be ready to go.  If you can test it for me I would appreciate it.  Otherwise it would probably sit untested until June.
>
> If there are issues with the code, John can probably patch it while I'm out (he'll be listening to this thread).  Otherwise, it will be a top priority when I return.
>

> You can download the file from here:http://bitbucket.org/nrg/xdat_release_1_4/raw/d5ea94c466cb/plugin-res...

> ...
>
> read more »

[Adam Harding]

I installed the new authenticator and left logging on debug from prior
testing. A login attempt using the problem account now logs:

ERROR: the incorrect DN and exception when the LDAP server reports
user not found
...
DEBUG: FILTER: and the filter with CN=literalUsernameString
...
Info: LDAP Server has a new DN for this user. Attempting
authentication with new DN CN=etc...
...
DEBUG: FILTER: and the filter with CN=literalUsernameString
...
INFO: LDAP authentication succeeded with updated DN CN=etc...

The login succeeds transparently from the user's perspective.
Subsequent login attempts succeed as well, and xdat.log also shows:
INFO: [http-8080-5] INFO org.nrg.xnat.security.LDAPAuthenticator -
theusername: verified versus cache password. EXPIRES:<datestamp a bit
less than an hour away>

I think the problem is solved from a behavior point of view. Thanks
for the fix.

Adam