Opencast 2.2.0 LDAP integration

[Justin Davis]

Hello All.

I'm having some problems with Opencast 2.2.0 and LDAP integration. I'm trying to implement LDAP as an authentication, authorization and user information provider (without CAS) as described in the documentation:

https://documentation.opencast.org/r/2.2.x/admin/configuration/security/

https://documentation.opencast.org/r/2.2.x/admin/configuration/security.ldap/

LDAP authentication was relatively easy to setup by configuring the security.xml as described. LDAP authorization and user info is still failing however.

When a user defined solely in LDAP accesses the media player, the user can authenticate but no roles seem to get populated and the logs on the directory server indicate that the additional LDAP attributes used to build the authorization rules are never queried for.

The following error also shows up in the Opencast logs:

2016-07-20 16:20:37,695 | WARN  | (ServletHandler:563) - /j_spring_security_check

org.springframework.security.core.userdetails.UsernameNotFoundException: USERNAME

        at org.opencastproject.kernel.userdirectory.UserAndRoleDirectoryServiceImpl.loadUserByUsername(UserAndRoleDirectoryServiceImpl.java:278)[161:matterhorn-kernel:2.2.0] 

I suspect this has something to do with the following snippet at the bottom of the initial security documentation:

        The set of user and role providers can be configured. If you do

  not want to keep users and passwords in Opencast's database, you can

  replace the JpaUserAndRoleProvider with the LdapUserProvider by

  replacing the matterhorn-userdirectory-jpa jar with the

  matterhorn-userdirectory-ldap jar.

The "matterhorn-userdirectory-ldap" module doesn't appear to be built/installed by default. Is there more documentation somewhere on this last step? Attempting to manually build and add the matterhorn-userdirectory-ldap module has proved fruitless thus far. When I build it and drop it in the ${MATTERHORN_HOME}/deploy folder, the following errors show up in the Opencast log:

2016-07-21 16:25:38,346 | INFO  | (BundleInfoLogger:110) - Bundle matterhorn-userdirectory-ldap, id 288, version 2.2.0, build number n/a

2016-07-21 16:25:38,412 | WARN  | (?:?) - Error while starting bundle: file:/opt/opencast/deploy/matterhorn-userdirectory-ldap-2.2.0.jar

    org.osgi.framework.BundleException: Unresolved constraint in bundle matterhorn-userdirectory-ldap [288]: Unable to resolve 288.0: missing requirement [288.0] osgi.wiring.package; (osgi.wiring.package=net.sf.cglib.proxy)

Any assistance would be greatly appreciated!

Thanks,

Justin

[Burhan Loqueman]

I have similar issues building cleanly from scratch on Centos 7. I was hoping the official release of the 2.2 RPMs would fix what seems to be poor support for LDAP / ActiveDirectory in the product which I find quite amazing really.

[Rubén Pérez]

Hello Justin,

I do not quite understand what you mean by "dropping the files in the deploy folder". In 2.2.x, a build folder should be created after a successful build, which contains several .zip files and a subfolder. Each of these zip files contains standalone Karaf instances for different Opencast profiles (admin, worker and presentation servers, as well as an "all in one" version, also unzipped). This mechanism is completely different as version prior to 2.0 inclusive.

I have just tried to run the "all-in-one" profile, using our local LDAP configuration, and I can successfully log in. The fact that your error is caused by a missing "wiring dependency" is also a hint in the direction that you may not be deploying the system as you should. Please make sure you are following the instructions in [1].

On the other hand, I must warn you that talking about "LDAP integration" in Opencast is an overstatement. Formerly, one could only read users from an LDAP instances by setting up a CAS server for authenticating the users. I have "patched" that in 2.2.0, so that your users can also log without a CAS if desired, but you still need to specify your LDAP configuration in two different places and in different formats. Also, while I can guarantee that the current state works for our particular scenario (in the University of Cologne), it may well not work for others.

Do not hesitate to ask any further questions you may have.

Best regards
Rubén

[1] https://documentation.opencast.org/r/2.2.x/admin/installation/

--
You received this message because you are subscribed to the Google Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

--
Rubén Pérez Vázquez

Universität zu Köln
Regionales Rechenzentrum (RRZK)
Weyertal 121, Raum 4.05
D-50931 Köln
✆: +49-221-470-89603

[Burhan Loqueman]

Hi.

Sorry for answering in place the original requester here but

a) I have also installed as per the RPM the all-in-one profile for 2.2

b) have following "the documentation" as per "the documentation" - but I can only find one reference to where LDAP settings are input - not two and your statement " but you still need to specify your LDAP configuration in two different places and in different formats," indicates that something else is missing in the official documentation about this point - where are the 'two locations' you mention apart from the one refenced below:

https://documentation.opencast.org/r/2.2.x/admin/configuration/security.ldap/

[Justin Davis]

Hi! Thank you for the detailed response.

> I do not quite understand what you mean by "dropping the files in
> the deploy folder". In 2.2.x, a build folder should be created after a
> successful build, which contains several .zip files and a subfolder. Each of
> these zip files contains standalone Karaf instances for different Opencast
> profiles (admin, worker and presentation servers, as well as an "all in one"
> version, also unzipped). This mechanism is completely different as version prior
> to 2.0 inclusive.

By "dropping the files in the deploy folder" I was trying to describe my
attempts to install/deploy a new *.jar (bundle?) into a running Opencast
instance that was deployed from a source build on an alternate system.

From the root of the source folder, I copied
"build/opencast-dist-allinone-2.2.0/" from my build workstation to
"/opt/opencast/" on ServerA. Within that there is a "deploy" folder
(/opt/opencast/deploy) -- that's where I was copying my custom built modules.
From watching the Opencast logs and digging through the Karaf web console I can
see that this did load the matterhorn-userdirectory-ldap (after some
trial-and-error).

> I have just tried to run the "all-in-one" profile, using our local LDAP
> configuration, and I can successfully log in. The fact that your error is caused
> by a missing "wiring dependency" is also a hint in the direction that you may
> not be deploying the system as you should. Please make sure you are following
> the instructions in [1].

I did run into some dependency issues when trying to install this bundle & did
flounder a bit trying to circumvent them.

Some of the specific missing libraries were mentioned in
${OPENCAST_HOME}/etc/custom.properties

# To enable the LDAP user provider, you must uncomment this line,
and run on a JVM that provides the
# "com.sun.jndi.ldap" package.
#org.osgi.framework.system.packages.extra=com.sun.jndi.ldap,com.sun.net.ssl.internal.ssl

So I tried commenting out the line in question, causing all kinds of woes and
errors.

Further investigation indicated that defining the extra system packages in
"custom.properties" overrides the same config option in "jre.properties" (those
lines in custom.properties should probably be corrected/removed), so I
re-commented those lines in custom.properties and instead added the libs to the
same config option in "jre.properties".

The libs in question are:

net.sf.cglib.proxy
org.apache.commons.pool
org.apache.commons.pool.impl

Which worked!

Mostly -- the bundle was successfully deployed and I'm able to verify that the
bundle did successfully pick up the LDAP connection details from my configs, but
when logging in to the Engage UI, there is an error that pops up in red at the
lower right of the page indicating a failed login -- however if I reload the
page, I do see that I'm logged in with the LDAP user, and checking the
"/info/me.json" REST endpoint shows that the LDAP user info and roles are
correctly defined.

The semi-successful login triggers the following warning in the opencast.log:

2016-07-27 11:45:10,350 | WARN | (ServletHandler:563) -
/j_spring_security_check
org.springframework.security.core.userdetails.UsernameNotFoundException:
LDAP_USER

> On the other hand, I must warn you that talking about "LDAP integration" in
> Opencast is an overstatement. Formerly, one could only read users from an LDAP
> instances by setting up a CAS server for authenticating the users. I have
> "patched" that in 2.2.0, so that your users can also log without a CAS if
> desired, but you still need to specify your LDAP configuration in two different
> places and in different formats. Also, while I can guarantee that the current
> state works for our particular scenario (in the University of Cologne), it may
> well not work for others. Do not hesitate to ask any further questions you may
> have.

I do have the LDAP info defined in both places as defined in the docs --
"mh_security.xml" and "org.opencastproject.userdirectory.ldap.cfg".

I guess at this point I'm a little lost as to what all functionality I can
leverage from using the provided LDAP tools. My basic goal was to be able to use
OpenLDAP as an authentication source, and within that, use group membership to
map to roles for authorization.

Once again, my sincere thanks.

Cheers!
Justin



--

justin davis
sivadw...@gmail.com

[Justin Davis]

Hey Burhan.

No worries -- more answers mean a more detailed response (ultimately)!

The two places the LDAP configuration is defined are in
"${OPENCAST_HOME}/etc/security/mh_security.xml" (for a single tenant
org) and "${OPENCAST_HOME}/etc/org.opencastproject.userdirectory.ldap.cfg".
This is described in Step 2 of the "Authorization" section on the page
you linked to -- however it does say "This step is only necessary in
Opencast deployments along multiple machines."

- Justin

justin davis
sivadw...@gmail.com

[Rubén Pérez]

Hi Justin,

Please find my answers after the relevant parts of your mail.

On 27/07/16 18:54, Justin Davis wrote:

By "dropping the files in the deploy folder" I was trying to describe my
attempts to install/deploy a new *.jar (bundle?) into a running Opencast
instance that was deployed from a source build on an alternate system.

From the root of the source folder, I copied
"build/opencast-dist-allinone-2.2.0/" from my build workstation to
"/opt/opencast/" on ServerA. Within that there is a "deploy" folder
(/opt/opencast/deploy) -- that's where I was copying my custom built modules.
From watching the Opencast logs and digging through the Karaf web console I can
see that this did load the matterhorn-userdirectory-ldap (after some
trial-and-error).

Yes, I must have been sleeping when I answered the first time. I did not realized you were talking about *the* deploy folder :P. As it turns out (and you could experience), this is not a good idea, because the LDAP bundle is tied to some rather obscure and complicated dependencies (hello, SpringSecurity!) and it does not work on its own.

I'm really intrigued why you should need these dependencies. What OS are you using? Which Java version? Are you running the Opencast version 2.2.0 without modifications? I makes no sense that the same bundle would require extra dependencies with no reason.

Mostly -- the bundle was successfully deployed and I'm able to verify that the
bundle did successfully pick up the LDAP connection details from my configs, but
when logging in to the Engage UI, there is an error that pops up in red at the
lower right of the page indicating a failed login -- however if I reload the
page, I do see that I'm logged in with the LDAP user, and checking the
"/info/me.json" REST endpoint shows that the LDAP user info and roles are
correctly defined.

The semi-successful login triggers the following warning in the opencast.log:

  2016-07-27 11:45:10,350 | WARN  | (ServletHandler:563) -
/j_spring_security_check
    org.springframework.security.core.userdetails.UsernameNotFoundException:
LDAP_USER

That sounds weird. Are you using one or several machines? (i.e. are you running the "Engage UI" in the same machine as the admin UI). If so, have you used the same LDAP configuration in the two files? Can you please write here the list of roles that your user has? Can you see any errors in the Javascript console, or in the list of requests from your browser to the server when you log in?

My guess here is, you have configured the mh_default_org.xml file correctly, but you may have made a mistake (or provided incomplete information) in the .cfg file. I must say, I find this configuration not particularly clear, and it is easy to make a mistake.

On the other hand, I must warn you that talking about "LDAP integration" in
Opencast is an overstatement. Formerly, one could only read users from an LDAP
instances by setting up a CAS server for authenticating the users. I have
"patched" that in 2.2.0, so that your users can also log without a CAS if
desired, but you still need to specify your LDAP configuration in two different
places and in different formats. Also, while I can guarantee that the current
state works for our particular scenario (in the University of Cologne), it may
well not work for others.  Do not hesitate to ask any further questions you may
have.

I do have the LDAP info defined in both places as defined in the docs --
"mh_security.xml" and "org.opencastproject.userdirectory.ldap.cfg".

I guess at this point I'm a little lost as to what all functionality I can
leverage from using the provided LDAP tools. My basic goal was to be able to use
OpenLDAP as an authentication source, and within that, use group membership to
map to roles for authorization.

That much is (should be) possible. The problem is that configuring it is overkill and then you need to add the insanity of the roles and groups that the new admin UI has brought into the system. The role groups don't seem to work as one might have expected, too, and they do not seem to get well with roles read from LDAP also.

For instance, imagine you want to give some users limited access to the UI. The roles are so fine grained (which is fine --no pun intended--), so you may end up adding your LDAP user into several dozens of roles to provide the functionality you required. You may well think "I can create several role groups with the different levels of access I want to provide. Every group generates an implicit role 'ROLE_GROUP_<GROUPNAME>', so I can assign my LDAP users to a group 'GROUP_<GROUPNAME>' and that shall do it". Turns out that it doesn't, and role groups are for some reason never expanded into the roles they contain, making then rather useless.

This, of course, has nothing to do with the LDAP "integration", but it directly affects how functional it can be.


Best regards

[Sven Laudel]

Hi Justin,

just found your LDAP thread now.

Here is what i found out.

I'm using the standard JpaUserAndRoleProvider and did not replace it by the LdapUserProvider. The user in the LDAP has one additional attribute which contains the opencast role "ROLE_ADMIN".

Sniffing on the network with wireshark turned out that there is communication between opencast and the LDAP AD Server. The bind user is successfully bound to the LDAP and the search for the user how tries to login successfully returns this user. The next bind with the login user ends successfully too and all attributes are returned from the LDAP to opencast.

So the user is being successfully authenticated. I can verify this, after opencast tells me that the authentication failed with "UserNotFound", by reloading the opencast admin index page, which shows me i am logged in. Also as you described when checking the 

"/info/me.json" REST endpoint i can see my username and the role which came from LDAP.

Next i connect my IntelliJ Idea to the running JVM to debug the process. But unfortunately i'm not that deep in java/spring development in general and opencast development in special. I can see what happens but can't tell why it goes wrong. It seems to me, opencast looks if the LDAP user exists in the user database which is not.

Maybe someone can tell me what happens when LDAP works correctly. Will the LDAP user be created in the opencast database?

Best regards

Sven

[Ruth Lang]

Hi Sven,

if you are talking about the LDAP part Rubén has shared with the community. Here are the restrictions.

1. the LDAP user is not created in the opencast database. If this would be the case I cannot see the advantage of using LDAP.
2. the LDAP user can use any role Opencast is offering as long as this role is attached to the LDAP user either in the LDAP or in the security file.
3. the LDAP user can not use own roles, which are not in the Opencast database or you have to define/add them additionally in the DB. 
4. the LDAP user can not use „grouped“ roles because this roles are handled differently. For using grouped roles you need to attach a local user that group,

Regards

Ruth

--
You received this message because you are subscribed to the Google Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

_______________________________

Universität zu Köln

Regionales Rechenzentrum (RRZK)
Weyertal 121, Raum 4.08
D-50931 Köln
✆:  +49-221-470-89618

rrzk.uni-koeln.de

facebook.com/rrzkoeln

[Sven Laudel]

Hi Ruth,

thank you for your response.

I now understand the role limitations better which maybe can be handled.

But at first i want the LDAP user being correctly authenticated without any "UserNotFound" exception and error message.

I'm using the attribute "extensionAttribute1 = ROLE_ADMIN" which seems to be applied correctly. If i don't set the attribute, the login fails completely and i'm not able to see the admin interface.

Best regards

Sven

[Ruth Lang]

Hi Sven,

this behavior is correct and "as designed“.

But this will happen also to every user defined in Opencast itself. 

An user in OC 2.2.x will see the adminUI

1. with the admin role ROLE_ADMIN (everything like the admin)
2. with the ROLE_ADMIN_UI and a lot of roles responsible for the access to different parts of the adminUI
3. with a grouped role, that contains the roles under 2.)

Unfortunately part 3 is not available for LDAP user because of an other treatment of grouped roles.

Regards

Ruth

You can define several roles and ROLE_ADMIN_UI

[Rubén Pérez]

Hi Sven,

I'm curious to know whether or not you were successful to access the UI using an LDAP user. The bottom line of Ruth's email is that you should, at least, provide your LDAP user with the role "ROLE_ADMIN_UI". Using different parts of the admin UI may require additional roles (which, unfortunately, I must say, are mostly not documented).

Best regards

--
Rubén Pérez Vázquez

Universität zu Köln
Regionales Rechenzentrum (RRZK)

Weyertal 121, Raum 4.05
D-50931 Köln
✆: +49-221-470-89603

[Sven Laudel]

Hi Rubén,

thanks for your reply.

Meanwhile i had a telephone call with Ruth, but we couldn't really fix the LDAP issue.

As i wrote in my first post in this thread, the login partly works. I'll describe it in a few words:

My LDAP user has an attribute which holds the additional opencast roles (in this example ROLE_ADMIN). Logging into the admin ui with the users credentials throws the mentioned "UsernameNotFound"-Exception. But loading "/admin-ng/index.html" after this exception has been thrown, i can see the full admin ui as i would expect it. Also calling "/info/me.json" rest endpoint shows me, i'm logged in. It tells me the username and the roles. The admin ui also shows the correct name top right. This corresponds with my network sniffing, which showed successful communication between opencast and active directory/LDAP.

But, it seems there are some more issues, because after uploading a recording it can't be processed, because the worker nodes do not know the user (sorry, don't remember the exact error message).

At the moment i postponed the further LDAP configuration because i don't know what to change in my configuration.

Best regards

Sven

[René Brandl]

Hi Sven!

Did you figure out your problem ?
I'm stuck somewhere on the same point as you ( at least it seams to be)

So i'm curious if you could provide any information on that matter

With regards
René