Missing documentation for ranger-druid-security.xml

Gabriele D'Arrigo

unread,

Dec 14, 2021, 7:53:32 AM12/14/21

to Druid User

Hi guys,
I'm trying to configure Druid to work with Apache Ranger by following this guide:

https://druid.apache.org/docs/latest/development/extensions-core/druid-ranger-security.html

Coming to the "Configuring the connection to Apache Range"r section, the guide says (bold is mine):

The Apache Ranger authorization extension will read several configuration files. Discussing the contents of those files is beyond the scope of this document. Depending on your needs you will need to create them. The minimum you will need to have is a ranger-druid-security.xml file that you will need to put in the classpath (e.g. _common). For auditing, the configuration is in ranger-druid-audit.xml.

And gives no other information on what is the format of ranger-druid-security.xml and how to configure it.
I wasn't able to find any other documentation on this config file, both on the official Druid documentation, that by simply searching on the Internet.
Am I missing something?
Thank you

Gabriele

Sergio Ferragut

unread,

Dec 14, 2021, 2:30:19 PM12/14/21

to Druid User

Hi Gabriele,

I have not done this myself, so take this with a grain of salt, but this is an example of that file I found on apache/druid github :

https://github.com/apache/druid/blob/master/extensions-core/druid-ranger-security/src/test/resources/ranger-druid-security.xml

Let us know how it goes.

Sergio

Gabriele D'Arrigo

unread,

Dec 15, 2021, 3:25:11 PM12/15/21

to Druid User

Ciao Sergio,
nice catch, and thank you so much for the response!
So, it seems that I'm not lucky with this Druid-Ranger integration:
I correctly mount ranger-druid-security.xml under /opt/druid/conf/druid/cluster/_common, but when I start the cluster I receive the following error:

Problem parsing object at prefix[druid.auth.authorizer.ranger]: Cannot construct instance of `org.apache.druid.security.ranger.authorizer.RangerAuthorizer`, problem: `java.lang.NullPointerException`

at [Source: UNKNOWN; line: -1, column: -1].

at org.apache.druid.server.initialization.AuthorizerMapperModule.configure(AuthorizerMapperModule.java:57) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.initialization.AuthorizerMapperModule)

while locating org.apache.druid.server.security.AuthorizerMapper

for the 1st parameter of org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater.<init>(CoordinatorBasicAuthorizerMetadataStorageUpdater.java:116)

at org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater.class(CoordinatorBasicAuthorizerMetadataStorageUpdater.java:77)

while locating org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater

at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerStorageUpdater(BasicSecurityDruidModule.java:158) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)

while locating org.apache.druid.security.basic.authorization.db.updater.BasicAuthorizerMetadataStorageUpdater

This is my current configuration (via an environment file, I'm working on a Docker compose basis):

https://gist.github.com/gabrieledarrigo/f03869706a968f4d05c0412076a668fb

Got any ideas?

G.

Sergio Ferragut

unread,

Dec 16, 2021, 2:52:15 PM12/16/21

to Druid User

More responses to your question are here: https://www.druidforum.org/t/druid-user-missing-documentation-for-ranger-druid-security-xml/6901/5

I'm reposting my response here as well:

Just checking. Did you customize the values in the config file for your environment?

the two settings in there that stand out at me as needing change are

The name of your ranger service:

<property> <name>ranger.plugin.druid.service.name</name>

<value>cl1_druid</value>. <== CHANGE THIS

<description> Name of the Ranger service containing policies for this SampleApp instance </description> </property>

and maybe also this one:

<property> <name>ranger.plugin.druid.policy.cache.dir</name>

<value>${project.build.directory}</value>. <==== CHANGE THIS

<description> Directory where Ranger policies are cached after successful retrieval from the source </description> </property>

Gabriele D'Arrigo

unread,

Dec 17, 2021, 10:13:38 AM12/17/21

to Druid User

Good morning Sergio,
I've seen the other responses in the forum and I subscribed, by I cannot reply to that thread (maybe 'cause I'm a newbie?).
First, forget the error of my last response, it was late and I noticed that I named ranger-druid-security.xml wrongly and it was not recognized by the components of the cluster.
In any case, I've got some news and a better understanding of the Druid - Ranger integration.
Let's break it into three parts:

1. ranger-druid-security.xml Configuration

Then in regards to the XML file that you found in the Druid repository, it's correct but partial since it contains only the service name and other configurations related to the policy cache.
What is missing is the URL of the Policy server (ie: Ranger Admin) that Druid can use to load the policies.
After some digging, I found that the format of the XML file is the same for each Ranger plugin, and is documented here (last section, Install and configure plugin in the service)

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207

So, with the XML file correctly configured (by adding the correct Ranger Admin URL, and an arbitrary service name) I faced a second (and biggest problem):

2. Service definition for Apache Druid to Apache Ranger

By following the Apache Ranger Security documentation the next step is to configure the service definition in Apache Ranger.
The documentation says:

> At the time of writing of this document Apache Ranger (2.0) does not include an out of the box service and service definition for Druid. You can add the service definition to Apache Ranger by entering the following command:

curl -u <user>:<password> -d "@ranger-servicedef-druid.json" -X POST -H "Accept: application/json" -H "Content-Type: application/json" http://localhost:6080/service/public/v2/api/servicedef/

So I thought, damn! Another missing documentation for a configuration file!
I found that the test package of the druid-ranger-security where you found the XML, contains a druid-policies.json file.
Under the serviceDef key, you can find the standard JSON that can be used to configure a new service in Apache Ranger.
Eureka! I POSTed the JSON to my Ranger Admin instance , opened the Admin UI, tried to create druid service (the same name used in ranger-druid-security.xml) and:

> Druid failed to find service class org.apache.ranger.service.druid.RangerDruidService. Resource lookup will not be available. Please make sure plugin jar is in the correct place.

Argh! Elementary, my dear Watson!
The fact is that even if configured, Druid can talk to Apache Ranger, but Ranger hasn't a plugin that implements an org.apache.ranger.service.druid.RangerDruidService.
With some backward thinking, it's exactly what the official Druid documentation says:

> At the time of writing of this document Apache Ranger (2.0) does not include an out of the box service and service definition for Druid.

But I find that this sentence is quite misleading:

> You can add the service definition to Apache Ranger by entering the following command:

Because it seems that, by doing the POST HTTP call to the Ranger Admin, the service will be automatically created!

So, I searched for some Apache Ranger Druid plugin, and I found this issue:

https://issues.apache.org/jira/browse/RANGER-1480

And a repository that implements an unofficial plugin: https://github.com/almeidajeff/ranger-druid/tree/master/druid

But at this point, I decided to stop my proof of concept of the integration

3. Conclusion

It's quite strange that Druid is shipped with an official extension for Ranger, but the latter still hasn't an official plugin to support Druid.
Maybe the author of https://github.com/apache/druid/pull/9579 tested it against a plugin or can give us a better understanding.

Saying this, in regards to the Druid documentation:

I guess that the "Configuring the connection to the Apache Ranger" section can be improved by adding an example of the ranger-druid-security.xml, maybe
with a link to the Ranger official documentation with the XML spec (on this, I'm really tempted to open a PR)
The "Adding the service definition for Apache Druid to Apache Ranger" is quite misleading and I guess it can be improved by describing how a service definition can be added to
Ranger if the latter has no support for Druid (unless a user is going to implement, compile and integrate a custom plugin for Ranger)

That's all!
Greetings

Gabriele

Sergio Ferragut

unread,

Dec 17, 2021, 12:28:16 PM12/17/21

to Druid User

Hi Gabriele,

I love the detailed post. Thank you.

I think you should fall into temptation and do the PR.

I want to go try this now too. In the end, was the authorization mechanism enabled ? i.e. after the using the REST API to register the service definition, do you see Druid in the Ranger UI? Can you define authorization rules? Do they control access?