AccessControlException for getClassLoader in custom AuthenticationBackend

[James Beckett]

I'm preparing an internal proof-of-concept for a custom authentication backend for Search Guard to integrate with our own credentials source. This would be one of our prerequisites for justifying purchase of an Enterprise licence.

I've read https://docs.search-guard.com/latest/custom-authentication-modules and the linked Javadoc, and the source, and I have a project which is working as far as its own unit tests, but fails to initialize when configured in sg_config.yml. My authentication module is built as a Maven artifact with dependencies extracted to the output "fat jar" and dropped into the search-guard-6 directory next to the plugin jars. I had a bit of a fight with "Jar Hell" but it's getting past that now.

At initialization it's throwing what a SecurityManager AccessControlException while trying to load JPA/Hibernate (4.3.11). I've seen that Elasticsearch builds its own Policy but this failure seems odd as the getClassLoader permission is already included in the Search Guard plugin security policy.

I've tried wrapping the createEntityManagerFactory call with the AccessController.doPrivileged pattern but that actually fails earlier in the call chain - probably the wrong approach.

I've tried running with -Djava.security.debug=access,failure but it provides no further insights.

Is there something else which needs adding to the plugin policy to allow this class loading process? Is there any way that a drop-in backend module can augment the policy or are we stuck with a manual edit? (if so, would it be better to package it as a separate Elasticsearch plugin?)

* Elasticsearch 6.2.2; Search Guard 22.1 (Kibana plugin v13)

* Using built-in enterprise modules only

* JVM 1.8.0_161 on Windows 7 Pro SP1

* Search Guard configuration files (some names changed to obscure IP details):

searchguard:
  dynamic:
  ...
    authc:
      my-auth:
        enabled: true
        http_enabled: true
        transport_enabled: false
        order: 5
        http_authenticator:
          type: "basic"
          challenge: false
        authentication_backend:
          type: "com.example.es.sg.auth.MyAuthenticationBackend"
          config: ...

* Elasticsearch log messages

[2018-06-13T13:38:37,259][DEBUG][c.f.s.c.IndexBaseConfigurationRepository] Notify com.floragunn.searchguard.auth.BackendRegistry@525b8922 listener about change configuration with type config
[2018-06-13T13:38:37,269][DEBUG][c.f.s.s.ReflectionHelper ] Loaded module Module [type=INTERNAL_USERS_AUTHENTICATION_BACKEND, implementing class=com.floragunn.searchguard.auth.internal.InternalAuthenticationBackend]
[2018-06-13T13:38:37,273][DEBUG][c.f.s.s.ReflectionHelper ] Loaded module Module [type=HTTP_BASIC_AUTHENTICATOR, implementing class=com.floragunn.searchguard.http.HTTPBasicAuthenticator]
[2018-06-13T13:38:37,274][ERROR][c.f.s.a.BackendRegistry  ] Not yet initialized (you may need to run sgadmin)
[2018-06-13T13:38:37,285][INFO ][c.e.e.s.a.MyAuthenticationBackend] Starting up authentication backend: { /* settings from elasticsearch.yml and sg_config.yml section */}, C:\dev\elastic\6.2.2\elasticsearch-6.2.2\config
[2018-06-13T13:38:37,304][WARN ][c.f.s.s.ReflectionHelper ] Unable to enable 'com.example.es.sg.auth.MyAuthenticationBackend' due to java.lang.reflect.InvocationTargetException
[2018-06-13T13:38:37,306][ERROR][c.f.s.a.BackendRegistry  ] Unable to initialize auth domain my-auth due to ElasticsearchException[java.lang.reflect.InvocationTargetException]; nested: InvocationTargetException; nested: AccessControlException[access denied ("java.lang.RuntimePermission" "getClassLoader")];
org.elasticsearch.ElasticsearchException: java.lang.reflect.InvocationTargetException
 at com.floragunn.searchguard.support.ReflectionHelper.instantiateAAA(ReflectionHelper.java:184) ~[search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
 at com.floragunn.searchguard.auth.BackendRegistry.newInstance(BackendRegistry.java:668) ~[search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
 at com.floragunn.searchguard.auth.BackendRegistry.onChange(BackendRegistry.java:230) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
 at com.floragunn.searchguard.configuration.IndexBaseConfigurationRepository.notifyAboutChanges(IndexBaseConfigurationRepository.java:361) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
 at com.floragunn.searchguard.configuration.IndexBaseConfigurationRepository.reloadConfiguration(IndexBaseConfigurationRepository.java:311) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
 at com.floragunn.searchguard.configuration.IndexBaseConfigurationRepository$1$1.run(IndexBaseConfigurationRepository.java:171) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]

Caused by: java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
 at com.floragunn.searchguard.support.ReflectionHelper.instantiateAAA(ReflectionHelper.java:176) ~[?:?]
 ... 6 more

Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getClassLoader")
 at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_161]
 at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_161]
 at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_161]
 at java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:1528) ~[?:1.8.0_161]
 at java.lang.Thread.getContextClassLoader(Thread.java:1443) ~[?:1.8.0_161]
 at javax.persistence.spi.PersistenceProviderResolverHolder$PersistenceProviderResolverPerClassLoader.getContextualClassLoader(PersistenceProviderResolverHolder.java:101) ~[?:?]
 at javax.persistence.spi.PersistenceProviderResolverHolder$PersistenceProviderResolverPerClassLoader.getPersistenceProviders(PersistenceProviderResolverHolder.java:76) ~[?:?]
 at javax.persistence.Persistence.getProviders(Persistence.java:69) ~[?:?]
 at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:53) ~[?:?]
 at com.example.es.sg.auth.db.Database.<init>(Database.java:21) ~[?:?]
 at com.example.es.sg.auth.MyAuthenticationBackend.<init>(MyAuthenticationBackend.java:66) ~[?:?]
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
 at com.floragunn.searchguard.support.ReflectionHelper.instantiateAAA(ReflectionHelper.java:176) ~[?:?]
 ... 6 more
[2018-06-13T13:38:37,356][INFO ][c.f.s.c.IndexBaseConfigurationRepository] Search Guard License Info: ...

Thanks,

James

[SG]

Can you probably share your code? You can send a pgp encrypted mail or put them in a private github repo?
Or maybe a stripped down part so that we can reproduce the error?

Wrapping the createEntityManagerFactory call with the AccessController.doPrivileged pattern sounds like the correct approach. Can you provide more details whats exactly happen when you do that?
We do similar in our own classes, pls have a look here https://github.com/floragunncom/search-guard-enterprise-modules/blob/master/src/main/java/com/floragunn/dlic/auth/http/kerberos/HTTPSpnegoAuthenticator.java

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/0803e31a-c080-4ad2-9355-6c1727f9d8b9%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[James Beckett]

Stripped down version using H2 now on GitHub: https://github.com/hackery/search-guard-authbackend-example-jpa/releases/tag/v1.0

The test case creates users; when run from Search Guard it won't populate these, but fails before it gets to calling the authentication method.

With the call to createEntityManagerFactory wrapped with AccessController.doPrivileged, it fails earlier:

[2018-06-13T19:21:21,890][ERROR][c.f.s.a.BackendRegistry  ] Unable to initialize auth domain my-auth due to ElasticsearchException[java.lang.reflect.InvocationTargetException]; nested: InvocationTargetException; nested: PersistenceException[No Persistence provider for EntityManager named com.example.db.users];

org.elasticsearch.ElasticsearchException: java.lang.reflect.InvocationTargetException
        at com.floragunn.searchguard.support.ReflectionHelper.instantiateAAA(ReflectionHelper.java:184) ~[search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
        at com.floragunn.searchguard.auth.BackendRegistry.newInstance(BackendRegistry.java:668) ~[search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
        at com.floragunn.searchguard.auth.BackendRegistry.onChange(BackendRegistry.java:230) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
        at com.floragunn.searchguard.configuration.IndexBaseConfigurationRepository.notifyAboutChanges(IndexBaseConfigurationRepository.java:361) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
        at com.floragunn.searchguard.configuration.IndexBaseConfigurationRepository.reloadConfiguration(IndexBaseConfigurationRepository.java:311) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
        at com.floragunn.searchguard.configuration.IndexBaseConfigurationRepository$1$1.run(IndexBaseConfigurationRepository.java:171) [search-guard-6-6.2.2-22.1.jar:6.2.2-22.1]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]

Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
        at com.floragunn.searchguard.support.ReflectionHelper.instantiateAAA(ReflectionHelper.java:176) ~[?:?]
        ... 6 more

Caused by: javax.persistence.PersistenceException: No Persistence provider for EntityManager named com.example.db.users
        at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:61) ~[?:?]
        at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:39) ~[?:?]
        at com.example.es.sg.auth.db.Database.lambda$new$0(Database.java:23) ~[?:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_161]

        at com.example.es.sg.auth.db.Database.<init>(Database.java:21) ~[?:?]

        at com.example.es.sg.auth.MyAuthenticationBackend.<init>(MyAuthenticationBackend.java:31) ~[?:?]

[SG]

see https://github.com/floragunncom/search-guard-authbackend-example-jpa/tree/sg and https://github.com/floragunncom/search-guard-authbackend-example-jpa/commits/sg

In general it looks to me that JPA (and especially Hibernate) is not working well in environments with uncommon classloader situations and with security manager enabled.
I would recommend to to check if another JPA implementation behaves better and maybe does not need the extra permissions

permission java.lang.RuntimePermission "createClassLoader";
permission java.util.PropertyPermission "jboss.i18n.generate-proxies","write";

Or you can skip JPA completely and use a more lightweight approach.

> Am 13.06.2018 um 20:13 schrieb James Beckett <choccy...@gmail.com>:
>
> Stripped down version using H2 now on GitHub: https://github.com/hackery/search-guard-authbackend-example-jpa/releases/tag/v1.0
> The test case creates users; when run from Search Guard it won't populate these, but fails before it gets to calling the authentication method.
>

> --
> You received this message because you are subscribed to the Google Groups "Search Guard Community Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to search-guard...@googlegroups.com.
> To post to this group, send email to search...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/search-guard/be26c646-0c93-4318-902c-d64e1335a560%40googlegroups.com.

[James Beckett]

Thanks a mil for that - I now have a working (loading) version of both the example.com version and my full version (albeit having unrelated issues).

Looks like the call to setContextClassLoader and the extra permissions were the critical missing pieces. I'm not terribly familiar with the security manager.

I did start with a non-JPA version but it felt a bit too old-school even for me :)

Incidentally, while squirrelling around code for JPA, I found that the jpa-spec version internally invokes doPrivileged ...

https://github.com/javaee/jpa-spec/blob/master/javax.persistence-api/src/main/java/javax/persistence/spi/PersistenceProviderResolverHolder.java#L152

I suspect the Hibernate JPA API has some differences there.