Jenkins plugin's classloader isolation

[Kirill]

Hi,

In our company we have created 2 plugins: plugin A and plugin B. Both of them use a library XX, but different versions of it.
The question is: are plugins (they do not depend on each other) using separate isolated class loaders? It's not obvious from reading https://wiki.jenkins-ci.org/display/JENKINS/Plugin+Structure.
Will there be a problem when both plugins' extensions are used in the same Jenkins job (build steps)?

Regards,
Kirill.

[Stephen Connolly]

So what will happen is that, unless you configure the hpi plugin correctly, they will be use the plugin uberclassloader which will be selecting one of those library versions as the winner.

The normal way to handle this issue is to create a mostly empty plugin that the other plugins can depend on directly and let that plugin drive the version alignment...

If you must use different versions then you need to ask the hpi plugin to give you classloader child-first classloading or ask it to isolate specific packages and hope you can slice and dice correctly as neither option is particularly pretty or without its associated pain/risks

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/587fa8e7-cf44-4850-aa90-178618e1c2cf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kirill]

So you actually say that plugins share their libraries unless the child-first classloading is defined?
That's odd. For example, many plugins use Apache Commons or Guava, or some XML parsing libs. If they'd share their libs, every Jenkins instance would be a "JAR hell" - but it's not.

[Jesse Glick]

On Fri, Feb 6, 2015 at 7:08 AM, Kirill <yam...@gmail.com> wrote:
> So you actually say that plugins share their libraries unless the
> child-first classloading is defined?

No, I think several possible cases are being mixed up in this discussion.

If the library is bundled in Jenkins core, you have a problem, because
the sloppy design here lets every plugin “see” libraries used by core,
even if it only intended to use them internally (rather than
reëxporting some of their types). To use a different version of the
library from your plugin, you need to (a) declare a dependency on it
from your POM, ensuring it is bundled in WEB-INF/lib/*.jar, (b) select
plugin-first class loading mode for this plugin (a dangerous option
disabled by default).

If the library is just something your plugins bundle that core does
not, and neither plugin depends on the other, normally you are fine.
The potential problems:

· Some Jelly view etc. tries to load a resource such as an icon by
package path; one version or the other is chosen arbitrarily. Maybe
OK.
· The author of the library (Xerces…) was a raving madman and defined
some extension points with default implementations which are loaded by
the equivalent of calling
Thread.currentThread().getContextClassLoader().loadClass("the.known.default.impl.which.is.actually.in.the.same.JAR").
You will get mysterious ClassCastException’s. The workaround is to
wrap any call to the library from your plugin in a try-finally block
which sets the CCL to the ClassLoader of some class in the plugin or
its bundled library.

If one or the other plugin depends on the other, or some third plugin
needs to depend on both, then the only option is to align versions of
the library. Merely picking the same version in each plugin is *not*
enough; you will still get linkage errors, because the same bytecode
will be loaded twice. You need to factor out the library into its own
otherwise empty plugin, and declare a regular plugin dependency on it.

> If they'd share their libs, every Jenkins instance would be a "JAR hell" - but it's not.

Well, most Jenkins instances are not JAR hells. But some are.

[Kirill]

Thanks for a very descriptive answer, Jesse.

Actually the library XX is our internal library, so it's definitely not among the Jenkins core libs.

The problem is that the users of Plugin B recently started to complain that they regularly get errors "java.lang.ClassCastException: com.foo.EventImpl cannot be cast to com.foo.Event". Library XX contains an interface com.foo.Event and an implementing class com.foo.EventImpl. So the error means that class and interface were loaded by different classloaders from different library versions.

Plugin A and Plugin B do not have dependencies on other plugins. So according to what you said I still assume that their libs are isolated from each other. And the problem is probably happening because Plugin B contains few dependencies that in their turn depend on different library XX versions. That seems to be the only possible option.

select plugin-first class loading mode for this plugin (a dangerous option disabled by default).

What's dangerous about it? Possibility to override some Jenkins core libs to incompatible ones?

[Nigel Magnay]

What's the current story with guava?

I.E: if you're pluginFirstClassLoader and a later-than-core guava in WEB-INF/lib, will this work? Or are you saying this implementation will 'leak' into other plugins?

I notice jclouds-plugin goes to lengths to shade its dependency. It'd be nice if the jenkins plugin manager could do that (the shading) at deploy-time..

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr107f2%2Bb2YSM_kWUk25wObzz3Yc3Q7kORX7fy%3DN0bmDBw%40mail.gmail.com.

[Jesse Glick]

On Fri, Feb 6, 2015 at 10:00 AM, Kirill <yam...@gmail.com> wrote:
> The problem is that the users of Plugin B recently started to complain that
> they regularly get errors "java.lang.ClassCastException: com.foo.EventImpl
> cannot be cast to com.foo.Event". Library XX contains an interface
> com.foo.Event and an implementing class com.foo.EventImpl. So the error
> means that class and interface were loaded by different classloaders from
> different library versions.

Then somebody somewhere is loading classes incorrectly and you have to
track down why. Typically this is the result of reflective calls to
ClassLoader.loadClass (or Class.forName), which might be visible in
code near lines in the stack trace.

>> select plugin-first class loading mode for this plugin (a dangerous option
>> disabled by default).
>
> What's dangerous about it?

If you do not know what you are doing, you can unintentionally mask
something you should not have masked, and wind up with linkage errors.

[Jesse Glick]

On Fri, Feb 6, 2015 at 10:05 AM, Nigel Magnay <nigel....@gmail.com> wrote:
> What's the current story with guava?

Hell.

> I.E: if you're pluginFirstClassLoader and a later-than-core guava in
> WEB-INF/lib, will this work?

If you are very careful.

> I notice jclouds-plugin goes to lengths to shade its dependency. It'd be
> nice if the jenkins plugin manager could do that (the shading) at
> deploy-time

It is easy enough to shade dependencies using the Maven plugin, there
is no need for any runtime tricks.

[Vaclav Tunka]

Folks,

maybe it's tangential to your discussion or event irrelevant, however
what about considering JBoss Modules?

TL;DR; Modular classloading instead of hierarchical.

Hierarchical classloading was giving us headaches in JBoss AS 5.x,
modular classloading used since AS7 and in Wildfly solves all various pains.

You can basically separate various parts of the platform, isolate
dependencies in multiple modules, design what will be exposed, what not.

It's basically very very very much simplified OSGi, that gives you
certain modularity and separation.

Doc:
https://docs.jboss.org/author/display/MODULES/Introduction

Github:
https://github.com/jboss-modules/jboss-modules

Or have a look at older presentation, since slide 25 to understand the
concept graphically:
http://www.slideshare.net/dandreadis/jboss-as7-reloaded

HTH

Cheers,
Vaclav

--
Vaclav Tunka
Enterprise Application Platforms
JBoss by Red Hat

[Kanstantsin Shautsou]

imho jenkins core definitely must provide good core and exclude such class mess. I also mentioned JBOSS modules on #jenkins as idea..

Is there any exising document that has comparison of various existed application, their technologies and problems? For example what is used in Jira, artifactory and other applications that provide plugins? I think it will be good to compare and choose something suitable for jenkins enhancement

[Jesse Glick]

On Tue, Feb 10, 2015 at 9:19 AM, Vaclav Tunka <vtu...@redhat.com> wrote:
> what about considering JBoss Modules?
>
> TL;DR; Modular classloading instead of hierarchical.

Jenkins *does* use modular class loading at the plugin level; for
historical reasons this is not applied to libraries shipped in core.

If we were starting from scratch, certainly JBoss Modules, OSGi (at
least the module layer), etc. etc. would be the only sane way to
manage a thousand plugins. (By way of background, I worked for years
on the NetBeans module system, which has roughly similar
capabilities.) But the question is how such a change could be
retrofitted into the complex existing system in Jenkins without major
compatibility problems.

[ejo...@gmail.com]

Hi, I do belong to same organisation as Kirill and responsible for one of the two mentioned plugins. We did locate one fault where we used the Jenkins XmlFile class to save on file. When adding a XStream with a proper ClassLoader defined we solved this part:

final XStream xs = new XStream();
xs.setClassLoader(this.getClass().getClassloader());
final XmlFile xFile = new XmlFile(xs, file);

But we also have problems with the Jenkins saved job configurations files (config.xml). For instance we have a trigger implemented in plugin A which have a variable that is defined in the library that we share with the plugin B (different versions of the lib). From config.xml file:

<triggers>
  <com.xxx.yyy.triggering.MessageTrigger plugin="e...@21.0.8-SNAPSHOT">
    <consumers>
      <com.xxx.yyy.triggering.consumer.JobFinishedEventConsumer>
        <jobInstanceName>Trigger</jobInstanceName>
        <results>
            <com.xxx.yyy.datawrappers.ResultCode plugin="e...@21.0.8-SNAPSHOT">
              <name>SUCCESS</name>
            </com.xxx.yyy.datawrappers.ResultCode>
          </results>
      </com.xxx.yyy.triggering.consumer.JobFinishedEventConsumer>
    </consumers>
  </com.xxx.yyy.triggering.MessageTrigger>
</triggers>

When we do startup and act upon the onLoaded() we do something like this (simplified):

for (AbstractProject<?, ?> project : jenkins.getAllItems(AbstractProject.class)) {
  for (Trigger<?> t : project.getTriggers().values()) {
     .....
    JobFinishedEventConsumer consumer = ((MessageTrigger)t).getConsumers().get(0);
   
If we then try to retreive the consumer value for ResultCode we get a class cast exception if we have both plugins installed.

I have not been able to figure out how/where I should set the correct class loader for this. This is similar to the XmlFile problem we located but here we can not set the XStream class loader... or can we?




 
 

[Jesse Glick]

On Fri, Feb 20, 2015 at 10:09 AM, <ejo...@gmail.com> wrote:
> This is similar to the XmlFile problem we located but here we can not set the XStream class loader

Do not really follow your example but it sounds like the XStream form
of your plugin’s settings includes this third-party library which
another plugin also bundles. So, stop using those types in your
settings. Use only classes defined in your plugin.

[Kirill]

Jesse, thanks a million for your comments and clarifications!

As for your recommendations on settings, I have a question. So, there are 2 plugins installed on the same Jenkins instance - plugin A and plugin B, both of which use library C that is embedded in both of them.
Build step, introduced by plugin A, has private fields with the types from library C. So when the appropriate job config is saved, these types are referred in config.xml. That's probably what you suggested to avoid.
The question is: what class loader is used to load config.xml? Probably, uber class loader?

Regards,
Kirill.

пятница, 20 февраля 2015 г., 20:50:48 UTC пользователь Jesse Glick написал:

[Jesse Glick]

On Thu, Feb 26, 2015 at 4:25 AM, Kirill <yam...@gmail.com> wrote:
> Build step, introduced by plugin A, has private fields with the types from
> library C. So when the appropriate job config is saved, these types are
> referred in config.xml. That's probably what you suggested to avoid.

Yes, exactly.

> The question is: what class loader is used to load config.xml? Probably,
> uber class loader?

I believe so.

FWIW, when working on the NetBeans module system I eventually just
made the equivalent of UberClassLoader throw a (descriptive)
ClassNotFoundException if it were ever asked to load a class defined
in two different modules (~ plugins). Thus, rather than having class
loading succeed but *sometimes* result in a ClassCastException
(depending on which plugin happened to be asked to load the class
first), you could *consistently* get this error whenever both plugins
were installed at once. Thus you would be alerted earlier to the need
to factor out the library into its own “wrapper” plugin that both
feature plugins depend on, or pass a specific (plugin) ClassLoader to
any deserialization-type calls, etc.

[ejo...@gmail.com]

We just realized that we can overcome the problem bu using the XSTRAEM2 similar as when we replace a class for backwards compatibility. We did see problems with the classloader both in job configuration (config.xml) and during job builds. Just by using the addCompatibilityAlias to the original class it seems that we get the correct classloader.

 Items.XSTREAM2.addCompatibilityAlias("com.xxx.yyy.ResultCode", ResultCode.class);     
 Run.XSTREAM2.addCompatibilityAlias("com.xxx.yyy.EventId", EventId.class);

[Jesse Glick]

On Fri, Feb 27, 2015 at 2:56 AM, <ejo...@gmail.com> wrote:
> Items.XSTREAM2.addCompatibilityAlias("com.xxx.yyy.ResultCode", ResultCode.class);

This is not very safe since if the *other* plugin also tries to use
XStream, it will fail.

[Сергей Шадрин]

Hi,

did you solve your problem with plugin isolation? 

Regards,

Sergey.

пятница, 6 февраля 2015 г., 14:42:05 UTC+3 пользователь Kirill написал: