What is the InjectedTest actually doing?

[Christoph Kutzinski]

Hi,

so I've developed Jenkins plugins for all these years and still I've got
no clue what the magical InjectTest is actually doing - excepting
stealing some of my time:

Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.365
sec - in InjectedTest

And actually I cannot remember that it ever caught any error in my plugin.
So can someone explain, what it is good for?

cheers
Kutzi

[Jesse Glick]

On Thu, Oct 30, 2014 at 5:14 PM, Christoph Kutzinski <ku...@gmx.de> wrote:
> So can someone explain, what it is good for?

You can browse its source code in maven-hpi-plugin. Checks things like
well-formedness of Jelly views and so on. It is possible to disable it
but we do not recommend that.

[Christoph Kutzinski]

Thanks Jesse,

I think some transparency what these tests are doing would be appreciated.
E.g. naming them in a way that describes what they are testing and not
how they are being run ('injected').
Maybe also a description on the Wiki pages for plugin development.

Also, I must be too stupid because I cannot find the test class in
https://github.com/jenkinsci/maven-hpi-plugin

[Jesse Glick]

On Thu, Oct 30, 2014 at 5:27 PM, Christoph Kutzinski <ku...@gmx.de> wrote:
> I think some transparency what these tests are doing would be appreciated.

Feel free to improve.

> Also, I must be too stupid because I cannot find the test class in
> https://github.com/jenkinsci/maven-hpi-plugin

Sorry, here:

https://github.com/jenkinsci/jenkins/blob/master/test/src/main/java/org/jvnet/hudson/test/PluginAutomaticTestBuilder.java

[Ulli Hafner]

If you want to skip these test you can add the following to your pom:

<plugin>
<groupId>org.jenkins-ci.tools</groupId>
<artifactId>maven-hpi-plugin</artifactId>
<configuration>
<disabledTestInjection>true</disabledTestInjection>
</configuration>
</plugin>

This is default in all my plug-ins since I also did not see a benefit in these integration tests yet… Maybe we should move such integration tests to the acceptance test harness project.

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Jesse Glick]

On Thu, Oct 30, 2014 at 6:47 PM, Ulli Hafner <ullrich...@gmail.com> wrote:
> Maybe we should move such integration tests to the acceptance test harness project.

These are not integration tests and do not belong in acceptance-test-harness.

I do notice that JellyTestSuiteBuilder runs with a live Jenkins
instance (inside a request handling thread). This is probably
unnecessary; I suspect it could be modified to run as a unit test,
essentially instantaneous.

CliSanityTest also runs with a live Jenkins instance but I wonder if
that could be optimized, for example by first checking if there are
any CLICommand’s or CLIMethod’s registered in the plugin, and quickly
returning if not.

Disabling JellyTestSuiteBuilder would be dangerous as one thing it
checks is that you are consistently using

<?jelly escape-by-default='true'?>

without which your plugin is highly vulnerable to XSS bugs. Although
perhaps at this point we can switch the default in Stapler Jelly to
assume escaping? (On the rare occasions where you do trust the source
and want to emit unescaped HTML, you should make this explicit with
<j:out>.)