Speeding up JenkinsRule tests?

[Andrew Bayer]

My tests in https://github.com/jenkinsci/pipeline-model-definition-plugin take...an ungodly amount of time. In part, that's 'cos I'm doing a *lot* of tests, and I'm using parameterized tests to make sure that failures are clear in some cases. But I gotta feel like things could be sped up. Does anyone know of any obvious tricks with JenkinsRule that can speed things up for a bunch of tests in a single class?

A.

[Andrew Bayer]

huh, initial experiments with turning the JenkinsRule into a @ClassRule rather than a @Rule have been...surprisingly positive. 4+x decrease in one of the parameterized tests (from 95s to 21s). I'll experiment more in that direction.

A.

[Andrew Bayer]

Yipes! Even better for the full test run - from 51m47s end-to-end on ci.jenkins.io to 6m8s with @ClassRule. Looks like that actually is the magic bullet here.

A.

[Stephen Connolly]

Yes that is the exact use case that drove me to write JenkinsRule in the first place... the ability to use @ClassRule for the tests that do not need the instance wiped clean in-between

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdOaWfXQ3vW1LQbSqFTj3NfL24MZScM6%2B%3DLwLPHuxy9A5_A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Jesse Glick]

On Mon, Sep 26, 2016 at 4:37 PM, Andrew Bayer <andrew...@gmail.com> wrote:
> turning the JenkinsRule into a @ClassRule rather than a @Rule

Be very, very careful. You can easily get yourself into trouble this way.

[Gavin Mogan]

Yea I just tried it, it made a bunch of my tests way faster but ran into a strange one on the pipeline tests

java.lang.IllegalStateException: Jenkins has not been started, or was already shut down

at com.saucelabs.jenkins.pipeline.SauceStepTest.sauceTest(SauceStepTest.java:53)

So not a drop in replacement at all :(

[Stephen Connolly]

You do need to know what you are doing to use @ClassRule

Typically I end up with *two* test classes, one has @ClassRule and has the more unity tests. The other has @Rule with the more integrationy tests.

If your tests don't clean up after themselves, @ClassRule is not for you. But very often I have a large chunk of tests that would not cross interact and @ClassRule is perfect for them

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/248236ce-0e2f-4bd1-8c74-a25a2b5da182%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Sent from my phone

[Jesse Glick]

On Thu, Sep 29, 2016 at 1:30 AM, Stephen Connolly
<stephen.al...@gmail.com> wrote:
> one has @ClassRule and has the

> more unity tests. […]

For example, they could be actual unit tests (no `JenkinsRule`) if it
were not for the fact that something in the call chain is loading
`@Extension`s.

> If your tests don't clean up after themselves, @ClassRule is not for you.

I would go so far as to say that if you are tests *have* anything to
clean up—created `Item`s, for example—using `@ClassRule JenkinsRule`
is too risky.

[Stephen Connolly]

As a principal of advice, yes. But for somebody that is looking to push the edge *and* likes driving their car at speed without a seatbelt and with a 12" metal spike protruding from the steering wheel, the option is there and it's not in and of itself bad...

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2A2FYdtx35p09VtCcXAH-CsDHj6iqk9m1noZigGjWKNQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.