Testing Jenkins Plugins

66 views
Skip to first unread message

Valerie Fernandes

unread,
Oct 24, 2017, 9:00:52 AM10/24/17
to Jenkins Developers
Hi Guys,

I need your advise to understand what is the best means to test Jenkins plugins. 
We have some plugins that were developed in house. To test the functionality we have selenium tests per plugin.
Problem is that, these tests are time consuming, and often fail due to timeout issues or element not found and could succeed on a rerun.
Additionally, changes in UI(could be due to Jenkins upgrade 1.x --> 2.x) means we need to make changes in the tests as well. :( 

I see most Jenkins plugins have JUnit's. Can you suggest?

Mark Waite

unread,
Oct 24, 2017, 9:14:00 AM10/24/17
to jenkin...@googlegroups.com
There is a great thread earlier in the list which offers insights into the different facilities available to test a plugin, the concepts involved, and the related compromises.


Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Valerie Fernandes

unread,
Oct 25, 2017, 5:04:08 AM10/25/17
to Jenkins Developers
Thanks Mark,

So what I understand is,
  • Primarily use JUnits based on JTH in order to run most of the tests for functionality. 
  • Use ATH minimally for testing the integration of plugins with each other
Please correct me if I have understood incorrectly.

Mark Waite

unread,
Oct 25, 2017, 6:11:42 AM10/25/17
to jenkin...@googlegroups.com
You've described the strategy that I prefer.

Mark Waite

Jesse Glick

unread,
Oct 25, 2017, 12:31:47 PM10/25/17
to Jenkins Dev
On Wed, Oct 25, 2017 at 3:15 AM, Valerie Fernandes
<valer...@gmail.com> wrote:
> Use ATH minimally for testing the integration of plugins with each other

I actually prefer to do most integration testing using `JenkinsRule`
as well—you simply add a `test`-scoped dependency on another plugin
(if not already in your dependency tree anyway), and you are good to
go.

ATH is good for tests that specifically cover the UI, or that demand
an unusually high fidelity to production behaviors in certain areas
such as class loading.

Mark Waite

unread,
Oct 25, 2017, 12:37:32 PM10/25/17
to jenkin...@googlegroups.com
To further support Jesse's comment, I use JTH based tests with JenkinsRule for git plugin and git client plugin tests.  I don't use any ATH based tests for git plugin or git client plugin.  There are others who have written ATH tests of git plugin and git client plugin.

Mark Waite
 
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

Jesse Glick

unread,
Oct 25, 2017, 12:58:42 PM10/25/17
to Jenkins Dev
On Wed, Oct 25, 2017 at 12:31 PM, Jesse Glick <jgl...@cloudbees.com> wrote:
> I actually prefer to do most integration testing using `JenkinsRule`

I suppose I should clarify that this refers to testing a specific
feature or fix with awareness of its point of implementation. There is
also a use case for ATH in terms of smoke-testing an overall user
workflow from end to end looking for otherwise unforeseeable
regressions or poor interactions among numerous components, but that
is probably not what you were thinking of when you asked about testing
a particular plugin.

Hemant Sharma

unread,
Nov 28, 2017, 8:02:04 AM11/28/17
to Jenkins Developers
hi,

Is there a way to check or update the configuration of jenkins instance that is created for each test?
Reply all
Reply to author
Forward
0 new messages