[GSoC 2018] Introduction for Improvements to the Jenkins Acceptance Test Harness

106 views
Skip to first unread message

Sahil Khan

unread,
Mar 12, 2018, 4:25:14 PM3/12/18
to Jenkins Developers
Hello
I am Sahil Khan, I am pursuing my 3rd year engineering from Lakshmi Narain College of Technology and Science Bhopal.I am really passionate about Linux.I have been a keen evangelist for Dockers. I have been using and playing around with Dockers for 2 years and also taken various sessions in my Local City(https://www.null.co.in/profile/5878-sahil-khan).I have also worked on some Docker-based project one of them is Documented on my Blog which can be seen here.( http://sahilkhan.me/Dockers-Virtual-Hosting/ ).
I have also contributed and worked on some Open Source Projects on Github (https://github.com/docker-scripts/privoxy).
I am interested to work on the project "Improvements to the Jenkins Acceptance Test Harness".
The reason I am Interested in that project is that I contribute to Open Source projects and once I was contributing to OWASP ZAP Proxy where I wanted that the Jenkins plugin to be able to use the ZAP docker images.So this built up my interest in Jenkins and Docker Integrations.
Looking forward to your response.


martinda

unread,
Mar 12, 2018, 9:19:23 PM3/12/18
to Jenkins Developers
Hello Sahil,

Thanks for your interest in this Jenkins GSoC project.

The best way to get acquainted with the Jenkins ATH is to actually try it out by yourself. There are instructions to run it inside a docker container. The ATH is used mostly for integration tests of plugins. If you look under https://github.com/jenkinsci/acceptance-test-harness/blob/master/src/test/java/plugins/, you will see tests for many plugins. For example, the External Workspace Manager plugin tests are here. The tests are written as a jenkins pipeline script. This is really powerful because using this model, users can write tests for their pipeline build scripts and test them using the ATH. Pipeline library code can be tested the same way.

The ATH is one of the better ways to test user build script and user pipeline libraries, because it runs in a live in-memory Jenkins instance. Users can select the jenkins core version and the plugin versions. But it can be slow to run. Another drawback is that tests will download plugins from the update center, which can fail sometimes when the update center is busy or not available. The good thing is that it is actively maintained and has lot of documentation.

To get started, I suggest you try to run the tests of a plugin that you know. You could also use the ATH to test your own build script or your own pipeline library. Then you can start thinking on how the ATH performance or usability could be improved, and you can create a project proposal based your observations.

Best Regards,
Martin d'Anjou

martinda

unread,
Mar 12, 2018, 9:37:02 PM3/12/18
to Jenkins Developers
Sahil,

I forgot to say that we invite all students to participate in office hours where you can meet mentors and ask questions. The office hours schedule is on the main Jenkins GSoC 2018 page.
Unfortunately I cannot participate in the office hours this week, but I will be there next week. Steven Christou is the other mentor for this project.

Martin
Message has been deleted

Sahil Khan

unread,
Mar 13, 2018, 1:30:03 PM3/13/18
to Jenkins Developers
Sure..
I will test and make a report accordingly.I saw the GitHub repository so why don't we use Docker in swarm mode or kubernates.Docker swarm mode can really make it faster but I need to do some test in that.


Oleg Nenashev

unread,
Mar 14, 2018, 6:00:13 AM3/14/18
to Jenkins Developers
Yes, it is something doable.

Actually Jenkins Test Harness and Acceptance Test Harness already include some Docker integrations even now, and it would be great to expand them. We also had discussions about introducing testcontainers within Jenkins test frameworks, and it's also a good opportunity for increasing Docker adoption.

Swarm/K8s would be also fine. We have use-cases for such kinds of testing in plugins, and any proposals on this front will be appreciated.

Best regards,
Oleg

Sahil Khan

unread,
Mar 20, 2018, 2:21:59 PM3/20/18
to Jenkins Developers
Hello,
Apologies for the late response. I was refreshing my Jenkins knowledge and trying to test ATH as much as I can.
So I did some tests some screenshots of me using ATH inside Docker container are shown here.
I also watched the youtube video by Kohsuke also GitHub is pretty well Documented and that's really appreciable.
Reports:
  1. I ran the first test BROWSER=firefox JENKINS_VERSION=2.73 mvn test.So this test ran for more than 10 hours in my system(Was not even completed due to power loss).
  2. I ran the test for the Single plugin which used to run for an average time of about 20-30mins(May vary depending on Plugin).
  3. I also tried to write the test for own Pipeline Build thanks to Kohsuke Video still I'm not that much good with that.
Observations:
  1. I saw all the test and their details can be seen in logs file of jenkins/blueocean container.So I can understand how Docker sockets are doing their work.
  2. ATH test Suite working inside Docker is really Slow :(

I was confused with proposing of which container technology whether Docker swarm,Kubernates or Testcontainer would be best.It's really difficult for me to figure this out at this stage as I need a lot more time for testing all of them and implementing them with Test Suite(But GSoC Submission Deadline is Approaching).Also If this is successful I can also implement the same with Test-Harness also and I would like to work on it too.If I can figure this out then Impretty sure can easily complete the work within SoC Final weeks.

I am looking forward to your response


Oleg Nenashev

unread,
Mar 20, 2018, 7:04:09 PM3/20/18
to Jenkins Developers
Hi Sahil,

Yes, the ATH test suite is not designed to be executed on a single local machine.
We run it on CI instances. But yes, there is an interest to speedup and parallelize the tests.


I was confused with proposing of which container technology whether Docker swarm,Kubernates or Testcontainer would be best.

We have a community bonding period, and in your proposal you could schedule selecting the engine to this phase. But it would be still useful to understand what is your plan for this project. Many deliverables do not really depend on the underlying implementation, so IMHO it would be preferable to create a proposal draft and reflect the main objectives/deliverables, at least on the top level.

Best regards,
Oleg

Sahil Khan

unread,
Mar 21, 2018, 3:48:15 AM3/21/18
to Jenkins Developers
Sure Olev
Will revert you back when completed with proposal.
Thanks for your response.


Sahil Khan

unread,
Mar 23, 2018, 4:30:01 PM3/23/18
to Jenkins Developers
Hey,
I have submitted my draft proposal on GSoC Platform and looking forward to your feedback to improve my Proposal.
 

martinda

unread,
Mar 24, 2018, 8:33:34 AM3/24/18
to Jenkins Developers
Hello Sahil,

Thanks for submitting your proposal.

Sorry for not sharing this sooner, but I just remembered about it: there was a discussion on speeding up the ATH in this group. You can find it here:

Best Regards,
Martin

Oleg Nenashev

unread,
Mar 24, 2018, 9:36:48 AM3/24/18
to JenkinsCI Developers
Added my comments in the proposal.
IMHO the weakest part in it is the technical design. It is not clear to me how you are going to achieve the declared goals.


--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/M8ZjJVCz2rA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/030748ce-f494-4889-85eb-a81da5e8dc2b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Sahil Khan

unread,
Mar 24, 2018, 10:47:41 AM3/24/18
to Jenkins Developers
I saw the comments and I will work on this to improve as much as possible.Thanks for your feedback and time.Will revert you back soon when I have improved that.

Sahil Khan

unread,
Mar 24, 2018, 4:11:59 PM3/24/18
to jenkin...@googlegroups.com
I saw the comments and I will work on this to improve as much as possible.Thanks for your feedback and time.Will revert you back soon when I have improved that.

With Regards ,
Sahil Khan

martinda

unread,
Dec 22, 2018, 11:55:02 AM12/22/18
to Jenkins Developers
Hello Sahil,

This project is still available for GSoC 2019. There has been a recent discussion with an excellent summary made recently.

If you are still interested please join the discussion!

Martin d'Anjou
Jenkins GSoC 2019 Org Admin

Sahil Khan

unread,
Dec 25, 2018, 3:56:49 AM12/25/18
to jenkin...@googlegroups.com
Hello,
I am still interested in the project and can work on making a better proposal for next GSoC year and also ready to start with the work prior to GSoC.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/M8ZjJVCz2rA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/81aad53b-afdd-457d-9924-e6436106fe7d%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages