[GSoC2016] External Workspace Manager plugin development

[somai.a...@gmail.com]

Hello,

My name is Alexandru Șomai, I am a second year master's degree student at the Faculty of Mathematics and Computer Science at Babes-Bolyai University of Cluj-Napoca, Romania. I would like to begin my contribution to the open source community and I think that this could be a great opportunity for me to get started. 

I am interested in working on the "External Workspace Manager plugin development" project. By building a plugin that can share the same workspace for different jobs would come up with advantages such as: decreasing the time taken by a job to run, resources would be used efficiently. I have shared the draft proposal on GSoC web app. When you have the time, please review it.

I believe that I have all the required skills for taking this challenge:

• I have more than two years hands-on experience with Java. 
• While I was working at Betfair (September 2014 - November 2015) I gained experience with Jenkins as a user, part of my work was to configure and create Jenkins jobs for building a specific project or for running integration tests on targeted environments. 
• During my other job, I had to build an Update Manager application in three months using Groovy and Grails framework. 
• I don't have knowledge yet of how to build my own Jenkins plugin, but I am a fast learner and I am eager to learn. In the next days I am planning to make a Jenkins plugin to improve my knowledge in this area. 
  

I am looking forward to hear from you!

Kind Regards,

Alex Șomai

LinkedIn

GitHub

Twitter

[somai.a...@gmail.com]

The draft proposal is available here, so that everyone can see it.

[Oleg Nenashev]

Hi,

Thanks a lot for contacting us!
The proposal looks good to me in general, I've added several comments to the document.

Best regards,
Oleg Nenashev

пятница, 18 марта 2016 г., 12:17:50 UTC+1 пользователь somai.a...@gmail.com написал:

[somai.a...@gmail.com]

Hi Oleg,

Thank you very much for your feedback, it was very useful to me.

Kind regards,

Alex

[martinda]

Alex,

Thank you for your proposal. The proposal looks good to me as well. At this time I do not have comments. I have added a feature to the plugin description after some comments were received off-line. The feature is to optionally copy the workspace. As such, I have made a few changes to the README file. Also, at this time, I do not have a strong preference with regards to the global configuration being inside or external to Jenkins.

I have several unit testing examples to share when the time comes should you need them.

Martin

[somai.a...@gmail.com]

Hello Martin,

Thank you for your feedback.

I have allocated a couple of days in my proposal timeline for implementing and testing the 'copy workspace' functionality. 

Related to the unit testing examples, the sooner you share them the better. I can look over them early to get accustomed to.

Alex

[martinda]

Hello Alex,

On Sunday, March 20, 2016 at 6:09:38 PM UTC-4, somai.a...@gmail.com wrote:

Related to the unit testing examples, the sooner you share them the better. I can look over them early to get accustomed to.

Basic test example:
https://github.com/martinda/workflow-migration-demo/blob/master/src/test/java/org/jenkins/plugins/demo/Demo1Test.java

The most recent way of writing the pom.xml that I use is in this project (also incudes some basic test examples - roundtrip and global config access from plugin test):
https://github.com/martinda/jenkins-plugin-globalconfig-demo1/blob/master/src/test/java/org/jenkins/plugins/demo/Demo1Test.java

More elaborate test examples:
https://github.com/martinda/http-request-plugin/tree/feature/workflow-integration/src/test/java/jenkins/plugins/http_request

Martin

[somai.a...@gmail.com]

Oleg, Martin,

Can you please have one final look over my project proposal before I will submit it?

Thank you,

Alex

[martinda]

Alex,

Please submit your final proposal as per the Google process. Google has a firm deadline, and I was told that their site can be slow to respond as the deadline approaches.

Thanks,
Martin

[martinda]

Alex,

Forgot to CC you. Please submit your final proposal as per the Google process. Google has a firm deadline, and I was told that their site can be slow to respond as the deadline approaches.

Best Regards,
Martin d'Anjou

[somai.a...@gmail.com]

I have uploaded my final proposal version.

Thank you all for your support and good advices!

Kind Regards,

Alex

[somai.a...@gmail.com]

Hello,

I am glad that my project proposal has been accepted for the GSoC 2016. I would like to thank everyone for their support and good advices. I am very excited to be working on this project!

The next period (by May 22nd) is allocated for community bonding. During this period I am planning to get more familiar with Pipeline script jobs and afterwards I could try writing a proof of concept with the basic usage of the external workspace manager plugin.

So far I have found these resources related to the Pipeline Plugin: 

• https://github.com/jenkinsci/pipeline-plugin
  
• https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md

From my understanding, to get started, what I have to do is to create a custom step, that would be externalWorkspace. I have found some helping guides and examples about creating custom steps in the pipeline script:

• https://github.com/jenkinsci/pipeline-plugin/blob/master/DEVGUIDE.md
  
• https://github.com/jenkinsci/workflow-step-api-plugin
  
• https://github.com/jenkinsci/workflow-scm-step-plugin
  

I will work on a forked repository from the initial project proposal https://github.com/martinda/external-workspace-manager. I will come up with updates as soon I have something functional. On IRC you can find me as "alexsomai".

Please let me know if I am on good track and if there are any other resources or materials that could be useful to me.

Regards,

Alex

[martinda]

Hi Alex,

Welcome, and glad to have you on board. You can find me as mda14 on the IRC channel.

One of the challenges I had with writing plugins was to come up with a good pom.xml. It took me a while to get it right, so here is my latest one. The findbugs command was not obvious to me at first, so here it is: mvn org.codehaus.mojo:findbugs-maven-plugin:3.0.3:check. I am sure there is an easier way of running findbugs, but I have not found it.

Also, I suggest getting a skeleton plugin working at first. This will help you distinguish between the Jenkins side of the plugin (the boiler plate stuff), vs. the actual features of the plugin. Learn how to create an empty plugin with just enough code to be able to see it in the Pipeline Snippet Generator. Then add arguments to learn how to create jelly files, and corresponding help files.

I have also read these at one point or another:
https://github.com/fbelzunc/jenkinsci-helloworld-builder-globalconfiguration-complex
https://wiki.jenkins-ci.org/display/JENKINS/Plugin+tutorial

Martin

[somai.a...@gmail.com]

Hello everyone,

I do have a prototype version of the External Workspace Manager plugin. The repository is on GitHub and can be viewed here: https://github.com/alexsomai/external-workspace-manager/tree/basic-usage. Currently it's supported the basic usage, and is working having Jenkins set on a single node (master).

I did have a meeting today with my mentors Oleg and Martin where I have shown them a demo of the prototype version. After the discussions that we had, we agreed that the plan for the next week is to have the prototype work on multiple nodes, so this is what I'll do up next.

At the moment I am having weekly meetings with Oleg and Martin on every Monday at 12:00 UTC in which we'll be discussing the current progress and the future working plan. Everyone is invited to join our meetings, the next one will be on Monday May 9th at 12:00 UTC. If you are interested to join, please let us know, or you can join the Jenkins hangout channel: http://jenkins-ci.org/hangout.

Have a good week!

Alex

[somai.a...@gmail.com]

Hello everyone,

I would like to share with the community another proposal of the external workspace manager plugin. This one contains the design details along with some new features that the plugin shall have. The document may be accessed on the following link: https://docs.google.com/document/d/1yiisnsR7qg3XEEvch8vocWbitSUCZcoQ-pfzEVFg1eA/edit?usp=sharing.

I do appreciate if the people within the community takes a little bit of their time to review it and provide feedback.

I will be having the next meeting with my mentors Oleg and Martin tomorrow, May 16th at 12 PM UTC where we are going to discuss future implementation details about the plugin. If anyone from the community has any questions regarding the plugin, has any improvement ideas, or just wants to say hello, feel free to join us on Jenkins hangout: http://jenkins-ci.org/hangout.

Regards,

Alex

[martinda]

Hi Alex,

I am going to answer your question about the configuration button here. To have a button without a pull down, you can use a repeatable element: <f:repeatable field="fieldName"> </f:repeatable>

There is an example of that in this file. You will also see the repeatableDeleteButton in there. The table markup in that file should be optional, but in this case I think it helped with the layout.

Martin

[somai.a...@gmail.com]

Hello Martin,

Great, I think this should work.

Thank you,

Alex

[somai.a...@gmail.com]

Hello everyone,

I haven't posted in this mail thread for a while, therefore I though it would be time to give a status update about my GSoC 2016 work.

I have managed to implement a proof of concept version of the External Workspace Manager plugin. You may find the source code here: https://github.com/jenkinsci/external-workspace-manager-plugin. There is still a lot of working and polishing to be done until the release version, but at least it's something :)

Currently I'm working with my mentors, Oleg and Martin, to enhance the plugin help files. You can join the discussions on the shared Google Doc. Maybe in the next couple of days we'll be releasing an alpha version of the plugin.

And don't forget, if you are interested in this subject, we are having meetings twice a week regarding the plugin. The meetings are taking place on the Jenkins hangout, every Monday at 12 PM UTC and every Thursday at 5 PM UTC.

Best Regards,

Alex

[Alexandru Somai]

I'd like to announce that an alpha version of the External Workspace Manager plugin has been released last week! It's public available for testing.

You may find additional info on the blog post announcing the release: https://jenkins.io/blog/2016/06/30/ewm-alpha-version/

If you get the chance to try the plugin, any feedback is welcome!

Regards,

Alex

[Alexandru Somai]

Hello everyone,

I'd like to make a new announcement: a beta version of the External Workspace Manager Plugin has been released.

A summary of the features added to the previous alpha version:

• Support for the Run Selector Plugin, so you can provide different run selection strategies when allocating a disk from the upstream job. e.g. here.

• Automatic workspace cleanup by using the Workspace Cleanup Plugin. Pipeline example here.

• User can provide custom workspace path to be used on the disk. Example here.

• Disk Pool restriction policies by integrating the Job Restrictions Plugin. (I still need to write documentation for this feature)

Currently I'm working to provide multiple disk allocation strategies.

Afterwards the plugin will enter a final testing stage.

If everything goes well, soon I'll perform the stable version release.

Regards,

Alex

[Alexandru Somai]

Hello,

A few days ago I've released the External Workspace Manager Plugin version 1.0.0: https://wiki.jenkins-ci.org/display/JENKINS/External+Workspace+Manager+Plugin

Usage examples and explanations are on the plugin's documentation page.

You can find more details on the blog post that announces the release: https://jenkins.io/blog/2016/08/22/ewm-stable-release/

This was a Google Summer of Code 2016 project and more details about the work that I've done during this period can be seen at the submitted page: https://alexsomai.github.io/gsoc-2016/. Many thanks to my mentors, Oleg and Martin, who helped me a lot in achieving all this work.

There are still a lot of improvements and features that can be added to the plugin, and I'm planning to continue my contributions. I've grouped them in the phase 3 EPIC: JENKINS-37543.

I also invite you to watch the GSoC 2016 project presentations where I'll present my project, the External Workspace Manager Plugin.

Regards,

Alex Somai