[GSOC 2018] [Student Introduction & query] Simple Pull-Request Job Plugin

[Abhishek Gautam]

Hello Everyone,

I am a 3rd-year Computer Science student from India. I found GSOC project Simple Pull-Request Job Plugin very interesting and want to work on it.

But I have some concerns.

1. In project description, it's been stated as following

"Specifically, this plugin does not create jobs and does not detect branches automatically. The users are responsible for creating the jobs they need. This type of jobs have to be triggered via the existing methods (e.g. an http post to the Jenkins REST API, or via the UI)."

But then for every pull request administrator of the project needs to create a job and then trigger a build. I think this will be cumbersome.

Please let me know if I am wrong.

2. In project description, it's been stated that the pull-request job types have these input parameters:

• PR Number

• From Repository URL

• From Branch

• Target Repository URL

• Target Branch

But I think this should not be the case. I have done some research on payload GitHub sends through webhooks, and I found that pull request payloads have all of the above things, so we can extract them from the pull request payload itself.

I have gone through the code of below two plugins and I think they are pretty simple.

https://github.com/jenkinsci/bitbucket-plugin/tree/master/src/main/java/com/cloudbees/jenkins/plugins

and

https://github.com/jenkinsci/generic-webhook-trigger-plugin

We just need to get input from YAML file and do things according to that.

There is a java lib available to parse YAML file: https://github.com/FasterXML/jackson-dataformats-text/tree/master/yaml

Tutorial for above lib: https://dzone.com/articles/read-yaml-in-java-with-jackson

Please let me know what do you think.

[martinda]

Hello Abhishek,

Thanks for your interest in the Jenkins GSoC projects.

> But then for every pull request administrator of the project needs to create a job and then trigger a build. I think this will be cumbersome.

Yes that is a model that I have seen. But there there is another model which is to have a job that handles all the pull requests of a git repo to a given branch. For example say there is a git repository called jupiter/juno.git (jupiter is the name of the github org or of the bitbucket project). Then the pull requests destined to the master branch of the juno git repository could have a correspondig job called jupiter/juno/master. There could be other models too.

> pull request payloads have all of the above things, so we can extract them from the pull request payload itself.

I did not know this. How do the PR coordinates (source and destination branches and repositories) get into the variables of a build if they are not input parameters? Does this require the installation of plugins in other systems?

I would suggest that you consider preparing an application with your ideas and suggestions in a google doc document.

Best Regards,
Martin d'Anjou

[Oleg Nenashev]

Hi Abhishek,

+1 to what you say. Any comments about the project proposal will be appreciated. Actually we expect students to make their own proposals, so do not consider the project idea description as a specification.

We just need to get input from YAML file and do things according to that.

There is a java lib available to parse YAML file: https://github.com/FasterXML/jackson-dataformats-text/tree/master/yaml

According to my experience, I would rather recommend Snake YAML. Actually we use FasterXML as well in many plugins... Generally we tend to package such libs as API plugins, like this one: https://github.com/jenkinsci/jackson2-api-plugin/blob/master/pom.xml . Any library can be used.

Best regards,
Oleg

[James Nord]

On Friday, March 16, 2018 at 10:12:33 AM UTC, Oleg Nenashev wrote:

We just need to get input from YAML file and do things according to that.

There is a java lib available to parse YAML file: https://github.com/FasterXML/jackson-dataformats-text/tree/master/yaml

According to my experience, I would rather recommend Snake YAML. Actually we use FasterXML as well in many plugins... Generally we tend to package such libs as API plugins, like this one: https://github.com/jenkinsci/jackson2-api-plugin/blob/master/pom.xml . Any library can be used.

I have been using snakeyaml in a few plugins - and It would probably benefit from being in a library plugin rather than pulled in - (but at the same time plugin dependency hell and well....)

As for Jackson and YAML - I raised 2 JIRAs yesterday for that plugin JENKINS-50202 is the one for the YAML parser in dataformats and the related JENKINS-50201 for using jax-b annotations with jackson.

/James

[Jesse Glick]

On Thu, Mar 15, 2018 at 10:22 PM, martinda <martin....@gmail.com> wrote:
> there there is another model which is to have a job that handles all the pull requests of a git repo to a given branch. For example say there is a git repository called jupiter/juno.git (jupiter is the name of the github org or of the bitbucket project). Then the pull requests destined to the master branch of the juno git repository could have a correspondig job called jupiter/juno/master.

Which is already a standard part of Jenkins 2, including GitHub and
Bitbucket support. The only thing distinct from the proposal as it
stands is that using the default set of plugins, the jobs are
configured from a `Jenkinsfile` whereas you are proposing some YAML
format—easily handled by an existing pair of extension points in
`workflow-multibranch`. I have discussed this is another thread.

[Abhishek Gautam]

Hello Everyone,

Thank you for the clarifications.

Oleg Nenashev and James Nord thank you for the recommendation of "Snake YAML".

@martinda 

While my research I came across something called EnvironmentContributor in Jenkins. Do you think that we can use this to pass required variables to the build processes?

[martinda]

On Friday, March 16, 2018 at 5:39:09 PM UTC-4, Abhishek Gautam wrote:

@martinda 

While my research I came across something called EnvironmentContributor in Jenkins. Do you think that we can use this to pass required variables to the build processes?

Hi Abhishek,

I have never used the EnvironmentContributor, but it would be one of the possibilities. Injecting environment variables is convenient as all sub-commands would inherit them (no explicit argument passing required), it is an implementation detail which can be decided later.

I suggest that you try the existing functionality (see Jesse's reply), and this will help you to write your own project proposal. The existing plugins that I think Jesse is referring to are https://jenkins.io/doc/book/pipeline/multibranch/ and https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/index.html#bitbucket. The example projects that seem to go along with these plugins are https://github.com/cloudbeers/multibranch-demo and https://github.com/cloudbeers/multibranch-demo-lib

Like Oleg said, the project idea description is not a specification, it does not have to be followed to the letter.

BR,
Martin

[martinda]

On Friday, March 16, 2018 at 11:17:02 AM UTC-4, Jesse Glick wrote:

you are proposing some YAML
format—easily handled by an existing pair of extension points in
`workflow-multibranch`. I have discussed this is another thread.

Hi Jesse,

I could be wrong but I have the impression that you are saying that this GSoC project idea is a bit too shallow on the quantity of code for GSoC. If that is the case, do you have recommendations to make it more substantial? Maybe some enhancements to the existing PR plugins?

Note that in addition to building the PR, the proposed plugin would have some automatic detection of reports like JUnit XML reports, HTML reports, summary display reports, etc. I also though about adding a default merge before build, a default of having PR builds triggered by the user (rather than automatic), some default way to set the build description, etc.

Thanks,
Martin

[martinda]

Hello Abhishek,

I forgot to mention that we have office hours and all students are invited to participate and ask questions.
See https://jenkins.io/projects/gsoc/#office-hours for specific times.

Martin

[Abhishek Gautam]

Hello Jesse,

Thank you for your reply. Can you please provide the link to the thread where you have discussed the thing you mentioned.

[Abhishek Gautam]

Hello Everyone,

@martinda, I have prepared my proposal for the project Simple Pull-Request Job Plugin and shared it on the official GSOC website.

The link to the project proposal: https://docs.google.com/document/d/1cuC0AvQG3e4GCjIoCwK3J0tcJVAz1eNDKV8d_zXxlO8/edit?usp=sharing

Please let me know if there is something that needs more clarification or if I have missed something.

Thank you.

[martinda]

Hello Abhishek,

Thank you for sharing the proposal. I have added some comments for your consideration.

Best,
Martin

[Abhishek Gautam]

I have made changes according to your comments.

Thank you very much.

[Jesse Glick]

On Fri, Mar 16, 2018 at 9:11 PM, martinda <martin....@gmail.com> wrote:
> I have the impression that you are saying that this
> GSoC project idea is a bit too shallow on the quantity of code for GSoC.

Perhaps. Depends on what approach you want the feature to take—adding
a convenience veneer atop the existing Jenkins features, or writing
something with a very different design from scratch.

> do you have recommendations to make it more substantial?

Not offhand. There are hundreds of issues I follow in the general area
of multibranch, but I have no idea which you would consider important
in this context.

> some automatic detection of reports like JUnit XML reports, HTML reports,
> summary display reports, etc.

FYI the `pipeline-maven` plugin does some things like this—for Maven
builds only, of course.

> a default merge before build

Already a standard part of Jenkins branch sources.

> a default of having PR builds triggered by the user (rather
> than automatic)

There are some RFEs in this area.

[Oleg Nenashev]

All my comments have been addressed.

Of course the proposal will need adjustement during the community bonding if accepted.

So +1 from me.

Thanks for finalizing it!

On Tue, Mar 27, 2018 at 4:02 PM, Abhishek Gautam <gautam.a...@gmail.com> wrote:

Hello everyone

 

As you all know only 2 hours are remaining for submission of GSOC proposal, hence I request all to look and comment on my proposal for the last time.

Proposal link: https://docs.google.com/document/d/1cuC0AvQG3e4GCjIoCwK3J0tcJVAz1eNDKV8d_zXxlO8/edit

 

Thank you. 

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/gdRXZ2OuOc8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8d41048a-fb18-4e0e-ba23-6c1e2fd3b784%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Abhishek Gautam]

Sure thing,
Thanks you everyone for your valuable feedback, specially Oleg and Martin.

I hope that I get to work with you guys.

Abhishek Gautam