Is it possible to build and test 50 similarly named Github repos with a singe job?

43 views
Skip to first unread message

Ted Cowan

unread,
Jul 19, 2018, 11:26:19 PM7/19/18
to Jenkins Users
I am a college instructor that is automating the submission of programming assignments using GitHub Classroom.  Typically we have 50 students, each submitting 9 assignments per semester.  All of the repos are part of one organization, and I wish to trigger a build and test when each student pushes to their private repo.

Is this possible in Jenkins, and if so, how?  There are myriad examples of how to trigger a build on a single repo, but I need to trigger the same build when any of 50 repos are changed.  I hope this doesn't mean I have to add the names of all 50 repos to each job, but that might be doable if I edit the config.xml outside of Jenkins.

...ted

Mark Waite

unread,
Jul 19, 2018, 11:43:48 PM7/19/18
to jenkins...@googlegroups.com
I think you want a "GitHub Organization" project.  It will scan an entire organization then create and destroy jobs based on the presence of a Jenkinsfile in the root directory of one or more branches in the repositories that it finds.

Mark Waite.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/56757380-01d9-45ef-9123-b0b732ab819b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jan Monterrubio

unread,
Jul 20, 2018, 8:28:01 AM7/20/18
to jenkins...@googlegroups.com
This definitely sounds like what Mark suggested. If you set up the GitHub service and webhook, Jenkins can build on every push to any branch in a repository. 

You’ll need to make sure the repositories have a jenkinsfile: 

You’ll probably need some plugins like the GitHub branch source and maybe GitHub organization. 

Ted Cowan

unread,
Jul 20, 2018, 4:19:32 PM7/20/18
to jenkins...@googlegroups.com
Thank you for your suggestions.

I am new to Jenkins, but I did look at the Pipeline approach with a Jenkinsfile.  It occurred to me that an "innocent" student might choose to change the content of the file.  That might not be a good thing.  I am also concerned that the Github Organization plugin appears to be at end-of-life. 

I have just discovered the Generic Webhook Trigger, and have successfully implemented it on my Jenkins instance and with a webhook at the organization level in Github.  Seems to work.  I have successfully passed four bits of information from the JSON data in the webhook payload to my build script, and now I just need to put in some front-end code to identify which assignment is being submitted.  That information can come from the prefix of the repo name.  

I will continue to look at using Pipelines (mainly because that is all the Jenkins docs want to talk about) but i will go with my Generic Trigger solution until I my requirements force me in a different direction.

...ted

On Fri, Jul 20, 2018 at 6:25 AM, Jan Monterrubio <janmont...@gmail.com> wrote:
This definitely sounds like what Mark suggested. If you set up the GitHub service and webhook, Jenkins can build on every push to any branch in a repository. 

You’ll need to make sure the repositories have a jenkinsfile: 

You’ll probably need some plugins like the GitHub branch source and maybe GitHub organization. 
On Thu, Jul 19, 2018 at 22:43 Mark Waite <mark.ea...@gmail.com> wrote:
I think you want a "GitHub Organization" project.  It will scan an entire organization then create and destroy jobs based on the presence of a Jenkinsfile in the root directory of one or more branches in the repositories that it finds.

Mark Waite.

On Thu, Jul 19, 2018 at 9:26 PM Ted Cowan <tco...@my321.net> wrote:
I am a college instructor that is automating the submission of programming assignments using GitHub Classroom.  Typically we have 50 students, each submitting 9 assignments per semester.  All of the repos are part of one organization, and I wish to trigger a build and test when each student pushes to their private repo.

Is this possible in Jenkins, and if so, how?  There are myriad examples of how to trigger a build on a single repo, but I need to trigger the same build when any of 50 repos are changed.  I hope this doesn't mean I have to add the names of all 50 repos to each job, but that might be doable if I edit the config.xml outside of Jenkins.

...ted

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/4xUNCOLdpfI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CADgiF9Kd0cvtmMQyUXs4a9FOtetevTBSU50vt%3DiNDaJLmsVCbQ%40mail.gmail.com.

Mark

unread,
Jul 20, 2018, 4:24:10 PM7/20/18
to jenkins...@googlegroups.com
Multipipeline branch options the way to go here. But Buck the bugs answer a few down might help you create multiple stages from a flat file of your students names to help you create the branches. 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CADgiF9Kd0cvtmMQyUXs4a9FOtetevTBSU50vt%3DiNDaJLmsVCbQ%40mail.gmail.com.

Mark Waite

unread,
Jul 20, 2018, 4:54:22 PM7/20/18
to jenkins...@googlegroups.com
I think you're making your life more difficult than it needs to be.

The "GitHub Organization" plugin is no longer being developed because the "GitHub Organization" project functionality has been included in other plugins.  The "GitHub Organization" project type is alive and working very well for many of us.  I suspect it is even the project type that is used on ci.jenkins.io to automatically create a new folder of jobs each time a new Jenkins plugin is added and each time a Jenkins plugin adds a Jenkinsfile.

Yes, you can probably find a way to use webhooks and triggers to approximate some of the things that the GitHub Organization project does.  I don't think those webhooks and triggers will match what your students will find as they become working programmers.  They will tend to see GitHub Organization projects when they work at a company that uses GitHub.  They will tend to see Bitbucket branch source projects when they work at a company that uses Bitbucket.

You expressed concern that a student might change the contents of the Jenkinsfile in their repository.  I think that is a good thing.  If they decide that they want more tests or different tests or different types of tests, they can add them.  If they decide that they want static analysis with findbugs or spotbugs, they can extend the Jenkinsfile.  Since each student has their own repository, I think you want the student to have the autonomy to choose how their repository is built.

Mark Waite

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CA%2BGaY5iZJgKbbqzb_rTzM1B5CnM_xVtTW_-WQ5D-QJsXzbXTLQ%40mail.gmail.com.

anillingutla

unread,
Jul 21, 2018, 9:40:30 AM7/21/18
to jenkins...@googlegroups.com
I agree with Mark. Webhooks and triggers is sort of old terminology. Adapting pipeline as a code is a good way. Use Jenkins for each repo. Keeping in simple adding the code to add more tests in jenkins file. I even went further and kept more simple using scripted pipeline instead of Declarative pipelines. You might find . good examples 


Declarative pipelines - 

Scripted pipelines - 


Configure each repo as a Job on Jenkins and you should be good..

best
Anil Lingutla

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtFf1xMSfz8EdKbSoOynfxgd5xKhJFv7tRER4Hz5HM1b4Q%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages