Project with several Git repositories

53 views
Skip to first unread message

Sébastien Hinderer

unread,
May 3, 2018, 1:16:21 AM5/3/18
to jenkins...@googlegroups.com
Dear all,

Say a project has two repositories: one for its source code and one for
its test suite.

How would you do CI for such a project?

I mean, I did try to define several repositories but it seems that
Jenkins only clones the most recently modified rather than cloning all
of them side-by-side. iS there another way to achieve this?

Many thanks in advance,

Sébastien.

Vivekanand S V

unread,
May 3, 2018, 1:53:57 AM5/3/18
to jenkins...@googlegroups.com
I do that using https://wiki.jenkins.io/display/JENKINS/Multiple+SCMs+Plugin

This plugin should help you.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20180502152621.GA7844%40prajna.paris.inria.fr.
For more options, visit https://groups.google.com/d/optout.

Vivekanand S V

unread,
May 3, 2018, 1:55:36 AM5/3/18
to jenkins...@googlegroups.com
Okay, I just saw its deprecated :( ... the page points to Pipeline Plugin though, do check it out. I don't have experience using that.

Sébastien Hinderer

unread,
May 3, 2018, 2:45:05 AM5/3/18
to jenkins...@googlegroups.com
OK many thanks for your response! Will definitely have a look to it!

Andrei Gheorghiu

unread,
May 3, 2018, 8:35:00 AM5/3/18
to Jenkins Users
You could use git hooks on any number of repositories to start the (same) Jenkins project.

Sébastien Hinderer

unread,
May 3, 2018, 9:38:22 AM5/3/18
to Jenkins Users
Dear Andrei,

Many thanks for your suggestion!

Andrei Gheorghiu (2018/05/03 05:15 -0700):
> You could use git hooks on any number of repositories to start the (same)
> Jenkins project.

Yeah, that's actually the solution I am playing with currently.

So just to make sure we are talking about the same thing, that means
checking the "None" radio button in the souce code managemnet area, then
check the "run builds remotely (from scripts)" checkbox and then,
finally, in the commands used to build, clone all the required
repositories.

Is that what you have in mind?

John Mellor

unread,
May 3, 2018, 10:18:23 AM5/3/18
to jenkins...@googlegroups.com
On Thu, 2018-05-03 at 15:38 +0200, Sébastien Hinderer wrote:
> . . .
>
> So just to make sure we are talking about the same thing, that means
> checking the "None" radio button in the souce code managemnet area,
> then
> check the "run builds remotely (from scripts)" checkbox and then,
> finally, in the commands used to build, clone all the required
> repositories.

I have a bit of a different situation, where I want to use a common
Jenkinsfile across a large number of git repos. Duplicating the
Jenkinsfile everywhere is a very poor solution, so I checkout the
project containing the Jenkinsfile, delete the .git directory in the
WORKSPACE, and then use the scm directives in the Jenkinsfile to
checkout the real project on top. Its far from ideal, but for some
reason it looks like this very common scenario is not supported.

If you have a better way to do this...

Sébastien Hinderer

unread,
May 3, 2018, 10:20:29 AM5/3/18
to jenkins...@googlegroups.com
Hi John,

John Mellor (2018/05/03 14:18 +0000):
No, no better alternative, sorry. I am not using pipelines yet, allas.

Sébastien.

Mark Waite

unread,
May 3, 2018, 12:26:28 PM5/3/18
to jenkins...@googlegroups.com
It seems like you're describing a very good use case for a Pipeline shared library.  Each repository and each branch then has a very simple Jenkinsfile which loads the shared library and calls it.  That's the technique used for Jenkins plugin development.

Mark Waite
 
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

John Mellor

unread,
May 3, 2018, 2:11:03 PM5/3/18
to jenkins...@googlegroups.com
On Thu, 2018-05-03 at 16:26 +0000, Mark Waite wrote:
. . .It seems like you're describing a very good use case for a Pipeline shared library.  Each repository and each branch then has a very simple Jenkinsfile which loads the shared library and calls it.  That's the technique used for Jenkins plugin development

Correct me if I am mistaken, but that would imply that I would still need to insert an identical Jenkinsfile into 155 projects in order to actually use the @library. That's not improving anything. I want to specify the Jenkinsfile from a different git repo and not alter all the other 155 repos, and there is no means of doing this as the pipeline plugins are currently written. That should be a base use-case, and it seems to have been missed. As a result, every project build that is changed over to pipelines needs to be modified for very questionable reasons.

Mark Waite

unread,
May 3, 2018, 2:18:57 PM5/3/18
to jenkins...@googlegroups.com
No, you're not mistaken.  You'd need to insert a Jenkinsfile into each repository to describe how that repository is built.

I don't think the use case you're describing should be a base use-case.

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

Sam Beckwith III

unread,
May 11, 2018, 1:42:28 PM5/11/18
to Jenkins Users
I second this request but not for the reasons given by Mr. John Mellor. Instead I request this because our team does not know or understand Jenkins nor the infrastructure available to it, and should not be allowed to make changes to the jenkins process without devops oversight. Putting the Jenkinsfile in a separate git repo allows devops to review that which concerns their ability to service the team (Jenkins), and keeping the Jenkinsfile out of the development team's repos allows the dev team to focus on what they know and need to progress on.

Currently we use Job DSL plugin for our Jenkins jobs however there is the desire to use pipelines, especially multi-branch pipelines, yet we cannot delegate the Jenkins build process to a Jenkinsfile in the microservices' repo. I see how that would be useful in many cases but I also see how it is not useful for teams like ourselves.

Would it be possible to setup a multi-branch pipeline where the Jenkinsfile is provided outside the target repo's source? I ask this after already looking several times for hours into this over the past year.

Victor Martinez

unread,
May 11, 2018, 4:34:55 PM5/11/18
to Jenkins Users
Maybe it sounds a bit crazy, but what about using git submodules. The jenkinsfile logic is outside the target repo's source as your initial requirement, although my experience with git submodules is not really something I'd say I enjoyed it... but it might be something to explore if it helps.

Besides of that, It looks like your requirement matches quite well with one of the enterprise plugins of CloudBees:

Cheers

Martin d'Anjou

unread,
May 12, 2018, 3:22:36 PM5/12/18
to Jenkins Users
You can write a Jenkinsfile that creates a clone of any git repository (calling out to Git with sh step for example), hence Jenkinsfiles can be central for however many repositories you have. No need for 100's of copies of the same Jenkinsfile in 100's of git repositories. You can create and manipulate Jenkins Jobs using the REST API from a loop on the command line. For Java projects, you can use this nice library called jenkins-rest to talk to the Jenkins REST endpoints. If your devops is python/curl based or other, see Remote access API.
Reply all
Reply to author
Forward
0 new messages