Pull two repos into one build

[Pete Kane]

Title says it all really - I have an existing Pipeline SCM job setup which utilises a Jenkins file in my Github repo ( the Jenkinsfile is itself a pipeline ) All works perfectly.

I want to split my current repo ( seperation of concerns reasons ) into two repos but still use the Jenkinsfile to pull them both into one build ( does this make sense ? ) is this the correct way of doing things ?

TIA

[Mark Waite]

That's a very reasonable way of doing things.  Be sure that the checkout of the additional repository is performed in a different directory than the original checkout.  For example, it could be in a new subdirectory inside the original checkout directory or it could be in another workspace.

dir('my-subdirectory') {

    checkout .... needed options ...

}

There are various examples available.  I have a scripted pipeline example and a declarative pipeline example (though that one uses the outmoded `git` step instead of the modern `checkout` step) in case they help you.

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/354e3634-2e36-4de8-9664-af47450fe906%40googlegroups.com.

[Felix Dorner]

I used 

options { skipDefaultCheckout() } 

and then clone my projects into subdirectories with:

checkout([$class: 'GitSCM',
	branches: [[name: "*/${env.BRANCH_NAME}"]],
	doGenerateSubmoduleConfigurations: false,
	extensions: [[$class: 'RelativeTargetDirectory',
	relativeTargetDir: subdir1],
	]] ///...

 

However, I did change that because I dont know how to handle pull requests in a simple way with this approach. I now use git submodules, 

so one is a submodule of the other.

Felix

--

[Pete Kane]

Hi Mark, thanks for replying I should of mentioned I'm a newbie to Jenkins so your examples are a tad overwhelming :-) I actually got Jenkins to build successfuly but it aint pretty.

pipeline{
    agent any
    environment {
        APIproject = "./MyAPI//API.csproj"
        APIPublishPath = "//home//myname//Jenkins//Linux-ARM64//"
        Commonproject = "./Common.csproj"
        CommonPublishPath = "//home//myname//Jenkins//Linux-ARM64//Common//"
   }
       
  stages{
  stage('Checkout Common') {
        steps {
               git 'g...@github.com:myname/Common.git'
         }
    }
   
    stage('Build Common'){
     steps{
      sh "dotnet build ${env.Commonproject} -c Release -r linux-arm64 -o ${env.CommonPublishPath}"
     }
    }
    stage('Checkout API') {
        steps {
            dir('API')
            {
         git 'g...@github.com:myname/API.git'
            }
        }
    }
    stage('Build API'){
     steps{
      sh "dotnet build ${env.APIproject} -c Release -r linux-arm64 -o ${env.APIPublishPath}"
     }
    }
  }
}

What I had to do which I don't like doing is add a hard coded reference in my API.csproj to the Common.dll in my buils output folder - but it works and hopefully I'll get some tips on here which will help me achieve this in an elegant manner

 

On Tuesday, 19 May 2020 13:34:38 UTC+1, Mark Waite wrote:

That's a very reasonable way of doing things.  Be sure that the checkout of the additional repository is performed in a different directory than the original checkout.  For example, it could be in a new subdirectory inside the original checkout directory or it could be in another workspace.

dir('my-subdirectory') {

    checkout .... needed options ...

}

There are various examples available.  I have a scripted pipeline example and a declarative pipeline example (though that one uses the outmoded `git` step instead of the modern `checkout` step) in case they help you.

Mark Waite

On Tue, May 19, 2020 at 6:13 AM Pete Kane <pe...@pjksolutions.com> wrote:

Title says it all really - I have an existing Pipeline SCM job setup which utilises a Jenkins file in my Github repo ( the Jenkinsfile is itself a pipeline ) All works perfectly.

I want to split my current repo ( seperation of concerns reasons ) into two repos but still use the Jenkinsfile to pull them both into one build ( does this make sense ? ) is this the correct way of doing things ?

TIA

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.

[Pete Kane]

Hi Felix, thanks for replying I think I have a steep learning curve ahead learning this scripting syntax - I've been a paid programmer / analyst / developer / software engineer for 35 + years but I've never had to use Jenkins or its ilk - I have to say it's a cool piece of kit

On Tuesday, 19 May 2020 15:11:13 UTC+1, Felix Dorner wrote:

I used 

options { skipDefaultCheckout() } 

and then clone my projects into subdirectories with:

checkout([$class: 'GitSCM',
	branches: [[name: "*/${env.BRANCH_NAME}"]],
	doGenerateSubmoduleConfigurations: false,
	extensions: [[$class: 'RelativeTargetDirectory',
	relativeTargetDir: subdir1],
	]] ///...

 

However, I did change that because I dont know how to handle pull requests in a simple way with this approach. I now use git submodules, 

so one is a submodule of the other.

Felix

On Tue, May 19, 2020, 14:13 Pete Kane <pe...@pjksolutions.com> wrote:

Title says it all really - I have an existing Pipeline SCM job setup which utilises a Jenkins file in my Github repo ( the Jenkinsfile is itself a pipeline ) All works perfectly.

I want to split my current repo ( seperation of concerns reasons ) into two repos but still use the Jenkinsfile to pull them both into one build ( does this make sense ? ) is this the correct way of doing things ?

TIA

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.

[Mark Waite]

Glad you're using Jenkins and declarative pipeline.  You may want to spend an hour watching a Declarative Pipeline video that Kohsuke Kawaguchi hosted with Tyler Johnson and me presenting some ideas that make it easier to succeed with declarative pipeline.

Specific comments are placed inline.

On Tue, May 19, 2020 at 8:31 AM Pete Kane <pe...@pjksolutions.com> wrote:

Hi Mark, thanks for replying I should of mentioned I'm a newbie to Jenkins so your examples are a tad overwhelming :-) I actually got Jenkins to build successfuly but it aint pretty.

pipeline{
    agent any
    environment {
        APIproject = "./MyAPI//API.csproj"
        APIPublishPath = "//home//myname//Jenkins//Linux-ARM64//"
        Commonproject = "./Common.csproj"
        CommonPublishPath = "//home//myname//Jenkins//Linux-ARM64//Common//"
   }
       
  stages{
  stage('Checkout Common') {
        steps {
               git 'g...@github.com:myname/Common.git'

Since you're using declarative pipeline, there is an implied checkout that happens already.  You don't need this 'git' step because declarative pipeline already assumed you would want a checkout and performed the initial checkout into the workspace.

Calling the git step at this location did not harm you, it just performed another 'fetch' (or two) that had been done only seconds before.

 

         }
    }
   
    stage('Build Common'){
     steps{
      sh "dotnet build ${env.Commonproject} -c Release -r linux-arm64 -o ${env.CommonPublishPath}"
     }
    }
    stage('Checkout API') {
        steps {
            dir('API')
            {
         git 'g...@github.com:myname/API.git'

The 'git' step has been superseded by the more powerful and more capable 'checkout' step.  The Jenkins pipeline syntax generator ( https://your-jenkins.example.com/pipeline-syntax  ) will allow you to define the configuration you want and then generate the syntax you should use.  I believe that video clip includes a section on the pipeline syntax generator.

Since this is using 'ssh' protocol to access the git repository (g...@github.com:xx/yy.git), you should define a Jenkins credential which includes the private key used to access that repository.  Then reference that credential in this context.  If you don't do that, then all the agents will need to be configured to silently allow ssh access to the GitHub repository.  That works for small installations but does not work well for medium size or large installations.

Thanks,

Mark Waite

[Jérôme Godbout]

Simply checkout into Agent workspace subfolder:

 

dir(“source1”){

   checkout(…);

}

 

dir(“source2”){

  checkout(…);  

}

 

I do 3-4 checkout for my builds (my common jenkins tools libraries repos, my provisioning repos, my source code and his submodules and sometimes some unit tests repos). My jenkinsfile is also into the CI repos so I can update my C and apply it on any version of the source code.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALG%2B_hAUn49Kp38C%3DD-T4803fe4Mz2OuTrJ%3D43yZSpBgcb%3DedA%40mail.gmail.com.

[Pete Kane]

Hi Mark, I should have mentioned this is a new pipeline not my original SCM one ( which as you say has an implicit checkout )

[Pete Kane]

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/354e3634-2e36-4de8-9664-af47450fe906%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.