different github checkout revisions

[unli...@gmail.com]

There is such a jenkins-code:

checkout ([ ...
extensions: [[$class: 'CheckoutOption', timeout: 180],[$class: 'CloneOption', depth: 1, noTags: true, reference: '', shallow: false, timeout: 60]],
... )]

Why do I get different revisions when I restart the same pipeline, without any changes in SCM (github)?
The latest versions of git & jenkins are installed. Thank you.
It's a bug in Jenkins? I ask for help and any ideas, because this is very urgent and important!

[Mark Waite]

If it is very urgent and important, then it should be important enough to provide a detailed description of the precise steps you're taking, a complete list of the checkout step (without eliding important values like the branch name), the things you've explored, what you saw in those explorations, and more.

If there are no new changes in SCM, then I would expect the same commit to be used each time you run that job on that branch.  Wiithout new commits on the branch, running the job should use the most recent commit on the branch that is being used for the checkout.  However, you didn't tell us if this is a Pipeline job with the script inside the job, a Pipeline job that is getting the script from SCM, or a Pipeline job that is being defined as part of a multibranch pipeline.

When you say "restart the same pipeline", do you mean that you press the "Replay" button for the pipeline?  Do you mean that you press the "Build now" link?  Do you mean that you're starting the job from a command line interface call?  Are you starting the job from another job?

Is there something special about the local file system that causes you to set the checkout timeout (a local file system operation) to allow more time that the clone timeout (which is a network operation)?

 

Mark Waite

[unli...@gmail.com]

Thanks for your reply.
Temporarily, I have to roll back to the old, but working successfully solution with git:

git(
        url: 'https://github.com/enterpr/myrepo.git',
        credentialsId: 'github-PAT',
        branch: "${params.branch_name}"
    )

But I would like to solve my problem with checkout and incorrect revisions on each run.
Here is the full code with checkout, which randomly gives different revisions:

checkout ([
       $class: 'GitSCM',
       branches: [[name: '"*/"${params.branch_name}']],
       doGenerateSubmoduleConfigurations: 'false',
       extensions: [[$class: 'CheckoutOption', timeout: 180],[$class: 'CloneOption', depth: 1, noTags: true, reference: '', shallow: true, timeout: 60]],
       submoduleCfg: [],
       userRemoteConfigs: [[url: 'https://github.com/enterpr/myrepo.git', credentialsId: 'github-PAT']]])

{Params. branch_name} always has the same value - branch 'dev'

>However, you didn't tell us if this is a Pipeline job with the script inside the job, a Pipeline job that is getting the script from SCM...
My pipeline is loaded from SCM (github) as a script code (Pipeline) (enabling or disabling Lightweight checkout in the pipeline settings does not affect issue with different revisions).

>  When you say "restart the same pipeline", do you mean that you press the "Replay" button for the pipeline?

Any launch "Build with parameters" or "Restart from stage", because in each stage I have code with "checkout" call.

>Is there something special about the local file system that causes you to set the checkout timeout (a local file system operation) to allow more time that the clone timeout (which is a network operation)?

If You mean the set values of [timeout: 180 & timeout: 60] in my code, then these parameters were added intentionally, because earlier when using git, where this default value is [timeout: 10], there were regular problems when loading my large repository.

I am sure that this is an error in checkout, because without any changes in the SCM or code, restarting the same pipeline can take different revisions.
How do I always ensure the latest revision?

-- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtEy-jdP2VXGC%3Dum%2B9e7%2BBbEPsrrum6UGLvkYk1sofCN0w%40mail.gmail.com.

[Mark Waite]

On Wed, Dec 2, 2020 at 12:56 AM <unli...@gmail.com> wrote:

Thanks for your reply.
Temporarily, I have to roll back to the old, but working successfully solution with git:

git(
        url: 'https://github.com/enterpr/myrepo.git',
        credentialsId: 'github-PAT',
        branch: "${params.branch_name}"
    )

But I would like to solve my problem with checkout and incorrect revisions on each run.
Here is the full code with checkout, which randomly gives different revisions:

checkout ([
       $class: 'GitSCM',
       branches: [[name: '"*/"${params.branch_name}']],
       doGenerateSubmoduleConfigurations: 'false',
       extensions: [[$class: 'CheckoutOption', timeout: 180],[$class: 'CloneOption', depth: 1, noTags: true, reference: '', shallow: true, timeout: 60]],
       submoduleCfg: [],
       userRemoteConfigs: [[url: 'https://github.com/enterpr/myrepo.git', credentialsId: 'github-PAT']]])

{Params. branch_name} always has the same value - branch 'dev'

Try this instead:

  checkout ([
       $class: 'GitSCM',

       branches: [[name: params.branch_name]],

       extensions: [[$class: 'CheckoutOption', timeout: 180],[$class: 'CloneOption', depth: 1, noTags: true, reference: '', shallow: true, timeout: 60]],

       userRemoteConfigs: [[url: 'https://github.com/enterpr/myrepo.git', credentialsId: 'github-PAT']]])

Since you're cloning from a single repository, you don't need the leading '*/' to precede the branch name.

Since you want to use the value of params.branch_name you don't need to embed that in single quotes.

Since you're using the default values for doGenerateSubmoduleConfigurations and  submoduleCfg , you don't need to specify them.

If your checkout only requires one branch, then you can reduce the work on the remote server and the disc space on the agent by using the "honor refspec on initial clone" along with a specifically defined refspec that uses only the specific branch.  Examples of intentionally narrowed refspecs are available in several places, including this repository. 

 

>However, you didn't tell us if this is a Pipeline job with the script inside the job, a Pipeline job that is getting the script from SCM...
My pipeline is loaded from SCM (github) as a script code (Pipeline) (enabling or disabling Lightweight checkout in the pipeline settings does not affect issue with different revisions).

>  When you say "restart the same pipeline", do you mean that you press the "Replay" button for the pipeline?
Any launch "Build with parameters" or "Restart from stage", because in each stage I have code with "checkout" call.

>Is there something special about the local file system that causes you to set the checkout timeout (a local file system operation) to allow more time that the clone timeout (which is a network operation)?
If You mean the set values of [timeout: 180 & timeout: 60] in my code, then these parameters were added intentionally, because earlier when using git, where this default value is [timeout: 10], there were regular problems when loading my large repository.

Yes, those are exactly what I mean.  If you're using a git repository that is large enough to exceed the default 10 minute timeout when cloning the repository, you probably should review the "Git in the Large" video segment that will point you to the things you can do on Jenkins agents to significantly reduce the time and the disc space cost of large repositories.  The key items (in priority order) are:

1. Reference repository on the agent in a known location (almost always effective at reducing data transfer and data storage on agents)
2. Narrow refspec for the job and honor refspec on initial clone (effective for repositories with many branches that have largely independent branches, less effective for large repositories with interdependent branches0
3. Shallow clone (not nearly as effective as I'd hoped)

Your current settings allow up to 60 minutes to pull the repository data from the remote git server, then allow up to 180 minutes to perform the local checkout on the agent file system.  That is a surprising combination of values.  Access to the agent file system is usually significantly faster than access to the remote git repository.  If that is not the case in your installation, then you may need to investigate why file system access to the agent file system is slower than network access to a git repository.

 

I am sure that this is an error in checkout, because without any changes in the SCM or code, restarting the same pipeline can take different revisions.
How do I always ensure the latest revision?

On Tue, Dec 1, 2020 at 5:32 AM <unli...@gmail.com> wrote: There is such a jenkins-code: checkout ([ ... extensions: [[$class: 'CheckoutOption', timeout: 180],[$class: 'CloneOption', depth: 1, noTags: true, reference: '', shallow: false, timeout: 60]], ... )] Why do I get different revisions when I restart the same pipeline, without any changes in SCM (github)? The latest versions of git & jenkins are installed. Thank you. It's a bug in Jenkins? I ask for help and any ideas, because this is very urgent and important! If it is very urgent and important, then it should be important enough to provide a detailed description of the precise steps you're taking, a complete list of the checkout step (without eliding important values like the branch name), the things you've explored, what you saw in those explorations, and more. If there are no new changes in SCM, then I would expect the same commit to be used each time you run that job on that branch.  Wiithout new commits on the branch, running the job should use the most recent commit on the branch that is being used for the checkout.  However, you didn't tell us if this is a Pipeline job with the script inside the job, a Pipeline job that is getting the script from SCM, or a Pipeline job that is being defined as part of a multibranch pipeline. When you say "restart the same pipeline", do you mean that you press the "Replay" button for the pipeline?  Do you mean that you press the "Build now" link?  Do you mean that you're starting the job from a command line interface call?  Are you starting the job from another job? Is there something special about the local file system that causes you to set the checkout timeout (a local file system operation) to allow more time that the clone timeout (which is a network operation)?   Mark Waite -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtEy-jdP2VXGC%3Dum%2B9e7%2BBbEPsrrum6UGLvkYk1sofCN0w%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/435403385.20201202105609%40gmail.com.

[unli...@gmail.com]

Please help with the working instructions for installing SSL (https) on Jenkins.
Is it possible to install an SSL certificate on Jenkins without a reverse proxy (nginx)?

I tried these instructions below, but they didn't help (the certificate doesn't work) and I get in the browser:

Your connection is not private
NET::ERR_CERT_AUTHORITY_INVALID

p.s. My network with Jenkins does not have access to the Internet, so solutions in letsencrypt and similar will not work for me.

links:
https://wiki.jenkins.io/pages/viewpage.action?pageId=135468777
https://mohitgoyal.co/2017/02/08/securing-your-jenkins-environment-and-configure-for-auditing/
https://wiki.jenkins.io/display/JENKINS/Running+Jenkins+behind+Nginx
https://agirlhasnona.me/ops-tutorial-ssl-jenkins/
https://jansipke.nl/enable-https-jenkins/

Thanks!

[Gianluca]

Hi,

let’s separate two completely different tasks of what you are trying to achieve:

1) Configure Jenkins server to serve itself over SSL / HTTPS

2) Use a certificate that your browser recognise as valid

The documents you listed, tell you how to do point 1) and in general is left to you (or to other guides) how to do point 2)

Now, from the error you see on the browser, “CERT_AUTHORITY_INVALID”, I’m pretty sure that you achieved 1)

So, you actually did it !!

But maybe you don’t know exactly what the error means. So, I’ll try to details to best of my knowledge (I’m not an expert of certificate authorities):

- the browser tries to reach your Jenkins over port 443 (HTTPS)

- Jenkins reply correctly and the browser is happy and they start to establish an SSL connection between them (that means they successfully exchanged the SSL certificate and they can encrypt the connection)

- But the browser wants to be sure that the certificate has been made by some “known” trusted authority … let’s think of it like a known valid institution that produce valid certificates

- … but when it looks inside the certificates, it can’t find a certificate authority that knows … and then, for safety, it stops the communication and raise an error saying that it can’t “trust” the certificate because it’s not produced by an authority that it knows.

So, Jenkins is setup correctly and you did everything correctly on configure it.

The last bit that remains for you is to get a “real” valid certificate from a real valid certificate authority … if you can’t, then you can’t avoid the browser giving the CERT_AUTHORITY_INVALID error.

I hope that’s clear.

Cheers,

Gianluca.

P.S.: The fact Jenkins does not have access to the Internet, doesn’t prevent you to get a valid certificate from a valid authority, like letsencrypt or other certificate authorities like Comodo SSL, and so on.

We have many servers in our infrastructure without access to Internet but with valid certificate from Comodo SSL and LetsEncrypt

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/459881610.20201211234224%40gmail.com.

[unli...@gmail.com]

Thanks, Gianluca.
This group is very helpful in resolving issues with Jenkins.

The problem is that the SSL certificate installed and certified by the root CA certificate of my Corporation still does not work with the error specified earlier. Therefore, it is important for me to get a good proven and working instruction that will allow me to follow the steps to create, sign, and install the certificate and settings in Jenkins.
Please share this link to the description or video. Thanks.

Hi, let’s separate two completely different tasks of what you are trying to achieve: 1) Configure Jenkins server to serve itself over SSL / HTTPS 2) Use a certificate that your browser recognise as valid The documents you listed, tell you how to do point 1) and in general is left to you (or to other guides) how to do point 2) Now, from the error you see on the browser, “CERT_AUTHORITY_INVALID”, I’m pretty sure that you achieved 1) So, you actually did it !! But maybe you don’t know exactly what the error means. So, I’ll try to details to best of my knowledge (I’m not an expert of certificate authorities): - the browser tries to reach your Jenkins over port 443 (HTTPS) - Jenkins reply correctly and the browser is happy and they start to establish an SSL connection between them (that means they successfully exchanged the SSL certificate and they can encrypt the connection) - But the browser wants to be sure that the certificate has been made by some “known” trusted authority … let’s think of it like a known valid institution that produce valid certificates - … but when it looks inside the certificates, it can’t find a certificate authority that knows … and then, for safety, it stops the communication and raise an error saying that it can’t “trust” the certificate because it’s not produced by an authority that it knows. So, Jenkins is setup correctly and you did everything correctly on configure it. The last bit that remains for you is to get a “real” valid certificate from a real valid certificate authority … if you can’t, then you can’t avoid the browser giving the CERT_AUTHORITY_INVALID error. I hope that’s clear. Cheers, Gianluca. P.S.: The fact Jenkins does not have access to the Internet, doesn’t prevent you to get a valid certificate from a valid authority, like letsencrypt or other certificate authorities like Comodo SSL, and so on. We have many servers in our infrastructure without access to Internet but with valid certificate from Comodo SSL and LetsEncrypt

On 11 Dec 2020, at 20:42, unli...@gmail.com wrote:

SSL native (self) Jenkins

Please help with the working instructions for installing SSL (https) on Jenkins. Is it possible to install an SSL certificate on Jenkins without a reverse proxy (nginx)? I tried these instructions below, but they didn't help (the certificate doesn't work) and I get in the browser: Your connection is not private NET::ERR_CERT_AUTHORITY_INVALID p.s. My network with Jenkins does not have access to the Internet, so solutions in letsencrypt and similar will not work for me. links: https://wiki.jenkins.io/pages/viewpage.action?pageId=135468777 https://mohitgoyal.co/2017/02/08/securing-your-jenkins-environment-and-configure-for-auditing/ https://wiki.jenkins.io/display/JENKINS/Running+Jenkins+behind+Nginx https://agirlhasnona.me/ops-tutorial-ssl-jenkins/ https://jansipke.nl/enable-https-jenkins/ Thanks! -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/459881610.20201211234224%40gmail.com.

-- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6422CCE5-7F24-4671-A5D7-9DE43A47721E%40gmail.com.

[geoffroy...@gmail.com]

Hello

if the root CA of your company is not issued from a trusted provider, then you will need to deploy the root certificate to all the user browser.

We had such issue when a company was issuing its own certificates using a private PKI.

Not user-friendly solution, i would recommend to go for a "real" signing authority like Digicert or others.

Other possible solution, maybe you are not providing to Jenkins the full SSL certificate, and it is missing the intermediate company certificate... In that case you should concatenate final & intermediate (& even root) into a single cert file.

Reg. letsencrypt, my understanding was you need your server to be accessible from letsencrypt so that you can prove you own it. Alternative was using a dedicated entry from your company DNS. But maybe i've misunderstood the concept...

Regards

[unli...@gmail.com]

Hello, Jenkins community!

Pipelines started to crash when trying to clone from github due to networking issues. Restarting the Pipeline solves the problem. What are the ways to increase the number of attempts or time to clone from github (plugins, configs...) ?

Error code:
fatal: unable to access 'https://github.com/enterpr/mygit.git/': TCP connection reset by peer

The second problem that occurs sometimes:
"ERROR: Maximum checkout retry attempts reached, aborting"

I would be grateful for sharing experience in solving these problems with github. Thank you.

Happy New Year!

[Ivan Fernandez Calvo]

We overwrote the checkout step in our pipeline shared library to introduce retires and a delay between retries, this is the step

https://github.com/elastic/apm-pipeline-library/blob/master/vars/checkout.groovy