[JIRA] (JENKINS-41050) Do checkout scm always outside of container

Do checkout scm always outside of container

Issue Type:	Improvement
Assignee:	Andrew Bayer
Components:	pipeline-model-definition-plugin
Created:	2017/Jan/12 10:28 PM
Priority:	Major
Reporter:	Michael Neale

Few docker images have git installed, and encouraging off the shelf image usage is good.

This causes checkout scm to fail in many scenarios as there is no git available (for example).

If the checkout scm was done outside before running docker.inside, this would be less surprising.

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)

mneale@cloudbees.com (JIRA)

unread,

Jan 12, 2017, 5:29:01 PM1/12/17

to jenkinsc...@googlegroups.com

Michael Neale updated an issue

Jenkins /

Do checkout scm always outside of container

Change By:	Michael Neale
Labels:	mvp

andrew.bayer@gmail.com (JIRA)

unread,

Jan 12, 2017, 6:25:01 PM1/12/17

to jenkinsc...@googlegroups.com

Andrew Bayer commented on

So to do this, I have to move the checkout logic into the DeclarativeAgent scripts themselves...but that might not be a bad thing, since right now, label is the only one that would actually do the checkout (any is just syntactic sugar for label, and docker/dockerfile use label on the backend)...

andrew.bayer@gmail.com (JIRA)

unread,

Jan 12, 2017, 6:28:01 PM1/12/17

to jenkinsc...@googlegroups.com

Andrew Bayer commented on

I will need to figure out how to pass some additional information to the agent that it didn't have previously - right now, that's just whether skipDefaultCheckout is set and whether we're at the top-level rather than a stage agent, but it could be more in time.

andrew.bayer@gmail.com (JIRA)

unread,

Jan 12, 2017, 6:47:02 PM1/12/17

to jenkinsc...@googlegroups.com

Andrew Bayer started work on

Change By:	Andrew Bayer
Status:	Open In Progress

andrew.bayer@gmail.com (JIRA)

unread,

Jan 12, 2017, 6:47:02 PM1/12/17

to jenkinsc...@googlegroups.com

Andrew Bayer commented on

Very preliminary PR up at https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/94 - mainly to see how the tests go.

rsandell@cloudbees.com (JIRA)

unread,

Jan 18, 2017, 6:34:01 AM1/18/17

to jenkinsc...@googlegroups.com

rsandell commented on

At the same time we tend to encurage users to use docker images instead of tool installers to have a consistent and source controlled build environment. If we change it to this it would mean all build tools except the scm tool goes in docker, but the scm tool needs either a tool installer or otherwise be available on the agent.
Kind of inconsistent IMHO.

andrew.bayer@gmail.com (JIRA)

unread,

Jan 18, 2017, 11:38:02 AM1/18/17

to jenkinsc...@googlegroups.com

Andrew Bayer commented on

ping Michael Neale for his thoughts on rsandell's comment.

mneale@cloudbees.com (JIRA)

unread,

Jan 19, 2017, 7:27:02 PM1/19/17

to jenkinsc...@googlegroups.com

Michael Neale commented on

OK have thought about this and talked with Andrew.

I somewhat can appreciate philosphically what Bobby says, however I think this should proceed. The reason is that the pull of "use any docker container" is far stronger than the push back of "well you need your scm on your agent".
Some docker plugins have solved this by using a "side car" container with the tools, and maybe someday there will be support for those (as opposed to docker pipeline, which it uses now). So I think this should def. go ahead.

So - +10000 for me. The risk of not doing this is people will use bad, unmaintained images just to get the scm they need (which is what has happened in the past). They already have to have the jenkins agent running (and jvm).

mneale@cloudbees.com (JIRA)

unread,

Jan 19, 2017, 7:34:01 PM1/19/17

to jenkinsc...@googlegroups.com

Michael Neale edited a comment on

One suggestion by James was that this could be "smart" and attempt checkout scm in on agent host , and fail over to checkout scm inside a container if possible. Give the best chance to find the tool without the user telling.

This absolutely nuts idea may work, and could be a good enhancement. Or be terrible. Often the best ideas are both.

mneale@cloudbees.com (JIRA)

unread,

Jan 19, 2017, 7:34:01 PM1/19/17

to jenkinsc...@googlegroups.com

Michael Neale commented on

One suggestion by James was that this could be "smart" and attempt checkout scm in agent, and fail over to checkout scm inside a container if possible. Give the best chance to find the tool without the user telling.

This absolutely nuts idea may work, and could be a good enhancement. Or be terrible. Often the best ideas are both.

mneale@cloudbees.com (JIRA)

unread,

Jan 19, 2017, 7:35:04 PM1/19/17

to jenkinsc...@googlegroups.com

Michael Neale edited a comment on

OK have thought about this and talked with Andrew.

I somewhat can appreciate philosphically what Bobby says, however I think this should proceed. The reason is that the pull of "use any docker container" is far stronger than the push back of "well you need your scm on your agent".
Some docker plugins have solved this by using a "side car" container with the tools, and maybe someday there will be support for those (as opposed to docker pipeline, which it uses now). So I think this should def. go ahead.

So - +10000 for me. The risk of not doing this is people will use bad, unmaintained images just to get the scm they need (which is what has happened in the past). They already have to have the jenkins agent running (and jvm). Bind mounting is the price of entry with docker pipeline right now, so this is how it needs to work.

scm_issue_link@java.net (JIRA)

unread,

Jan 23, 2017, 9:28:02 AM1/23/17

to jenkinsc...@googlegroups.com

SCM/JIRA link daemon commented on

Code changed in jenkins
User: Andrew Bayer
Path:
pipeline-model-declarative-agent/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/DeclarativeAgent.java
pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Agent.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/AnyScript.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineFromDockerfileScript.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/LabelScript.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/2f7fb17a89016f8f8aa5829a8f3d69212077da01
Log:
JENKINS-41050 First work on checkout always being on node

i.e., not in the container. Need to see how this works in practice and
figure out how to test it.

scm_issue_link@java.net (JIRA)

unread,

Jan 23, 2017, 9:28:02 AM1/23/17

to jenkinsc...@googlegroups.com

SCM/JIRA link daemon commented on