Kubernetes Plugin: Option to run pod as different user (e.g. as root)
Torsten Gippert
Hello everybody,
this question is about implementing an option to run a build pod (using the kubernetes plugin from Carlos Sanchez) as a different user.
@Carlos: Great work, we love the ability to define our own pod templates!
Software versions:
-
Jenkins version: 2.89.2
-
Kubernetes Plugin version: 1.1.2
-
Durable Task Plugin Version: 1.17
First, let's assume the following scenario:
One JNLP container and 2 containers with different images in a podTemplate.
-
jnlp container - image jenkins/jnlp-slave:alpine - running as user 'jenkins' with uid 10000 (ten thousand)
-
node container - image node:9.3-alpine - running as root user with uid 0
-
gradle container - image gradle:4.4-alpine - running as user 'gradle' with uid 1000 (one thousand).
podTemplate from Jenkinsfile:
podTemplate(label: nodeLabel, containers: [
containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat')
])
Switching
into container 'gradle' is not possible due to permission denied
errors (seems to be caused by the durable task plugin, but I am not
sure about that).
Root cause seems to be that the jnlp container runs
with uid 10000 (yes, ten thousand - the 'old' image jenkins/jnlp-slave:2.62 ran with uid 1000) and therefore the working directory has file/directory
permissions that prevent other users except uid 10000 and root (uid
0) from accessing the working directory.
I also added a pod yaml file (see attachment pod-permission-denied.yaml) that shows the "problem":
1. You can run that pod (kubectl
apply -f pod-permission-denied.yaml) and
2. start a
shell in the different containers (kubectl
exec -ti -c [jnlp|node|gradle] permission-denied sh) to
3.
run some commands like ‘id’
or ‘whoami’ within a
container to show you the different user-ids the containers are
running with.
See
attachments
-
Jenkinsfile-permission-denied.groovy (contains Jenkinsfile to reproduce the “problem”)
-
Jenkinsfile-permission-denied_output.txt (contains the stdout of the Jenkins build job)
-
pod-permission-denied.yaml (simplified k8s pod declaration)
Suggested solution:
Add options to set 'runAsUser' and 'fsGroup' at podTemplate level to be able to get rid of permission problems.
'runAsUser' and 'fsGroup' stand for the user id (uid) and match the options from podSecurityContext at pod level in kubernetes pod specification (see https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
suggested
podTemplate from Jenkinsfile:
podTemplate(label: nodeLabel, runAsUser: 0, fsGroup: 0,
containers: [
containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat')
])
The pod ‘run-as-root’ (see attachment pod-run-as-root.yaml) is a minimal modified version of the pod ‘permission-denied’ (see above) with a different pod security context:
securityContext:
runAsUser: 0
fsGroup:
0
1.
You can run that pod (kubectl apply -f
pod-run-as-root.yaml)
and
2. start a shell in the different containers (kubectl
exec -ti -c [jnlp|node|gradle] run-as-root
sh) to
3.
run some commands like ‘id’
or ‘whoami’
within a container to show you that
each container is running as root now.
Therefore no
permission denied problems should
occur if this will be implemented in the kubernetes plugin.
See
attachments
-
pod-run-as-root.yaml (simplified k8s pod declaration with podSecurityContext set to root user)
What
do you think? Could this be a proper way to “override”
the the container specific default users?
Carlos Sanchez
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e06a9c7b-79bd-43cc-9428-498432f0fcfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Surjit Bains
def label = "jnlp-slave"
podTemplate(label: label, containers: [
containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat', workingDir: '/var/lib/jenkins')
])
{
node(label) {
stage('test') {
container('gradle') {
stage('test') {
try {
sh 'gradle clean test'
} catch (err) {
println err.message
} finally {
sleep 1
}
}
}
}
}
}
Agent specification [Kubernetes Pod Template] (jnlp-slave): * [gradle] gradle:4.4-alpine(resourceRequestCpu: , resourceRequestMemory: , resourceLimitCpu: , resourceLimitMemory: ) Running on jnlp-slave-hgbsp in /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ [Pipeline] { [Pipeline] stage [Pipeline] { (test) [Pipeline] container [Pipeline] { [Pipeline] stage [Pipeline] { (test) [Pipeline] sh [roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ] Running shell script sh: can't create /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied sh: can't create /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-result.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied touch: /home/jenkins/workspace/roductcatalogue_development-DN7AGMLQXLAWWWMLAFESYWBNB4L6VPMJ3GRJRRRYC5D5ODGXGLSQ@tmp/durable-5bff21c2/jenkins-log.txt: Permission denied [Pipeline] echo
script returned exit code -2
