No credentials/secrets pulled from master into PodTemplate/ContainerTemplate by Jenkins Kubernetes Plugin during pipeline execution?

[Vedran Lerenc]

Hi,

Setting the scene:

I have set up a multi-node Kubernetes cluster [1] and deployed the Jenkins Helm Chart [2] with the Jenkins Kubernetes plugin [3]. We run (company-)internally an Enterprise GitHub installation and we have multiple private and public repos. The builds are implemented using non-declarative Jenkins pipelines [4] (many features are missing in the plugin for declarative pipelines [5]). The builds don't run on the Jenkins master, but every job run consumes its own pod for obvious reasons (maximum scale-out and total isolation/no side effects) via podTemplates/containerTemplates [6].

The problem/question:

I found no way or description how to fetch from private repos or, more generally spoken, I found no way to pull any credentials/secrets from the Jenkins master (where I like to maintain them centrally) into the pods/containers that the plugin creates for my job runs, i.e. I miss in those pods/containers e.g. the credentials/secrets to pull from a private repo. How can I make those available in the dynamically spawned pods/containers? Is that even possible?

Some additional information:

On my Jenkins master I can perfectly fetch from private repos, have access to everything and all is fine. It's only lately that I tried to use the Kubernetes plugin and get more out of my cluster. Also, I `kubectl exec`'d into the JNLP slave container (and its siblings based on my containerTemplates) and couldn’t find anything. Not in the ENV, not in files. It is not clear to me how my credentials/secrets would get injected, and what I need to do for it.

Dirty solutions I already use, but I like to replace:

To overcome the problem, I put my credentials into the JenkinsFile, but that's bad because I now smear them across my repos and it's no solution for the public repos either. What I also did was to bake them into my images for the pod/container templates, but that's ugly for similar reasons as I put them now into DockerFiles (directly or indirectly during the build) and can't use off-the-shelf images anymore and can't put mine into public image repos anymore, too. I guess it would be also possible to modify my top-most podTemplate/containerTemplate and manually add ENV vars with the credentials/secrets, but that's ugly as well as I wouldn't make use of the Jenkins master credentials/secrets store anymore.

Can someone please help? I look for a clean solution to the problem. I hope, it's possible to bring my credentials/secrets from the Jenkins master into my dynamically spawned pods/containers that I also like to keep (no static slaves, but dynamic ones for each and every job run).

Thanks in advance, Vedran

[1] https://kubernetes.io

[2] https://github.com/kubernetes/charts/tree/master/stable/jenkins

[3] https://github.com/jenkinsci/kubernetes-plugin

[4] https://jenkins.io/doc/book/pipeline

[5] https://groups.google.com/forum/#!topic/jenkinsci-users/DEwTX-C5ct4

[6] https://github.com/jenkinsci/kubernetes-plugin#pod-and-container-template-configuration

[Carlos Sanchez]

To pull images from private registries you need the imagePullSecrets option (not yet released for pipeline, will be in 1.0)

For credentials you would need to do something with them, ie. put them in an environment variable and then use it in the containers. This has also been improved/fixed for 1.0

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1caaec44-8c6f-4499-a32c-e29145908c4d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Leshc]

Hi,

i have v1.2 of Kubernetes plugin, but checkout still fails -- apparently due to not finding credentials -- when i point at any node other than the master. any pointers as to what i am doing wrong or how to workaround the issue?

thanks,
dan

pipeline:

podTemplate(
    label: 'svntest',
    containers: [containerTemplate(name: 'jnlp', image: 'jenkins/jnlp-slave:alpine', args: '${computer.jnlpmac} ${computer.name}')],
    nodeSelector: 'kubernetes.io/hostname=cluster-member.some-company.com')
{
    node('svntest')
    {
        checkout([$class: 'SubversionSCM', additionalCredentials: [[credentialsId: 'svnlogin', realm: '<https://svn.some-company.com:443> Some-Company ActiveDirectory ID']], excludedCommitMessages: '', excludedRegions: '', excludedRevprop: '', excludedUsers: '', filterChangelog: false, ignoreDirPropChanges: false, includedRegions: '', locations: [[credentialsId: 'svnlogin', depthOption: 'immediates', ignoreExternalsOption: true, local: '.', remote: 'https://svn.some-company.com/svn/repo']], quietOperation: true, workspaceUpdater: [$class: 'UpdateUpdater']])
    }
}

 log:

Started by user anonymous
Running in Durability level: PERFORMANCE_OPTIMIZED
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
jenkins-slave-19lx3-6p5t0 is offline
Running on jenkins-slave-0ztg1-5w07r in /home/jenkins/workspace/scmtest
[Pipeline] {
[Pipeline] checkout
Checking out a fresh workspace because /home/jenkins/workspace/scmtest doesn't exist
Cleaning local Directory .
Checking out https://svn.some-company.com/svn/repo at revision '2018-02-08T00:15:43.521 +0000' --quiet
ERROR: Failed to check out https://svn.some-company.com/svn/repo
org.tmatesoft.svn.core.SVNException: svn: E175002: timed out waiting for server
svn: E175002: OPTIONS request failed on '/svn/repo'
    at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:112)

on master -- where checkout works -- checkout lines look like:

Checking out https://svn.some-company.com/svn/repo at revision '2018-02-08T00:15:43.521 +0000' --quiet
Found credentials some-user/****** in realm ‘<https://svn.some-company.com:443> Some-Company ActiveDirectory ID’

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

[Carlos Sanchez]

"timed out waiting for server" means the pod can't connect to your svn server 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4843615b-c526-4d93-93f9-e64ae703a747%40googlegroups.com.