YAML config repos and git credentials

[Mike Culbertson]

Hey all.  

I've got gocd working generally fine, on kubernetes (GKE) via the helm chart but I'm blundering around a bit on getting materials to use credentials to clone private git repos.  

Specifically, what is the actual "best" practice as where to put credentials, particularly if we're defining the pipelines in a yaml/json config repo? Env vars? Parameters? How do I communicate them to the material?

I am *really* not a fan of putting credentials in the YAML itself (even if encrypted) as that binds changes to credentials to the source repo, and makes it much harder to automate/orchestrate credential changes.  Normally, I'd expect to be able to set username/token as env vars, then reference them in the config but so far that doesn't seem to be working.  I can dig into details if needed but more generally:

If one is using YAML based pipeline configs, where are you putting credentials, and how do you reference them in the YAML?

Thanks!

-Mike

[Chad Wilson]

Hi Mike

One philosophy I've previously used was to consider GoCD itself as a system requiring an identity identity. Accordingly, we used SSH clones for all repos, and GoCD identified itself with an SSH key which was mounted into both server and agents with a regular Kubernetes volume mount, where the private key was managed as a Kubernete secret. Teams were responsible to grant GoCD's SSH identity read access on their repos to allow it to clone. Rotating the key had co-ordination issues; however to some extent I felt this was unavoidable if the GoCD-admin-team is a type of platform team independent of the teams using it - credential and identity rotation and the co-ordination required is harder than it should be!

Given the ability to configure arbitrary repos as materials for pipelines, I'm not sure there is a perfect answer here which deals with all the rotation concerns. Even if the secrets are stored separately, GoCD would need permission to retrieve those secrets via some sort of identity.

Aside from encrypted passwords in the YAML (secure_variables), have you had a look at the Secret Params/Management capability? https://docs.gocd.org/current/configuration/secrets_management.html

This way the YAML materials should be able to have reference to a secret (with syntax like {{SECRET:[secret_config_id][secret_key]}} ), where the actual secrets are stored and maintained elsewhere - in a file on the GoCD server, or Vault/Secrets Manager/Kubernetes via plugins. I haven't personally used these plugins actively or assessed their capability for automation but believe this works when interpolating a secret into either material username/password or regular environment variables.

-Chad

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/eabe48e7-2f87-4f3a-b220-88f96cc524can%40googlegroups.com.