GoCD and private GitHub repo

[vv-fork]

Hello colleagues!

What is the best way to connect on-prem goCD with GitHub private repo in cloud? I was smoking docs and manuals for quite a while, but what people say it’s to install ssh keys to both GitHub and goCD, which won’t work, since I am using github.com, so i suppose i can’t install ssh key there.

I’ve installed github-oauth-authorization-plugin and set it as described (connection ok in authorisation configuration step), and restarted the server, however it’s still throwing that standard error “fatal: could not read Username for ‘https://github.com’ meaning that the access is still closed.

What else can be done as you think?

[Sriram Narayanan]

Please see:

https://docs.github.com/en/authentication/connecting-to-github-with-ssh

The gocd server runs as a particular user account. That user account needs access to the ssh private keys used to authenticate with GitHub.

The go agent too needs the same access.

Assuming you are on Linux and installer gocd via rpm, then you would set this key in the home directory ( /var/lib/go-server/.ssh/myprivatekey.id_rsa)

Permissions for .ssh would be 600, and for the key would be 400, with the gocd process user owning the directory and The identity file.

— Sriram

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c5e6b7d72f4n%40googlegroups.com.

[Chad Wilson]

To add on to Sriram's comments, the use of the github-oauth-authorization-plugin doesn't have any relationship with access to repository content on GitHub - it simply allows people to log onto GoCD using their Github identity, and optionally to have access to GoCD pipeline groups mapped to GitHub roles.

This is because materials/repositories need to be accessed in an identity known to the GoCD server/agents, not necessarily the individual user who happens to be logged in to GoCD. So even if you use that authorization plugin, you still need to decide how to provide GoCD itself access to repositories on Github.

You can use an SSH key linked to a GitHub user if you wish to use SSH access - no restrictions for private repos unless your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS access to repositories you have to fill in a username/"password" for each material you configure. That "password" would be a personal access token with at least read-only access to the 1 or more repositories you want to use.

If you want to share one personal access token across many materials (perhaps a single token has read-only access to many repositories), the easiest way is to use a GoCD Secrets Management plugin and refer to them in the username/"password" fields of each material using the special secrets interpolation syntax: https://docs.gocd.org/current/configuration/secrets_management.html This will work with either manually defined pipelines/materials, or those defined externally in source control.

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CANiY96azM2%3DaFO351d4PpExOatRCO%2BoaQju3Juvm2yAbQR2d5A%40mail.gmail.com.

[vv-fork]

Thank you guys Sriram and Chad for answering those! Now it's getting clearer to me

I was able to connect using token. It's fine.

Though I wasn't able to connect using SSH Certificate. 

The key has been generated and installed:

it seems i set proper permissions:

but i still get that error message when i test connection from go-server:

--- STANDARD ERR --- STDERR: fatal: could not read Username for 'https://github.com': No such device or address ---

So what am i doing wrong? May that be I have messed with permissions for go user?

Vlad.

[Sriram Narayanan]

On Fri, Oct 27, 2023 at 1:13 PM vv-fork <vakhl...@gmail.com> wrote:

Thank you guys Sriram and Chad for answering those! Now it's getting clearer to me

I was able to connect using token. It's fine.

Though I wasn't able to connect using SSH Certificate. 

The key has been generated and installed:

it seems i set proper permissions:

but i still get that error message when i test connection from go-server:

--- STANDARD ERR --- STDERR: fatal: could not read Username for 'https://github.com': No such device or address ---

So what am i doing wrong? May that be I have messed with permissions for go user?

Please change the key's owner to the user "go".

 

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/95da29ae-dfae-46cd-ace0-b928b9b1a556n%40googlegroups.com.

[Chad Wilson]

Based on the error message it looks like the clone URL you are using is still an HTTPS one - to use SSH auth, you need to change it to an ssh URL, e.g g...@github.com:gocd/gocd.git - it's an entirely different git transport, not just an auth mechanism so the URL needs to change accordingly :-)

If you're new to using SSH to talk to a git repo manager, you might want to try doing it separately on the command line with a git clone before getting it to work with GoCD as adding in the GoCD server and agent adds some extra complexity.

-Chad

On Fri, Oct 27, 2023 at 1:13 PM vv-fork <vakhl...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/95da29ae-dfae-46cd-ace0-b928b9b1a556n%40googlegroups.com.

[vv-fork]

Thank you very much for the hints!

I still wasn't able to make it work. Here is what i did: 

• changed the group and owner of the key to 'go' and of the .ssh directory.

[ec2-user@ ~]$ sudo ls -al /var/lib/go-server/.ssh/
total 4
drw-------.  2 root root  26 Oct 26 21:45 .
drwxr-x---. 12 go   go   169 Oct 27 03:16 ..
-r--------.  1 go   go   432 Oct 26 21:45 vladimir_key

[ec2-user@ ~]$ sudo ls -alh /var/lib/go-server/
total 102M
drwxr-x---. 12 go   go    169 Oct 27 03:16 .
drwxr-xr-x. 28 root root  16K Oct 25 03:21 ..
drw-------.  2 go   go     26 Oct 26 21:45 .ssh
drwxr-xr-x.  3 go   go     23 Oct 25 03:34 artifacts
-rw-r--r--.  1 go   go   102M Oct 27 03:16 cruise.war
<...>

• changed url to ssh like according to the instruction, where myuser is the user in link http://github.com/[myuser]/[myreponame], and myreponame is the name of the repo in the link (just substituted them here in order not to expose).

What i get according to the pic is 'key verification failed'.

What I am doing wrong again?

Vlad.

[Sriram Narayanan]

The known_hosts file for your GoCD user needs the GitHub server’s ssh key.

https://docs.github.com/en/authentication/troubleshooting-ssh/error-host-key-verification-failed#

Are you able to perform git clone using the command line as the gocd user?

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/2978fa19-08ec-4541-a185-4b6ebf4a9aadn%40googlegroups.com.

[vv-fork]

Greetings Sriram! Thank you for pointing that out.

Here is what i tried:

1. switched to go user and checked known_hosts

[go@ip-172... ~]$ cat .ssh/known_hosts
github.com,20.248.137.48 ecdsa-sha2-nistp256 A...........

2. added ssh-add into bashrc, so each connection to that user should now start with key adding to ssh agent

[ec2-user@ip-172... ~  ]$ sudo su - go
Last login: Thu Nov  2 00:10:17 UTC 2023 on pts/1
Agent pid 25465
Identity added: vladimir_key3

3. tried to git clone:

[go@ ip-172... ~ ]$ git clone g...@github.com:username/reponame
Cloning into 'reponame'...
remote: Enumerating objects: 3343, done.
remote: Counting objects: 100% (71/71), done.
remote: Compressing objects: 100% (56/56), done.
^CKilled by signal 2.6% (2207/3343), 1.68 MiB | 1.37 MiB/s

git clone works both for go and ec2-user

4. restarted go-server

The error message is still the same...