Connecting Azure Repo to gocd version 19.8.0

[Obiageli Adegbite]

Hello,

i am currently using gocd v19.8.0 and i want to connect to my Azure repo. When i put in my  credentials in the advanced settings the username and password(Personal Access Token(PAT)) and click connection it fails (pic1). However if i put it same PAT  in the repo url it works (pic2). The issue is the PAT is exposed. I also cannot use aws secret manager to hide the PAT in the repo url. If i don't want to use ssh is there something i am doing wrong

[Sriram Narayanan]

You may find this approach convenient (not as convenient as a ready-made UI, but convenient all the same).

1. Create a Secured Environment variable in a pipeline and set the value to be the PAT

2. View the config.xml from the admin menu to get the encrypted token strong that will start with "AES:"

3. Use this as the encryptedpassword value for this remote material - set in the pipeline XML or the yaml.

4. You can "Test Connection" to the material from the material menu as well.

-- Sriram

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/14651c28-7e39-45b3-946f-f4705830e134n%40googlegroups.com.

[Chad Wilson]

The way GoCD constructs the URLs for Git materials requires both username and password to have non-empty values since all it does is just construct a URL using these values and invokes the command line git with this URL.

What's the error you get if you remove both username and path from URL (e.g https://dev.azure.com/blah), put a dummy string in username, and the PAT in password?

The docs at https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows#use-a-pat are a bit confusing to me. It's possible to interpret that as saying it's not possible to use PATs on Azure DevOps with default Git URLs if Azure requires the username to be empty/blank; since it talks about using a custom constructed header with a blank username as a workaround (one that wouldn't be possible via GoCD).

GoCD would be effectively doing the same as "git clone https://dummyusername:P...@dev.azure.com/blah" so if this doesn't work from the command line, it might not be possible to make this work right now, and unfortunately for some historic reasons GoCD doesn't classify the username as "secret/secure" and so you can't put the PAT there either (in case that works).

-Chad

On Mon, May 27, 2024 at 11:15 PM Obiageli Adegbite <oec...@gmail.com> wrote:

--

[Obiageli Adegbite]

Thank you Chad and Sriram for your response,

I tried wihout the username and password in the url and got pic3 which shows the error. Then i remove both username and path from the url, put a dummy string in username, and the PAT in password the errors are in pic 4 and .

Something i noticed on commandline is both git clone https://dummyusername:P...@dev.azure.com/blah and  git clone https://P...@dev.azure.com/blah clone just fine. So puting my username in the username slot an PAT in my password should work but it seems GoCD is the error i got in pic3 when i did not use a username an password shows GoCD is trying to use https://******@dev.azure.com/******/blah/bla

[Chad Wilson]

> I tried without the username and password in the url and got pic3 which shows the error. Then i remove both username and path from the url, put a dummy string in username, and the PAT in password the errors are in pic 4 and pic5.

The error message "ambiguous credentials" implies GoCD thinks there is still a username or password in the URL as well as set separately - this is a GoCD validation failing. I can't see your actual input since you have redacted the input, but please double/triple check there is no username left in the URL? If you have definitely removed it, this is very confusing - possibly an old UI bug, but I'd be surprised.

> shows GoCD is trying to use https://******@dev.azure.com/******/blah/bla

This is possibly expected IF the username is set in the username field since the error message comes from the actual git command line invocation. GoCD builds the URL dynamically and inserts the "username:password@" before invoking the git command line. If the username was empty in GoCD, that is unusual and points to a bug where GoCD is confused about which URL and credentials to use.

• The ability to use separate username/password for Git materials (outside the URL) was relatively new in your version 19.8.0. That version is 5 years old, and there are many things likely to have been fixed since then, including on the UI. You should upgrade - it's not really possible to effectively support 5 year old versions.
  
• If you have multiple pipelines that have the same Git URL pointing at them which you are using to test, there could be situations where GoCD is confused as to which credentials to use and could keep using "old" credentials. If you only have one pipeline using the URL you are testing with this will not be the issue. Such bugs have been fixed since then, but will definitely be broken in your version. e.g https://github.com/gocd/gocd/issues/9153
  

After testing this empirically myself on latest GoCD 24.1.0, it worked fine for me to connect with a PAT to an Azure DevOps repository using a personal access token with a random username.

I'd suggest using https://www.gocd.org/test-drive-gocd.html to run a quick temporary test GoCD locally and try on a current version with the same repo/PAT etc. If you can't replicate a problem on the most recent version it's likely it's a bug long since fixed.

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/b272b4ab-cb19-4350-8301-d73d29f8291dn%40googlegroups.com.

[Obiageli Adegbite]

Hey Chad,

it works

well apparently when i was imputing the url, I was using the part https://abc@dev.azure.com/******/blah/bla (clone part from azure). Then in advanced settings i left username blank and password i added my PAT. It appears GoCD simple read abc@ as username:password thereby the error message ambiguous credentials. When i used it like you did it works.

Thank you for all your help.

[Chad Wilson]

OK, that's good to hear. And yes, Azure DevOps seems to include username in the clone URL by default, so has to be edited/removed.

I should have applied Occam's razor here....sometimes the simplest explanation is the best one!

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/5656e4a7-150c-4f75-a594-ede69a2a5c7fn%40googlegroups.com.

[Obiageli Adegbite]

so true, simplest explanations.

The next issue i'm getting is error message: Failed to run git clone command, the test connection is okay so this is strange.

[Obiageli Adegbite]

still having the git clone failure

[Chad Wilson]

Try setting a dummy value for the username rather than leaving it blank.

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/c994a2c9-2951-48ed-8b4c-ff337461584en%40googlegroups.com.

[Obiageli Adegbite]

i did put in user name in the username field and still got the same error. but the test connection shows as okay, can't even just upgrade seeing as i have over 100 pipelines running, but plan to. just hopingthere's a workaround.

look at that : before the ****** what is that about

STDERR: fatal: Authentication failed for 'https://:******@dev.azure.com/Taqauto/Migrate-Project-Test/_git/Migrate-Project-Test/'

[Chad Wilson]

From that error message it looks like it's using an empty username for some reason, rather than the dummy username you entered/expect.

Check if you have multiple pipelines with the same exact full "Migrate-Project-Test" URL in the Git Material (from Materials view). If you do, go through each one of the pipelines and update the username and password to the same "correct" ones. If you only have a single pipeline using that material, I don't know what could cause it.

Basically, this is a workaround for the issue I mentioned at https://github.com/gocd/gocd/issues/9153 and https://github.com/gocd/gocd/issues/8986 (fixed in GoCD 22.2.0 but definitely an issue in your old version). When doing actual work on agents, GoCD can get confused about which username/pass combination to use to authenticate to the repository. When you do "test connection" it uses the details directly in the web form, which is slightly different to how it works when distributing work to agents to do.

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/e167b3c9-d40d-4113-8e05-20b25a2f7fe9n%40googlegroups.com.

[Obiageli Adegbite]

Hello Chad,

Thank you for all you help so far so i used another project in my azure repo to connect and i still have the unable to clone error i test connection with username:ABC and PAT: ...., url :https://dev.azure.com/test/

test connection is okay

only 1 pipeline is using the Migrate test

and now i use even a different repo in the azure devops repo and still get the error

[Chad Wilson]

You can try the workaround at https://github.com/gocd/gocd/issues/9153#issuecomment-1048912342 and  https://groups.google.com/g/go-cd/c/z-g1hqX0XGA/m/fEL8jsOHAgAJ of changing capitalisation of a single letter in the domain of your material URL to ensure GoCD isn't caching credentials incorrectly.

If that workaround works for you, good, but confirms you are probably affected by a bug that requires upgrading your GoCD server to resolve properly.

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/e9ef2f31-b0fc-4438-98d7-4b5f017ef8b0n%40googlegroups.com.