Question about SSHUserPrivateKey snapshots and git checkouts

[James Robson]

Hello,

I’m trying to update the hashicorp-vault-plugin to use credential snapshots to prevent access from agents but doing scm checkouts with ssh keys and the command line git aren't working correctly and I can't tell why. You can look at the vault pr and the ssh plugin pr to see the change being made.

For some reason the git executable is not getting the key passphrase, despite it being put in a txt file in the workspace@tmp dir on the agent, and git will instead ask for the passphrase in the terminal where I launched the agent. JGit is able to get the passphrase and run a checkout successfully.

I have included debug details below, can anyone provide guidance on how to fix this or next steps for further investigation?

Adding some debugging the following env variables are set when git is called:

DISPLAY=:

GIT_ASKPASS=echo

GIT_SSH=/var/tmp/workspace/test-vault@tmp/jenkins-gitclient-ssh17057598793954356813.sh-copy

GIT_SSH_VARIANT=ssh

GIT_TERMINAL_PROMPT=false

SSH_ASKPASS=/var/tmp/workspace/test-vault@tmp/jenkins-gitclient-pass11065529797750620112.sh

Versions:

Jenkins: 2.354

Pipeline: SCM step 400.v6b_89a_1317c9a_

SSH Agent 295.v9ca_a_1c7cc3a_a_

Git client 3.11.0

Git 4.11.3

[jn...@cloudbees.com]

HI James

Have your test setup works ok before you introduced vault (ie with the standard credential helper?).

I would hazard a guess that your git installation was installed and configured to use a credential helper.
    git config --system --unset credential.helper

Regards

/James

[James Robson]

Hi,

I looks like it was some system configuration issue.

I had created set of vms on my laptop just for this work to avoid having the standard test instance unavailable, so it hadn't been working before. The vm running the controller for my scratch instance had been able to do checkouts and the agent vm should have been identical, so I don't know where the problem came from. I tried the command you suggested but that didn't help, but after applying my updated plugins to my preexisting test instance that was able to do checkout successfully.

Thanks for looking at this.

[Jesse Glick]

On Mon, Jun 13, 2022 at 12:58 PM James Robson <james....@secondmind.ai> wrote:

I had created set of vms on my laptop just for this work

I would strongly recommended writing an integration test using Testcontainers to do things like retrieve an SSH private key from Vault running in a Docker container (using a specific pinned version of all relevant software—let Dependabot keep your `Dockerfile` fresh) and then passing it to an agent running in another container (again with a specific version of Git, etc.) so as to check out from an authenticated Git server running in yet another container. This would allow you or anyone else working on the plugin to debug and examine the complete flow in a realistic environment, without concern that a test environment might have become corrupted or subtly altered; and ci.jenkins.io can run such a test as well if you supply the `docker` label in `Jenkinsfile`.