Failed to find 'git' in path with Helm based installation

[Andreas Hubert]

Hi all!

I just wanted to play and experiment a little bit with GoCD and tried to use the Helm chart for my own k8s cluster.

But when I try to add Material or work with the sample Material and test connection, I get this error:

Failed to find 'git' on your PATH. Please ensure 'git' is executable by the Go Server and on the Go Agents where this material will be used.

If I check the resources in my namespace, it seems the agent is not coming up. Could this be related? 

NAME                                   READY   STATUS    RESTARTS   AGE
pod/gocd-app-server-5c9dd5b56c-646pn   1/1     Running   0          44m

NAME                      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/gocd-app-server   NodePort   10.43.100.193   <none>        8153:30760/TCP   44m

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/gocd-app-agent    0/0     0            0           44m
deployment.apps/gocd-app-server   1/1     1            1           44m

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/gocd-app-agent-54b5bdc767    0         0         0       44m
replicaset.apps/gocd-app-server-5c9dd5b56c   1         1         1       44m

Thanks for any hint!

[Chad Wilson]

The core error regarding git you are seeing is not directly related to the agent not coming up, but they may have the same root cause.

What operating system, hardware architecture and Kubernetes variant are you deploying the Helm chart to?

At a guess, is this perhaps a local cluster on an M1 Mac?

-Chad

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/6ad3fb0c-a828-43fc-b103-e086cf7b293cn%40googlegroups.com.

[Andreas Hubert]

> At a guess, is this perhaps a local cluster on an M1 Mac?

Good guess ;)

When I check the logs from the pod, I get this error upon checking connection for sample Material:

jvm 1    | 2023-07-19 14:52:28,756  INFO [166@MessageListener for ServerPingListener] p.c.g.c.e.k.c.g.c.e.KubernetesPlugin:72 [plugin-cd.go.contrib.elasticagent.kubernetes] - [refresh-pod-state] Pod information successfully synced. All(Running/Pending) pod count is 0.
jvm 1    | 2023-07-19 14:52:30,015 ERROR [124@MessageListener for MaterialUpdateListener] ProcessManager:102 - [Command Line] Failed executing [git clone --branch master --no-checkout https://github.com/gocd-contrib/getting-started-repo /go-working-dir/pipelines/flyweight/8ad0eaec-5e2d-4f61-bfd6-dc26f7f67818]
jvm 1    | 2023-07-19 14:52:30,015 ERROR [124@MessageListener for MaterialUpdateListener] ProcessManager:103 - [Command Line] Agent's Environment Variables: {GOCD_APP_SERVER_SERVICE_PORT_HTTP=8153, PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, WRAPPER_JAVA_VERSION_MINOR=0, WRAPPER_HOSTNAME=gocd-app-server-5c9dd5b56c-646pn, WRAPPER_BITS=64, WRAPPER_VERSION=3.5.51, WRAPPER_BASE_NAME=wrapper, GOCD_APP_SERVER_SERVICE_PORT=8153, WRAPPER_HOST_NAME=gocd-app-server-5c9dd5b56c-646pn, WRAPPER_JAVA_VENDOR=OpenJDK, PWD=/, KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443, LANGUAGE=en_US:en, GOCD_PLUGIN_INSTALL_docker-registry-artifact-plugin=https://github.com/gocd/docker-registry-artifact-plugin/releases/download/v1.3.1-485/docker-registry-artifact-plugin-1.3.1-485.jar, WRAPPER_EDITION=Standard, GOCD_APP_SERVER_PORT_8153_TCP_PROTO=tcp, LC_ALL=en_US.UTF-8, WRAPPER_JAVA_VERSION_REVISION=6, WRAPPER_JAVA_VERSION=17.0.6, KUBERNETES_SERVICE_PORT_HTTPS=443, SHLVL=1, WRAPPER_PID=115, WRAPPER_WORKING_DIR=/go-working-dir, WRAPPER_OS=linux, KUBERNETES_PORT=tcp://10.43.0.1:443, GOCD_APP_SERVER_SERVICE_HOST=10.43.100.193, KUBERNETES_SERVICE_HOST=10.43.0.1, LANG=en_US.UTF-8, WRAPPER_BIN_DIR=/go-server/wrapper, WRAPPER_CONF_DIR=/go-server/wrapper-config, WRAPPER_LANG=en, GOCD_APP_SERVER_PORT_8153_TCP=tcp://10.43.100.193:8153, WRAPPER_FILE_SEPARATOR=/, WRAPPER_INIT_DIR=/, KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1, GOCD_APP_SERVER_PORT_8153_TCP_ADDR=10.43.100.193, GOCD_PLUGIN_INSTALL_kubernetes-elastic-agents=https://github.com/gocd/kubernetes-elastic-agents/releases/download/v3.9.0-442/kubernetes-elastic-agent-3.9.0-442.jar, GO_JAVA_HOME=/gocd-jre, WRAPPER_PATH_SEPARATOR=:, KUBERNETES_PORT_443_TCP_PROTO=tcp, KUBERNETES_SERVICE_PORT=443, GOCD_APP_SERVER_PORT=tcp://10.43.100.193:8153, HOSTNAME=gocd-app-server-5c9dd5b56c-646pn, WRAPPER_JAVA_VERSION_MAJOR=17, WRAPPER_RUN_MODE=console, WRAPPER_ARCH=x86, GOCD_APP_SERVER_PORT_8153_TCP_PORT=8153, KUBERNETES_PORT_443_TCP_PORT=443, HOME=/home/go}

Which is weird, because if I just run those commands directly with git, it's fine:

kubectl exec --stdin --tty  gocd-app-server-5c9dd5b56c-646pn  -n homelab -- git version
git version 2.38.4

I am working with Rancher Desktop. I also just enabled some experimental features for Apple Silicon virtualisation.... Do you have any recommendations when running things on M1 Mac?

Thanks!

[Andreas Hubert]

Okay, so it did not worked in Rancher Desktop when I enabled Virtual Machine Emulation VZ with enabled Rosetta Option. It also did not worked with Virtual Machine Emulation QEMU. But finally it works now with VZ, but Rosetta option unchecked.

Thanks for the hint Chad!

[Chad Wilson]

Yeah, the default images are Alpine based and not built for arm64/aarch64, see this for nasty detail.

I use colima rather than rancher but it is horrifically slow under QEMU emulation, and still weirdly unstable like you experienced (although never seen that git error as a result). Never bothered with Rosetta, but for some reason qemu doesn't like the JVM very much on the GoCD images. It is possibly a whole lot of problems mixed up together.

A much higher performing and more stable alternative for you will be to add --set server.image.repository=gocd/gocd-server-centos-9 --set agent.image.repository=gocd/gocd-agent-centos-9 or equivalent in values overrides when you install/upgrade the chart. This switches to the CentOS-based images which are a bit bigger, but perfectly stable and built multi-arch including arm64. For the agent, all the non-alpine images (ubuntu, debian, centos) have been built multi-arch since 23.1.0.

This is how I test/validate/develop locally, which is also on an Apple Silicon Mac.

Probably could do with being better documented on the Helm Chart itself, PRs welcome 😅

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/4e870585-e6d6-4912-97ba-e9954d0d17bbn%40googlegroups.com.

[Andreas Hubert]

I also tried colima before... everything in this container world became horrible with M1. I tried to test with Oracle database and another Linux container using systemd.. I ended up to run docker in a compute instance in GCP, since I got 300$ free credits there. Recently I also bought a mini-PC, but couldn't get things run there either on Ubuntu 22. K8s keeps failing and with docker container I have some weird issue that network interface gets interrupted. 

Don't you think it would make sense in the Helm chart to set Agent replica to 1 at least? As far as I understood the documentation, you need an agent anyway to have things working.