VCS Stamping - How To Use It? How to debug failures?

[Adam Kaplan (He / Him / His)]

My team recently started using podman and the UBI9 go-toolset image to containerize our golang apps [1]. We found that if our Dockerfile copied the entire source tree in (including the .git directory), `go build` would fail unless we set the `-buildvcs=false` flag [2].

Since this wasn't happening on my local machine (Fedora 39), I tried to debug further only to find a surprising lack of documentation on this feature. I couldn't find anything in the `go` command documentation beyond the short note on what the `-buildvcs` flag does [3]. Nor could I easily find information on how developers can consume the VCS stamp information. My hope is that this feature would/could replace the `-ldflags` technique many golang developers have used to inject source control information into  applications.

Is there any documentation that exists here? And for instances when `-buildvcs=auto|true` fails, how can developers debug and find the root cause?

Pertinent version information for those who want to investigate further:

Go version (in the container): go1.20.12 linux/amd64

Git version (container): 2.39.3

Podman version (Fedora 39): 

Client:       Podman Engine
Version:      4.9.4
API Version:  4.9.4
Go Version:   go1.21.8
Built:        Tue Mar 26 05:39:52 2024
OS/Arch:      linux/amd64

[1] https://catalog.redhat.com/software/containers/ubi9/go-toolset/61e5c00b4ec9945c18787690?q=go%20toolset&architecture=amd64&image=6627073123047dd3dc857edb

[2] https://issues.redhat.com/browse/RHEL-36097

[3] https://pkg.go.dev/cmd/go

[Jason E. Aten]

> how can developers debug and find the root cause?

If it was me, I would start by going into the container (whatever the podman equivalent of docker exec -it containernumber bash) and try to run 'git status' or 'git log' and see why the git query is giving an error.  You could also try strace to see what git command specifically is being execed, then try to get that command working manually.

[Adam Kaplan]

Great suggestion Jason - adding `git status` took me in a very unexpected direction, and ultimately a solution.

tl;dr if your build's base container image does not use root/uid 0, git commands won't work unless you add the `--chown=<uid>` flag to your `COPY` instruction. Go builds need this if you want `-buildvcs=auto|true` to succeed.

When I changed my build command to `RUN git status && go build` in the Dockerfile, I got the following output:

```

$ podman build -t localhost/sclorg/hello-openshift:latest .
[1/2] STEP 1/3: FROM registry.redhat.io/ubi9/go-toolset:1.20.12 AS builder
[1/2] STEP 2/3: COPY . .
--> 10a13b463199
[1/2] STEP 3/3: RUN git status && go build -o /tmp/hello
fatal: detected dubious ownership in repository at '/opt/app-root/src'
To add an exception for this directory, call:

        git config --global --add safe.directory /opt/app-root/src
Error: building at STEP "RUN git status && go build -o /tmp/hello": while running runtime: exit status 128

```

This was a new and different error message for me - but same exit code as before. A quick Google search brought me to CVE-2022-24765 [1], whose fix introduced this "dubious ownership" message/protection.

I was finally able to piece everything together with a few more debug builds and internet searches:

1. On Fedora 39, podman runs in "rootless" mode. Files owned by me show up as owned by "root" in containers.

2. For Linux containers, `COPY` commands in Dockerfiles copy files as UID/GID 0 unless the `--chown` flag is passed. [2].

3. As part of the mitigation for CVE-2022-24765, git commands will succeed only if:

  a. The `.git` directory is owned by the same user executing the `.git` command OR

  b. The parent directory marked "safe" in the git configuration.

Using `COPY --chown=default . .` instead of `COPY . .` works for the UBI go-toolset image referenced previously in this thread. Your results may vary using other golang "builder" images.

[1] https://github.blog/2022-04-12-git-security-vulnerability-announced/

[2] https://docs.docker.com/reference/dockerfile/#copy---chown---chmod

On Sat, May 11, 2024 at 11:43 AM Jason E. Aten <j.e....@gmail.com> wrote:

> how can developers debug and find the root cause?

If it was me, I would start by going into the container (whatever the podman equivalent of docker exec -it containernumber bash) and try to run 'git status' or 'git log' and see why the git query is giving an error.  You could also try strace to see what git command specifically is being execed, then try to get that command working manually.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/LZbM2WlZoJM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ca680397-1497-4b3a-83ce-301c936308c1n%40googlegroups.com.

--

Adam Kaplan

He/Him

Principal Software Engineer

Red Hat

100 E. Davie Street

adam....@redhat.com    T: 1-919-754-4843    

[Adam Kaplan]

I forgot to add one more detail - the go-toolset image defaults to running as user "default" (UID 1001).

Adding `USER root` right after the `FROM` declaration is another way to work around the issue I described, but from a security perspective I would advise against it.