Need help with Skaffold and monorepo

[Tim Hardy]

I'm having a very difficult time getting Skaffold to work with my monorepo, and I see zero documentation, blogs, guides, etc addressing the use of Skaffold and monorepos. The precious few examples I've seen have no shared packages in the monorepo, meaning they aren't helpful at all.

The issue is that my Dockerfiles need to be built from the root of the monorepo in order to grab shared packages and to determine the exact files needed for a particular application. I handle the latter by using turbo repo's "prune" command, which is working great. In order for this to work though, it needs access to the entire monorepo, meaning one of the first things I do in any app's Dockerfile is "COPY . ." from the root. Turbo's prune can then build a package-lock.json and a list of the exact files I need specifically for that app, so I can then run an "npm ci" and build that particular app in subsequent stages in the Dockerfile.

The problem is that setting context to "." in my skaffold.yaml for every app causes every app to get rebuilt when any file changes anywhere in the monorepo.

Is there a way to more manually customize how/when a build is triggered for an app and still run my docker build from the root of my monorepo?

Please help. I've spent several days working on this to no avail. It makes me think anyone using monorepos with one or more shared packages simply abandons the use of Skaffold, which I am close to doing.

[Tim Hardy]

Here's an example of what all the AI code helpers have tried suggesting, using "requires", but they're obviously wrong about its usage...

apiVersion: skaffold/v4beta12

kind: Config

manifests:

  kustomize:

    paths:

      - ./k8s/base

build:

  local:

    push: false

  artifacts:

    - image: my-app/api

      context: .

      docker:

        dockerfile: api/Dockerfile.dev

        noCache: true

      requires:

        - path: api/**/*

        - path: packages/api-common/**/*

        - path: packages/common-models/**/*

      sync:

        manual:

          - src: 'api/src/**/*.ts'

            dest: .

          - src: 'api/src/**/*.json'

            dest: .

This "requires" syntax is invalid, but if it did work, this would be EXACTLY what I need - the ability to trigger an artifact build based on a manually specified list of paths. 

If there is some way to do this, please let me know because I haven't been able to find it.

[Brian de Alwis]

Hi Tim.

Skaffold infers the paths to be monitored by analyzing the dockerfile and looking for `ADD` and `COPY` commands.  So your `COPY . .` is thoroughly confusing it.

You could try the `custom` builder to invoke Docker manually.  You can explicitly specify the paths to be monitored there.

The effort to maintain these rules in the `skaffold.yaml` doesn't seem any easier than having explicit `ADD` and `COPY` commands.  There is a risk that you miss dependencies and so end up with stale images.  The advantage of explicit `ADD` and `COPY` commands is that missing dependencies are surfaced as build-time errors.

Brian.

--
You received this message because you are subscribed to the Google Groups "skaffold-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skaffold-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/skaffold-users/dc7271ef-6ef6-4308-b604-60ba4bff1c57n%40googlegroups.com.

[Tim Hardy]

Thank you for your response. I got it working using a custom builders for my apps in skaffold.yaml. I ultimately decided on this path because it's a little complexity in one file instead of spreading that complexity across multiple Dockerfiles. My Dockerfiles leverage turbo repo's prune capability to have any and all package dependency changes "just work" through intelligently reading the actual dependencies. I'm happy with the result.