Docker Build issue with Dind - MTU size

[Rabea]

I have a go-cd deployment on top of K8s cluster that runs inside a VM (esxi vm)
I configured a pipeline to build a docker image from a Dockerfile
I have the Elastic agent profile configured to use gocd/gocd-agent-docker-dind:v22.3.0

While running docker build all steps pass except for 
a curl command that downloads a scripts from github ( I get a timeout and connection reset by peer)

the same curl command does work inside the pod that runs the docker inside it 
and the curl work also from the VM itself
and looking at the network configuration inside the agent pod 
I can see that my agent pod interface has mtu of 1450 (which is correct as my flannel cni sets it to 1450)
while the docker interface (inside the pod ) that is used for docker containers has mtu size of 1500 that that is the problem because the inner docker container has higher mtu size that the outer pod mtu

How can I configure the MTU size for the docker daemon inside the gocd agent ?
Can I build docker image by using other agent image ? maybe not a dind image ?


see this image that I found that explains the issue.

https://blog.zespre.com/assets/images/dind-mtu-size-matters/the-relationship-between-two-containers.png

[Chad Wilson]

The GoCD dind image is just an extension of the official Alpine-based Docker one at docker:dind: https://github.com/gocd/docker-gocd-agent-dind/blob/master/Dockerfile#L29 ... so anything you can do with dockerd or the regular dind image should be possible on GoCD.

Unfortunately, it looks like changing MTUs might be a bit more difficult than expected? Not sure though, as some of the tickets seem old.

https://docs.docker.com/engine/reference/commandline/dockerd/ (see --mtu)
https://github.com/docker-library/docker/issues/102

https://github.com/moby/moby/issues/36659

In any case, you could try building a child image and/or overwriting /run-docker-daemon.sh to see if setting the MTU there works now - or experiment with the base docker:dind image first to see if you can overcome your MTU challenges.

-Chad

PS: It's worth noting though that I understand building docker images from within k8s in the traditional DIND way is I imagine likely to disappear due to removal of default support for Docker itself within later Kubernetes versions, and so folks will likely need to use different approaches to build images when running inside Kubernetes. I imagine these would be things like podman/kaniko/buildah etc. I haven't yet given enough thought to how we GoCD agent images might best support this with (possibly opinionated?) OOTB tooling, and support for various rootless and non-privileged modes for building images.

Depending on how old your K8s version is and when you plan to upgrade, you might want to consider how you will do such builds in a world without Docker?

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/89d63d5d-e346-4e91-887c-287b66756956n%40googlegroups.com.

[Rabea]

Thanks for the help.
for the time being I think I will go with the traditional way and build me a dind image with --mtu flag based on the gocd agent dind image 
but I am planning to use kaniko in the near future if it will work with go-cd ( did not explore that yet)