Building multiple docker images using the same kaniko executor gives 'file exists' error

[Brendan Hatton]

Hi,

I am using Kaniko as part of a jenkins pipeline, using dynamic kubernetes slaves to perform the build steps. This has worked great for the first couple of projects I have migrated to this pattern, but we have one project which generates two docker images as part of the build. It seems any way I try to use kaniko to build two similar images, it gives an error about symlinks already exist. Is it expected that I should be able to make multiple calls to the kaniko executor?

This is the error:

time="2018-08-29T06:02:14Z" level=info msg="Downloading base image openjdk:10-jre"
2018/08/29 06:02:14 No matching credentials were found, falling back on anonymous
time="2018-08-29T06:02:16Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/net_prio,net_cls /sys/fs/cgroup/devices /sys/fs/cgroup/cpuacct,cpu /sys/fs/cgroup/perf_event /sys/fs/cgroup/blkio /sys/fs/cgroup/pids /sys/fs/cgroup/hugetlb /sys/fs/cgroup/freezer /sys/fs/cgroup/cpuset /sys/fs/cgroup/memory /dev/mqueue /tmp /run/secrets /root/.aws /root/.docker /kaniko/.docker /root/.kube /dev/termination-log /home/jenkins /dev/shm /etc/hosts /etc/resolv.conf /etc/hostname /home/jenkins/.docker /home/jenkins/.aws /var/run/secrets/kubernetes.io/serviceaccount /dev/console /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/timer_list /proc/timer_stats /proc/sched_debug /proc/scsi /sys/firmware]"
time="2018-08-29T06:02:17Z" level=info msg="Unpacking layer: 6"
Error: error building image: symlink /usr/lib/jvm/java-10-openjdk-amd64/bin/java /etc/alternatives/java: file exists

and here is a simplified snipper of the Jenkinsfile:

                stage('publish submit') {
                  container(name: 'kaniko', shell: '/busybox/sh') {
                      sh """
                         /kaniko/executor -f `pwd`/${submitProjectName}/Dockerfile -c `pwd`/${submitProjectName} --destination=${submitProjectName}:${BUILD_NUMBER}
                      """
                  }
                }

                stage('publish validate') {
                  container(name: 'kaniko', shell: '/busybox/sh') {
                      sh """
                         /kaniko/executor -f `pwd`/${validateProjectName}/Dockerfile -c `pwd`/${validateProjectName} --destination=${validateProjectName}:${BUILD_NUMBER}
                      """
                  }
                }

The images have different Dockerfiles but both use the same base image.

Is there an easy way to achieve something like this?

Cheers

[Priya Wadhwa]

Hi Brendan,

To clarify, you're trying to build two images in the same container, correct? If so, this probably won't work -- when executing the second image kaniko tries to extract the file system of the base image (openjdk:10-jre), but runs into errors because the file system from the first image already exists.

Would you be able to run two separate kaniko builds, one for each of the images?

Priya

[Brendan Hatton]

Hi Priya,

Yes, I was trying to use a single container. My very low-tech workaround is to define two containers and build one image in each:

podTemplate(label: label,  containers: [
        containerTemplate(name: 'kaniko', image: "gcr.io/kaniko-project/executor:debug", command: '/busybox/cat', ttyEnabled: true),
        containerTemplate(name: 'kaniko-2', image: "gcr.io/kaniko-project/executor:debug", command: '/busybox/cat', ttyEnabled: true),
       
        volumes: [
                secretVolume(mountPath: '/home/jenkins/.aws/',  secretName: 'aws-secret'),
                configMapVolume(mountPath: '/kaniko/.docker/', configMapName: 'docker-config')
        {
            node(label){

                ...

                stage('publish submit') {
                  container(name: 'kaniko', shell: '/busybox/sh') {
                      sh """
                         /kaniko/executor -f `pwd`/${submitProjectName}/Dockerfile -c `pwd`/${submitProjectName} --destination=${submitProjectName}:${BUILD_NUMBER}
                      """
                  }
                }

                stage('publish validate') {

                  container(name: 'kaniko-2', shell: '/busybox/sh') {

                      sh """
                         /kaniko/executor -f `pwd`/${validateProjectName}/Dockerfile -c `pwd`/${validateProjectName} --destination=${validateProjectName}:${BUILD_NUMBER}
                      """
                  }
                }

                ...

            }
        }

It just feels .... like I am doing something wrong. But that sounds like what you are suggesting, so maybe it's not so wrong?

Cheers

Brendan

[Priya Wadhwa]

Hi Brendan,

Yah that's correct! kaniko is meant to build one image per container.

Priya 

[Vincent Behar]

Hi,

just a quick update on this, because I had the same issue (jenkins, and building multiple images with a single kaniko container): with https://github.com/GoogleContainerTools/kaniko/pull/370 there is now a new flag "--cleanup" which will clean the filesystem at the end of the build, leaving the kaniko container ready to build another image.

So kaniko can now build more than 1 image per container.

Vincent