new-app Docker build problems

[Just Marvin]

Hi,

    I'm getting the following output from my build spawned off from a docker build started by new-app:

STEP 1: FROM websphere-liberty@sha256:62932a6f5d972f15fefb28d62312840c20240908ce7577a563dc0ec768c735e2

STEP 2: RUN mkdir -p /tmp/src

time="2019-05-19T15:16:49Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah937126128/mnt/rootfs\": error checking if \"/tmp/buildah937126128/mnt/rootfs/sys/fs/cgroup/memory\" is mounted: no such file or directory"

.....What is it complaining about? My FROM is based on https://hub.docker.com/_/websphere-liberty/ , and I have the container running locally as non-root. I was able to open a bash shell into the container and validate that I could create the /tmp/src directory. How do I fix this?

Regards,

Marvin

[Just Marvin]

Hi,

    I have been able to use new-app and point to the image stream for websphere-liberty and launch a pod that successfully runs it. I then rsh'd into the running container and verified that "mkdir -p /tmp/src" worked. So this has got to be some sort of problem with the build pod. 

    There is one reference in the documentation to "buildah" - which talks about how one can use it to enable custom builds. That doesn't seem to apply here. buildah doesn't appear to be installed on that container. Does it need to be? And if yes, then why? It would be strange to require that tool to support a mkdir command in a dockerfile.

Regards,

Marvin

[Just Marvin]

Hi,

    This seems to be a problem with writes within that container. I rewrote the Dockerfile to not need the mkdir. The "RUN cp". below that works. But not other cp commands which target a different area of the filesystem. Here is the build output:

STEP 1: FROM websphere-liberty@sha256:62932a6f5d972f15fefb28d62312840c20240908ce7577a563dc0ec768c735e2

STEP 2: COPY . /tmp/src/

STEP 3: WORKDIR /tmp/src

STEP 4: RUN cp /tmp/src/dist/jpetstore.war /opt/ibm/wlp/usr/servers/defaultServer/apps/ && cp /tmp/src/server.xml /opt/ibm/wlp/usr/servers/defaultServer/

time="2019-05-19T19:16:22Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah185967529/mnt/rootfs\": error checking if \"/tmp/buildah185967529/mnt/rootfs/sys/fs/cgroup/memory\" is mounted: no such file or directory"

time="2019-05-19T19:16:22Z" level=warning msg="pkg/bind: error unmounting \"/tmp/buildah185967529/mnt/rootfs\": error checking if \"/tmp/buildah185967529/mnt/rootfs/sys/fs/cgroup/memory\" is mounted: no such file or directory"

STEP 5: RUN mkdir -p /config/lib/global

time="2019-05-19T19:16:23Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah813596378/mnt/rootfs\": error checking if \"/tmp/buildah813596378/mnt/rootfs/sys/fs/cgroup/memory\" is mounted: no such file or directory"

time="2019-05-19T19:16:23Z" level=warning msg="pkg/bind: error unmounting \"/tmp/buildah813596378/mnt/rootfs\": error checking if \"/tmp/buildah813596378/mnt/rootfs/sys/fs/cgroup/memory\" is mounted: no such file or directory"

STEP 6: COPY lib/mysql-connector-java-3.0.17-ga-bin.jar /config/lib/global

error: build error: error building at STEP "COPY lib/mysql-connector-java-3.0.17-ga-bin.jar /config/lib/global": error copying "/tmp/build/inputs/jpetstore/lib/mysql-connector-java-3.0.17-ga-bin.jar" to "/var/lib/containers/storage/overlay/43275671b07ae9b9e7ebc93bc0d39a4d4320c120c19f7b1d9fc05c27f7cb3ea1/merged/config/lib/global": mkdir /var/lib/containers/storage/overlay/43275671b07ae9b9e7ebc93bc0d39a4d4320c120c19f7b1d9fc05c27f7cb3ea1/merged/config: file exists

[friar@oc6344180105 jpetstore]$ 

    And within a locally running container of websphere-liberty, I see:

Filesystem                                                                                        1K-blocks     Used Available Use% Mounted on

/dev/mapper/docker-253:1-1697390-fa7c0a3df1f813e48f39bbf16b7da96d12ad6c1006b1062d8cc8fe710735d677  10474496   529680   9944816   6% /

tmpfs                                                                                               4086760        0   4086760   0% /dev

tmpfs                                                                                               4086760        0   4086760   0% /sys/fs/cgroup

/dev/vda1                                                                                          49551932 41832536   5179328  89% /etc/hosts

shm                                                                                                   65536        0     65536   0% /dev/shm

tmpfs                                                                                               4086760        0   4086760   0% /proc/acpi

tmpfs                                                                                               4086760        0   4086760   0% /proc/scsi

tmpfs                                                                                               4086760        0   4086760   0% /sys/firmware

I have no name!@7f81302d3913:/$ 

.....both /usr and /tmp appear to be on the same filesystem ('/'), and its a mystery to me why the COPY worked, but the RUN cp didn't.

Regards,

Marvin

[ben.p...@gmail.com]

On Sunday, May 19, 2019 at 12:21:41 PM UTC-4, Just Marvin wrote:

Hi,

    I have been able to use new-app and point to the image stream for websphere-liberty and launch a pod that successfully runs it. I then rsh'd into the running container and verified that "mkdir -p /tmp/src" worked. So this has got to be some sort of problem with the build pod. 

    There is one reference in the documentation to "buildah" - which talks about how one can use it to enable custom builds. That doesn't seem to apply here. buildah doesn't appear to be installed on that container. Does it need to be? And if yes, then why? It would be strange to require that tool to support a mkdir command in a dockerfile.

you're not doing a custom build (presumably) so you can ignore that.  The discussion of buildah is for people who previously used the custom build strategy in conjunction with a docker invocation to perform docker actions (build, push) from within a custom build.  Since docker is no longer present on the nodes, this is not possible, and the alternative is to use "buildah" instead.

[ben.p...@gmail.com]

On Sunday, May 19, 2019 at 11:26:39 AM UTC-4, Just Marvin wrote:

Hi,

    I'm getting the following output from my build spawned off from a docker build started by new-app:

STEP 1: FROM websphere-liberty@sha256:62932a6f5d972f15fefb28d62312840c20240908ce7577a563dc0ec768c735e2

STEP 2: RUN mkdir -p /tmp/src

time="2019-05-19T15:16:49Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah937126128/mnt/rootfs\": error checking if \"/tmp/buildah937126128/mnt/rootfs/sys/fs/cgroup/memory\" is mounted: no such file or directory"

did the "/tmp/src" directory get created?  did the build ultimately succeed or fail?  This message looks like a warning, not a failure.

Do you have the rest of the build log output?

I've asked the buildah team to weigh in on this warning message to explain it further.

[ben.p...@gmail.com]

On Sunday, May 19, 2019 at 3:33:32 PM UTC-4, Just Marvin wrote:

Hi,

I read the output the other way around.  The "RUN cp" succeeded (with some warning output).  The COPY failed because the target file already existed.

Since "cp" happily overwrites content, that makes sense to me.  I'm not 100% certain on docker COPY behavior if the target exists.

[Just Marvin]

---------- Forwarded message ---------
From: Just Marvin <marvin.the.c...@gmail.com>
Date: Mon, May 20, 2019 at 9:30 AM
Subject: Re: new-app Docker build problems
To: <ben.p...@gmail.com>

Ben,

    Thanks for the response. I don’t follow why you believe that the Docker copy is the problem - after all, the error is on step 4 - which is the Unix cp. in any case, If you have suggestions on how to track down the cause or come up with a fix, that would be much appreciated. 

Regards,

Marvin

--
You received this message because you are subscribed to the Google Groups "OpenShift 4 Developer Preview" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openshift-4-dev-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openshift-4-dev-preview/dd3f76d5-fbe1-431c-a63f-e5f6a1566d25%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ben.p...@gmail.com]

On Monday, May 20, 2019 at 10:36:04 AM UTC-4, Just Marvin wrote:

---------- Forwarded message ---------
From: Just Marvin <marvin.the...@gmail.com>
Date: Mon, May 20, 2019 at 9:30 AM
Subject: Re: new-app Docker build problems
To: <ben....@gmail.com>

Ben,

    Thanks for the response. I don’t follow why you believe that the Docker copy is the problem - after all, the error is on step 4 - which is the Unix cp.

you got a warning on step 4.  If you'd gotten an error, you would not have gotten to step 6.

you got an error on step 6, which is what failed the build.

 

in any case, If you have suggestions on how to track down the cause or come up with a fix, that would be much appreciated. 

I suggest you check what already exists in the target location of the COPY command (including whatever the earlier steps in the dockerfile maybe have already created at that location).

 

Regards,

Marvin

To unsubscribe from this group and stop receiving emails from it, send an email to openshift-4-dev-preview+unsub...@googlegroups.com.

[Just Marvin]

Ben,

    Now that you described to me what was wrong, it was fairly straightforward to figure out the answer. It turns out that the Docker copy copies over the full path of the source. So "COPY lib/<jar file> /config/lib/global" really was trying to create /config/lib/global/lib/<jarfile> and complaining that /config/lib/global/lib didn't exist.

    I would still like to understand why I'm getting those warning messages. They are pretty loud and makes me suspicious that something subtle is broken. Any insight would be appreciated.

Thanks,

Marvin

Regards,

Marvin

To unsubscribe from this group and stop receiving emails from it, send an email to openshift-4-dev-p...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openshift-4-dev-preview/dd3f76d5-fbe1-431c-a63f-e5f6a1566d25%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "OpenShift 4 Developer Preview" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openshift-4-dev-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openshift-4-dev-preview/0baae8a8-0205-43b6-a9c7-e34607d04570%40googlegroups.com.

[Just Marvin]

Hi,

    My theory was incorrect. It appears to be a weird interaction of the docker COPY command with the target - where the target was defined via a sym link. When I do a COPY to a regular filesystem (such as /tmp), "COPY lib/<filename>...." copies just the file. Anyway, my Dockerfile is working correctly now. Thanks for the pointers. Would appreciate any insight into what those weird warnings mean,

    It strikes me that the COPY problem w.r.t sym links might be a problem with the implementation that exists on the cluster that runs docker commands......

Regards,

Marvin

Regards,

Marvin

To unsubscribe from this group and stop receiving emails from it, send an email to openshift-4-dev-preview+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openshift-4-dev-preview/dd3f76d5-fbe1-431c-a63f-e5f6a1566d25%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "OpenShift 4 Developer Preview" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openshift-4-dev-preview+unsub...@googlegroups.com.

[ben.p...@gmail.com]

On Tuesday, May 21, 2019 at 7:11:59 AM UTC-4, Just Marvin wrote:

Hi,

    My theory was incorrect. It appears to be a weird interaction of the docker COPY command with the target - where the target was defined via a sym link. When I do a COPY to a regular filesystem (such as /tmp), "COPY lib/<filename>...." copies just the file. Anyway, my Dockerfile is working correctly now. Thanks for the pointers. Would appreciate any insight into what those weird warnings mean,

    It strikes me that the COPY problem w.r.t sym links might be a problem with the implementation that exists on the cluster that runs docker commands......

It sounds like you ran into https://bugzilla.redhat.com/show_bug.cgi?id=1710008 which was a bug related to copying content that included symlinks.  It should be fixed in 4.1GA.

 

Regards,

Marvin

On Monday, May 20, 2019 at 5:05:46 PM UTC-4, Just Marvin wrote:

Ben,

    Now that you described to me what was wrong, it was fairly straightforward to figure out the answer. It turns out that the Docker copy copies over the full path of the source. So "COPY lib/<jar file> /config/lib/global" really was trying to create /config/lib/global/lib/<jarfile> and complaining that /config/lib/global/lib didn't exist.

    I would still like to understand why I'm getting those warning messages. They are pretty loud and makes me suspicious that something subtle is broken. Any insight would be appreciated.

Thanks,

Marvin