AWX-EE Deployment Fails with Error: error building at STEP "RUN assemble"
1,357 views
Skip to first unread message
Andrei Mihai
Sep 19, 2022, 10:50:09 AM9/19/22
to AWX Project
Hi AWX Team,
```
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ ls -la
total 32
drwxrwxr-x. 4 sysadmin sysadmin 166 Sep 19 10:21 .
drwx------. 17 sysadmin sysadmin 4096 Sep 19 10:15 ..
-rw-rw-r--. 1 sysadmin sysadmin 1686 Sep 19 10:21 Dockerfile
drwxrwxr-x. 2 sysadmin sysadmin 72 Sep 19 10:21 _build
-rwxrwxr-x. 1 sysadmin sysadmin 114 Sep 19 06:33 bindep.txt
drwxrwxr-x. 5 sysadmin sysadmin 74 Sep 19 10:20 builder
-rwxrwxr-x. 1 sysadmin sysadmin 888 Sep 19 10:10 execution-environment.yml
-rwxrwxr-x. 1 sysadmin sysadmin 25 Sep 19 09:41 requirements.txt
-rwxrwxr-x. 1 sysadmin sysadmin 2617 Sep 19 10:15 requirements.yml
-rwxrwxr-x. 1 sysadmin sysadmin 63 Sep 19 06:20 run.sh
(builder) [sysadmin@dev-ansible-02 ansible-builder]$
```
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat bindep.txt
python39-devel [platform:rpm compile]
subversion [platform:rpm]
subversion [platform:dpkg]
git-lfs [platform:rpm]
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat requirements.txt
awxkit==21.6.0
urllib3
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat Dockerfile
ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:latest
ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest
FROM $EE_BASE_IMAGE as galaxy
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS=
USER root
ADD _build /build
WORKDIR /build
RUN ansible-galaxy role install -r requirements.yml --roles-path "/usr/share/ansible/roles"
RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"
FROM $EE_BUILDER_IMAGE as builder
COPY --from=galaxy /usr/share/ansible /usr/share/ansible
ADD _build/requirements.txt requirements.txt
ADD _build/bindep.txt bindep.txt
RUN ansible-builder introspect --sanitize --user-pip=requirements.txt --user-bindep=bindep.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
RUN assemble
FROM $EE_BASE_IMAGE
USER root
COPY --from=galaxy /usr/share/ansible /usr/share/ansible
COPY --from=builder /output/ /output/
RUN /output/install-from-bindep && rm -rf /output/wheels
RUN yum upgrade -y
RUN yum install -y python39
RUN yum install -y python39-pip
RUN yum install -y krb5-devel
RUN yum install -y python39-devel
RUN yum install -y gcc
RUN alternatives --set python /usr/bin/python3.9
RUN pip3.9 install --upgrade --force pip
RUN python -m pip install --upgrade pip
RUN pip uninstall --yes ansible-runner && pip install ansible-runner==2.1.4
RUN ansible-galaxy collection install azure.azcollection
RUN pip install pypsrp[kerberos]
COPY --from=quay.io/project-receptor/receptor:1.0.0 /usr/bin/receptor /usr/bin/receptor
RUN mkdir -p /var/run/receptor
USER 1000
ADD run.sh /run.sh
CMD /run.sh
RUN git lfs install
I'm running into some issues with the creation of a new AWX-EE.
I followed this this guide and this guide in order to get a AWX-EE going.
Everything runs seemingly fine, until I run into this issue:
```
Collecting azure-mgmt-nspkg==2.0.0
Downloading azure_mgmt_nspkg-2.0.0-py2.py3-none-any.whl (2.8 kB)
Collecting azure-mgmt-privatedns==0.1.0
Downloading azure_mgmt_privatedns-0.1.0-py2.py3-none-any.whl (44 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.6/44.6 KB 9.7 MB/s eta 0:00:00
Collecting azure-mgmt-redis==13.0.0
Downloading azure_mgmt_redis-13.0.0-py2.py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 KB 14.6 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement azure-mgmt-resource==10.2.0,>=16.1.0 (from versions: 0.20.0rc1, 0.20.0rc2, 0.20.0, 0.20.1, 0.30.0a1, 0.30.0rc1, 0.30.0rc2, 0.30.0rc3, 0.30.0rc4, 0.30.0rc5, 0.30.0rc6, 0.30.0, 0.30.1, 0.30.2, 0.31.0, 1.0.0rc1, 1.0.0rc2, 1.0.0rc3, 1.1.0rc1, 1.1.0rc2, 1.1.0, 1.2.0rc1, 1.2.0rc2, 1.2.0rc3, 1.2.0, 1.2.1, 1.2.2, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.1.0, 2.2.0, 3.0.0, 3.1.0, 4.0.0, 5.0.0, 5.1.0, 6.0.0, 7.0.0, 8.0.0, 8.0.1, 9.0.0, 10.0.0, 10.1.0, 10.2.0, 10.3.0, 11.0.0, 12.0.0, 12.1.0, 13.0.0, 15.0.0b1, 15.0.0, 16.0.0, 16.1.0, 17.0.0, 18.0.0, 18.1.0, 19.0.0, 20.0.0, 20.1.0, 21.0.0, 21.1.0b1, 21.1.0)
ERROR: No matching distribution found for azure-mgmt-resource==10.2.0,>=16.1.0
WARNING: You are using pip version 22.0.3; however, version 22.2.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
Error: error building at STEP "RUN assemble": error while running runtime: exit status 1
An error occured (rc=1), see output line(s) above for details.
```
This is the command I run:
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ ansible-builder build --tag quay.io/andreichnla/awx-ee:latest --context ./ --container-runtime docker --verbosity 3
I'm running on Oracle 8 so I don't necessarily need to run --container-runtime docker, however I get to the same error even when podman is used for the build.
Collecting azure-mgmt-nspkg==2.0.0
Downloading azure_mgmt_nspkg-2.0.0-py2.py3-none-any.whl (2.8 kB)
Collecting azure-mgmt-privatedns==0.1.0
Downloading azure_mgmt_privatedns-0.1.0-py2.py3-none-any.whl (44 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.6/44.6 KB 9.7 MB/s eta 0:00:00
Collecting azure-mgmt-redis==13.0.0
Downloading azure_mgmt_redis-13.0.0-py2.py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 KB 14.6 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement azure-mgmt-resource==10.2.0,>=16.1.0 (from versions: 0.20.0rc1, 0.20.0rc2, 0.20.0, 0.20.1, 0.30.0a1, 0.30.0rc1, 0.30.0rc2, 0.30.0rc3, 0.30.0rc4, 0.30.0rc5, 0.30.0rc6, 0.30.0, 0.30.1, 0.30.2, 0.31.0, 1.0.0rc1, 1.0.0rc2, 1.0.0rc3, 1.1.0rc1, 1.1.0rc2, 1.1.0, 1.2.0rc1, 1.2.0rc2, 1.2.0rc3, 1.2.0, 1.2.1, 1.2.2, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.1.0, 2.2.0, 3.0.0, 3.1.0, 4.0.0, 5.0.0, 5.1.0, 6.0.0, 7.0.0, 8.0.0, 8.0.1, 9.0.0, 10.0.0, 10.1.0, 10.2.0, 10.3.0, 11.0.0, 12.0.0, 12.1.0, 13.0.0, 15.0.0b1, 15.0.0, 16.0.0, 16.1.0, 17.0.0, 18.0.0, 18.1.0, 19.0.0, 20.0.0, 20.1.0, 21.0.0, 21.1.0b1, 21.1.0)
ERROR: No matching distribution found for azure-mgmt-resource==10.2.0,>=16.1.0
WARNING: You are using pip version 22.0.3; however, version 22.2.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
Error: error building at STEP "RUN assemble": error while running runtime: exit status 1
An error occured (rc=1), see output line(s) above for details.
```
This is the command I run:
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ ansible-builder build --tag quay.io/andreichnla/awx-ee:latest --context ./ --container-runtime docker --verbosity 3
I'm running on Oracle 8 so I don't necessarily need to run --container-runtime docker, however I get to the same error even when podman is used for the build.
here are my files.
```
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ ls -la
total 32
drwxrwxr-x. 4 sysadmin sysadmin 166 Sep 19 10:21 .
drwx------. 17 sysadmin sysadmin 4096 Sep 19 10:15 ..
-rw-rw-r--. 1 sysadmin sysadmin 1686 Sep 19 10:21 Dockerfile
drwxrwxr-x. 2 sysadmin sysadmin 72 Sep 19 10:21 _build
-rwxrwxr-x. 1 sysadmin sysadmin 114 Sep 19 06:33 bindep.txt
drwxrwxr-x. 5 sysadmin sysadmin 74 Sep 19 10:20 builder
-rwxrwxr-x. 1 sysadmin sysadmin 888 Sep 19 10:10 execution-environment.yml
-rwxrwxr-x. 1 sysadmin sysadmin 25 Sep 19 09:41 requirements.txt
-rwxrwxr-x. 1 sysadmin sysadmin 2617 Sep 19 10:15 requirements.yml
-rwxrwxr-x. 1 sysadmin sysadmin 63 Sep 19 06:20 run.sh
(builder) [sysadmin@dev-ansible-02 ansible-builder]$
```
Below are the execution-environment.yml, requirements.txt and bindep.txt as well as the Dockerfile.
```
```
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat execution-environment.yml
---
version: 1
dependencies:
galaxy: requirements.yml
python: requirements.txt
system: bindep.txt
additional_build_steps:
append:
- RUN yum upgrade -y
- RUN yum install -y python39
- RUN yum install -y python39-pip
- RUN yum install -y krb5-devel
- RUN yum install -y python39-devel
- RUN yum install -y gcc
- RUN alternatives --set python /usr/bin/python3.9
- RUN pip3.9 install --upgrade --force pip
- RUN python -m pip install --upgrade pip
- RUN pip uninstall --yes ansible-runner && pip install ansible-runner==2.1.4
- RUN pip install pypsrp[kerberos]
- COPY --from=quay.io/project-receptor/receptor:latest /usr/bin/receptor /usr/bin/receptor
- RUN mkdir -p /var/run/receptor
- USER 1000
- ADD run.sh /run.sh
- CMD /run.sh
- RUN git lfs install
---
version: 1
dependencies:
galaxy: requirements.yml
python: requirements.txt
system: bindep.txt
additional_build_steps:
append:
- RUN yum upgrade -y
- RUN yum install -y python39
- RUN yum install -y python39-pip
- RUN yum install -y krb5-devel
- RUN yum install -y python39-devel
- RUN yum install -y gcc
- RUN alternatives --set python /usr/bin/python3.9
- RUN pip3.9 install --upgrade --force pip
- RUN python -m pip install --upgrade pip
- RUN pip uninstall --yes ansible-runner && pip install ansible-runner==2.1.4
- RUN pip install pypsrp[kerberos]
- COPY --from=quay.io/project-receptor/receptor:latest /usr/bin/receptor /usr/bin/receptor
- RUN mkdir -p /var/run/receptor
- USER 1000
- ADD run.sh /run.sh
- CMD /run.sh
- RUN git lfs install
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat bindep.txt
python39-devel [platform:rpm compile]
subversion [platform:rpm]
subversion [platform:dpkg]
git-lfs [platform:rpm]
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat requirements.txt
awxkit==21.6.0
urllib3
(builder) [sysadmin@dev-ansible-02 ansible-builder]$ cat Dockerfile
ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:latest
ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest
FROM $EE_BASE_IMAGE as galaxy
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS=
USER root
ADD _build /build
WORKDIR /build
RUN ansible-galaxy role install -r requirements.yml --roles-path "/usr/share/ansible/roles"
RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"
FROM $EE_BUILDER_IMAGE as builder
COPY --from=galaxy /usr/share/ansible /usr/share/ansible
ADD _build/requirements.txt requirements.txt
ADD _build/bindep.txt bindep.txt
RUN ansible-builder introspect --sanitize --user-pip=requirements.txt --user-bindep=bindep.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
RUN assemble
FROM $EE_BASE_IMAGE
USER root
COPY --from=galaxy /usr/share/ansible /usr/share/ansible
COPY --from=builder /output/ /output/
RUN /output/install-from-bindep && rm -rf /output/wheels
RUN yum upgrade -y
RUN yum install -y python39
RUN yum install -y python39-pip
RUN yum install -y krb5-devel
RUN yum install -y python39-devel
RUN yum install -y gcc
RUN alternatives --set python /usr/bin/python3.9
RUN pip3.9 install --upgrade --force pip
RUN python -m pip install --upgrade pip
RUN pip uninstall --yes ansible-runner && pip install ansible-runner==2.1.4
RUN ansible-galaxy collection install azure.azcollection
RUN pip install pypsrp[kerberos]
COPY --from=quay.io/project-receptor/receptor:1.0.0 /usr/bin/receptor /usr/bin/receptor
RUN mkdir -p /var/run/receptor
USER 1000
ADD run.sh /run.sh
CMD /run.sh
RUN git lfs install
```
The requirements.yml file does not contain any azure related collections. I removed them all.
However, I noticed the awx-ee image itself contains these packages, which means that is what's failing.
But I do not understand why and how to prevent this.
But I do not understand why and how to prevent this.
Andrei Mihai
Sep 20, 2022, 2:55:16 AM9/20/22
to AWX Project
Come to think of it, this is not quite the place where I should ask this, as it's not an AWX issue.
Anyway, if anyone comes across this and knows what the issue is, before I post the solution, that would be great.
Andrei Mihai
Sep 20, 2022, 4:42:19 AM9/20/22
to AWX Project
So I have performed the following:
1. removed everything from requirements.txt
2. kept only the following requirements in requirements.yaml - before, I had almost all available collections, with maybe 3-4 exceptions.
collections:
- name: ansible.posix
- name: ansible.utils
- name: ansible.windows
- name: community.general
- name: community.windows
- name: ansible.posix
- name: ansible.utils
- name: ansible.windows
- name: community.general
- name: community.windows
3. stripped down execution-environment.yaml to this:
---
version: 1
dependencies:
galaxy: requirements.yml
python: requirements.txt
system: bindep.txt
additional_build_steps:
append:
version: 1
dependencies:
galaxy: requirements.yml
python: requirements.txt
system: bindep.txt
additional_build_steps:
append:
- COPY --from=quay.io/project-receptor/receptor:latest /usr/bin/receptor /usr/bin/receptor
- RUN mkdir -p /var/run/receptor
- USER 1000
- ADD run.sh /run.sh
- CMD /run.sh
- RUN git lfs install
- RUN mkdir -p /var/run/receptor
- USER 1000
- ADD run.sh /run.sh
- CMD /run.sh
- RUN git lfs install
4. re-ran the deployment -> SUCCESS!
However, this is not what I want, so:
5. I re-used the original requirements.txt and deployed again. --> SUCCESS!
6. I then added back the following commands in execution-environment.yaml.
- RUN yum upgrade -y
- RUN yum install -y krb5-devel
- RUN alternatives --set python /usr/bin/python3.9
- RUN pip3.9 install --upgrade --force pip
- RUN python -m pip install --upgrade pip
- RUN pip uninstall --yes ansible-runner && pip install ansible-runner==2.1.4
- RUN pip install pypsrp[kerberos]
7. The re-ran the deployment --> SUCCESS!
8. I analysed the collections, checked which ones are really needed in our environment and used those.
9. I re-ran the deployment with the collections I needed for our environment and --> SUCCESS!!
It turns out it was the netapp.azure collection that was busting things. And this is one collection we definitely don't need.
So now I have a new EE with everything I need on it and a quick way to add/remove more if I need.
NICE!!
Reply all
Reply to author
Forward
0 new messages