azure.azcollection azure_rm.yml inventory plugin failing to use az login credentials in execution environment
735 views
Skip to first unread message
Will McDonald
Jul 24, 2023, 7:21:40 AM7/24/23
to ansible...@googlegroups.com
Hi all,
[root@3209917451f4 runner]# python3 -m pip freeze | grep ansible
ansible-core==2.15.2
ansible-runner==2.3.3
I was looking for some help debugging an unusual issue I'm seeing with azure.azcollection and the azure-cli in an Ansible execution environment.
TL;DR, if I manually build a container, pip install ansible-core, ansible-galaxy install the azure.azcollection, then pip install its requirements, install azure-cli, setup a dynamic inventory plugin, az login and then run ansible-inventory, everything works.
If I use ansible-builder to accomplish the same end result, ansible-inventory fails with the following stack trace:
[root@3209917451f4 runner]# ansible-inventory -i inventory/azure_rm.yml --graph
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with auto plugin: Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (`az login`).
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
[WARNING]: Unable to parse /runner/inventory/azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
@all:
|--@ungrouped:
[root@3209917451f4 runner]# ansible-inventory -vvv -i inventory/azure_rm.yml --graph
ansible-inventory [core 2.15.2]
config file = /runner/project/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible-inventory
python version = 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] (/usr/bin/python3)
jinja version = 3.1.2
libyaml = True
Using /runner/project/ansible.cfg as config file
Using inventory plugin 'ansible_collections.azure.azcollection.plugins.inventory.azure_rm' to process inventory source '/runner/inventory/azure_rm.yml'
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with auto plugin: Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (`az login`).
File "/usr/local/lib/python3.11/site-packages/ansible/inventory/manager.py", line 293, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
plugin.parse(inventory, loader, path, cache=cache)
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 221, in parse
self._credential_setup()
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 242, in _credential_setup
self.azure_auth = AzureRMAuth(**auth_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1514, in __init__
self.fail("Failed to get credentials. Either pass as parameters, set environment variables, "
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1640, in fail
self._fail_impl(msg)
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1643, in _default_fail_impl
raise AzureRMAuthException(msg)
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
File "/usr/local/lib/python3.11/site-packages/ansible/inventory/manager.py", line 293, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/inventory/ini.py", line 137, in parse
raise AnsibleParserError(e)
[WARNING]: Unable to parse /runner/inventory/azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
@all:
|--@ungrouped:
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with auto plugin: Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (`az login`).
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
[WARNING]: Unable to parse /runner/inventory/azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
@all:
|--@ungrouped:
[root@3209917451f4 runner]# ansible-inventory -vvv -i inventory/azure_rm.yml --graph
ansible-inventory [core 2.15.2]
config file = /runner/project/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible-inventory
python version = 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] (/usr/bin/python3)
jinja version = 3.1.2
libyaml = True
Using /runner/project/ansible.cfg as config file
Using inventory plugin 'ansible_collections.azure.azcollection.plugins.inventory.azure_rm' to process inventory source '/runner/inventory/azure_rm.yml'
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with auto plugin: Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (`az login`).
File "/usr/local/lib/python3.11/site-packages/ansible/inventory/manager.py", line 293, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
plugin.parse(inventory, loader, path, cache=cache)
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 221, in parse
self._credential_setup()
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 242, in _credential_setup
self.azure_auth = AzureRMAuth(**auth_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1514, in __init__
self.fail("Failed to get credentials. Either pass as parameters, set environment variables, "
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1640, in fail
self._fail_impl(msg)
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1643, in _default_fail_impl
raise AzureRMAuthException(msg)
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
File "/usr/local/lib/python3.11/site-packages/ansible/inventory/manager.py", line 293, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/inventory/ini.py", line 137, in parse
raise AnsibleParserError(e)
[WARNING]: Unable to parse /runner/inventory/azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
@all:
|--@ungrouped:
This is in a running instance of the ansible-builder built container, az login has successfully run, I can az account show/az vm list and see the subscription, tennant ID, resources etc.
I've done all the usual Googling and as much RTFMing as I can but haven't found anything that would explain the difference in behaviour. More debug info below from the non-working ansible-builder container in case it helps. The behaviour persists whether podman or docker is used for the build. The only thing that seems obviously different between the two scenarios is that ansible-builder is building the EE with dumb-init?
Any suggestions are greatly appreciated.
Cheers,
Will.
--
[root@3209917451f4 runner]# cat inventory/azure_rm.yml
plugin: azure.azcollection.azure_rm
auth_source: auto
plugin: azure.azcollection.azure_rm
auth_source: auto
[root@3209917451f4 runner]# cat project/ansible.cfg
[inventory]
enable_plugins = auto, ini
[inventory]
enable_plugins = auto, ini
[root@3209917451f4 runner]# ansible-galaxy collection list
# /usr/share/ansible/collections/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.16.0
# /usr/share/ansible/collections/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.16.0
[root@3209917451f4 runner]# python3 -m pip freeze | grep ansible
ansible-core==2.15.2
ansible-runner==2.3.3
[root@3209917451f4 runner]# python3 -m pip freeze
adal==1.2.7
ansible-core==2.15.2
ansible-runner==2.3.3
antlr4-python3-runtime==4.10.1
applicationinsights==0.11.9
argcomplete==1.12.3
azure-appconfiguration==1.1.1
azure-batch==13.0.0
azure-cli==2.49.0
azure-cli-core==2.34.0
azure-cli-telemetry==1.0.6
azure-common==1.1.11
azure-containerregistry==1.1.0
azure-core==1.25.1
azure-cosmos==3.2.0
azure-data-tables==12.4.0
azure-datalake-store==0.0.49
azure-graphrbac==0.61.1
azure-identity==1.7.0
azure-keyvault==1.1.0
azure-keyvault-administration==4.3.0
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.8.0b2
azure-keyvault-secrets==4.7.0
azure-loganalytics==0.1.0
azure-mgmt-advisor==9.0.0
azure-mgmt-apimanagement==3.0.0
azure-mgmt-appconfiguration==3.0.0
azure-mgmt-appcontainers==2.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-authorization==2.0.0
azure-mgmt-automation==1.0.0
azure-mgmt-batch==5.0.1
azure-mgmt-batchai==7.0.0b1
azure-mgmt-billing==6.0.0
azure-mgmt-botservice==2.0.0b3
azure-mgmt-cdn==11.0.0
azure-mgmt-cognitiveservices==13.3.0
azure-mgmt-compute==26.1.0
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==9.0.0
azure-mgmt-containerregistry==9.1.0
azure-mgmt-containerservice==20.0.0
azure-mgmt-core==1.3.0
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-databoxedge==1.0.0
azure-mgmt-datafactory==2.0.0
azure-mgmt-datalake-analytics==0.2.1
azure-mgmt-datalake-store==1.0.0
azure-mgmt-datamigration==10.0.0
azure-mgmt-devtestlabs==9.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-eventgrid==10.2.0b2
azure-mgmt-eventhub==10.1.0
azure-mgmt-extendedlocation==1.0.0b2
azure-mgmt-hdinsight==9.0.0
azure-mgmt-imagebuilder==1.2.0
azure-mgmt-iotcentral==10.0.0b1
azure-mgmt-iothub==2.2.0
azure-mgmt-iothubprovisioningservices==1.1.0
azure-mgmt-keyvault==10.0.0
azure-mgmt-kusto==0.3.0
azure-mgmt-loganalytics==12.0.0
azure-mgmt-managedservices==6.0.0
azure-mgmt-managementgroups==1.0.0
azure-mgmt-maps==2.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-media==9.0.0
azure-mgmt-monitor==3.0.0
azure-mgmt-msi==7.0.0
azure-mgmt-netapp==10.0.0
azure-mgmt-network==19.1.0
azure-mgmt-notificationhubs==7.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-policyinsights==1.1.0b2
azure-mgmt-privatedns==1.0.0
azure-mgmt-rdbms==10.0.0
azure-mgmt-recoveryservices==2.0.0
azure-mgmt-recoveryservicesbackup==3.0.0
azure-mgmt-redhatopenshift==1.2.0
azure-mgmt-redis==13.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-resource==21.1.0
azure-mgmt-search==8.0.0
azure-mgmt-security==3.0.0
azure-mgmt-servicebus==7.1.0
azure-mgmt-servicefabric==1.0.0
azure-mgmt-servicefabricmanagedclusters==1.0.0
azure-mgmt-servicelinker==1.2.0b1
azure-mgmt-signalr==1.1.0
azure-mgmt-sql==3.0.1
azure-mgmt-sqlvirtualmachine==1.0.0b5
azure-mgmt-storage==19.0.0
azure-mgmt-synapse==2.1.0b5
azure-mgmt-trafficmanager==1.0.0b1
azure-mgmt-web==6.1.0
azure-multiapi-storage==1.1.0
azure-nspkg==2.0.0
azure-storage-blob==12.11.0
azure-storage-common==1.4.2
azure-synapse-accesscontrol==0.5.0
azure-synapse-artifacts==0.15.0
azure-synapse-managedprivateendpoints==0.4.0
azure-synapse-spark==0.2.0
bcrypt==3.2.2
certifi==2022.9.24
cffi==1.15.1
chardet==5.1.0
charset-normalizer==3.1.0
colorama==0.4.6
cryptography==40.0.2
Deprecated==1.2.14
distro==1.8.0
docutils==0.20.1
dumb-init==1.2.5
fabric==3.0.0
fluidity-sm==0.2.0
gpg==1.17.1
humanfriendly==10.0
idna==3.4
invoke==2.0.0
isodate==0.6.1
javaproperties==0.8.1
Jinja2==3.1.2
jmespath==1.0.1
jsondiff==2.0.0
knack==0.9.0
lexicon==2.0.1
libcomps==0.1.18
lockfile==0.12.2
MarkupSafe==2.1.3
msal==1.20.0
msal-extensions==0.3.1
msrest==0.7.1
msrestazure==0.6.4
oauthlib==3.2.1
packaging==21.3
paramiko==2.12.0
pexpect==4.8.0
pkginfo==1.9.6
ply==3.11
portalocker==1.7.1
psutil==5.9.2
ptyprocess==0.7.0
pyasn1==0.4.8
pycparser==2.21
PyGithub==1.58.2
Pygments==2.14.0
PyJWT==2.6.0
PyNaCl==1.5.0
pyOpenSSL==21.0.0
pyparsing==3.1.0
PySocks==1.7.1
python-daemon==3.0.1
python-dateutil==2.8.2
PyYAML==6.0.1
requests==2.28.2
requests-oauthlib==1.3.1
resolvelib==1.0.1
rpm==4.18.1
scp==0.14.5
semver==2.13.0
six==1.16.0
sshtunnel==0.4.0
tabulate==0.9.0
typing_extensions==4.5.0
urllib3==1.26.16
wcwidth==0.2.5
websocket-client==1.3.3
wrapt==1.14.1
xmltodict==0.12.0
adal==1.2.7
ansible-core==2.15.2
ansible-runner==2.3.3
antlr4-python3-runtime==4.10.1
applicationinsights==0.11.9
argcomplete==1.12.3
azure-appconfiguration==1.1.1
azure-batch==13.0.0
azure-cli==2.49.0
azure-cli-core==2.34.0
azure-cli-telemetry==1.0.6
azure-common==1.1.11
azure-containerregistry==1.1.0
azure-core==1.25.1
azure-cosmos==3.2.0
azure-data-tables==12.4.0
azure-datalake-store==0.0.49
azure-graphrbac==0.61.1
azure-identity==1.7.0
azure-keyvault==1.1.0
azure-keyvault-administration==4.3.0
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.8.0b2
azure-keyvault-secrets==4.7.0
azure-loganalytics==0.1.0
azure-mgmt-advisor==9.0.0
azure-mgmt-apimanagement==3.0.0
azure-mgmt-appconfiguration==3.0.0
azure-mgmt-appcontainers==2.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-authorization==2.0.0
azure-mgmt-automation==1.0.0
azure-mgmt-batch==5.0.1
azure-mgmt-batchai==7.0.0b1
azure-mgmt-billing==6.0.0
azure-mgmt-botservice==2.0.0b3
azure-mgmt-cdn==11.0.0
azure-mgmt-cognitiveservices==13.3.0
azure-mgmt-compute==26.1.0
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==9.0.0
azure-mgmt-containerregistry==9.1.0
azure-mgmt-containerservice==20.0.0
azure-mgmt-core==1.3.0
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-databoxedge==1.0.0
azure-mgmt-datafactory==2.0.0
azure-mgmt-datalake-analytics==0.2.1
azure-mgmt-datalake-store==1.0.0
azure-mgmt-datamigration==10.0.0
azure-mgmt-devtestlabs==9.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-eventgrid==10.2.0b2
azure-mgmt-eventhub==10.1.0
azure-mgmt-extendedlocation==1.0.0b2
azure-mgmt-hdinsight==9.0.0
azure-mgmt-imagebuilder==1.2.0
azure-mgmt-iotcentral==10.0.0b1
azure-mgmt-iothub==2.2.0
azure-mgmt-iothubprovisioningservices==1.1.0
azure-mgmt-keyvault==10.0.0
azure-mgmt-kusto==0.3.0
azure-mgmt-loganalytics==12.0.0
azure-mgmt-managedservices==6.0.0
azure-mgmt-managementgroups==1.0.0
azure-mgmt-maps==2.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-media==9.0.0
azure-mgmt-monitor==3.0.0
azure-mgmt-msi==7.0.0
azure-mgmt-netapp==10.0.0
azure-mgmt-network==19.1.0
azure-mgmt-notificationhubs==7.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-policyinsights==1.1.0b2
azure-mgmt-privatedns==1.0.0
azure-mgmt-rdbms==10.0.0
azure-mgmt-recoveryservices==2.0.0
azure-mgmt-recoveryservicesbackup==3.0.0
azure-mgmt-redhatopenshift==1.2.0
azure-mgmt-redis==13.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-resource==21.1.0
azure-mgmt-search==8.0.0
azure-mgmt-security==3.0.0
azure-mgmt-servicebus==7.1.0
azure-mgmt-servicefabric==1.0.0
azure-mgmt-servicefabricmanagedclusters==1.0.0
azure-mgmt-servicelinker==1.2.0b1
azure-mgmt-signalr==1.1.0
azure-mgmt-sql==3.0.1
azure-mgmt-sqlvirtualmachine==1.0.0b5
azure-mgmt-storage==19.0.0
azure-mgmt-synapse==2.1.0b5
azure-mgmt-trafficmanager==1.0.0b1
azure-mgmt-web==6.1.0
azure-multiapi-storage==1.1.0
azure-nspkg==2.0.0
azure-storage-blob==12.11.0
azure-storage-common==1.4.2
azure-synapse-accesscontrol==0.5.0
azure-synapse-artifacts==0.15.0
azure-synapse-managedprivateendpoints==0.4.0
azure-synapse-spark==0.2.0
bcrypt==3.2.2
certifi==2022.9.24
cffi==1.15.1
chardet==5.1.0
charset-normalizer==3.1.0
colorama==0.4.6
cryptography==40.0.2
Deprecated==1.2.14
distro==1.8.0
docutils==0.20.1
dumb-init==1.2.5
fabric==3.0.0
fluidity-sm==0.2.0
gpg==1.17.1
humanfriendly==10.0
idna==3.4
invoke==2.0.0
isodate==0.6.1
javaproperties==0.8.1
Jinja2==3.1.2
jmespath==1.0.1
jsondiff==2.0.0
knack==0.9.0
lexicon==2.0.1
libcomps==0.1.18
lockfile==0.12.2
MarkupSafe==2.1.3
msal==1.20.0
msal-extensions==0.3.1
msrest==0.7.1
msrestazure==0.6.4
oauthlib==3.2.1
packaging==21.3
paramiko==2.12.0
pexpect==4.8.0
pkginfo==1.9.6
ply==3.11
portalocker==1.7.1
psutil==5.9.2
ptyprocess==0.7.0
pyasn1==0.4.8
pycparser==2.21
PyGithub==1.58.2
Pygments==2.14.0
PyJWT==2.6.0
PyNaCl==1.5.0
pyOpenSSL==21.0.0
pyparsing==3.1.0
PySocks==1.7.1
python-daemon==3.0.1
python-dateutil==2.8.2
PyYAML==6.0.1
requests==2.28.2
requests-oauthlib==1.3.1
resolvelib==1.0.1
rpm==4.18.1
scp==0.14.5
semver==2.13.0
six==1.16.0
sshtunnel==0.4.0
tabulate==0.9.0
typing_extensions==4.5.0
urllib3==1.26.16
wcwidth==0.2.5
websocket-client==1.3.3
wrapt==1.14.1
xmltodict==0.12.0
[root@3209917451f4 runner]# az --version
azure-cli 2.49.0 *
core 2.49.0 *
telemetry 1.0.8
Dependencies:
msal 1.20.0
azure-mgmt-resource 22.0.0
Python location '/usr/bin/python3'
Extensions directory '/root/.azure/cliextensions'
Python (Linux) 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)]
Legal docs and information: aka.ms/AzureCliLegal
azure-cli 2.49.0 *
core 2.49.0 *
telemetry 1.0.8
Dependencies:
msal 1.20.0
azure-mgmt-resource 22.0.0
Python location '/usr/bin/python3'
Extensions directory '/root/.azure/cliextensions'
Python (Linux) 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)]
Legal docs and information: aka.ms/AzureCliLegal
You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade
I'm building an initial EE in case we want to maintain separate EEs for different purposes, then layering in the Azure CLI and azure.azcollection dependencies into a subsequent EE build:
- name: Install ansible-builder python requirements
run: |
mkdir -p ~/venv/ee
python3 -m venv ~/venv/ee/
. ~/venv/ee/bin/activate
python3 -m pip install --upgrade pip
pip install ansible-builder
- name: Prepare baseline execution environment config
run: |
cat > ~/baseline-execution-environment.yml <<EOF
version: 3
images:
base_image:
name: registry.fedoraproject.org/fedora:38
build_arg_defaults:
ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '-vvv'
dependencies:
ansible_core:
package_pip: ansible-core
ansible_runner:
package_pip: ansible-runner
EOF
- name: Build baseline execution environment image
run: |
. ~/venv/ee/bin/activate
ansible-builder build -f ~/baseline-execution-environment.yml -t ee-baseline:latest -v3 --container-runtime docker
- name: Push baseline execution environment image
run: |
docker tag ee-baseline:latest ${{ env.CONTAINER_REGISTRY_URL }}/ansible/ee-baseline:latest
docker push ${{ env.CONTAINER_REGISTRY_URL }}/ansible/ee-baseline:latest
- name: Prepare Azure execution environment config
run: |
cat > ~/azure-execution-environment.yml <<EOF
version: 3
images:
base_image:
name: ${{ env.CONTAINER_REGISTRY_URL }}/ansible/ee-baseline:latest
build_arg_defaults:
ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '-vvv'
dependencies:
ansible_core:
package_pip: ansible-core
ansible_runner:
package_pip: ansible-runner
galaxy:
collections:
- azure.azcollection
additional_build_steps:
prepend_final: |
RUN rpm --import https://packages.microsoft.com/keys/microsoft.asc
RUN dnf install -y https://packages.microsoft.com/config/fedora/38/packages-microsoft-prod.rpm
RUN dnf -y install azure-cli
EOF
- name: Build azure execution environment image
run: |
. ~/venv/ee/bin/activate
ansible-builder build -f ~/azure-execution-environment.yml -t ee-azure:latest -v3 --container-runtime docker
run: |
mkdir -p ~/venv/ee
python3 -m venv ~/venv/ee/
. ~/venv/ee/bin/activate
python3 -m pip install --upgrade pip
pip install ansible-builder
- name: Prepare baseline execution environment config
run: |
cat > ~/baseline-execution-environment.yml <<EOF
version: 3
images:
base_image:
name: registry.fedoraproject.org/fedora:38
build_arg_defaults:
ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '-vvv'
dependencies:
ansible_core:
package_pip: ansible-core
ansible_runner:
package_pip: ansible-runner
EOF
- name: Build baseline execution environment image
run: |
. ~/venv/ee/bin/activate
ansible-builder build -f ~/baseline-execution-environment.yml -t ee-baseline:latest -v3 --container-runtime docker
- name: Push baseline execution environment image
run: |
docker tag ee-baseline:latest ${{ env.CONTAINER_REGISTRY_URL }}/ansible/ee-baseline:latest
docker push ${{ env.CONTAINER_REGISTRY_URL }}/ansible/ee-baseline:latest
- name: Prepare Azure execution environment config
run: |
cat > ~/azure-execution-environment.yml <<EOF
version: 3
images:
base_image:
name: ${{ env.CONTAINER_REGISTRY_URL }}/ansible/ee-baseline:latest
build_arg_defaults:
ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '-vvv'
dependencies:
ansible_core:
package_pip: ansible-core
ansible_runner:
package_pip: ansible-runner
galaxy:
collections:
- azure.azcollection
additional_build_steps:
prepend_final: |
RUN rpm --import https://packages.microsoft.com/keys/microsoft.asc
RUN dnf install -y https://packages.microsoft.com/config/fedora/38/packages-microsoft-prod.rpm
RUN dnf -y install azure-cli
EOF
- name: Build azure execution environment image
run: |
. ~/venv/ee/bin/activate
ansible-builder build -f ~/azure-execution-environment.yml -t ee-azure:latest -v3 --container-runtime docker
Will McDonald
Jul 24, 2023, 7:28:11 AM7/24/23
to ansible...@googlegroups.com
Oh, and one more potentially useful data point, if I switch auth_source to explicitly 'cli' in the inventory plugin config I see the following:
[root@3209917451f4 runner]# cat inventory/azure_rm.yml
plugin: azure.azcollection.azure_rm
plugin: azure.azcollection.azure_rm
auth_source: cli
[root@3209917451f4 runner]# ansible-inventory -vvv -i inventory/azure_rm.yml --graph
ansible-inventory [core 2.15.2]
config file = /runner/project/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible-inventory
python version = 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] (/usr/bin/python3)
jinja version = 3.1.2
libyaml = True
Using /runner/project/ansible.cfg as config file
Using inventory plugin 'ansible_collections.azure.azcollection.plugins.inventory.azure_rm' to process inventory source '/runner/inventory/azure_rm.yml'
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with auto plugin: Failed to import the required Python library (azure-cli) on 3209917451f4's Python /usr/bin/python3. This is required for `cli` auth_source. Please read the module documentation and install it in the
appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter
appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter
File "/usr/local/lib/python3.11/site-packages/ansible/inventory/manager.py", line 293, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
plugin.parse(inventory, loader, path, cache=cache)
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 221, in parse
self._credential_setup()
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 242, in _credential_setup
self.azure_auth = AzureRMAuth(**auth_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1493, in __init__
self.credentials = self._get_credentials(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1759, in _get_credentials
self.fail(msg=missing_required_lib('azure-cli', reason='for `cli` auth_source'),
self.credentials = self._get_credentials(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1759, in _get_credentials
self.fail(msg=missing_required_lib('azure-cli', reason='for `cli` auth_source'),
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1640, in fail
self._fail_impl(msg)
File "/usr/share/ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1643, in _default_fail_impl
raise AzureRMAuthException(msg)
[WARNING]: * Failed to parse /runner/inventory/azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
File "/usr/local/lib/python3.11/site-packages/ansible/inventory/manager.py", line 293, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/local/lib/python3.11/site-packages/ansible/plugins/inventory/ini.py", line 137, in parse
raise AnsibleParserError(e)
[WARNING]: Unable to parse /runner/inventory/azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
@all:
|--@ungrouped:
If I run python3 in the container, I can successfully import azure.core.cli modules:
[root@3209917451f4 runner]# python3
Python 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import azure.cli.core
>>> dir(azure.cli.core)
['ALWAYS_LOADED_EXTENSIONS', 'ALWAYS_LOADED_MODULES', 'ARGCOMPLETE_ENV_NAME', 'ArgumentsContext', 'AzCLILocalContext', 'AzCli', 'AzCommandsLoader', 'CLI', 'CLICommandsLoader', 'CLIError', 'CaseInsensitiveList', 'CommandIndex', 'EVENT_FAILED_EXTENSION_LOAD', 'EXCLUDED_PARAMS', 'ExperimentalItem', 'LocalContextAction', 'MainCommandsLoader', 'ModExtensionSuppress', 'PreviewItem', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_configure_knack', 'extract_args_from_signature', 'extract_full_summary_from_signature', 'get_default_cli', 'get_logger', 'local_context', 'logger', 'os', 'sys', 'timeit']
>>> dir(azure.cli.core.AzCli)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_configure_style', '_print_init_log', '_should_enable_color', '_should_show_version', 'exception_handler', 'get_cli_version', 'get_progress_controller', 'get_runtime_version', 'invoke', 'raise_event', 'refresh_request_id', 'register_event', 'save_local_context', 'show_version', 'unregister_event']
Python 3.11.4 (main, Jun 7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import azure.cli.core
>>> dir(azure.cli.core)
['ALWAYS_LOADED_EXTENSIONS', 'ALWAYS_LOADED_MODULES', 'ARGCOMPLETE_ENV_NAME', 'ArgumentsContext', 'AzCLILocalContext', 'AzCli', 'AzCommandsLoader', 'CLI', 'CLICommandsLoader', 'CLIError', 'CaseInsensitiveList', 'CommandIndex', 'EVENT_FAILED_EXTENSION_LOAD', 'EXCLUDED_PARAMS', 'ExperimentalItem', 'LocalContextAction', 'MainCommandsLoader', 'ModExtensionSuppress', 'PreviewItem', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_configure_knack', 'extract_args_from_signature', 'extract_full_summary_from_signature', 'get_default_cli', 'get_logger', 'local_context', 'logger', 'os', 'sys', 'timeit']
>>> dir(azure.cli.core.AzCli)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_configure_style', '_print_init_log', '_should_enable_color', '_should_show_version', 'exception_handler', 'get_cli_version', 'get_progress_controller', 'get_runtime_version', 'invoke', 'raise_event', 'refresh_request_id', 'register_event', 'save_local_context', 'show_version', 'unregister_event']
Will McDonald
Jul 25, 2023, 7:45:58 AM7/25/23
to ansible...@googlegroups.com
Just in case it's of any use to someone Googling or trawling the archives, switching from prepend_final to append_final in additional_build_steps in the EE definition seems to resolve /whatever/ the issue is:
additional_build_steps:
append_final: |
append_final: |
RUN rpm --import https://packages.microsoft.com/keys/microsoft.asc
RUN dnf install -y https://packages.microsoft.com/config/fedora/38/packages-microsoft-prod.rpm
RUN dnf -y install azure-cli
It seems like the azure-cli RPM-based install pulls in all its packaged dependencies correctly and shoves them into /usr/lib/<python-ver>/site-packages.
And the pip install ansible-builder calls populates /usr/local/lib/<python-ver>/site-packages.
So I'm not sure quite what's happening in the sequencing of package/build that would be significantly different but switching prepend to append seems to do the trick. <shrug>
Reply all
Reply to author
Forward
0 new messages