Issue with import hostvars Option in Ansible Dynamic Inventory Parsing In Rundeck
kedar jadhav
Description : When using the import hostvars option with Ansible dynamic inventory (Azure RM Inventory Plugin), Rundeck fails to correctly parse and identify hosts. Without this option, the inventory is parsed and hosts are identified correctly.
Steps to Reproduce:
- Configure Azure RM Inventory Plugin in Ansible with import hostvars option.
- Run the Ansible inventory command via Rundeck.
- Observe that hosts are not identified correctly.
Expected Behavior: Hosts should be correctly identified and parsed with the import hostvars option.
Environment:
Rundeck Version: 5.3.0
Ansible Version: 2.16.7
Ansible Inventory Plugin: azure.azcollection.azure_rm
OS: [Ubuntu 22.04.4 LTS]
Python Version: 3.10.12
Additional Information: The same configuration works correctly without the import hostvars option.
Dynamic Inventory file which rundeck uses to create its own inventory file.
------
plugin: azure.azcollection.azure_rm # Specifies the Azure dynamic inventory plugin
cloud_environment : "AzureCloud"
auth_source: auto # Automatically determines the authentication source
include_vm_resource_groups: # List of Azure resource groups to include
- "decops-aero"
regions: # List of Azure regions to include
- centralindia
plain_host_names: yes # Use plain host names instead of the FQDN
strict_permissions: false # Do not enforce strict permissions
allow_duplicated_hosts: false # Do not allow duplicated hosts
# Conditional groups based on VM tags and other properties
conditional_groups:
logstash_hosts: '"logstash_hosts" in (tags and tags["tools_groups"])'
kafka_hosts: '"kafka_hosts" in (tags and tags["tools_groups"])'
cra_hosts: '"cra_hosts" in (tags and tags["tools_groups"])' #
httpd_hosts: '"httpd_hosts" in (tags and tags["tools_groups"])'
java_hosts: '"java_hosts" in (tags and tags["tools_groups"])'
cra_core: '"cra_core" in (tags and tags["service_groups"])'
cra_webapps: '"cra_webapps" in (tags and tags["service_groups"])'
cra_analytics: '"cra_analytics" in (tags and tags["service_groups"])'
cra_agents_hosts: '"cra_agents_hosts" in (tags and tags["service_groups"])'
cra_sme_ai_hosts: '"cra_sme_ai_hosts" in (tags and tags["service_groups"])'
cra_datastats_hosts: '"cra_datastats_hosts" in (tags and tags["service_groups"])'
crate_hosts: '"crate_hosts" in (tags and tags["db_groups"])'
mysqldb_hosts: '"mysqldb_hosts" in (tags and tags["db_groups"])'
cra_database_hosts: '"cra_database_hosts" in (tags and tags["db_groups"])'
# Filters to apply to the inventory
filters:
powerState:
- running # Include only running VMs
tags.env:
- dev # Include only VMs with the 'dev' environment tag
use_extra_vars: true # Allow use of extra variables in the inventory
# Compose variables to set for each host
compose:
ansible_host: public_ip_address[0].ipv4_address
ansible_user: "'azureuser'"
zookeeper_id: tags.zookeeper_id # Set the zookeeper_id from the tags
kafka_broker_id: tags.kafka_broker_id # Set the kafka_broker_id from the tags
crate_node_master: tags.crate_node_master # Set the crate_node_master from the tags
cluster_name: "'cracluster'"
opt_dir: "'/opt'"
data_dir: "'/data'"
spark_dir: "'/opt/spark'"
cra_dir: "'/opt/cra'"
cra_nexus_repo_url: "'https://jfrog.independent.com/repository'"
cra_local_repo_url: "'/opt/cra/artifacts'"
ansible_managed: "'ansible managed file'"
service_file_path: "'/etc/systemd/system/'"
cra_schema_name: "'cra'"
cra_env_files_path: "'/etc/sysconfig'"
java_17_home: "'/usr/lib/jvm/temurin-17-jre'"
java_8_home: "'/usr/lib/jvm/temurin-8-jre'"
deploymentType: "'cra-services'"
platform_tools: "'cra-rpms'"
mysql_user: "'root'"
mysql_password: "'Pass2024'"
mysql_port: 3306
mysql_schema_name: "'cra'"
configureFirewall: true
db_script_location: "'/opt/cra/database-scripts'"
crate_heap_size: "'2G'"
crate_file_max: 500000
crate_max_map_count: 262144
logstash_jvm_options: |
'-Xms500m -Xmx500m'
-----
- Working fine without using import host vars option .
PFA
rac...@rundeck.com
rac...@rundeck.com
Hi,
I tried using a basic example (an NMAP module-based dynamic inventory) as follows (nmap.yml). Could you test it in your environment to discard a Rundeck bug?
The dynamic inventory is configured on the Rundeck Model Source (with the “import host vars” activated). Take a look.
# nmap dynamic inventory example # installed with the following command: # ansible-galaxy collection install community.general --- plugin: nmap address: 192.168.56.0/24 strict: False ipv4: yes ports: yes ansible_user: vagrant groups: appliance: "'Amazon' in hostname" regular: "'host' in hostname"I used the following command to test the dynamic inventory:
ansible-inventory -i nmap.yml --listObtaining the following inventory:
{ "_meta": { "hostvars": { "192.168.56.1": { "ip": "192.168.56.1", "name": "192.168.56.1" }, "192.168.56.20": { "ip": "192.168.56.20", "name": "192.168.56.20", "ports": [ { "port": "22", "protocol": "tcp", "service": "ssh", "state": "open" } ] }, "192.168.56.21": { "ip": "192.168.56.21", "name": "192.168.56.21", "ports": [ { "port": "22", "protocol": "tcp", "service": "ssh", "state": "open" } ] }, "192.168.56.22": { "ip": "192.168.56.22", "name": "192.168.56.22", "ports": [ { "port": "22", "protocol": "tcp", "service": "ssh", "state": "open" } ] } } }, "all": { "children": [ "ungrouped" ] }, "ungrouped": { "hosts": [ "192.168.56.1", "192.168.56.20", "192.168.56.21", "192.168.56.22" ] } }The playbook calls some rundeck options and the ip hostvar as follows (task 3):
--- - name: just an ansible test playbook hosts: all tasks: - name: ansible step one (rundeck option) debug: msg: "first string is {{ string_1 }}" - name: ansible step two (rundeck option) debug: msg: "second string is {{ string_2 }}" - name: ansible step three (hostvar) debug: msg: "the ip address is {{ ip }}"This generates this node-set.
On Rundeck I used the following job that calls that playbook:
- defaultTab: nodes description: Test host vars executionEnabled: true id: b7930cec-02f2-4046-9ffc-550a9de6e22a loglevel: INFO name: hostvarstest nodeFilterEditable: false nodefilters: dispatch: excludePrecedence: true keepgoing: false rankOrder: ascending successOnEmptyNodeFilter: false threadcount: '1' filter: 'ip: 192.*' nodesSelectedByDefault: true options: - name: string_1 required: true sortValues: true value: hello - name: string_2 required: true sortValues: true value: world plugins: ExecutionLifecycle: {} scheduleEnabled: true sequence: commands: - configuration: ansible-base-dir-path: /home/user/Downloads/ ansible-become: 'false' ansible-binaries-dir-path: /home/user/.local/bin/ ansible-disable-limit: 'false' ansible-encrypt-extra-vars: 'false' ansible-extra-vars: |- string_1: ${option.string_1} string_2: ${option.string_2} ansible-playbook: example.yml ansible-ssh-passphrase-option: option.password ansible-ssh-use-agent: 'false' nodeStep: false type: com.batix.rundeck.plugins.AnsiblePlaybookWorkflowStep keepgoing: false strategy: node-first uuid: b7930cec-02f2-4046-9ffc-550a9de6e22aGenerates the following result:
PLAY [just an ansible test playbook] ******************************************** TASK [Gathering Facts] ********************************************************* ok: [192.168.56.21] ok: [192.168.56.20] ok: [192.168.56.22] TASK [ansible step one (rundeck option)] *************************************** ok: [192.168.56.20] => { "msg": "first string is hello" } ok: [192.168.56.21] => { "msg": "first string is hello" } ok: [192.168.56.22] => { "msg": "first string is hello" } TASK [ansible step two (rundeck option)] *************************************** ok: [192.168.56.20] => { "msg": "second string is world" } ok: [192.168.56.21] => { "msg": "second string is world" } ok: [192.168.56.22] => { "msg": "second string is world" } TASK [ansible step three (hostvar)] ******************************************** ok: [192.168.56.20] => { "msg": "the ip address is 192.168.56.20" } ok: [192.168.56.21] => { "msg": "the ip address is 192.168.56.21" } ok: [192.168.56.22] => { "msg": "the ip address is 192.168.56.22" } PLAY RECAP ********************************************************************* 192.168.56.20 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 192.168.56.21 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 192.168.56.22 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0After deactivating the “Import host vars” option on the Ansible Model Source, the ip hostvar is gone and of course, the filter fails:
com.dtolabs.rundeck.core.NodesetEmptyException: No matched nodes: NodeSet{includes={dominant=false, attributesMap={ip=192.*}}} at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowExecutor.validateNodeSet(BaseWorkflowExecutor.java:880) at com.dtolabs.rundeck.core.execution.workflow.NodeFirstWorkflowExecutor.executeWorkflowImpl(NodeFirstWorkflowExecutor.java:92) at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowExecutor.executeWorkflow(BaseWorkflowExecutor.java:220) at com.dtolabs.rundeck.core.execution.WorkflowExecutionServiceThread.runWorkflow(WorkflowExecutionServiceThread.java:95) at com.dtolabs.rundeck.core.logging.LoggingManagerImpl$MyPluginLoggingManager.runWith(LoggingManagerImpl.java:146) at com.dtolabs.rundeck.core.execution.WorkflowExecutionServiceThread.run(WorkflowExecutionServiceThread.java:77) Exception: class com.dtolabs.rundeck.core.NodesetEmptyException: No matched nodes: NodeSet{includes={dominant=false, attributesMap={ip=192.*}}} No matched nodes: NodeSet{includes={dominant=false, attributesMap={ip=192.*}}}Could you please test my basic example? Maybe I’m missing something (probably in my playbook). It’s possibly related to the Azure RM Inventory Plugin.
Also, do you see something related in your service.log when executing that job?
Regards!
kedar jadhav
[2024-06-13T16:38:00,643] ERROR services.NotificationService - Cannot send notification email: Property ${globals.rundeck-email-notif} could not be resolved in template: ${globals.rundeck-email-notif}, context: user: admin, job: platform-tools/kafka
/usr/lib/python3/dist-packages/requests/__init__.py:87: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (5.2.0) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
PLAY [all] *********************************************************************
TASK [Gathering Facts] *********************************************************
Thursday 13 June 2024 16:38:15 +0530 (0:00:00.020) 0:00:00.020 *********
ok: [devops-aero1]
TASK [Ensure tmpdir data directory] ********************************************
Thursday 13 June 2024 16:38:18 +0530 (0:00:02.292) 0:00:02.312 *********
changed: [devops-aero1 -> localhost]
TASK [Template the gathered facts] *********************************************
Thursday 13 June 2024 16:38:18 +0530 (0:00:00.674) 0:00:02.986 *********
changed: [devops-aero1 -> localhost]
PLAY RECAP *********************************************************************
devops-aero1 : ok=3 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Thursday 13 June 2024 16:38:19 +0530 (0:00:00.969) 0:00:03.956 *********
===============================================================================
Gathering Facts --------------------------------------------------------- 2.29s
Template the gathered facts --------------------------------------------- 0.97s
Ensure tmpdir data directory -------------------------------------------- 0.67s
[2024-06-13T16:38:32,671] ERROR services.NotificationService - Cannot send notification email: Property ${globals.rundeck-email-notif} could not be resolved in template: ${globals.rundeck-email-notif}, context: user: admin, job: platform-tools/kafka
[2024-06-13T16:38:32,678] ERROR services.NotificationService - Cannot send notification email: Property ${globals.rundeck-email-notif} could not be resolved in template: ${globals.rundeck-email-notif}, context: user: admin, job: platform-tools/kafka