Not able to use Foreman as ENC - "YAML data failed to parse"

[JS]

Hello all. I am having some major headaches attempting to get parameters from foreman into salt, for use as an ENC. I am trying to get a host parameter to show up in the pillar data, but it does not. 

For an example, I have added a test parameter on a host (Ive tried adding states, as well). I have ran salt '*' saltutil.refresh_pillar but alas the parameter does not show up when running salt <minion> pillar.items. I see YAML data from /usr/bin/foreman-node failed to parse in the salt logs, so it appears salt never knows about this information. However, when I run /usr/bin/foreman-node <minion> I do indeed see the parameter. Ive been working on this for the better part of 3 weeks and scoured the deepest parts of the internet, to no avail.

Anyone have any clue what is going on here?  It seems like something is broken on the salt side of things, understanding the data coming from Foreman.

Thanks.

[Phipps, Thomas]

unfortunately without any actual information to work from there is no way to troubleshoot this. please post some configs and the actual error message.

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/921add4b-6a8a-402f-b4e3-8ac7a28538e2n%40googlegroups.com.

[JS]

Apologies.

Foreman stuffs:

[root@10-222-76-237 usr]# cat /etc/salt/master.d/foreman.conf

# /etc/salt/master.d/foreman.config Master configuration
#
# This file summarizes configurations for the salt-master.
# Have a look at the [Foreman Salt Plugin Documentation](https://theforeman.org/plugins/foreman_salt/) for detailed explanations.
#
# After editing this file, run the following command to active the changes:
# systemctl restart salt-master

##
# Autosign
autosign_grains_dir: /var/lib/foreman-proxy/salt/grains
autosign_file: /etc/salt/autosign.conf
# Uncomment the next line to make use of the autosign host name file (not recommended)
permissive_pki_access: True

##
# Node classifier
master_tops:
  ext_nodes: /usr/bin/foreman-node

##
# Pillar data access
ext_pillar:
  - puppet: /usr/bin/foreman-node

##
# Salt API access
external_auth:
  pam:
    salt:
      - '@runner'

rest_cherrypy:
  port: 9191
  ssl_key: /etc/pki/tls/certs/localhost.key
  ssl_crt: /etc/pki/tls/certs/localhost.crt

##
# Remote execution provider
publisher_acl:
  foreman-proxy:
    - state.template_str

##
# Reactors
reactor:
  - 'salt/auth': # Autosign reactor
    - /usr/share/foreman-proxy/salt/reactors/foreman_minion_auth.sls
  - 'salt/job/*/ret/*': # Report reactor
    - /usr/share/foreman-proxy/salt/reactors/foreman_report_upload.sls

Error upon running  salt <minion> pillar.items (I do get our external pillar data back, but not Foreman ENC data):

2024-03-04 14:55:24,712 [salt.loaded.int.pillar.puppet:27  ][CRITICAL][1169822] YAML data from /usr/bin/foreman-node failed to parse

Return when running /usr/bin/foreman-node <minion>:

{:name=>"10-222-10-5.redacted-corp.cloud", :facts=>{"cwd"=>"/", "ip_gw"=>true, "ip4_gw"=>"10.222.10.1", "ip6_gw"=>false, "dns::nameservers::0"=>"170.40.0.100", "dns::nameservers::1"=>"170.40.127.100", "dns::ip4_nameservers::0"=>"170.40.0.100", "dns::ip4_nameservers::1"=>"170.40.127.100", "dns::domain"=>"", "dns::search::0"=>"redacted-corp.cloud", "fqdns::0"=>"10-222-10-5.redacted-corp.cloud", "machine_id"=>"5c7a98d039f74057ad4df61e043c5215", "master"=>"sparrow-sb-master.redacted-corp.cloud", "server_id"=>269576512, "localhost"=>"10-222-10-5.redacted-corp.cloud", "fqdn"=>"10-222-10-5.redacted-corp.cloud", "host"=>"10-222-10-5", "domain"=>"redacted-corp.cloud", "hwaddr_interfaces::lo"=>"00:00:00:00:00:00", "hwaddr_interfaces::eth0"=>"00:50:56:bc:7f:c3", "id"=>"10-222-10-5.redacted-corp.cloud", "ip4_interfaces::lo::0"=>"127.0.0.1", "ip4_interfaces::eth0::0"=>"10.222.10.5", "ipv4::0"=>"10.222.10.5", "ipv4::1"=>"127.0.0.1", "fqdn_ip4::0"=>"10.222.10.5", "ip_interfaces::lo::0"=>"127.0.0.1", "ip_interfaces::eth0::0"=>"10.222.10.5", "kernelparams::0::0"=>"BOOT_IMAGE", "kernelparams::0::1"=>"(hd0,msdos1)/vmlinuz-4.18.0-532.el8.x86_64", "kernelparams::1::0"=>"root", "kernelparams::1::1"=>"/dev/mapper/rootvg-rootvol", "kernelparams::2::0"=>"ro", "kernelparams::2::1"=>nil, "kernelparams::3::0"=>"nofb", "kernelparams::3::1"=>nil, "kernelparams::4::0"=>"splash", "kernelparams::4::1"=>"quiet", "kernelparams::5::0"=>"crashkernel", "kernelparams::5::1"=>"auto", "kernelparams::6::0"=>"resume", "kernelparams::6::1"=>"/dev/mapper/rootvg-swap", "kernelparams::7::0"=>"rd.lvm.lv", "kernelparams::7::1"=>"rootvg/rootvol", "kernelparams::8::0"=>"rd.lvm.lv", "kernelparams::8::1"=>"rootvg/swap", "kernelparams::9::0"=>"rhgb", "kernelparams::9::1"=>nil, "kernelparams::10::0"=>"quiet", "kernelparams::10::1"=>nil, "locale_info::defaultlanguage"=>"en_US", "locale_info::defaultencoding"=>"UTF-8", "locale_info::detectedencoding"=>"UTF-8", "locale_info::timezone"=>"UTC", "num_gpus"=>1, "gpus::0::vendor"=>"vmware", "gpus::0::model"=>"SVGA II Adapter", "kernel"=>"Linux", "nodename"=>"10-222-10-5.redacted-corp.cloud", "kernelrelease"=>"4.18.0-532.el8.x86_64", "kernelversion"=>"#1 SMP Thu Dec 21 12:11:59 UTC 2023", "cpuarch"=>"x86_64", "selinux::enabled"=>true, "selinux::enforced"=>"Permissive", "systemd::version"=>"239", "systemd::features"=>"+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy", "init"=>"systemd", "lsb_distrib_id"=>"CentOS Stream", "lsb_distrib_release"=>"8", "lsb_distrib_codename"=>"CentOS Stream 8", "osfullname"=>"CentOS Stream", "osrelease"=>"8", "oscodename"=>"CentOS Stream 8", "os"=>"CentOS Stream", "num_cpus"=>6, "cpu_model"=>"Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz", "cpu_flags::0"=>"fpu", "cpu_flags::1"=>"vme", "cpu_flags::2"=>"de", "cpu_flags::3"=>"pse", "cpu_flags::4"=>"tsc", "cpu_flags::5"=>"msr", "cpu_flags::6"=>"pae", "cpu_flags::7"=>"mce", "cpu_flags::8"=>"cx8", "cpu_flags::9"=>"apic", "cpu_flags::10"=>"sep", "cpu_flags::11"=>"mtrr", "cpu_flags::12"=>"pge", "cpu_flags::13"=>"mca", "cpu_flags::14"=>"cmov", "cpu_flags::15"=>"pat", "cpu_flags::16"=>"pse36", "cpu_flags::17"=>"clflush", "cpu_flags::18"=>"dts", "cpu_flags::19"=>"mmx", "cpu_flags::20"=>"fxsr", "cpu_flags::21"=>"sse", "cpu_flags::22"=>"sse2", "cpu_flags::23"=>"ss", "cpu_flags::24"=>"syscall", "cpu_flags::25"=>"nx", "cpu_flags::26"=>"pdpe1gb", "cpu_flags::27"=>"rdtscp", "cpu_flags::28"=>"lm", "cpu_flags::29"=>"constant_tsc", "cpu_flags::30"=>"arch_perfmon", "cpu_flags::31"=>"pebs", "cpu_flags::32"=>"bts", "cpu_flags::33"=>"nopl", "cpu_flags::34"=>"xtopology", "cpu_flags::35"=>"tsc_reliable", "cpu_flags::36"=>"nonstop_tsc", "cpu_flags::37"=>"cpuid", "cpu_flags::38"=>"pni", "cpu_flags::39"=>"pclmulqdq", "cpu_flags::40"=>"monitor", "cpu_flags::41"=>"vmx", "cpu_flags::42"=>"ssse3", "cpu_flags::43"=>"fma", "cpu_flags::44"=>"cx16", "cpu_flags::45"=>"pcid", "cpu_flags::46"=>"sse4_1", "cpu_flags::47"=>"sse4_2", "cpu_flags::48"=>"x2apic", "cpu_flags::49"=>"movbe", "cpu_flags::50"=>"popcnt", "cpu_flags::51"=>"tsc_deadline_timer", "cpu_flags::52"=>"aes", "cpu_flags::53"=>"xsave", "cpu_flags::54"=>"avx", "cpu_flags::55"=>"f16c", "cpu_flags::56"=>"rdrand", "cpu_flags::57"=>"hypervisor", "cpu_flags::58"=>"lahf_lm", "cpu_flags::59"=>"abm", "cpu_flags::60"=>"3dnowprefetch", "cpu_flags::61"=>"invpcid_single", "cpu_flags::62"=>"pti", "cpu_flags::63"=>"ssbd", "cpu_flags::64"=>"ibrs", "cpu_flags::65"=>"ibpb", "cpu_flags::66"=>"stibp", "cpu_flags::67"=>"tpr_shadow", "cpu_flags::68"=>"vnmi", "cpu_flags::69"=>"ept", "cpu_flags::70"=>"vpid", "cpu_flags::71"=>"ept_ad", "cpu_flags::72"=>"fsgsbase", "cpu_flags::73"=>"tsc_adjust", "cpu_flags::74"=>"bmi1", "cpu_flags::75"=>"avx2", "cpu_flags::76"=>"smep", "cpu_flags::77"=>"bmi2", "cpu_flags::78"=>"invpcid", "cpu_flags::79"=>"avx512f", "cpu_flags::80"=>"avx512dq", "cpu_flags::81"=>"rdseed", "cpu_flags::82"=>"adx", "cpu_flags::83"=>"smap", "cpu_flags::84"=>"clflushopt", "cpu_flags::85"=>"clwb", "cpu_flags::86"=>"avx512cd", "cpu_flags::87"=>"avx512bw", "cpu_flags::88"=>"avx512vl", "cpu_flags::89"=>"xsaveopt", "cpu_flags::90"=>"xsavec", "cpu_flags::91"=>"xsaves", "cpu_flags::92"=>"arat", "cpu_flags::93"=>"pku", "cpu_flags::94"=>"ospke", "cpu_flags::95"=>"md_clear", "cpu_flags::96"=>"flush_l1d", "cpu_flags::97"=>"arch_capabilities", "os_family"=>"RedHat", "osarch"=>"x86_64", "mem_total"=>9681, "swap_total"=>1999, "biosversion"=>"6.00", "productname"=>"VMware Virtual Platform", "manufacturer"=>"VMware, Inc.", "biosreleasedate"=>"11/12/2020", "uuid"=>"ccfb3c42-d47a-fa4b-7967-25535c7d40e2", "serialnumber"=>"VMware-42 3c fb cc 7a d4 4b fa-79 67 25 53 5c 7d 40 e2", "virtual"=>"VMware", "ps"=>"ps -efHww", "osrelease_info::0"=>8, "osmajorrelease"=>8, "osfinger"=>"CentOS Stream-8", "path"=>"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin", "systempath::0"=>"/usr/local/sbin", "systempath::1"=>"/usr/local/bin", "systempath::2"=>"/usr/sbin", "systempath::3"=>"/usr/bin", "pythonexecutable"=>"/usr/bin/python3", "pythonpath::0"=>"/usr/bin", "pythonpath::1"=>"/usr/lib64/python36.zip", "pythonpath::2"=>"/usr/lib64/python3.6", "pythonpath::3"=>"/usr/lib64/python3.6/lib-dynload", "pythonpath::4"=>"/usr/local/lib/python3.6/site-packages", "pythonpath::5"=>"/usr/lib64/python3.6/site-packages", "pythonpath::6"=>"/usr/lib/python3.6/site-packages", "pythonversion::0"=>3, "pythonversion::1"=>6, "pythonversion::2"=>8, "pythonversion::3"=>"final", "pythonversion::4"=>0, "saltpath"=>"/usr/lib/python3.6/site-packages/salt", "saltversion"=>"3005.1", "saltversioninfo::0"=>3005, "saltversioninfo::1"=>1, "zmqversion"=>"4.3.4", "redacted_sssd_ldap_uri"=>"ldaps://ldaps.redacted-corp.global", "redacted_auth_method"=>"sssd", "highstate_last_run"=>"20240304144052962609", "disks::0"=>"sr0", "disks::1"=>"sda", "redacted_app"=>"ES - SaltStack Enterprise Test", "redacted_app_id"=>"18410", "redacted_auditable"=>"False", "redacted_cloud_account_id"=>"account-676de612-6c6f-47fd-af7a-1137333fc921", "redacted_cloud_account_name"=>"ES - Managed Services", "redacted_cloud_id"=>"i-2733cd54-50c0-4e82-81d4-fbf0afa2d562", "redacted_cloud_image"=>"redacted-cloud-el8-base", "redacted_cloud_name"=>"sparrow-centos8-1", "redacted_cloud_network_tier_id"=>"", "redacted_cloud_network_tier_name"=>"", "redacted_cloud_platform"=>"prod", "redacted_cloud_project_id"=>"project-033564a8-fa66-4196-8bb6-94c22cd967ed", "redacted_cloud_project_name"=>"ES - SaltStack Enterprise Test", "redacted_cloud_subproject_id"=>"subproject-e4f69a61-56e9-4dde-9420-03a99d1e9426", "redacted_cloud_subproject_name"=>"Minions", "redacted_datacenter"=>"na-central-kc", "redacted_environment"=>"sandbox", "redacted_fireeye_id"=>"33n6aQm9QhffF38cbBJFy7", "redacted_org"=>"redacted", "shell"=>"/bin/sh", "transactional"=>false, "efi"=>false, "efi-secure-boot"=>false, "lvm::rootvg::0"=>"homevol", "lvm::rootvg::1"=>"logvol", "lvm::rootvg::2"=>"rootvol", "lvm::rootvg::3"=>"swap", "lvm::rootvg::4"=>"tmpvol", "lvm::rootvg::5"=>"vartmpvol", "username"=>"root", "groupname"=>"root", "pid"=>3385598, "gid"=>0, "uid"=>0, "zfs_support"=>false, "zfs_feature_flags"=>false, "redacted_cloud_tags::workload_type"=>"DEFAULT", "redacted_servicenow_id"=>"5db41f6a1b88fd1cb384c950604bcb07", "redacted_patching_group"=>"3rd-tues-10pm-4am", "redacted_cloud_environment"=>"dev", "redacted_cloud_metadata::content::name"=>"sparrow-centos8-1", "redacted_cloud_metadata::content::id"=>"i-2733cd54-50c0-4e82-81d4-fbf0afa2d562", "redacted_cloud_metadata::content::patchingGroup"=>"3rd-tues-10pm-4am", "redacted_cloud_metadata::content::subproject::id"=>"subproject-e4f69a61-56e9-4dde-9420-03a99d1e9426", "redacted_cloud_metadata::content::subproject::name"=>"Minions", "redacted_cloud_metadata::content::subproject::environment"=>"dev", "redacted_cloud_metadata::content::subproject::auditable"=>false, "redacted_cloud_metadata::content::project::name"=>"ES - SaltStack Enterprise Test", "redacted_cloud_metadata::content::project::managed"=>false, "redacted_cloud_metadata::content::account::id"=>"account-676de612-6c6f-47fd-af7a-1137333fc921", "redacted_cloud_metadata::content::account::name"=>"ES - Managed Services", "redacted_cloud_metadata::content::cloudApplicationId"=>"18410", "redacted_cloud_metadata::content::posixGroup"=>"6u6jd2641zrpigssypnwaigd-967edN", "redacted_cloud_metadata::content::serviceNowClass"=>"cmdb_ci_linux_server", "redacted_cloud_metadata::content::serviceNowId"=>"5db41f6a1b88fd1cb384c950604bcb07", "redacted_cloud_metadata::content::lastPatchedDate"=>"2024-01-17 04:13:53.0", "redacted_cloud_metadata::content::lastOsPatchedDate"=>"2023-08-21 12:53:06.0", "redacted_cloud_metadata::content::scheduledForPatching"=>true, "redacted_cloud_metadata::content::enterpriseDatabase"=>false, "redacted_cloud_metadata::content::enterpriseCluster"=>false, "redacted_master_pair_value"=>1, "redacted_master_pair_count"=>4, :_timestamp=>2024-03-04 17:56:46.507402778 +0000, :_type=>"foreman_salt"}}
---
classes:
- motd
parameters:
  foreman_config_groups: []
  puppetmaster: ''
  foreman_env: production
  foreman_hostname: 10-222-10-5
  foreman_fqdn: 10-222-10-5.redacted-corp.cloud
  root_pw:
  foreman_subnets: []
  foreman_interfaces:
  - ip: 10.222.10.5
    ip6:
    mac: 00:50:56:bc:7f:c3
    name: 10-222-10-5.redacted-corp.cloud
    attrs: {}
    virtual: false
    link: true
    identifier: eth0
    managed: true
    primary: true
    provision: true
    subnet:
    subnet6:
    tag:
    attached_to:
    type: Interface
  location: na-central-kc
  location_title: na-central-kc
  organization: Solutions Engineering
  organization_title: Solutions Engineering
  domainname: redacted-corp.cloud
  owner_name: Admin User
  owner_email: ro...@redacted-corp.cloud
  ssh_authorized_keys: []
  foreman_users:
    admin:
      firstname: Admin
      lastname: User
      mail: ro...@redacted-corp.cloud
      description: ''
      fullname: Admin User
      name: admin
      ssh_authorized_keys: []
  host_packages: ''
  host_registration_insights: false
  host_registration_remote_execution: true
  vendor_name: foreman
  test: test
  remote_execution_ssh_keys:
  - ssh-rsa <redacted>
    forema...@10-222-76-237.redacted-corp.cloud
  remote_execution_ssh_user: root
  remote_execution_effective_user_method: sudo
  remote_execution_connect_by_ip: false
  salt_master: 10-222-76-237.redacted-corp.cloud
  saltenv: __env__
environment: __env__

Im beginning to mess around with: /usr/lib/python3.6/site-packages/salt/pillar/puppet.py in an attempt to debug what its actually getting hung up on, but no luck yet.

If there is anything else needed, please let me know.  

Thanks so much. 

[Phipps, Thomas]

you haven't actually put that through a yaml processor yet i take it. cause it blows up. and like the error says. it isn't parsable by YAML.

here is the yamllint output for it. [doesn't even parse enough to be able to formulate output.

```

output.yaml
  1:1       warning  missing document start "---"  (document-start)
  1:2       error    syntax error: expected the node content, but found ':' (syntax)
  1:43      error    too many spaces before colon  (colons)
  1:44      error    cannot infer indentation: unexpected token  (indentation)
  1:1106    error    too few spaces after comma  (commas)
  1:9976    error    too many spaces before colon  (colons)
  1:9977    error    cannot infer indentation: unexpected token  (indentation)
  1:10026   error    too many spaces before colon  (colons)
  1:10027   error    cannot infer indentation: unexpected token  (indentation)
  1:81      error    line too long (10050 > 80 characters)  (line-length)
  4:1       error    wrong indentation: expected at least 1  (indentation)
  14:3      error    wrong indentation: expected 4 but found 2  (indentation)
  53:3      error    wrong indentation: expected 4 but found 2  (indentation)

```

the first warning about missing document start can be ignored. but the rest can not. that output is not valid yaml.

have you tried using https://docs.saltproject.io/en/latest/ref/pillar/all/salt.pillar.foreman.html instead.

To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/771d48c2-39a6-4e84-9c3f-29176b1a4bcan%40googlegroups.com.

[JS]

I didnt run the facts block through a linter because I didnt think salt would be looking at that.  Considering the `puppet.py` file says:

```

#!/usr/bin/python
# -*- coding: utf-8 -*-

"""
Execute an unmodified puppet_node_classifier and read the output as YAML. The YAML data is then directly overlaid onto the minion's Pillar data.
"""

import logging
import salt.utils.yaml

# Don't "fix" the above docstring to put it on two lines, as the sphinx
# autosummary pulls only the first line for its description.

# Set up logging

log = logging.getLogger(__name__)
log.critical('Just doing some testing')


def ext_pillar(minion_id, pillar, command):  # pylint: disable=W0613
    """
    Execute an unmodified puppet_node_classifier and read the output as YAML
    """

    try:
        log.critical('try block')
        data = salt.utils.yaml.safe_load(__salt__['cmd.run'
                ]('{} {}'.format(command, minion_id)))
        log.critical(data)
        print data
        return data['parameters']
    except Exception:
                       # pylint: disable=broad-except
        log.critical('YAML data from %s failed to parse', command)
        return {}
```

Which is returning the `parameters` block, which is correctly styled yaml.  No, I have not seen that foreman ext_pillar yet.  Wow, thanks.  Going to look at it, but still wondering whats going on, considering what Im doing is exactly how Foreman states to do this.  

[JS]

Bless you @why....  Just spent 15 mins getting the foreman ext_pillar setup, and its working fine.  Why foreman themselves dont state to use this, is beyond me. 

[Phipps, Thomas]

so it isn't justa block that needs to be correctly styled yaml it has to be the whole thing.

```

  data = salt.utils.yaml.safe_load(__salt__['cmd.run'
                ]('{} {}'.format(command, minion_id)))

```

this was the line causing you problems. the output from foreman-node was NOT valid yaml. that line took the output as is and put the whole thing into the yaml loader. not just parts of it but all of the output.

I ran the entire output you pasted into a yamllinter and the results said it wasn't valid yaml. which was the original problem. the error was spot on that it couldn't parse YAML.

but i am glad you got it going. with the foreman ext_pillar. much better and less intense solution all around anyway.

To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/9e1c3ce3-40c7-4a2e-ab8b-cf8861abfc5an%40googlegroups.com.

[JS]

Thanks again.  However, Ive now discovered if using the ext_pillar of foreman, you dont get any of the salt information (salt_env, salt_master, classes). So that renders it only somewhat helpful.  I guess its back to the drawing board.  Apparently Ill need to edit the scripts to remove the first block of facts, to make the yaml correct.  

[JS]

Solution:  Im an idiot.  I had edited the foreman-node script many weeks back, with additional `puts` in it to see what it was doing.  I didnt realize that the ext_pillar system for salt/puppet was literally reading the exact output of the script.  I changed the file back, and all works now.  But hey, I sure learned a lot!