Foreman (Katello) and separate salt smart proxy - SSL issue

127 views
Skip to first unread message

Dylan Baars

unread,
Mar 22, 2016, 1:14:06 PM3/22/16
to Salt-users
Hi all,

I posted this to the foreman users group before finding this (salt-users group) - I hope it's OK to cross-post this
Original post:
https://groups.google.com/forum/#!topic/foreman-users/B3DlDpP3ImY

Hi all,

I am attempting to integrate a new Saltstack server and an existing Katello installation. I have Katello 2.4 installed, which is running foreman 1.10.2. I have been following http://theforeman.org/plugins/foreman_salt/4.0/index.html#1.ForemanSalt4.0Manual to configure things.

On my salt master I am running 2015.8.7-1.el7. It has foreman-proxy 1.10.2-1.e17 installed
On my Katello server, it has tfm-rubygem-foreman_salt installed as below

tfm-rubygem-foreman_salt.noarch      4.0.1-1.fm1_10.el7      @foreman-plugins
tfm-rubygem-hammer_cli_foreman_salt.noarch
tfm-rubygem-hammer_cli_foreman_salt-doc.noarch

I have a working cherrypy salt-API setup, running on port 8000. To prove that, after logging in and getting a token using curl (using the zsaltuser as seen further down), I can run the below

curl -ksi https://wellsaltdev.niwa.local:8000 -H "Accept: application/x-yaml" -H "X-Auth-Token: "780173c4e02c9ee4b18a32abe77c904e112727d3"" -d client='local' -d tgt='*' -d fun='test.ping'

HTTP/1.1 200 OK
Content-Length: 72
Access-Control-Expose-Headers: GET, POST
Cache-Control: private
Vary: Accept-Encoding
Server: CherryPy/3.2.2
Allow: GET, HEAD, POST
Access-Control-Allow-Credentials: true
Date: Mon, 21 Mar 2016 22:10:48 GMT
Access-Control-Allow-Origin: *
Content-Type: application/x-yaml
Set-Cookie: session_id=780173c4e02c9ee4b18a32abe77c904e112727d3; expires=Tue, 22 Mar 2016 08:10:48 GMT; Path=/

return:
- wellminiondev.niwa.local: true
  wellsaltdev.niwa.local: true

My /etc/salt/foreman.yaml is below

:proto: https
:host: wellkatellodev.niwa.local
:port: 443
:ssl_ca: "/var/lib/puppet/ssl/certs/ca.pem"
:ssl_cert: "/var/lib/puppet/ssl/certs/wellsaltdev.niwa.local.pem"
:ssl_key: "/var/lib/puppet/ssl/private_keys/wellsaltdev.niwa.local.pem"
:timeout:  10
:salt:  /usr/bin/salt
:upload_grains:  true

As per the documentation, I have configured CherryPy in /etc/salt/master as below

# Salt-API configuration
rest_cherrypy:
  port: 8000
  ssl_crt: /var/lib/puppet/ssl/certs/wellsaltdev.niwa.local.pem
  ssl_key: /var/lib/puppet/ssl/private_keys/wellsaltdev.niwa.local.pem


external_auth:
    zsaltuser:
      - .*

In /etc/foreman-proxy.settings.d/salt.yml configured the API-related settings

---
:enabled: true
:autosign_file: /etc/salt/autosign.conf
:salt_command_user: root
:use_api: true
:api_auth: ldap
:api_username: zsaltuser
:api_password: removed

and in /etc/foreman-proxy/settings.yml

:ssl_certificate: /var/lib/puppet/ssl/certs/wellsaltdev.niwa.local.pem
:ssl_ca_file: /var/lib/puppet/ssl/certs/ca.pem
:ssl_private_key: /var/lib/puppet/ssl/private_keys/wellsaltdev.niwa.local.pem
:trusted_hosts:
- wellkatellodev.niwa.local
:forward_verify: true
:daemon: true
:bind_host: '*'
:http_port: 9000
:https_port: 9001
:virsh_network: default
:log_level: DEBUG

To be sure, rebooted both servers. With the above configuration, if I login to the Katello website, Infrastructure > Smart Proxies, I can add the Salt smart proxy via HTTP - i.e. http://wellsaltdev.niwa.local:9000, however if I try and use HTTPS and "Refresh features" I get

Unable to communicate with the proxy: ERF12-2530 [ProxyAPI::ProxyException]: Unable to detect features ([Errno::EACCES]: Permission denied - connect(2)) for proxy https://wellsaltdev.niwa.local:9001/features and Please check the proxy is configured and running on the host.

No extra messages in the logs on either server

In the foreman-proxy logs on the server, when I start the proxy (systemctl start foreman-proxy) I get

D, [2016-03-22T11:32:31.223710 #7199] DEBUG -- : TCPServer.new(0.0.0.0, 9000)
D, [2016-03-22T11:32:31.224020 #7199] DEBUG -- : TCPServer.new(::, 9000)
W, [2016-03-22T11:32:31.224189 #7199]  WARN -- : TCPServer Error: Address already in use - bind(2)
D, [2016-03-22T11:32:31.224315 #7199] DEBUG -- : Rack::Handler::WEBrick is mounted on /.
I, [2016-03-22T11:32:31.224457 #7199]  INFO -- : WEBrick::HTTPServer#start: pid=7199 port=9000
D, [2016-03-22T11:32:31.224585 #7199] DEBUG -- : TCPServer.new(0.0.0.0, 9001)
D, [2016-03-22T11:32:31.224768 #7199] DEBUG -- : TCPServer.new(::, 9001)
W, [2016-03-22T11:32:31.224883 #7199]  WARN -- : TCPServer Error: Address already in use - bind(2)
I, [2016-03-22T11:32:31.225973 #7199]  INFO -- :

but checking using lsof, only the foreman-proxy is using the ports

[root@wellsaltdev foreman-proxy]# lsof -wni tcp:9000
COMMAND  PID          USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
ruby    7199 foreman-proxy   11u  IPv4  57748      0t0  TCP *:cslistener (LISTEN)
[root@wellsaltdev foreman-proxy]# lsof -wni tcp:9001
COMMAND  PID          USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
ruby    7199 foreman-proxy   12u  IPv4  58567      0t0  TCP *:etlservicemgr (LISTEN)

The /var/log/foreman-proxy/proxy.log also shows the Puppet certificate being loaded

I, [2016-03-22T11:32:31.225973 #7199]  INFO -- :
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 4 (0x4)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=Puppet CA: wellkatellodev.niwa.local
        Validity
            Not Before: Mar 13 04:18:19 2016 GMT
            Not After : Mar 13 04:18:19 2021 GMT
        Subject: CN=wellsaltdev.niwa.local
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
                Modulus:
                    00:98:bd:54:ee:ef:3c:4c:ed:08:81:5c:d8:09:bf:
                    28:3a:09:d8:3c:f2:e3:13:8a:10:a2:50:d6:4c:8b:
                    0a:62:2c:4e:20:b2:51:fc:34:c7:a3:80:22:11:95:
                    62:62:49:16:af:9e:34:5c:44:4d:2a:6e:56:22:00:
                    34:70:ea:56:6f:cb:a0:7e:4e:19:36:13:87:5d:42:
                    31:fb:f4:4c:fe:d4:a6:59:46:36:8a:8e:6f:e4:ef:
                    63:1b:8a:f8:d3:e6:98:a0:81:93:de:4e:81:d8:23:
                    3f:14:da:b5:11:6a:09:89:83:19:9e:0d:f1:72:a2:
                    d9:6c:c1:51:a8:9d:55:cc:7a:e4:57:70:05:5f:58:
                    ae:05:a6:85:02:3d:d4:82:73:b5:f8:75:24:bc:76:
                    26:09:90:18:3e:69:dc:87:6e:88:1e:c2:8b:44:a2:
                    2a:5b:2f:55:72:a7:f9:0c:6b:13:62:69:af:b7:b8:
                    f8:8d:2c:e7:cb:61:ab:5f:81:bd:25:8d:38:df:d5:
                    5b:ba:08:1e:49:ff:c2:54:29:61:31:dd:06:16:33:
                    e7:3a:4c:3a:cc:b1:ae:4d:b0:0b:f9:20:df:21:f3:
                    05:f5:59:eb:6f:6a:14:6a:27:59:8d:7b:14:b2:09:
                    21:1c:c1:92:79:c6:00:f6:29:bf:2b:a7:76:fc:5b:
                    34:fe:c2:18:01:49:ac:bc:93:dd:83:72:13:01:7e:
                    7a:d5:c5:8e:bb:9b:da:44:cd:51:b4:ee:58:cb:5e:
                    25:53:73:0c:f0:15:8f:ec:aa:c9:a3:71:97:63:77:
                    e7:ab:b3:c1:bd:b6:af:2d:d5:e3:aa:94:eb:81:3c:
                    87:25:93:41:8e:02:67:22:aa:ec:f3:b6:61:e5:61:
                    d4:aa:bd:3b:d0:fd:80:92:75:f1:b1:dc:32:7c:79:
                    8a:4d:0c:d4:90:d9:c6:81:cf:c3:a2:b7:26:7a:d3:
                    ce:ce:c8:52:f8:d8:0c:18:0e:2c:fc:47:f1:55:23:
                    e8:e4:ba:f8:ab:97:78:ab:99:eb:93:8c:1d:41:16:
                    9b:59:07:f1:11:de:ae:aa:a0:87:4b:b5:99:dc:3c:
                    30:80:0b:7e:5f:22:eb:ed:92:50:a6:d5:83:ce:94:
                    8e:29:d1:fd:5d:b7:d2:a4:79:46:dc:53:54:dc:2d:
                    20:b2:56:d3:cf:02:97:3c:02:d4:d4:36:b0:d3:68:
                    ba:b2:e7:11:67:ab:d0:1b:e7:b7:3f:21:ca:e2:03:
                    50:68:7f:3f:c3:07:46:1a:49:da:0e:0d:d4:7d:16:
                    5b:75:4a:36:42:4e:70:f3:79:64:27:77:34:a1:29:
                    58:ff:9e:9a:68:d2:ca:43:69:ee:db:5a:0f:45:d9:
                    80:e0:ff
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            Netscape Comment:
                Puppet Ruby/OpenSSL Internal Certificate
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: critical
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                7B:97:A4:69:AF:A1:08:F5:49:2A:FC:3D:02:32:2D:04:FF:D5:F5:BA
            X509v3 Authority Key Identifier:
                keyid:F7:A7:90:88:EC:2C:5A:C2:61:94:F7:83:E3:50:B0:66:C5:CD:D5:2D

    Signature Algorithm: sha256WithRSAEncryption
         61:25:a5:b6:3e:02:3d:4c:85:b5:e2:31:e9:03:ce:44:04:03:
         75:08:a2:2c:c8:c9:b1:c6:26:9c:42:9d:66:9d:64:17:19:bd:
         89:a3:1f:f5:02:67:05:8c:6b:9f:ef:e2:a7:34:93:fa:3b:d8:
         a7:29:3c:82:47:14:db:ef:57:34:a7:7f:52:50:05:16:ed:9e:
         de:34:4f:54:0c:8a:5d:d8:7e:d3:0a:f8:f0:36:37:4c:67:94:
         15:e6:5b:89:48:72:f5:05:79:f8:d0:26:bd:43:1f:6c:aa:7e:
         7b:fd:f3:cf:33:f1:02:6f:eb:2d:d9:e3:6f:ec:16:3e:ac:03:
         22:d7:d8:51:86:bd:84:37:94:be:32:c1:25:8f:d3:7c:89:3b:
         a9:d8:56:ca:5a:87:1a:76:38:30:be:23:e6:d5:ae:75:7c:a5:
         ca:2b:f1:82:31:ac:eb:86:06:cf:08:01:5a:b0:52:54:33:a5:
         8b:69:5e:cd:61:74:86:6d:75:7e:0e:d1:d3:57:9b:f7:6b:ed:
         d0:f9:d7:3f:a9:f0:ff:5c:8a:bf:a8:e7:fb:0a:75:fc:4f:a3:
         ec:a1:65:4d:b1:d1:2b:cb:3a:68:1a:e0:ff:ea:28:a0:e8:c2:
         39:cc:59:72:07:c9:cf:ae:94:cc:21:c5:ff:1a:21:9c:cc:4c:
         b5:73:5b:62:6d:23:6a:5c:56:da:13:f4:e9:bb:c4:c5:16:15:
         39:1e:49:d9:3a:b4:23:97:4e:0b:49:0b:37:c3:41:69:85:b5:
         17:aa:f9:0f:98:8d:8f:20:37:d9:a5:2e:ad:fd:c3:76:01:d3:
         25:2a:38:e6:68:96:81:2e:42:ff:72:a0:53:7a:fe:70:9a:54:
         8c:14:3a:ac:34:92:f8:01:ea:88:73:eb:e7:30:69:a6:5f:97:
         58:63:e9:06:f5:d6:32:b5:49:a0:63:ab:cf:2d:05:f1:79:f5:
         37:7d:71:b3:e1:9a:7d:58:f1:5c:f3:b8:f4:37:e9:5d:97:39:
         30:50:8c:a5:00:a7:52:63:db:9a:c3:24:46:c5:84:46:35:08:
         33:ef:fe:40:9d:6b:bc:62:0f:df:98:f7:51:65:aa:8a:de:ba:
         2c:f9:00:d4:16:82:56:c8:c6:07:3d:4d:78:73:a1:f5:69:a5:
         6a:25:f1:57:4b:1b:18:0f:99:ad:8c:b0:1f:87:f8:6a:95:9a:
         02:24:7e:3f:ab:cf:3a:5a:25:42:ce:25:cb:cc:c5:77:a1:8b:
         b7:bf:4f:11:e0:8c:e1:a4:62:25:94:17:58:b1:5f:03:87:f3:
         f6:7e:4f:fa:9a:d5:03:73:86:81:e0:97:9f:23:ed:3d:7a:4e:
         9c:17:78:1c:c9:bc:9d:46

D, [2016-03-22T11:32:31.226161 #7199] DEBUG -- : Rack::Handler::WEBrick is mounted on /.
I, [2016-03-22T11:32:31.226247 #7199]  INFO -- : WEBrick::HTTPServer#start: pid=7199 port=9001

Can anyone help figure out why I can't use SSL from the smart proxies page in Katello?

Thanks heaps :-)
Dylan


A possible spanner in the works, I have updated Katello to use our internal certificate authority for the web page (https://wellkatellodev.niwa.local) with this command, although this might be nothing:

katello-installer --certs-server-cert "/certs/wellkatellodev.niwa.local.crt"\
                      --certs-server-cert-req "/certs/wellkatellodev.niwa.local.csr"\
                      --certs-server-key "/certs/wellkatellodev.niwa.local.key"\
                      --certs-server-ca-cert "/certs/niwa_cacert.pem"\
                      --certs-update-server --certs-update-server-ca

Reply all
Reply to author
Forward
0 new messages