Rackspace and AWX

261 views
Skip to first unread message

Larry Singer

unread,
Sep 28, 2017, 7:26:13 PM9/28/17
to awx-p...@googlegroups.com

Hi,

I am trying to get AWX going with our Rackspace cloud. I already have Ansible & Tower scripts running for our cloud. At the moment I have two issues:

  1. I cannot provision new VMs because pyrax is not installed. This is a Python module. How do I add this to AWX?
  2. I cannot get an inventory. I can see the openstack inventory plugin, but I can't see any documentation on how to configure this for Rackspace. Should I be be using this plugin or a custom script?

Larry Singer

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au

Matthew Jones

unread,
Sep 28, 2017, 9:45:50 PM9/28/17
to Larry Singer, AWX Project
Howdy Larry,

At one time we did support Rackspace inventory and modules directly (in downstream Tower). This was deprecated in our last major release and was removed several months ago from our mainline development branch. We did this because we actually found it difficult to maintain the shared dependencies between pyrax and openstack.

It might be totally possible to support these but you'll need to update the the container image and rebuild (in the form of updating the requirements files). In the process of deprecating this the inventory source was removed. If you solve the dependency issues then you can import the inventory script into the custom inventory script system and add a username/password credential source for it.

If you can get it to work let us know... pyrax gave us a whole lot of grief.



--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/63de1727-3471-68cb-709c-4e306ef8f51a%40smarthealth.com.au.
For more options, visit https://groups.google.com/d/optout.



--
Matt Jones
Principal Software Engineer
Ansible Tower

Larry Singer

unread,
Sep 28, 2017, 10:23:15 PM9/28/17
to Matthew Jones, AWX Project

Hi Matthew,

Thank you for your reply.

I am happy to try to use the openstack inventory and Ansible modules. I can't find any doco on how to configure these. Are you able to point me in the right direction?

Larry Singer

Larry Singer

unread,
Sep 29, 2017, 1:24:56 AM9/29/17
to awx-p...@googlegroups.com

I am getting stack traces when running the openstack plugin:

2017-09-29 03:44:25,668 INFO awx.main.commands.inventory_import Updating inventory 2: Rackspace
2017-09-29 03:44:25,681 DEBUG awx.main.commands.inventory_import Using system install of ansible-inventory CLI: /usr/bin/ansible-inventory 2
017-09-29 03:44:25,681 INFO awx.main.commands.inventory_import Reading Ansible inventory source: /usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py
Traceback (most recent call last): File "/usr/bin/awx-manage", line 9, in <module> load_entry_point('awx==1.0.0.561', 'console_scripts', 'awx-manage')()
 File "/usr/lib/python2.7/site-packages/awx/__init__.py", line 107, in manage execute_from_command_line(sys.argv)
 File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line utility.execute()
 File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute self.fetch_command(subcommand).run_from_argv(self.argv)
 File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv self.execute(*args, **cmd_options)
 File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute output = self.handle(*args, **options)
 File "/var/lib/awx/venv/awx/lib/python2.7/site-packages/django/core/management/base.py", line 661, in handle return self.handle_noargs(**options)
 File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 979, in handle_noargs self.is_custom)
 File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 234, in load_inventory_source is_custom=is_custom).load()
 File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 179, in load data = self.command_to_json(base_args + ['--list'])
 File "/usr/lib/python2.7/site-packages/awx/main/management/commands/inventory_import.py", line 163, in command_to_json self.method, proc.returncode, stdout, stderr))
RuntimeError: ansible-inventory failed (rc=4) with stdout:

stderr:
ERROR! Could not parse inventory source /usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py with available plugins:
Plugin script failed: Inventory script (/usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py) had an execution error: Error fetching server list on devstack:: 

Plugin ini failed: /usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py:48: Expected key=value host variable assignment, got: argparse

I think this is because the Openstack inventory plugin imports packages os_client_config and shade and these are installed in the Docker container. I am not having any success in adding these either manually through the container shell, or via the install script.

To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au

Matthew Jones

unread,
Sep 29, 2017, 7:42:07 AM9/29/17
to Larry Singer, AWX Project
os-client-config is installed both in our AWX environment:


and in the ansible execution environment:


So you don't need to install it by hand... this looks like it could be some other issue.


-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/1aed7e17-5903-74cb-937a-13dbf800acdd%40smarthealth.com.au.

For more options, visit https://groups.google.com/d/optout.

Matthew Jones

unread,
Sep 29, 2017, 7:42:31 AM9/29/17
to Larry Singer, AWX Project
If you try to run the inventory script from inside of the container what does it look like?

Larry Singer

unread,
Oct 3, 2017, 12:24:03 AM10/3/17
to Matthew Jones, AWX Project

Matthew,

I run  sudo docker exec -it awx_task /bin/bash to get into the container, then:
[root@awx awx]# /usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py


Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py", line 60, in <module>
    import os_client_config
ImportError: No module named os_client_config

Is this running the wrong install of Python? I remember seeing something about running embedded python somewhere, but I can't find any doco on it.

Larry

Matthew Jones

unread,
Oct 3, 2017, 7:45:00 AM10/3/17
to Larry Singer, AWX Project
you'll need to activate the virtualenv:

```
$ source /var/lib/awx/venv/awx/bin/activate
$ /usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py
```

Larry Singer

unread,
Oct 3, 2017, 8:51:01 PM10/3/17
to Matthew Jones, AWX Project

Matthew,

I got the plugin running inside the container, but it is not running from the inventory. This appears to be related to Issue https://github.com/ansible/awx/issues/268 . This points me at the code I need to look at, which appears to only support devstack. I'll see what I can do to make this work.

Larry

Larry Singer

unread,
Oct 4, 2017, 12:26:08 AM10/4/17
to AWX Project

Matthew,

I have been trying to trace the execution of this by inserting logger statements. This is my understanding so far:

The Inventory Source refresh in the UI calls class AnsibleInventoryLoader in /awx/main/management/commands/inventory_import.py. This runs the command

    /usr/bin/ansible-inventory -i /usr/lib/python2.7/site-packages/awx/plugins/inventory/openstack.py --list

An environment variable OS_CLIENT_CONFIG_FILE points to a temporary file with the settings to get the inventory. (I need to change the contents of this file to support Rackspace.)

The class RunInventoryUpdate in /awx/main/tasks.py prepares the temporary settings file.

I cannot see RunInventoryUpdate being called.

I have changed the data that is output by RunInventoryUpdate (/usr/lib/python2.7/site-packages/awx/main/tasks.py in Docker container awx_task), but the temporary settings file is not changing when I refresh the inventory.

Can you tell me where RunInventoryUpdate (or some other code) is preparing the temporary settings file?

Larry

To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Matthew Jones

unread,
Oct 4, 2017, 8:09:00 AM10/4/17
to Larry Singer, AWX Project
The task container *will* need to be restarted before it picks up the changes that you are making in tasks.py

You might consider using the development environment rather than the production environment when working with this: https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md


-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/7935af0f-34cc-6cfc-90d4-4c4c8bd2f035%40smarthealth.com.au.

For more options, visit https://groups.google.com/d/optout.

Larry Singer

unread,
Oct 4, 2017, 6:56:15 PM10/4/17
to awx-p...@googlegroups.com

Matthew,

Thanks, restarting the container worked. I'll see if I can work out what needs be changed to support this in the codebase.

Larry

To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Larry Singer

unread,
Oct 5, 2017, 1:57:08 AM10/5/17
to awx-p...@googlegroups.com

Hi,

I would like some guidance on the best way to implement some changes to support Rackspace inventories using the openstack inventory plugin. This can be done without major changes to the codebase. The method build_private_data() in RunInventoryUpdate in awx/main/tasks.py needs to be changed to be able to generate a file that supports Rackspace (and retain backward compatibility for devstack). The file needs to contain the rax data as shown in awx/plugins/inventory/openstack.yml rather than the devstack data.

To do this we need to be able to tell the difference between a devstack inventory and a Rackspace inventory. I have identified a few ways this could be done:

  • Create a new credentials type for Rackspace and supply these to the inventory source.
  • Reuse the OpenStack credentials type: Username, Password and Project fields would remain the same, Host field would be "rackspace" to identify that these are Rackspace credentials, Domain Name field could be reused to hold region names.
  • Use variables on the inventory source to identify a Rackspace source: profile: rackspace, region_name: SYD

Thanks in advance.

Larry Singer


For more options, visit https://groups.google.com/d/optout.

Christopher Meyers

unread,
Oct 5, 2017, 2:43:46 PM10/5/17
to AWX Project
I'm a bit confused Larry. Do you want to get Rackspace Dynamic Inventory Script to work in Tower or OpenStack Dynamic Inventory Script?

FYI, this was the commit that removed support for Rackspace in Tower https://github.com/ansible/awx/commit/cfb633e8a643b0190fa07b6204b339a1d336cbb3
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.



--
Matt Jones
Principal Software Engineer
Ansible Tower

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au
--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au
--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.



--
Matt Jones
Principal Software Engineer
Ansible Tower



--
Matt Jones
Principal Software Engineer
Ansible Tower

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au



--
Matt Jones
Principal Software Engineer
Ansible Tower

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au
--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au
--
You received this message because you are subscribed to the Google Groups "AWX Project" group.

Larry Singer

unread,
Oct 5, 2017, 5:59:52 PM10/5/17
to awx-p...@googlegroups.com

Christopher,

I am trying to use terms from the AWX UI, rather than from Ansible Tower. The OpenStack inventory source (dynamic inventory) can support Rackspace if different parameters are passed to the inventory plugin. This is about the best way to get those parameters to the OpenStack plugin.

Larry

For more options, visit https://groups.google.com/d/optout.

Matthew Jones

unread,
Oct 6, 2017, 12:20:24 PM10/6/17
to Larry Singer, AWX Project
I reckon I'd prefer a standalone credential... but I will point out that we removed support for rax originally because of the dependency conflict. If you can make it work then I'm all ears.

To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/f06030b1-ba47-403a-896b-f2156e81590f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Larry Singer
Smart Health Solutions Pty Ltd
http://www.smarthealth.com.au

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/b7371313-4a4a-f9df-5623-511183398216%40smarthealth.com.au.

For more options, visit https://groups.google.com/d/optout.

Larry Singer

unread,
Oct 8, 2017, 7:34:45 PM10/8/17
to Matthew Jones, AWX Project

Matthew,

The old Rackspace inventory (and the ansible modules) used a package pyrax. This has been deprecated by Rackspace. The openstack inventory plugin in AWX does allow access to Rackspace without any extra packages. So we should be able to support Rackspace without Pyrax.

I will have a go at a separate credential for Rackspace.

Larry

Matthew Jones

unread,
Oct 8, 2017, 9:59:47 PM10/8/17
to Larry Singer, AWX Project
Excellent, carry on then!

To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/344e474c-e128-1767-f791-613f377691d1%40smarthealth.com.au.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages