Scaling AWX

[Jeb Baxley]

Hey Everyone,

I'm attempting to scale AWX, but am constantly running into road blocks.  I have a couple of questions and hope someone from the community can point me in the right direction:

1) Is there a scaling guide for AWX?  I'm sure a number of shops have run into this previously, so any best practices that the community has come up with would be awesome!

2) Is it possible to select a Container group for a job_template launch using the awx cli?  or maybe a curl command?  I'm attempting to isolate portions of my inventory to run on specific container groups so that we can scale.  I might have to put in more work, but it'll be pretty simple from a python scripting perspective.

That's it for now.  Thanks!

[Elijah DeLee]

On Mon, Jul 10, 2023 at 12:59 PM Jeb Baxley <jeb.b...@gmail.com> wrote:

Hey Everyone,

I'm attempting to scale AWX, but am constantly running into road blocks.  I have a couple of questions and hope someone from the community can point me in the right direction:

1) Is there a scaling guide for AWX?  I'm sure a number of shops have run into this previously, so any best practices that the community has come up with would be awesome!

 

I'm not aware of a scaling guide specifically for AWX.

For the downstream Red Hat offerings there exists:

https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/2.4/html/red_hat_ansible_automation_platform_performance_considerations_for_operator_based_installations/index?extIdCarryOver=true&sc_cid=701f2000001OH6uAAG 

And also useful, the reference architecture for AAP on Openshift:

https://access.redhat.com/documentation/en-us/reference_architectures/2023/html-single/deploying_ansible_automation_platform_2_on_red_hat_openshift/index#before_you_start 

 There are scatterings of many relevant settings/options for scaling the cluster in different ways. What are some of the road blocks you are running into? Maybe could link to more specific docs or use to help formulate what you are requesting.

 

2) Is it possible to select a Container group for a job_template launch using the awx cli?  or maybe a curl command?  I'm attempting to isolate portions of my inventory to run on specific container groups so that we can scale.  I might have to put in more work, but it'll be pretty simple from a python scripting perspective.

You can assign a container group to a job template or in most recent versions this is now a "prompt on launch" compatible attribute, so you could at launch time indicate the container group. Note that container groups in the API are referred to as instance groups, they are just a specific type of instance group. See https://github.com/ansible/awx/pull/12875 and https://docs.ansible.com/automation-controller/4.3/html/controllercli/reference.html#awx-job-templates-launch

Think it would be something along the lines of 

awx job_templates launch --instance_groups 1 7

that **should** launch job template id 7 with instance group id 1

granted you need to be on a recent enough version of CLI and awx to support this.

That's it for now.  Thanks!

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/7517027c-d51c-4984-92bb-a00d87770317n%40googlegroups.com.

--

Elijah DeLee, RHCSA

He/Him/His

associate manager

Performance and Scale, Red Hat Ansible Automation Platform

Red Hat

kde...@redhat.com   

[Jeb Baxley]

Hey thanks for answering!

—

There are scatterings of many relevant settings/options for scaling the cluster in different ways. What are some of the road blocks you are running into? Maybe could link to more specific docs or use to help formulate what you are requesting.

—

Big road blocks;

1) can’t deploy an instance group with the included ansible code, when I do get it deployed by adjusting it to work with Ubuntu, it won’t launch any jobs.  The certs come over fine, I don’t think that’s the issue.  Could just be that I need to update the operator/AWX.

2) awx job_templates launch --instance_groups 1 7 just doesn’t work.  I’ve tried a number of times, it could be an issue with the version of AWX, as the awx cli is newer, but it just keeps erroring out. 

Thanks for getting back!  And I’ll check out the material you’ve sent.  

My ask is simple, I’d would be amazing to have a TOWER / ANSIBLE variable that defined where the AWX job is run, which instance group.  That way, I can define it for each host or for a group of hosts.  

Assigning it for an inventory is great, unless you have a nearly endless inventory that is geographically dispersed.  Or in my case, just an incredibly massive number of hosts in separate regions that running with job slices isn’t anywhere close to enough. 

I do have one final question, how far have you scaled AWX?  Is RedHat looking at AWX/Ansible Automation Platform to be a better solution to scale than utilizing ansible-pull ? 

Thanks again!

Sent from my iPhone

On Jul 10, 2023, at 20:18, Elijah DeLee <kde...@redhat.com> wrote:



You received this message because you are subscribed to a topic in the Google Groups "AWX Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/awx-project/kPfHHic43fw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/CAHmbr-_8DhttEQCZA8HJn8GMejG3SYdq6V%2BVO0cmU0%3DNhHz%3Dsw%40mail.gmail.com.

[Jeb Baxley]

a little bit more.  

the hope / goal is to have a openshift/k8s/k3s cluster in each region/dc that can host jobs from awx and thus run the ansible much much much closer to the nodes.  ideally, I would have an instance group as close as a single switch away from the nodes.    

I want to get as close to running ansible-pull as I can with AWX. 

[AWX Project]

in case you haven't seen this link, here is a good place to read up on scaling AAP. Many of the concepts should apply to AWX on k8s as well

https://docs.ansible.com/automation-controller/latest/html/administration/performance.html

AWX Team

[Jeb Baxley]

thanks, reading up on this.  but also looking to update my awx instance, very worried about destroying this as it was deployed by another employee.  so trying to figure out awx backups and restoration.

[AWX Project]

Yes be sure to do a backup first

AWX Team

[Jeb Baxley]

ok, feature request, awx backup mechanism! lol

[AWX Project]

We do have mechanism in place for this, you can view that here:  https://github.com/ansible/awx-operator/blob/devel/roles/backup/README.md