Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Offline EE testing?

321 views
Skip to first unread message

Howard Jones

unread,
Apr 12, 2021, 10:04:56 AM4/12/21
to awx-p...@googlegroups.com
The various presentations I've seen about Execution Environments talk
about the benefits for testing: you can run your playbooks locally in
the same environment that AWX does. That sounded cool, so I got
ansible-builder to build my environment (eventually!) and push it to our
Docker registry.

Now - how do I get ansible-runner to actually use it? I set my
runner/env/settings to use process_isolation, and gave it my container
image, but it still seems to be running locally with bwrap. e.g. "az
account list" run from the playbook gives my locally-configured list of
accounts.

runner/env/settings:
   container_image: my-awx-ee-az-cli-helm
   process_isolation: true

ansible-runner run --playbook test-playbook.yaml runner -v

The only example I could find was
https://github.com/AlanCoding/awx-ee-test which also has
"process_isolation_executable: docker" but if I include that, docker
complains about command-line options that are meant for bwrap
(--die-with-parent).

How is it supposed to be done? Enabling our users to test their
awx-bound playbooks before committing would be very useful!


Vincent Rubiolo

unread,
Nov 26, 2021, 9:12:38 AM11/26/21
to AWX Project
Hi Howard,

I am looking into previous posts on this ML and found yours.

Did you eventually succeed in doing what you wanted? I am interested in doing the same but have difficulties finding which arguments to pass to ansible-runner.

Ideally I would like to know how it is invoked by AWX so that I can replicate that on my host.

Thanks for any info you might have !

Vincent

Wei-Yen Tan

unread,
Nov 26, 2021, 9:14:15 AM11/26/21
to Vincent Rubiolo, AWX Project
In your project/ template  you can choose the ee image 
From: awx-p...@googlegroups.com <awx-p...@googlegroups.com> on behalf of Vincent Rubiolo <vincent.rub...@gmail.com>
Sent: Saturday, November 27, 2021 3:12:37 AM
To: AWX Project <awx-p...@googlegroups.com>
Subject: [awx-project] Re: Offline EE testing?
 
--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/2d88f671-0ce0-4b9a-9034-8acc92016ab9n%40googlegroups.com.

Vincent Rubiolo

unread,
Nov 30, 2021, 8:04:04 AM11/30/21
to AWX Project
Hi Weiye,

Thanks for your answer. In my case, I am interested in using ansible-runner to run the jobs on the commandline with the same environment than on AWX.
I have added more information and my findings in the relevant thread at https://groups.google.com/g/awx-project/c/vYBIfSEq3vo

Vincent

Wei-Yen Tan

unread,
Nov 30, 2021, 8:09:04 AM11/30/21
to Vincent Rubiolo, AWX Project
If I understsnd correctly you want to loop through a job template with different inventory on command line. You can Use awx-cli to call the awx api. You can then script and call awx-cli in your workflow. You are then free to use any language you want to. 

Awx-cli can be installed as so 


Another way is it use workflow templates 
Sent: Wednesday, December 1, 2021 2:04:04 AM
To: AWX Project <awx-p...@googlegroups.com>
Subject: Re: [awx-project] Re: Offline EE testing?
 

Vincent Rubiolo

unread,
Nov 30, 2021, 8:35:06 AM11/30/21
to AWX Project
Weiye,

Thanks for taking the time to chime into the discussion (and for documenting your findings on your blog btw, I found it very useful too!).

I have replied to your comments inline below.

On Tuesday, November 30, 2021 at 2:09:04 PM UTC+1 weiye...@gmail.com wrote:
If I understsnd correctly you want to loop through a job template with different inventory on command line.

No, this is not exactly what I would like to do.

What I am doing is integrating existing Ansible jobs into AWX. They currently run (without AWX) on each developer machine (each developer might have a different version of Ansible and environment, etc). So I am bringing the developer environment into a custom EE on AWX. This isolates the dependencies required to run our playbooks and allows them to run into AWX.

What I want to do is to be able to run on my machine, without AWX, a given job that ran on AWX. I understand this is possible by passing the correct commandline arguments to ansible-runner (especially the EE) after cloning our playbook project repository. This is because with an EE, everything necessary to run the playbook is contained in the EE image itself.

Running locally would:
  1. allow any dev in our team to run the job on their machine (as they did previously). This would ease the transition to AWX.
  2. permit anyone to run the job locally on their machine with the guarantee that it behaves the same than on AWX.
  3. allow any new developer to ramp up and easily run the jobs w/o suffering environment setup issues (as all is contained in the EE).
  4. considerably shorten the debugging cycle when jobs run on AWX. For instance, right now, when something fails in AWX, I have to rerun the entire playbook since I have not found a way to use ansible-playbook --start-at with the UI to only rerun the part of the job which failed.
The ansible-runner doc explicitely mentions that ability to run locally w.r.t Execution Environments at https://ansible-runner.readthedocs.io/en/stable/execution_environments/:
+++
Execution Environments are meant to be a consistent, reproducible, portable, and sharable method to run Ansible Automation jobs in the exact same way on 
your laptop as they are executed in Ansible AWX. This aids in the development of automation jobs and Ansible Content that is meant to be run in Ansible AWX, 
+++
 
You can Use awx-cli to call the awx api. You can then script and call awx-cli in your workflow. You are then free to use any language you want to. 

Awx-cli can be installed as so 


Yes, I already have the CLI but this requires the job to exist on AWX. The job will also be run on AWX as opposed to on my own machine.

The CLI might allow to pass "--start-at" though, I have to check for that.
 


Another way is it use workflow templates 

That I have not used yet. My understanding was that was a tool to chain multiple playbooks together (which I have not yet done).

Thanks again for your assistance!

Vincent 

Wei-Yen Tan

unread,
Nov 30, 2021, 8:46:48 AM11/30/21
to Vincent Rubiolo, AWX Project
Taken from the same page have you Read the standalone command line tool?



I have not primarily used the runner in this way so I am not quite sure if I can be of much help. Sorry about that 
Sent: Wednesday, December 1, 2021 2:35:06 AM

To: AWX Project <awx-p...@googlegroups.com>
Subject: Re: [awx-project] Re: Offline EE testing?
--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.

Vincent Rubiolo

unread,
Nov 30, 2021, 9:12:39 AM11/30/21
to AWX Project
On Tuesday, November 30, 2021 at 2:46:48 PM UTC+1 weiye...@gmail.com wrote:
Taken from the same page have you Read the standalone command line tool?


Yes, this is exactly what I am looking at. What I need help on is how/where to find the actual arguments used within AWX to build a matching ansible-runner commandline that I can run locally.

The Runner doc mentions it's the Python interface which is used within AWX so not directly ansible-runner. I am looking at the logs on my nodes to see if I can find that.


I just need to find a was to pass down the AWX_CLEANUP_PATHS=False setting within the AWX containers so that it triggers the non-cleanup code.
 

I have not primarily used the runner in this way so I am not quite sure if I can be of much help. Sorry about that 

No worries. Thanks for being there and replying in any case! I will keep you informed on how it goes.

Vincent

Alan Rominger

unread,
Nov 30, 2021, 3:49:42 PM11/30/21
to Vincent Rubiolo, AWX Project
Vincent is onto the path which I would recommend. There are a bit too many moving parts to easily replicate on your own, but I would like to simplify it.

So the best option is to set AWX_CLEANUP_PATHS to False, run a job and then copy the folder back to your machine. This might not yet totally work because of some pipes waiting for a read (but I don't remember the result from my last experimentation), and those can require some manual tweaking. I would like to allow re-running it out-of-the-box if it doesn't work now.

The awx-operator has some instructions for how to change a setting before you deploy it, but I haven't tested this method.

Alan
github: AlanCoding


--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/74911a15-b0b0-4128-81a6-389a3d485171n%40googlegroups.com.

Vincent Rubiolo

unread,
Dec 1, 2021, 5:57:59 PM12/1/21
to AWX Project
Hi Alan,

Thanks for confirming this is the right path, this really helps! I will keep on checking AWX_CLEANUP_PATHS (via the operator, I see someone else has asked a similar question on how to change settings for an existing operator deployment).

In the meantime, through a stuck job, I have found that Runner is launched in Receptor mode on the job pod which is started. Indeed, the started pod runs ansible-runner worker --private-data-dir=/runner , as described at https://ansible-runner.readthedocs.io/en/stable/remote_jobs/#. I think this is related to the pipe issues you mentioned as in this case one ansible-runner streams the job while the other pends on its payload before running it.

This was through a stuck job so I could login on the node and see the /runner directory which indeed contains what is needed here. I think AWX_CLEANUP_PATHS would be close to what I need. Ideally I could publish the associated instructions so that a developer can rerun locally a job which failed in AWX, with the same settings.

Vincent
Reply all
Reply to author
Forward
0 new messages