Setting PYTHONPATH during playbook execution

[Cody John]

I have some lookup plugins that import some custom python modules, and I'm gettting python errors during execution because it can't find them.   I need a way to set the PYTHONPATH environment variable prior to playbook execution, or somehow achieve the same result.  Any ideas?

[Matthew Jones]

This is going to be super tricky because we also isolate the process using bubblewrap, so your chance of gaining access to another interpreter is pretty small unless you disable that. This is generally not something we've supported in any way in the past.

If you need to run some lookup plugins you can put them alongside your playbooks and install what dependencies you need in the /var/lib/awx/venv/awx virtualenv.

On Mon, Sep 18, 2017 at 1:36 PM, Cody John <cody....@gmail.com> wrote:

I have some lookup plugins that import some custom python modules, and I'm gettting python errors during execution because it can't find them.   I need a way to set the PYTHONPATH environment variable prior to playbook execution, or somehow achieve the same result.  Any ideas?

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.
To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/1b7238a9-8ee0-4d1e-8ed5-ea1a3ae6db64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Jones

Principal Software Engineer

Ansible Tower

[Cody John]

Hmm, that doesn't really work in this case because we manage our custom python code in RPMs using yum, not pip, so it's not possible to install them in the virtualenv.

I noticed under Settings -> Jobs, there is a field named "EXTRA ENVIRONMENT VARIABLES" but it's disabled because it says its managed in a config file somewhere.   Would adding PYTHONPATH there help in this case?

On Monday, September 18, 2017 at 11:43:48 AM UTC-6, Matthew Jones wrote:

This is going to be super tricky because we also isolate the process using bubblewrap, so your chance of gaining access to another interpreter is pretty small unless you disable that. This is generally not something we've supported in any way in the past.

If you need to run some lookup plugins you can put them alongside your playbooks and install what dependencies you need in the /var/lib/awx/venv/awx virtualenv.

On Mon, Sep 18, 2017 at 1:36 PM, Cody John <cody....@gmail.com> wrote:

I have some lookup plugins that import some custom python modules, and I'm gettting python errors during execution because it can't find them.   I need a way to set the PYTHONPATH environment variable prior to playbook execution, or somehow achieve the same result.  Any ideas?

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/1b7238a9-8ee0-4d1e-8ed5-ea1a3ae6db64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matthew Jones]

maybe, you can find the settings files in the container. You are definitely on your own for that one though... I wouldn't expect it to work very well at all :)

To unsubscribe from this group and stop receiving emails from it, send an email to awx-project+unsubscribe@googlegroups.com.

To post to this group, send email to awx-p...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/b92af6af-297c-41fd-9760-73c3cbc6fda2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Mike Biancaniello]

So ... here's what I've done:

1. Modified installer/local_docker/tasks/main.yml to include an optional set of volumes.

2. Created installer/group_vars/all to specify my custom settings.

I'm then taking advantage of site .pth files to point python at the right set of libs.

This should allow someone to install python packages on a host machine that can be used by the AWX container.

Custom vars:

$ cat installer/group_vars/all.yml 

---

custom_ansible_dir: "/opt/ansible"

task_volumes:

- "/opt/pypath.pth:/usr/lib/python2.7/site-packages/pypath.pth"

- "/opt/custom_pylib:/opt/custom_pylib"

pth file:

$ cat /opt/pypath.pth 

/opt/custom_pylib

Custom tasks:

  

$ git diff installer/local_docker/tasks/main.yml

diff --git a/installer/local_docker/tasks/main.yml b/installer/local_docker/tasks/main.yml

index 7eba9d357..0a58baaec 100644

--- a/installer/local_docker/tasks/main.yml

+++ b/installer/local_docker/tasks/main.yml

@@ -190,6 +190,16 @@

       MEMCACHED_HOST: "memcached"

       MEMCACHED_PORT: "11211"

 

+- name: Set volume for custom ansible directory in task container

+  set_fact:

+    volumes_ansible_dir: 

+    - "{{ custom_ansible_dir }}:/etc/ansible"

+  when: custom_ansible_dir is defined

+

+- name: show me volumes

+  debug:

+    msg: "Mounting volumes: {{ task_volumes|default([])|list + volumes_ansible_dir|default([])|list }}"

+

 - name: Activate AWX Task Container

   docker_container:

     name: awx_task

@@ -198,6 +208,7 @@

     links: "{{ awx_task_container_links|list }}"

     user: root

     hostname: awx

+    volumes: "{{ task_volumes|default([])|list + volumes_ansible_dir|default([])|list }}"

     env:

       SECRET_KEY: "{{ awx_secret_key }}"

       DATABASE_NAME: "{{ pg_database }}"