mysql_db and pymysql error

[Nick-H]

I am trying to move to python 3.6 on centos 7.8 and ansible 2.8.12 

Mysql_db module fails importing connections from PyMySQL pip module.

At least with recent versions ( 0.9 ,1.0.2 ) using 0.7 pass. 

the error seems to be related to a PEP recommendation E402 . 

As per PyMySQL doc , python3 is supported . 

here is my error: 

{

    "module_stdout": "",

    "module_stderr": "Traceback (most recent call last):\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1617903869.1782537-29212-149280496397207/AnsiballZ_mysql_db.py\", line 114, in <module>\n    _ansiballz_main()\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1617903869.1782537-29212-149280496397207/AnsiballZ_mysql_db.py\", line 106, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1617903869.1782537-29212-149280496397207/AnsiballZ_mysql_db.py\", line 49, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_mysql_db_payload_3i8jCj/__main__.py\", line 110, in <module>\n  File \"/tmp/ansible_mysql_db_payload_3i8jCj/ansible_mysql_db_payload.zip/ansible/module_utils/mysql.py\", line 33, in <module>\n  File \"/opt/venvs/py3_ansible28/lib/python3.6/site-packages/pymysql/__init__.py\", line 59, in <module>\n    from . import connections  # noqa: E402\n  File \"/opt/venvs/py3_ansible28/lib/python3.6/site-packages/pymysql/connections.py\", line 167\n    *,\n     ^\nSyntaxError: invalid syntax\n",

    "exception": "Traceback (most recent call last):\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1617903869.1782537-29212-149280496397207/AnsiballZ_mysql_db.py\", line 114, in <module>\n    _ansiballz_main()\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1617903869.1782537-29212-149280496397207/AnsiballZ_mysql_db.py\", line 106, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1617903869.1782537-29212-149280496397207/AnsiballZ_mysql_db.py\", line 49, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_mysql_db_payload_3i8jCj/__main__.py\", line 110, in <module>\n  File \"/tmp/ansible_mysql_db_payload_3i8jCj/ansible_mysql_db_payload.zip/ansible/module_utils/mysql.py\", line 33, in <module>\n  File \"/opt/venvs/py3_ansible28/lib/python3.6/site-packages/pymysql/__init__.py\", line 59, in <module>\n    from . import connections  # noqa: E402\n  File \"/opt/venvs/py3_ansible28/lib/python3.6/site-packages/pymysql/connections.py\", line 167\n    *,\n     ^\nSyntaxError: invalid syntax\n",

    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",

    "rc": 1,

    "_ansible_no_log": false,

    "changed": false

}

[Matt Martz]

If I had to guess, you are executing the module and subsequently pymysql with python2.  The paths seem to indicate a python3 virtualenv, but the error indicates that you are executing Python2, with potentially a PYTHONPATH pointing to the Python3 venv.

The error is in regards to the "keyword only marker" defined at:

https://github.com/PyMySQL/PyMySQL/blob/6ccbecc1a0dfd04065b081950d2d35b1dac0aaa8/pymysql/connections.py#L164-L166

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/c9a3c9a6-f31f-43fc-901f-e328d6d6b1een%40googlegroups.com.

--

Matt Martz
@sivel
sivel.net

[Nick-H]

Oh No.... I want to get rid of python2 for my ansible playbooks  :( been fighting this for weeks ... 

from ansible itself it seems to be running on python3 : 

ansible-playbook 2.8.12

config file = /var/lib/awx/tmp/.../project/ansible.cfg

configured module search path = ['/var/lib/awx/tmp/.../project/library']

ansible python module location = /opt/venvs/py3_ansible28/lib/python3.6/site-packages/ansible

executable location = /opt/venvs/py3_ansible28/bin/ansible-playbook

python version = 3.6.8 (default, Nov 16 2020, 16:55:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

Is that the KW marker '*' that make you find I run on python2 ? Is that not being used on python3 ?  

Now I don't really see how to make good use of python3 for ansible if the underlying system keeps using python2 , is there any good guideline I could use ?

thanks

[Matt Martz]

If you want a module to run with python3, you will generally need to set `ansible_python_interpreter` for that host to get it to use the interpreter you want.  We default and fallback to py2.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/1d2f2bc6-6434-429c-b840-30d596356bb2n%40googlegroups.com.

[Nick-H]

hmm ok but this runs on the controller ( awx ) , so I followed this doc :

https://docs.ansible.com/ansible/latest/reference_appendices/python_3_support.html#on-the-controller-side

Do I need to use the interpreter var in that case ? 

[Nick-H]

Now, I tried that in my ansible.cfg  and indeed it passed my errors , 

thank you Matt !

I am still a bit confused though , Why do we bother creating virtual environments if eventually ansible keeps trying to use a system-wide installed python ? 

really , why  is that ? 

thanks for enlightenment  

[Nick-H]

still doesn't make sense to me, or doc is not clear

My controller Tower/AWX :

 python3 installed on system

 virtual environment to run ansible installed as per the doc : python3 -m pip install ansible 

Yet, I see playbooks using python3 , but what is that usr/bin/python  ? 

Why is ansible still using python2 please ?

[Matt Martz]

The Python interpreter that you use to install ansible, is not necessarily the python used to execute modules, even when targeting localhost.

The controller code, and module code execute completely different, with different rules.

If you don't have `localhost` in your inventory, you can use something called implicit localhost, as described at https://docs.ansible.com/ansible/latest/inventory/implicit_localhost.html

But you must target that localhost explicitly in your play, or use `delegate_to: localhost` for that to work.

If you have `localhost` in your inventory, then you have to explicitly set the python interpreter to be used for that host, via `ansible_python_interpreter`

See also: https://docs.ansible.com/ansible/latest/reference_appendices/faq.html#how-do-i-handle-not-having-a-python-interpreter-at-usr-bin-python-on-a-remote-machine

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/87632da2-fcb1-4db6-85ea-2ec9206e2cedn%40googlegroups.com.

[Nick-H]

ok thanks, sorry but my tower/awx is not remote per say as it is the box that executes all playbooks , so it like my localhost  sure .

the link describe how to set the interpreter on remote hosts, which in my case are fine using their python2 ( legacy boxes) .

But if I define the interpreter on tower ansible.cfg (localhost)  in default section then   even remote hosts inherits that config . 

I d like to only run python3 on the controller (localhost)  itself  if that s doable . 

[Komal Suthar]

If you want to install the different version first install the python3 interpreter using yum then it bydefault comes with pip3 command so run pip3 install ansible. 

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/1ffd0881-eb45-47f7-8b84-b82fabc01930n%40googlegroups.com.