In a pickle...

[Chris Jefferies]

The error message:

ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object 

I was accessing a new collection, icinga.icinga, and it complained so I decided to update...  huge mistake.  Now I can't keep track of what I have and why this error is happening.  I'm guessing I need to route out the 2.9 version but wanted some advice before i go...  into the valley below.  ;)

Thanks for any tips...

Chris.

The current state of things:

# ansible --version
ansible 2.9.0
  config file = /Users/user/Documents/code/gitlab/devops/ansible/base/ansible.cfg
  configured module search path = ['/Users/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.9.13 (main, Aug  7 2022, 01:33:23) [Clang 13.1.6 (clang-1316.0.21.2.5)]
# python3 -m pip show ansible
Name: ansible
Version: 6.2.0
Summary: Radically simple IT automation
Home-page: https://ansible.com/
Author: Ansible, Inc.
Author-email: in...@ansible.com
License: GPLv3+
Location: /Users/user/Library/Python/3.9/lib/python/site-packages
Requires: ansible-core
Required-by:

# ~/Documents/code/gitlab/devops/ansible/base | python3 -m pip show ansible-core
Name: ansible-core
Version: 2.13.2
Summary: Radically simple IT automation
Home-page: https://ansible.com/
Author: Ansible, Inc.
Author-email: in...@ansible.com
License: GPLv3+
Location: /Users/user/Library/Python/3.9/lib/python/site-packages
Requires: cryptography, jinja2, packaging, PyYAML, resolvelib
Required-by: ansible

The Details:

I was running a simple ping just to test the accessibility of the hosts in my hosts file.  The same happens on any hosts file...

# ansible -vvv all -m ping -i hosts/myhosts.yml
ansible 2.9.0
  config file = /Users/user/Documents/code/gitlab/devops/ansible/base/ansible.cfg
  configured module search path = ['/Users/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.9.13 (main, Aug  7 2022, 01:33:23) [Clang 13.1.6 (clang-1316.0.21.2.5)]
Using /Users/user/Documents/code/gitlab/devops/ansible/base/ansible.cfg as config file
host_list declined parsing /Users/user/Documents/code/gitlab/devops/ansible/base/hosts/hosts_deva.yml as it did not pass its verify_file() method
script declined parsing /Users/user/Documents/code/gitlab/devops/ansible/base/hosts/hosts_deva.yml as it did not pass its verify_file() method
Parsed /Users/user/Documents/code/gitlab/devops/ansible/base/hosts/hosts_deva.yml inventory source with yaml plugin
META: ran handlers
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
the full traceback was:

Traceback (most recent call last):
  File "/usr/local/bin/ansible", line 123, in <module>
    exit_code = cli.run()
  File "/usr/local/lib/python3.9/site-packages/ansible/cli/adhoc.py", line 165, in run
    result = self._tqm.run(play)
  File "/usr/local/lib/python3.9/site-packages/ansible/executor/task_queue_manager.py", line 240, in run
    play_return = strategy.run(iterator, play_context)
  File "/usr/local/lib/python3.9/site-packages/ansible/plugins/strategy/linear.py", line 310, in run
    self._queue_task(host, task, task_vars, play_context)
  File "/usr/local/lib/python3.9/site-packages/ansible/plugins/strategy/__init__.py", line 343, in _queue_task
    worker_prc.start()
  File "/usr/local/lib/python3.9/site-packages/ansible/executor/process/worker.py", line 96, in start
    return super(WorkerProcess, self).start()
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'module' object

[Walter Rowe]

Do you have any outdated pip modules?

% pip list -o

Perhaps one/some of them need updating?

[Matt Martz]

That bug was addressed in v2.9.1 as part of https://github.com/ansible/ansible/pull/63591

I wouldn't suggest simply upgrading to 2.9.1.  2.9.1 was released over 2.5 years ago, and the 2.9 release is now end of life.  You should consider upgrading to at least ansible-core 2.12 (ansible 5.0)

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/26b3196b-47c5-42c4-985c-8df749fded77n%40googlegroups.com.

--

Matt Martz
@sivel
sivel.net

[Chris Jefferies]

Thank you.

What I have, and might have been lost in too much detail, is.

# ansible --version
ansible 2.9.0

ansible python module location = /usr/local/lib/python3.9/site-packages/ansible

executable location = /usr/local/bin/ansible

# python3 -m pip show ansible-core
Name: ansible-core
Version: 2.13.2

Location: /Users/user/Library/Python/3.9/lib/python/site-packages

# python3 -m pip show ansible
Name: ansible
Version: 6.2.0

Location: /Users/user/Library/Python/3.9/lib/python/site-packages

I thought 2.9 was gone, but it persists.  How do I inform my generic call to "ansible" to reference 2.13 vs 2.9?  I think I need to replace "/usr/local/bin/ansible".

Again, thank you for the tips.

[Matt Martz]

It looks like you need to follow the documentation from https://packaging.python.org/en/latest/tutorials/installing-packages/#installing-to-the-user-site

That URL is linked from our install guide:

> You may need to perform some additional configuration before you are able to run Ansible. See the Python documentation on installing to the user site for more information.

Effectively you need to run:

python3 -m site --user-base

Add `/bin` on the end, and insert that into your `PATH` environment variable.  Generally it will be `$HOME/.local/bin`.  That new bin path should be added before `/usr/local/bin`

To remove the old version, you likely just need to do a `python3 uninstall ansible`, just ensure the paths it informs you it will remove are within `/usr/local`

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/d46fef31-a1e9-48c1-affe-3380c8c5e84bn%40googlegroups.com.

[Chris Jefferies]

Ack.  Worse than I thought...  Here's what I see when executing some of the commands from the page you referenced (thank you).

On MacOS - Monterey 

# python -m site --user-base
/Users/user/Library/Python/2.7  <<==== ACK!

# python3 --version
Python 3.9.13

# python --version
Python 2.7.18

# python3 -m pip --version
pip 22.2.2 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

# echo $PATH (broken out to make it easier to see - I don't recall ever managing this PATH variable)
/usr/local/sbin:
/usr/local/bin/python3.9/bin:
/usr/local/bin:
/usr/bin:
/bin:
/usr/sbin:
/sbin:
/usr/local/go/bin:
/opt/X11/bin:
/Library/Frameworks/Mono.framework/Versions/Current/Commands

When I follow the path to python in /usr/local/bin/, I see the following:
python3 -> ../Cellar/python@3.9/3.9.13_2/bin/python3
python3.7 -> ../../../Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7
python3.9 -> ../Cellar/python@3.9/3.9.13_2/bin/python3.9

And here:

# ls -la /Users/user/Library/Python/
total 0
drwx------    6 user  staff   192 Jun 29 16:45 .
drwx------@ 104 user  staff  3328 Mar 11 09:25 ..
drwx------    4 user  staff   128 May 23  2019 2.7
drwx------    4 user  staff   128 Dec  2  2020 3.7
drwx------    4 user  staff   128 Aug  4  2021 3.8
drwx------    4 user  staff   128 Aug  9  2021 3.9

I think I used brew when I installed 3.9.  

Also, to confirm, `python3 uninstall ansible` will ask before uninstalling?

Thank you,

[Chris Jefferies]

The Fix - It came down to "actually" removing (deleting) the 2.9.0 version which was a hard file references as:  /usr/local/bin/ansible  

I then added a sym-link as follows:

/usr/local/bin/ansible -> ../Cellar/ansible/6.2.0/bin/ansible

All seems back to normal.

If an application was installed with a command, it should uninstall with the same command.  Perhaps there's a purge I missed.  Seems uninstall leaves some bits of flotsam...

Thanks for your help...

[Nico Kadel-Garcia]

On Wed, Aug 10, 2022 at 7:08 PM Chris Jefferies <chr...@gmail.com> wrote:
>
> The Fix - It came down to "actually" removing (deleting) the 2.9.0 version which was a hard file references as: /usr/local/bin/ansible
>
> I then added a sym-link as follows:
>
> /usr/local/bin/ansible -> ../Cellar/ansible/6.2.0/bin/ansible
>
> All seems back to normal.
>
> If an application was installed with a command, it should uninstall with the same command. Perhaps there's a purge I missed. Seems uninstall leaves some bits of flotsam...
>
> Thanks for your help...

What command created this hardlink?

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/039d7d59-536c-4598-b1ed-756157ce8d26n%40googlegroups.com.