Not detecting installed plugin when using verdi run

[Nathan Keilbart]

Hello everyone,

I recently had to redo my python environment when they upgraded the OS on the server I run AiiDA. We went from TOSS3 to TOSS4 and I've been running it on the Quartz supercomputer at LLNL which is an intel machine. I am currently on v2.3.1 and the develop branch of aiida-vasp plus using a personal version of aiida-nwchem.

Before the update I had aiida-vasp running with no issues but since I've redone my environment I haven't been able to get it working as it states:

2023-06-10 00:48:06 [641186 | ERROR]: Traceback (most recent call last):
  File "/usr/WS1/keilbart/AiiDA/aiida-core/aiida/engine/persistence.py", line 48, in load_object
    module = importlib.import_module(module_name)
  File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'aiida_vasp.workchains.vasp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/WS1/keilbart/AiiDA/aiida-core/aiida/manage/external/rmq/launcher.py", line 90, in _continue
    result = await super()._continue(communicator, pid, nowait, tag)
  File "/usr/workspace/keilbart/envs/aiida_quartz/lib/python3.10/site-packages/plumpy/process_comms.py", line 604, in _continue
  File "/usr/workspace/keilbart/envs/aiida_quartz/lib/python3.10/site-packages/plumpy/persistence.py", line 58, in unbundle
  File "/usr/workspace/keilbart/envs/aiida_quartz/lib/python3.10/site-packages/plumpy/persistence.py", line 446, in load
  File "/usr/WS1/keilbart/AiiDA/aiida-core/aiida/engine/persistence.py", line 50, in load_object
    raise ImportError(f"module '{module_name}' from identifier '{identifier}' could not be loaded")
ImportError: module 'aiida_vasp.workchains.vasp' from identifier 'aiida_vasp.workchains.vasp:VaspWorkChain' could not be loaded

When I run verdi plugin list aiida.calculations or aiida.workflows the vasp related items are listed and I'm able to load aiida_vasp while using verdi shell as well. When I combine all the information into a script and attempt to submit it with verdi run I get the outputted error above in the report related to the pk. I've attempted to reinstall all files and redo my python environment. Are there any suggestions to try and figure out what's going on? I've reached out to the aiida-vasp developers as well and they weren't too sure so I decided to check in here. We both initially thought the daemon needed to be restarted which I've done but that did not fix the issue. I am seeing this for both aiida-vasp and aiida-nwchem now. I had aiida-nwchem working but in trying to debug the aiida-vasp situation I updated my aiida-core version to the latest and now it is saying the same issue. Thanks.

Nathan

[Nathan Keilbart]

Alright so it seems that I checked that there weren't other daemons running on other login nodes but it seems there were some running on my current node that had somehow become decoupled as well. Killing those processes solved the issue. No need for help anymore. :)