my test started to fail at commit 09adc188e

[Atsushi Togo]

Dear aiidausers,

I found that my tests of the aiida-phonoxpy plugin started to fail
updating aiida-core.

(aiida-core commit hash)
09adc188e -> Fail
073b43108 -> OK

I don't understand well how to write plugin tests, and mainly
mimicking the tests from aiida-quantumespresso, though my tests were
not updated for a long time. So probably my way of writing it is
outdated. Or simply, I wrote tests in wrong way, but it was working
fortuitously. I would like to provide more information but I don't see
what information I should provide. If someone could kindly help me,
would you tell me necessary information that may tell the point.

On aiida-phonoxpy repository, if I run a test, the error message is
like below. Many tests fail similarly. This is just in case for some
information.

% pytest -sv tests/workflows/test_phono3py.py::test_initialize_with_dataset
================================== test session starts
===================================
platform linux -- Python 3.9.13, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
-- /home/togo/.miniforge/envs/demo/bin/python3.9
cachedir: .pytest_cache
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False
min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10
warmup=False warmup_iterations=100000)
rootdir: /home/togo/aiida-demo/aiida-phonoxpy
plugins: anyio-3.6.2, rerunfailures-9.1.1, timeout-1.4.2,
benchmark-3.4.1, datadir-1.4.1, cov-2.10.1, regressions-2.4.1,
asyncio-0.16.0
collected 1 item

tests/workflows/test_phono3py.py::test_initialize_with_dataset FAILED

======================================== FAILURES
========================================
______________________________ test_initialize_with_dataset
______________________________

generate_workchain = <function
generate_workchain.<locals>._generate_workchain at 0x14f3f80f4670>
generate_structure = <function
generate_structure.<locals>._generate_structure at 0x14f3f81c5550>
generate_displacement_dataset = <function
generate_displacement_dataset.<locals>._generate_displacement_dataset
at 0x14f3f81c5430>
generate_settings = <function
generate_settings.<locals>._generate_settings at 0x14f3f81c54c0>

def test_initialize_with_dataset(
generate_workchain,
generate_structure,
generate_displacement_dataset,
generate_settings,
):
"""Test of Phono3pyWorkChain.initialize() using NaCl data.

`displacement_dataset` is generated using phono3py.

"""
structure = generate_structure()
settings = generate_settings()
dataset = generate_displacement_dataset(structure_id="NaCl-fc3")
inputs = {"structure": structure, "settings": settings}
> wc = generate_workchain("phonoxpy.phono3py", inputs)

tests/workflows/test_phono3py.py:24:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _
tests/workflows/conftest.py:41: in _generate_workchain
process = instantiate_process(runner, process_class, **inputs)
../aiida-core/aiida/engine/utils.py:63: in instantiate_process
process = process_class(runner=runner, inputs=inputs)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/state_machine.py:194:
in __call__
inst.transition_to(inst.create_initial_state())
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/state_machine.py:339:
in transition_to
self.transition_failed(initial_state_label, label, *sys.exc_info()[1:])
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/processes.py:1003:
in transition_failed
raise exception.with_traceback(trace)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/state_machine.py:324:
in transition_to
self._enter_next_state(new_state)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/state_machine.py:384:
in _enter_next_state
self._fire_state_event(StateEventHook.ENTERING_STATE, next_state)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/state_machine.py:300:
in _fire_state_event
callback(self, hook, state)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/processes.py:329:
in <lambda>
lambda _s, _h, state: self.on_entering(cast(process_states.State, state)),
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/processes.py:683:
in on_entering
call_with_super_check(self.on_create)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/utils.py:29:
in call_with_super_check
wrapped(*args, **kwargs)
../aiida-core/aiida/engine/processes/process.py:396: in on_create
super().on_create()
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/base/utils.py:16:
in wrapper
wrapped(self, *args, **kwargs)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/processes.py:747:
in on_create
result = self.spec().inputs.validate(self._parsed_inputs)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/ports.py:650:
in validate
validation_error = self.validate_ports(port_values, breadcrumbs_local)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/ports.py:724:
in validate_ports
validation_error = port.validate(port_values.pop(name,
UNSPECIFIED), breadcrumbs)
../../.miniforge/envs/demo/lib/python3.9/site-packages/plumpy/ports.py:666:
in validate
message = self.validator(port_values_clone, self) # pylint:
disable=not-callable
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _

inputs = {'metadata': <AttributesFrozendict {'store_provenance': True,
'call_link_label': 'CALL', 'dry_run': False, 'options':
...ronment_variables_double_quotes': False, 'prepend_text': '',
'append_text': '', 'parser_name': 'phonoxpy.phono3py'}>}>}
ctx = <aiida.engine.processes.ports.PortNamespace object at 0x14f3f7d15460>

def validate_calc_job(inputs: Any, ctx: PortNamespace) ->
Optional[str]: # pylint: disable=too-many-return-statements
"""Validate the entire set of inputs passed to the `CalcJob`
constructor.

Reasons that will cause this validation to raise an
`InputValidationError`:

* No `Computer` has been specified, neither directly in
`metadata.computer` nor indirectly through the `Code` input
* The specified computer is not stored
* The `Computer` specified in `metadata.computer` is not the
same as that of the specified `Code`
* No `Code` has been specified and no `remote_folder` input
has been specified, i.e. this is no import run

:return: string with error message in case the inputs are invalid
"""
try:
ctx.get_port('code')
ctx.get_port('metadata.computer')
except ValueError:
# If the namespace no longer contains the `code` or
`metadata.computer` ports we skip validation
return None

remote_folder = inputs.get('remote_folder', None)

if remote_folder is not None:
# The `remote_folder` input has been specified and so this
concerns an import run, which means that neither
# a `Code` nor a `Computer` are required. However, they
are allowed to be specified but will not be explicitly
# checked for consistency.
return None

code = inputs.get('code', None)
> computer_from_code = code.computer
E AttributeError: 'NoneType' object has no attribute 'computer'

../aiida-core/aiida/engine/processes/calcjobs/calcjob.py:70: AttributeError
----------------------------------- Captured log setup
-----------------------------------
DEBUG pgsu:__init__.py:144 Trying to connect via "psycopg2"...
==================================== warnings summary
====================================
tests/workflows/test_phono3py.py::test_initialize_with_dataset
/home/togo/aiida-demo/aiida-core/aiida/manage/configuration/settings.py:59:
UserWarning: Creating AiiDA configuration folder
`/tmp/pytest-of-togo/pytest-59/config0/.aiida`.
warnings.warn(f'Creating AiiDA configuration folder `{path}`.')

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================================ short test summary info
=================================
FAILED tests/workflows/test_phono3py.py::test_initialize_with_dataset
- AttributeError:...
============================== 1 failed, 1 warning in 1.59s
==============================

Thanks,

--
Atsushi Togo

[Sebastiaan Huber]

Dear Togo,

Thanks for the report and taking the time to bisect the problem to a commit in `aiida-core`.
I have looked at the commits and don't see how the changes could be responsible for the sudden failure.
The commit you highlight just changed some dependencies, that shouldn't affect the behavior that is failing in the test.

Nevertheless, I think it should be easy to solve.
The test is creating/launching a CalcJob with some dummy inputs, that fail the validation.
The reason being that no input is provided for the `code`, and so `code.computer` fails with an AttributeError.
Usually, therefore, in these types of tests you want to provide a (fake) `Code` input.
This can be easily done using the `aiida_local_code_factory` fixture.
Here is how you can update your test:

    def test_initialize_with_dataset(
        generate_workchain,
        generate_structure,
        generate_displacement_dataset,
        generate_settings,
	aiida_local_code_factory
    ):
        """Test of Phono3pyWorkChain.initialize() using NaCl data.

        `displacement_dataset` is generated using phono3py.

        """
	code = aiida_local_code_factory('phonoxpy.phono3py', 'python') # Define the code entry point and executable. Executable can just be a dummy, unless the test _actually_ runs the code
        structure = generate_structure()
        settings = generate_settings()
        dataset = generate_displacement_dataset(structure_id="NaCl-fc3")
        inputs = {"structure": structure, "settings": settings, "code": code}

Now the inputs should contain a `Code` input and the exception should disappear.

Hope that helps,

Sebastiaan

[Atsushi Togo]

Dear Sebastiaan,

Thanks for your quick response and your help.
I tried your instruction about `code`, and also `fixture_code`
similarly written as aiida.quantumespresso, but it still fails with
almost the same error message. I guess that the elements of
`inputs`are somehow lost before reaching `validate_calc_job`. If the
highlighted commit is not clearly the reason, I do something very
strange in writing my workchain which is probably beyond your
imagination. I will investigate it by myself further.

Can I ask one thing related to this? Can my `generate_workchain`
fixture found at the link below be updated using a fixture provided
from aiida-core? Please ignore the `mock_*` fixtures given as
parameters of `generate_workchain`.

https://github.com/atztogo/aiida-phonoxpy/blob/b3e419c9cfd1f152c717ffb579ad9f187f4d41dc/tests/workflows/conftest.py#L6-L45

Thank you very much,

Togo

> --
> AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
>
> Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
> ---
> You received this message because you are subscribed to the Google Groups "aiidausers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/8bf87e2a-7a23-3e2f-09fb-9de2e184e071%40sphuber.net.



--
Atsushi Togo

[Sebastiaan Huber]

Dear Togo,

There is no built-in fixture that replaces your `generate_workchain`, but the implementation looks correct.
Having a second look at your code, I think I noticed the problem.
You are launching the workchain that wraps the `Phonopy3Calculation`, but it does so in the namespace `phono3py`.
This means the inputs have also be passed in that namespace.
So your test input dictionary should be something like:

inputs = {
    "phono3py": {

        structure": structure, "settings": settings, "code": code
    }
}

That might fix it.

Regards,

Sebastiaan

[Atsushi Togo]

Dear Sebastiaan,

Yes, the `expose_inputs` was the origin of the problem. Although, your
suggestion didn't work, finally if I commented out the following line:
https://github.com/atztogo/aiida-phonoxpy/blob/b3e419c9cfd1f152c717ffb579ad9f187f4d41dc/aiida_phonoxpy/workflows/phono3py.py#L24-L29
the test and real calculation went through.

Thanks again for taking your time!

Togo

> To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/346f9e85-73bf-fa49-8446-f5c4f6f74847%40sphuber.net.



--
Atsushi Togo