PwBandStructureWorkChain and A real world workchain example

[Atsushi Togo]

Dear AiiDA users and developers,

I am in trouble running "2020 AiiDA tutorial week “Virtual Edition”"
Sec. 6 "A real world workchain example: electronic band structure". I
am not very sure if this is my problem or not. If this is an issue
that should be fixed, I would like to know the appropriate place that
I can raise an issue. The issue is summarized as below.

I use QuantumMobile v21.05.1 (thanks Chris for the update!). The
PwBandStructureWorkChain calculation crashes. I attach the log at the
end of this e-mail.

The versions of the aiida plugins are:
aiida-core 1.6.1 /home/max/codes/aiida-core
aiida-pseudo 0.6.0
aiida-quantumespresso 3.4.1

It seems the change in PwBandStructureWorkChain
if not isinstance(upf, UpfData)
->
if not isinstance(upf, UpfData) and not isinstance(upf, LegacyUpfData):

passes this issue. But after this tentative change, I have another issue.
------------
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/process_states.py",
line 230, in execute
result = self.run_fn(*self.args, **self.kwargs)
File "/home/max/codes/aiida-core/aiida/engine/processes/workchains/workchain.py",
line 214, in _do_step
finished, stepper_result = self._stepper.step()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/workchains.py",
line 299, in step
finished, result = self._child_stepper.step()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/workchains.py",
line 250, in step
return True, self._fn(self._workchain)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/workflows/pw/relax.py",
line 362, in results
if self.inputs.base.pw.parameters['CONTROL']['calculation'] != 'scf':
KeyError: 'calculation'
------------

There seems to be no key 'calculation' in this dictionary. I could
not inspect how the self.inputs.base.pw is set quickly, but I doubt if
this check is necessary or not.

I would appreciate it if you could give me any advice to understand this issue.

Best regards,

Togo


-------
The log is given below:

$ verdi run run_band.py
05/07/2021 04:47:08 AM <6750>
aiida.orm.nodes.process.workflow.workchain.WorkChainNode: [REPORT]
[2135|PwBandStructureWorkChain|setup_protocol]: running the workchain
with the "theos-ht-1.0" protocol
/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/utils/pseudopotential.py:102:
AiidaDeprecationWarning: this function is deprecated and will be
removed in `v4.0.0`.
warnings.warn('this function is deprecated and will be removed in
`v4.0.0`.', AiidaDeprecationWarning)
05/07/2021 04:47:09 AM <6750>
aiida.orm.nodes.process.workflow.workchain.WorkChainNode: [REPORT]
[2135|PwBandStructureWorkChain|on_except]: Traceback (most recent call
last):
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/process_states.py",
line 230, in execute
result = self.run_fn(*self.args, **self.kwargs)
File "/home/max/codes/aiida-core/aiida/engine/processes/workchains/workchain.py",
line 214, in _do_step
finished, stepper_result = self._stepper.step()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/workchains.py",
line 299, in step
finished, result = self._child_stepper.step()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/workchains.py",
line 250, in step
return True, self._fn(self._workchain)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/workflows/pw/band_structure.py",
line 147, in run_bands
'base': get_common_inputs(),
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/workflows/pw/band_structure.py",
line 131, in get_common_inputs
'pseudos': get_pseudos_from_dict(self.inputs.structure, known_pseudos),
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/utils/pseudopotential.py",
line 121, in get_pseudos_from_dict
raise ValueError(f'Node with UUID {uuid} is not a UpfData')
ValueError: Node with UUID daed0023-e341-4b10-8df1-d8dda12c3ad8 is not a UpfData

Traceback (most recent call last):
File "/home/max/.virtualenvs/aiida/bin/verdi", line 12, in <module>
sys.exit(verdi())
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/click/core.py",
line 829, in __call__
return self.main(*args, **kwargs)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/click/core.py",
line 782, in main
rv = self.invoke(ctx)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/click/core.py",
line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/click/core.py",
line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/click/core.py",
line 610, in invoke
return callback(*args, **kwargs)
File "/home/max/codes/aiida-core/aiida/cmdline/utils/decorators.py",
line 65, in wrapper
return wrapped(*args, **kwargs)
File "/home/max/codes/aiida-core/aiida/cmdline/commands/cmd_run.py",
line 147, in run
exec(compile(handle.read(), scriptname, 'exec',
dont_inherit=True), globals_dict) # yapf: disable # pylint:
disable=exec-used
File "run_band.py", line 11, in <module>
structure=structure_Al
File "/home/max/codes/aiida-core/aiida/engine/launch.py", line 41, in run
return runner.run(process, *args, **inputs)
File "/home/max/codes/aiida-core/aiida/engine/runners.py", line 266, in run
result, _ = self._run(process, *args, **inputs)
File "/home/max/codes/aiida-core/aiida/engine/runners.py", line 250, in _run
process_inited.execute()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/processes.py",
line 81, in func_wrapper
return func(self, *args, **kwargs)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/processes.py",
line 1166, in execute
return self.future().result()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/process_states.py",
line 230, in execute
result = self.run_fn(*self.args, **self.kwargs)
File "/home/max/codes/aiida-core/aiida/engine/processes/workchains/workchain.py",
line 214, in _do_step
finished, stepper_result = self._stepper.step()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/workchains.py",
line 299, in step
finished, result = self._child_stepper.step()
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/plumpy/workchains.py",
line 250, in step
return True, self._fn(self._workchain)
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/workflows/pw/band_structure.py",
line 147, in run_bands
'base': get_common_inputs(),
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/workflows/pw/band_structure.py",
line 131, in get_common_inputs
'pseudos': get_pseudos_from_dict(self.inputs.structure, known_pseudos),
File "/home/max/.virtualenvs/aiida/lib/python3.7/site-packages/aiida_quantumespresso/utils/pseudopotential.py",
line 120, in get_pseudos_from_dict
raise ValueError(f'Node with UUID {uuid} is not a UpfData')
ValueError: Node with UUID daed0023-e341-4b10-8df1-d8dda12c3ad8 is not a UpfData

--
Atsushi Togo

[Marnik Bercx]

Dear Togo,

Thanks for the report! The `PwBandStructureWorkChain` is a older version of a band structure work chain with a pre-defined protocol, used for previous tutorials. As such, it should be deprecated, but I haven't gotten around to doing this so far. I've opened an issue for doing this as well as fixing the issues you mentioned and will add the fix to the v3.4.2 release:

https://github.com/aiidateam/aiida-quantumespresso/issues/687

The new work chain for running a band structure calculation is the `PwBandsWorkChain`. This work chain has now been provided a `get_builder_from_protocol()` method, that returns a fully populated builder based on the `structure` and `code` you want to run, for example:

```
from aiida.plugins import WorkflowFactory

PwBandsWorkChain = WorkflowFactory('quantumespresso.pw.bands')
code = orm.load_code(<CODE_ID>)
structure = orm.load_node(<STRUCTURE_ID>)

builder = PwBandsWorkChain.get_builder_from_protocol(
    code=code,
    structure=structure,
)

from aiida.engine import submit
submit(builder)

```

Let me know if this works for you, or if you have any other questions!

Best,

Marnik

--
AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).

Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/CAAfkA7f4vSBdg%2B34qVsM6SLeucyVmb%3Du9YU8PeFA9wGijtcq_Q%40mail.gmail.com.

[Atsushi Togo]

Dear Marnik,

Thanks for your quick reply.

I tried to use PwBandsWorkChain.get_builder_from_protocol() in the
tutorial case, I got the following error.

ValueError: required pseudo family `SSSP/1.1/PBE/efficiency` is not
installed. Please use `aiida-pseudo install` toinstall it.

In my Quantum Mobile shell,
$ verdi data upf listfamilies
Success: * SSSP [85 pseudos]

So I did

$ verdi group copy SSSP SSSP/1.1/PBE/efficiency

but it seems PwBandsWorkChain.get_builder_from_protocol() could not
find it well. Can I specify this 'SSSP' in
PwBandsWorkChain.get_builder_from_protocol()?

Maybe I simply have to run 'aiida-pseudo install sssp'. Does this work
behind proxy? There are many useful commands around AiiDA, but behind
proxy, I have difficulty using those commands connecting to the
internet to download data. CodDbImporter also. Is there any way to
specify proxy like HTTPS_PROXY? Maybe this proxy setting information
is written somewhere in the documentation, but I could not find it.
Because of this network problem, I usually take steps downloading the
data using wget (setting .wgetrc for http_proxy etc) and then import
the data locally. So if I could specify pseudo_family in
PwBandsWorkChain.get_builder_from_protocol(), it is straightforward to
me.

Best regards,

Togo

> To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/A4B55A8C-D700-432A-884D-CF1CFD87B4E9%40gmail.com.



--
Atsushi Togo

[Sebastiaan Huber]

Dear Togo,

You would indeed have to simply run

    aiida-pseudo install sssp

This will install the SSSP 1.1 PBE efficiency, if you want to run the
`precise` protocol, you need to install the precision version as well

    aiida-pseudo install sssp -p precision

Note that you may have to delete the `SSSP/1.1/PBE/efficiency` group
that you created manually first, because the label should be unique.
The upf family you already have installed won't work as it is a plain
`Group` instance, and the new protocols require the more specific
`SsspFamily` (which is a `Group` subclass, but comes with more features
such as providing recommended cutoffs).
Marnik has just released some beautiful documentation for this new
`aiida-pseudo` plugin: https://aiida-pseudo.readthedocs.io/en/latest/

If the install method for you doesn't work, you could try the workaround
that we installed for this very problem.
You can run the CLI command with:

    aiida-pseudo install sssp --download-only

This should just download the archive file that contains the
pseudopotentials and a JSON file containing the metadata (let's call it
metadata.json).
You might have to run this on another computer that is not behind a
proxy, and then you copy the two files to your computer.
Then you can create the family manually from those files as follows

    aiida-pseudo install family -F pseudo.family.sssp -P pseudo.upf 
SSSP_1.1_PBE_efficiency.tar.gz SSSP/1.1/PBE/efficiency

Note that the last argument is the label and should not be changed,
otherwise the QE protocol builders cannot find it.
The last thing to do is to add the recommended cutoffs:

    aiida-pseudo family cutoffs set SSSP/1.1/PBE/efficiency metadata.json

And that should be it.
As you can see, the automated commands are quite convenient because they
do all of this automatically, but if you cannot directly download this
is to do it manually.

HTH,

SPH

[Atsushi Togo]

Dear Sebastiaan,

Thanks for your detailed reply. Network proxy is bad. Unfortunately I
could not make it work.

I could download archive.tar.gz metadata.json by

$ aiida-pseudo install sssp --download-only

But your command didn't work:

$ aiida-pseudo install family -F pseudo.family.sssp -P pseudo.upf
SSSP_1.1_PBE_efficiency.tar.gz SSSP/1.1/PBE/efficiency

Initially, I have aiida-pseudo 0.6.0, where the options were
different. I updated it to 0.6.1. Then

$ aiida-pseudo install family -F pseudo.family.sssp -P pseudo.upf
SSSP_1.1_PBE_efficiency.tar.gz SSSP/1.1/PBE/efficiency
/home/max/codes/aiida-core/aiida/orm/groups.py:61: UserWarning: no
registered entry point for `CutoffsPseudoPotentialFamily` so its
instances will not be storable.
warnings.warn(message) # pylint: disable=no-member
Usage: aiida-pseudo install family [OPTIONS] ARCHIVE LABEL

Error: Invalid value for '-F' / '--family-type': `pseudo.family.sssp`
is not an accepted value for this option.

$ aiida-pseudo install family -T pseudo.family.sssp -P pseudo.upf
archive.tar.gz SSSP/1.1/PBE/efficiency
/home/max/codes/aiida-core/aiida/orm/groups.py:61: UserWarning: no
registered entry point for `CutoffsPseudoPotentialFamily` so its
instances will not be storable.
warnings.warn(message) # pylint: disable=no-member
Usage: aiida-pseudo install family [OPTIONS] ARCHIVE LABEL
Try 'aiida-pseudo install family -h' for help.

Error: no such option: -T

$ aiida-pseudo install family -F pseudo.family.sssp -P pseudo.upf
archive.tar.gz SSSP/1.1/PBE/efficiency
/home/max/codes/aiida-core/aiida/orm/groups.py:61: UserWarning: no
registered entry point for `CutoffsPseudoPotentialFamily` so its
instances will not be storable.
warnings.warn(message) # pylint: disable=no-member
Usage: aiida-pseudo install family [OPTIONS] ARCHIVE LABEL

Error: Invalid value for '-F' / '--family-type': `pseudo.family.sssp`
is not an accepted value for this option.

$ aiida-pseudo install family -P pseudo.upf archive.tar.gz
SSSP/1.1/PBE/efficiency
/home/max/codes/aiida-core/aiida/orm/groups.py:61: UserWarning: no
registered entry point for `CutoffsPseudoPotentialFamily` so its
instances will not be storable.
warnings.warn(message) # pylint: disable=no-member
Info: unpacking archive and parsing pseudos... [OK]
Success: installed `SSSP/1.1/PBE/efficiency` containing 85 pseudopotentials

$ aiida-pseudo family cutoffs set -s normal SSSP/1.1/PBE/efficiency
metadata.json
/home/max/codes/aiida-core/aiida/orm/groups.py:61: UserWarning: no
registered entry point for `CutoffsPseudoPotentialFamily` so its
instances will not be storable.
warnings.warn(message) # pylint: disable=no-member
Usage: aiida-pseudo family cutoffs set [OPTIONS] FAMILY CUTOFFS

Error: Invalid value: family
`PseudoPotentialFamily<SSSP/1.1/PBE/efficiency>` does not support
recommended cutoffs to be set.



So reverting to aiida-pseudo 0.6.0

$ aiida-pseudo install family -T pseudo.family.sssp archive.tar.gz
SSSP/1.1/PBE/efficiency

passes (I'm not sure if this went correctly or not). But next

$ aiida-pseudo family cutoffs set -s normal SSSP/1.1/PBE/efficiency
metadata.json
Usage: aiida-pseudo family cutoffs set [OPTIONS] FAMILY CUTOFFS

Error: Invalid value for CUTOFFS: `metadata.json` contains invalid
cutoffs: invalid cutoff keys for element Ag: {'filename':
'Ag_ONCV_PBE-1.0.oncvpsp.upf', 'md5':
'94f47bd0669c641108e45594df92fabc', 'pseudopotential': 'SG15',
'cutoff_wfc': 50.0, 'cutoff_rho': 200.0}

I guess I did something wrong.
This is more specific on aiida-pseudo, so may I discuss on the github
issues of aiida-pseudo?

Thanks,

Togo

> To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/68eda090-734a-7051-0cd1-a34916bab782%40epfl.ch.



--
Atsushi Togo

[Marnik Bercx]

Dear Togo,

Thanks a lot for the detailed report! You did nothing wrong, the oversight was on our end. I've opened two issues on `aiida-pseudo`:

* The manual install using `aiida-pseudo install family` doesn't accept `pseudo.family.sssp` for the `--family-type` option. The reasoning was that there is a dedicated install command for this family type, but this doesn't allow you to install from a local archive (yet), see:

https://github.com/aiidateam/aiida-pseudo/issues/92

* The `aiida-pseudo family cutoffs set ` command too strict in its validation of the dictionary, and will not accept other keys than `ecut_rho` and `ecut_wfc` for each element. See:

https://github.com/aiidateam/aiida-pseudo/issues/93

As a workaround for the workaround, you can use the Python API. From where you left off (after having installed the pseudo family with version 0.6.0), open a `verdi shell` and do:

```

# Load the metadata file

import json

with open('metadata.json', 'r') as file:
    metadata = json.load(file)

# Set up a dictionary with _only_ the cutoffs

cutoffs = {element: {'cutoff_wfc': values['cutoff_wfc'], 'cutoff_rho': values['cutoff_rho']} for element, values in metadata.items()}

# Load the pseudopotential family

pseudo_family = load_group('SSSP/1.1/PBE/efficiency')

# Set the recommended cutoffs based on the values obtained from the metadata file

pseudo_family.set_cutoffs(cutoffs, 'normal', unit='Ry')

```

Now the 'SSSP/1.1/PBE/efficiency' pseudopotential family, which you manually installed with `aiida-pseudo install family`, will have the recommended cutoffs configured. For example:

```

In [5]: pseudo_family.get_recommended_cutoffs(elements='Si')
Out[5]: (30.0, 240.0)

```

Hope that helps, and thanks again for reporting! Let me know if you run into any further issues.

Best,

Marnik

To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/CAAfkA7c_rLNEZDurz4vYvMg5cOtmnGJSuA1g0pB6VwzF5o2QNQ%40mail.gmail.com.