how to use verdi commands in python code

[Johannes Wasmer]

Hi,

I'd like to use verdi commands in python code.

My use case is that I want to search the output of verdi outputcat calcjob NODE_ID for some pattern. Why write my own code if I just could use that one if a solution exists I thought. In general.

In IPython, I can use the %%capture cell magic to get the string output.

But in python code, I would have to rewrite it / copy-paste it into my own function.

Cause when I just import the verdi function and call calcjob_outputcat(calcjob), I get an error TypeError: 'CalcJobNode' object is not iterable.

Is there a simple solution to reuse verdi commands programmatically?

Thanks

[Bröder, Jens]

Hi Johannes,

To use verdi commands directly from python, you have several options:

   

    1. In python you can use the subprocess module (see https://docs.python.org/3/library/subprocess.html) to call external programs and thus also verdi commands. This module also allows you to capture the output anywhere you like.

    2. in a notebook and ipython execute `!verdi command >> filename`" (notice the '!' ) here you could pipe the output to a file, which you can read afterwards.

   
Beyond executing the verdi command directly through subprocess, you could also checkout what the command calls in the aiida-core source code does and use one of these routines from python directly. Some command may not be modular enough to do this.

Hope that helped.

Best regards,

Jens

----------------------------------------------------------

Jens Bröder

Research fellow at PGI-1/IAS-1

Peter Grünberg Institut (PGI) und Institute for Advanced Simulation (IAS) Forschungszentrum Jülich

D-52425 Jülich

Tel.: 02461-61-9385

E-Mail: j.br...@fz-juelich.de

----------------------------------------------------------

---

From: aiida...@googlegroups.com <aiida...@googlegroups.com> on behalf of Johannes Wasmer <johanne...@gmail.com>
Sent: Thursday, January 14, 2021 2:02:07 PM
To: aiidausers
Subject: [aiidausers] how to use verdi commands in python code

 

--
AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
 
Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/6936a49e-2bc0-479d-a3fb-56c69089a1f5n%40googlegroups.com.

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

[Sebastiaan Huber]

Hi Johannes,

In addition to the two options specified by Jens, there is another option that I use myself now and again.
Since `verdi` is built with the `click` library, which essentially turns normal Python functions into the CLI commands, it is possible to call those functions directly from a shell as well.
It is not super straightforward, but sometimes it can be useful.
As an example, this is how you can call `verdi process list` from within a `verdi shell`:

import click
from aiida.cmdline.commands.cmd_process import process_list
click.Context(process_list).invoke(process_list)

This will now print the usual output.
You can even define the usual flags as arguments, e.g.:

click.Context(process_list).invoke(process_list, all_entries=True, past_days=5)

Note, however, that the translation of flag names to function arguments are not always 1-to-1, as here with `--all` -> `all_entries`.
You will have to read the source code or know it, to figure this out unfortunately.

Important: now, that being said, often the `verdi` commands are simple wrappers around AiiDA's normal Python API.
As Jens also already said, it may actually be better, and more efficient, to directly use the API.
For example, to do what `verdi calcjob outputcat` does through the API is as simple as:

    calcjob_node.outputs.retrieved.get_object_content('aiida.out')

This will load the output of the `aiida.out` output file into memory.
You can then use normal Python to search that string for the pattern of interest.
Especially in the long run it will be worthwhile to learn the Python API because it will be much faster.
The CLI will almost always have an overhead.

HTH,

SPH

To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/59906ea12565495ba2a0acb04f518e6f%40fz-juelich.de.

[Johannes Wasmer]

Thanks Jens and Sebastiaan, that has all been enlightening.

I do want to use/learn the Python API, sorry for not communicating this clearly.

I guessed I could do that also by looking into the cmdline source code.

But now it has become more obvious to me that this approach is not so sensible, if the 30 lines there in calcjob_outputcat() are actually equivalent to your one-liner above 😄.

Best, Johannes