Querying Calculations from parent workflows

45 views
Skip to first unread message

Jonathan Chico

unread,
Jun 14, 2021, 8:49:26 AM6/14/21
to aiidausers

Dear all

Right now I'm trying to visualize a bit the data from calculations that are currently running, e.g. I want to use AiiDA to find the underlying calculation that my workflow is running, get the remote folder and then by some ssh request obtain the data to plot the energy per scf step for example. Of this way I can monitor if my calculation "looks" correct, and I can cancel it before hand or just see how it evolves.

By using the QueryBuilder plus some extras tags in my workflows I'm able to find the parent workflows

_qb = orm.QueryBuilder()
_qb.append(
    orm.Node,
    filters={
        'and':[
            {'node_type': {'ilike':'process.workflow.workchain.WorkChainNode.%'}}
        ]
   },
tag='workchain'
)

I thought that by adding something like

_qb.append(orm.CalcJobNode, with_incoming='workchain')

I should be able to get the underlying calculations from my parent workflows, however, I obtain an empty list, I think this is because most parent workflows run several workchains inside of them and the Calculation is several layers below the parent.

Hence, I wonder is there a way to query for the CalcJobNode underlying a parent Workchain when the CalcJobNode is buried under several layers of workchains?

Thanks!

Cheers

Zhu, Bonan

unread,
Jun 14, 2021, 9:32:01 AM6/14/21
to aiida...@googlegroups.com

Hi Jonathan,

 

Once you get the WorkChainNode you can use the ` .called_descendants` property to access a list of children following the CALL links.

You can iterate though it and find the `CalcJobNode` that are still running (those with `.is_finshed` being `False`).

 

Alternatively, if all of your CalcJobNodes are nested one level down, you can add another WorkChainNode then the CalcJobNode in the query, this is probably faster than above.

 

Best wishes,

Bonan

--
AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
 
Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/9dfbea1f-3510-4c86-8fed-eb7eae6cc1cdn%40googlegroups.com.

Jonathan Chico

unread,
Jun 14, 2021, 9:47:24 AM6/14/21
to aiida...@googlegroups.com
Hi Bonan!

Thank you for your quire reply! Yes I think using the called descendants is a good idea ( I before thought with the get_outgoing().all() but the number of nodes is very large and many are unnecessary).

I was just wondering if I could do it directly with the QueryBuilder, without having to make the several calls to `append` with each of the children workchains, as depending on the workflow it could be that one has to go down one level or many levels to find the CalcJobNode.  i.e. is there a way for me to re-write the query

_qb = orm.QueryBuilder()
_qb.append(
    orm.Node,
    filters={
        'and':[
            {'node_type': {'ilike':'process.workflow.
workchain.WorkChainNode.%'}}
        ]
   },
tag='workchain'
)
_qb.append(orm.Node, with_incoming='workchain', tag='workchain_1')
_qb.append(orm.Node, with_incoming='workchain_1', tag='workchain_2')
_qb.append(orm.Node, with_incoming='workchain_2', tag='workchain_3')
_qb.append(orm.CalcJobNodes, with_incoming='workchain_3')

into something like

_qb = orm.QueryBuilder()
_qb.append(
    orm.Node,
    filters={
        'and':[
            {'node_type': {'ilike':'process.workflow.
workchain.WorkChainNode.%'}}
        ]
   },
tag='workchain'
)
_qb.append(some_very_clever_query, with_incoming='workchain', tag='clever_query')
_qb.append(orm.CalcJobNodes, with_incoming='clever_query')


Cheers



--
Jonathan Chico

Giovanni Pizzi

unread,
Jun 15, 2021, 2:35:18 AM6/15/21
to AiiDA users mailing list
Hi Jonathan,
I think what you are looking for is an equivalent for `with_descendants/with_ancestors` (see https://aiida.readthedocs.io/projects/aiida-core/en/latest/topics/database.html#topics-database-advancedquery) but following called links rather than INPUT_CALC/CREATE.
So your last two lines of your "clever_query" would be replaced just by one looking like this:
_qb.append(orm.CalcJobNode, with_calling_ancestor='workchain')

However this, at the moment, does not exist (even if I think it would be a very important and useful addition). I suggest you open an issue so we can discuss there the details. Note that this might not be very trivial to implement (e.g. running these queries at the DB level requires to write recursive SQL queries, and the performance is not always optimal, see e.g. https://github.com/aiidateam/aiida-core/issues/3755 but I think we can discuss there the technical details.

Thanks,
Giovanni
 

Reply all
Reply to author
Forward
0 new messages