How to write tests for workflows

[Philipp Rüßmann]

Dear all,

I am working on the aiida-kkr plugin and I am wondering how to test the workflows automatically (i.e. with travis). So far, for calculations I just use the submit_test which goes through the code without actually running a calculation and I test the parsers by importing a calculation into the database and parse the output again. 

For the workflows on the other hand I am unsure what the best way would be. Should I define a mock calculation to do this?

I appreciate your feedback.

Best regards,

Philipp

[Espen Flage-Larsen]

Dear Philipp,

Thanks for your efforts. A plugin for KKR sounds great.

This is still something that is not yet fully settled on how to do. If you want, please have a look at what we did in the VASP plugin: github.com/aiida-vasp/aiida-vasp. Look in the development branch.

There are still some problems with this, in particular when the workflow gets complicated and nested. E.g. Travis has a time limit. That can be avoided by a wait command, although we have not yet fully tested this.

Also, there is always the issue of keeping the package small (e.g. not include too much test data), obeying licensing issues (difficult to host code in public for test execution by anyone) and maintenance. We have not yet concluded on what is the best approach. Issues like this would be nice to discuss together. I know there is a plan for a meeting between all the plugin developers so these issues would definitely be important there.

What we will most likely do is to keep the workflow tests rather basic, but try to cover most of them, e.g. the most used routines. Then we will have VASP compiled internally and run full tests there. This would introduce two sets of tests, but I think that is fine. Ignoring to tests the workchains fully is something we would like to avoid at all cost. Better to at least do it internally during development.

Cheers,
Espen

________________________________________
From: aiida...@googlegroups.com <aiida...@googlegroups.com> on behalf of Philipp Rüßmann <p.rue...@fz-juelich.de>
Sent: Monday, November 26, 2018 10:08 AM
To: aiidausers
Subject: [aiidausers] How to write tests for workflows

--
Please see the guidelines at AiiDA mailing list information page (http://www.aiida.net/?page_id=356) before posting a new question.
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com<mailto:aiidausers+...@googlegroups.com>.
To post to this group, send email to aiida...@googlegroups.com<mailto:aiida...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/6cd92bb6-ae39-4c5f-98da-85babfa90443%40googlegroups.com<https://groups.google.com/d/msgid/aiidausers/6cd92bb6-ae39-4c5f-98da-85babfa90443%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.