The currently AiiDA code architecture design makes it likes a different programming language than python and will frustrate popular WingIDE for code analysis.

[Hongyi Zhao]

Hi,

See here for my full discussions with the Wingware Support group:

https://ask.wingware.com/question/1777/unexpected-auto-completer-behavior-when-working-with-aiida/?answer=1796#post-id-1796

Here are the comment excerpted from there given by the Wingware Support group:

--------------
OK, based on setting this up here I've got the explanation.

The short answer is that aiida implements an import hook that has
behavior that does not match what Python itself does when it processes
imports, and this is combining with some poor naming choices and
overuse of 'from x import *' to make you think you're being offered
the function calcfunction when in fact it would normally be importing
the module aiida-core/orm/nodes/process/calculation/calcfunction.py
through a long chain of 'from x import *'s.

The details:

The reason we're offering calcfunction in the completer for aiida.orm
is because of the following import chain:

orm/__init__.py has from .nodes import *
orm/nodes/__init__.py has from .process import *
orm/nodes/process/__init__.py has from .calculation import *
orm/nodes/process/calculation/__init__.py has from .calcfunction
import CalcFunctionNode
This imports from orm/nodes/process/calculation/calcfunction.py

Indeed, calcfunction is defined also at runtime if I stop in the
debugger after that import in
aiida/orm/nodes/process/calculation/__init__.py but module loading in
aiida is completely dynamic and it appears that its module load it's
not processing the import * in such a way that 'calcfunction' is
defined in the places that did the imports. CalcFunctionNode is
defined but not calcfunction. This clearly does not match what Python
normally does with imports. It would also import the symbol
calcfunction through the import *.

Goto-definition was presumably going to the like-named calcfunction
function definition because WIng was also seeing the second correct
import 'from aiida.engine import calcfunction' and finding it that
way. In fact calcfunction could be either that or the imported module
as described above, except of course that aiida has implemented
non-standard module loading. Indeed if I have code like the following,
Wing's Source Assistant shows that calcfunction could either be a
module or a function:

from aiida.engine import calcfunction
from aiida.orm import calcfunction

There's nothing we can do about this. If someone decides to change
module load behavior at runtime there's no way for us to know that.
Essentially, they're inventing a different programming language.
--------------

Regards
--
Hongyi Zhao <hongy...@gmail.com>

[Sebastiaan Huber]

Hi Hongyi,

I have not heard of the Wing IDE myself, but we have multiple users using VS Code and PyCharm (and other IDEs) that do not experience these issues.
When reading through the thread you mentioned, I notice someone else has mentioned this as well and the developer of Wing was going to look into it some more.
It is true that the `aiida-core` package is quite involved and has some deep module paths.
However, we are not doing anything out of the ordinary when it comes to import modules.
It is true that you will find the use of the style `from .. import *` to be discouraged by various sources online, because it is not quite clear by reading it what exactly will be imported into the namespace.
However, this is perfectly valid standard Python and not something we invented.
On top of that, whenever we use that paradigm, we make sure that the module defines the `__all__` attribute, in which case using the `import *` method is perfectly well-defined.
We use this method of exposing inputs such that users can always import relevant resources from the second level down, i.e. they can do

    from aiida.engine import calcfunction

instead of forcing them to go diving down and do

    from aiida.engine.processes.functions import calcfunction

This makes it easier for users to import and also makes clear what resources are intended to be used publicly and which methods are only for internal use.
This concept and the exact list of public resources is also listed in detail in the documentation [1]

HTH,

SPH


[1] https://aiida-core.readthedocs.io/en/latest/working_with_aiida/python_api.html#public-resources

[Hongyi Zhao]

Sebastiaan Huber <sebastia...@epfl.ch> 于2020年4月16日周四 下午3:23写道：

>
> Hi Hongyi,
>
> I have not heard of the Wing IDE myself, but we have multiple users using VS Code and PyCharm (and other IDEs) that do not experience these issues.
> When reading through the thread you mentioned, I notice someone else has mentioned this as well and the developer of Wing was going to look into it some more.
> It is true that the `aiida-core` package is quite involved and has some deep module paths.
> However, we are not doing anything out of the ordinary when it comes to import modules.
> It is true that you will find the use of the style `from .. import *` to be discouraged by various sources online, because it is not quite clear by reading it what exactly will be imported into the namespace.
> However, this is perfectly valid standard Python and not something we invented.
> On top of that, whenever we use that paradigm, we make sure that the module defines the `__all__` attribute, in which case using the `import *` method is perfectly well-defined.
> We use this method of exposing inputs such that users can always import relevant resources from the second level down, i.e. they can do
>
> from aiida.engine import calcfunction
>
> instead of forcing them to go diving down and do
>
> from aiida.engine.processes.functions import calcfunction
>
> This makes it easier for users to import and also makes clear what resources are intended to be used publicly and which methods are only for internal use.
> This concept and the exact list of public resources is also listed in detail in the documentation [1]

Thanks, maybe there is some bug in the WingIDE, so let we stay tuned.

Best,

> --
> AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
>
> Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
> ---
> You received this message because you are subscribed to the Google Groups "aiidausers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/ee029d11-87af-eb34-5048-993ce501b7f0%40epfl.ch.



--
Hongyi Zhao <hongy...@gmail.com>

[greschd]

Dear Hongyi,

Let me add the reason why 'calcfunction' is not imported in the way described in the linked post:

In the  import

	orm/nodes/process/__init__.py has from .calculation import *

the calcfunction module is imported into orm/nodes/process/calculation/__init__.py, but that file also defines an attribute

    __all__ = ('CalculationNode', 'CalcFunctionNode', 'CalcJobNode')

Because a wildcard (from something import *) import will import only the names defined in __all__, the "calcfunction" module is not exposed at the level of orm/nodes/process/__init__.py.

Maybe this explanation will help in figuring out what goes wrong with the autocompletion.

Best regards,

Dominik

[Hongyi Zhao]

greschd <gre...@hotmail.com> 于2020年4月16日周四 下午4:48写道：

>
> Dear Hongyi,
>
> Let me add the reason why 'calcfunction' is not imported in the way described in the linked post:
>
> In the import
>
> orm/nodes/process/__init__.py has from .calculation import *
>
> the calcfunction module is imported into orm/nodes/process/calculation/__init__.py, but that file also defines an attribute
>
> __all__ = ('CalculationNode', 'CalcFunctionNode', 'CalcJobNode')
>
> Because a wildcard (from something import *) import will import only the names defined in __all__, the "calcfunction" module is not exposed at the level of orm/nodes/process/__init__.py.
>
> Maybe this explanation will help in figuring out what goes wrong with the autocompletion.

Thanks again, Wingware Support people confirmed that this is a bug of
their software. See the following link for more info:
https://ask.wingware.com/question/1777/unexpected-auto-completer-behavior-when-working-with-aiida/?answer=1812#post-id-1812

---------------
I've removed my earlier incorrect answer and have tagged this a bug
report. It's due to Wing not understanding the __all__ in
orm/nodes/process/__init__.py which reads: __all__ =
(calculation.__all__ + process.__all__ + workflow.__all__)

We'll try to fix this soon. Thanks for reporting this and noticing
that our earlier analysis was incorrect!
---------------

Regards

--
Hongyi Zhao <hongy...@gmail.com>

[Hongyi Zhao]

Sebastiaan Huber <sebastia...@epfl.ch> 于2020年4月16日周四 下午3:23写道：
>

> Hi Hongyi,
>
> I have not heard of the Wing IDE myself,

See here for a list aimed to comparison different Python IDEs:
https://en.wikipedia.org/wiki/Wing_IDE

The Wing IDE is also listed there. I previously tried over
pycharm|spyder|vscode|vim and so on. Among them pycharm is the most
powerful one, but very slow for the starting process. The Wing IDE is
build from pure python and for python development, like spyder, but
more powerful than spyder.

It supports the dynamical type names auto-completion function when
using in debug mode which makes it very suitable for using with AiiDA.

Regards