AttributeError occurs even with --hidden-imports, hooks and runtime hooks

[Phil Tresadern]

I have a test example structured as:

   

    .

    ├─ mypkg

    │   ├─ __init__.py

    │   └─ mymod.py

    ├─ mycli.py

   

where mycli.py is 

    import mypkg.mymod

    mypkg.mymod.myfunc()

mymod.py is

    def myfunc():

        print("Hello, World")

        

and that all works perfectly.

A slightly different mycli.py,

    import mypkg

    if __name__ == "__main__":

        mypkg.mymod.myfunc()

bundles to an executable but fails to execute with the following Traceback:

    Traceback (most recent call last):

      File "mycli.py", line 6, in <module>

        mypkg.mymod.myfunc()

    AttributeError: module 'mypkg' has no attribute 'mymod'

    [83952] Failed to execute script mycli

(the same failure as you get with "python mycli.py").

    

My understanding is that using --hidden-import=mypkg.mymod has the effect of inserting "import mypkg.mymod" into the source code before running, which should make it equivalent to the first script (that worked), but the bundled executable still fails.

warn-mycli.txt lists a few missing modules but nothing related to mypkg.

With --hidden-import=mypkg.mymod, mypkg.mymod appears in the build/mycli/xref-mycli.html file, possibly suggesting that it was recognised, but the bundled executable still fails.

If I run with "--debug all" I get a dist/mycli/mypkg/mymod.pyc file, also suggesting it was finding and compiling the module correctly, but the bundled executable still fails.

I've also tried with an equivalent hook:

    hiddenimports = ["mypkg.mymod"]

which runs (as confirmed by adding a print statement), but the bundled executable still fails.

I also have the same problem when calling "python -m PyInstaller ..." rather than just "pyinstaller ..."

I'm using Python 3.5.10 & PyInstaller 4.0 on Ubuntu 20.04 under VirtualBox 6.1, and have the same problem with and without using a virtualenv (also trying with Python 3.8). 

Can anyone spot where I'm going wrong, please?

Thanks.

[bwoodsend]

That setup should fail even without PyInstaller because mypkg.mymod is never imported.

# This initialises mypkg.__init__ only.

 import mypkg

if __name__ == "__main__"

:
    # So even if mymod has been collected by PyInstaller, it won't have been loaded
    # and set as an attribute of `mypkg`.
    mypkg.mymod.myfunc()

Possibly what your after is the following line in __init__.py?

from . import mymod

That way, initialising mypkg (using import mypkg) will implicitly load mymod and you can safely use mypkg.mymod.myfunc(). Alternatively you can just import mypkg.mymod.

​

[Phil Tresadern]

Thank you for the prompt response. 

I agree with your explanation, but would like to avoid having to manually import submodules in every package's __init__.py if possible. I will if I must, but this was a toy example - the code base I'd like to bundle is far more complex and, up to now, everything has worked without hand-crafted imports.

If possible, I'd like to resolve it using PyInstaller's functionality. Specifically, I was under the impression that using --hidden-import (or, more likely, a custom hook) would solve the problem. Quoting the PyInstaller docs (https://pyinstaller.readthedocs.io/en/stable/hooks.html):

    Many hooks consist of only one statement, an assignment to hiddenimports. For example, the hook for the dnspython package, called hook-dns.rdata.py, has only this statement:

        hiddenimports = [

            "dns.rdtypes.*",

            "dns.rdtypes.ANY.*"

        ]

    When Analysis sees import dns.rdata or from dns import rdata it calls hook-dns.rdata.py and examines its value of hiddenimports. As a result, it is as if your source script also contained [my emphasis]:

        import dns.rdtypes.*

        import dsn.rdtypes.ANY.*

My question would then be: if I can solve it with "import mypkg.mymod" in my script, why can't I solve it with "--hidden-import=mypkg.mymod" in PyInstaller?

Thanks again. I do appreciate you taking the time to help me understand.

[bwoodsend]

Because import mypkg.mymod initialises it but --hidden-import=mypkg.mymod just says include it. Python doesn’t automatically import modules just because they exist otherwise it would be incredibly slow.

If you really don’t want to use from . import mymod then you can put the following in your __init__.py (note Python>=3.6 only) but it’s terrible programming practice.

def __getattr__(submodule):
    import importlib
    try:
        return importlib.import_module("mypkg." + submodule)
    except ImportError:
        raise AttributeError(submodule) 

​

[Phil Tresadern]

Righto. Thanks very much for your help - much appreciated.