circular import problem

[Syam Gadde]

Hi,

I'm running cython (cloned from github mid-May 2021) and Python 3.6/3.7.9.  I've encountered an issue with circular imports, one that Python runs without a problem.  At its simplest, it's the following sequence of operations:

import A.B         

  from . import C      # <-- in A/B.py

    from . import D    # <-- in A/C.py

      from . import C  # <-- in A/D.py

If you run this scenario in CPython, it runs without error, but if I compile all the .py files into C extensions using Cython, the last import fails with the error:

ImportError: cannot import name C

Each of the above statements seems to be implemented by Cython with a pair of __Pyx_Import and __Pyx_ImportFrom calls.  The __Pyx_Import call attempts to do the import, and returns the parent package (in all cases above, it returns A).  The __Pyx_ImportFrom call just looks inside the returned module for an attribute with the requested name ('C', or 'D' in the above examples).  If the requested name ('C') in __Pyx_ImportFrom is actually a module that has not finished initializing yet, then it is not yet set as an attribute in its parent ('A') so this fails.

In CPython, I think, if the above attribute access (i.e. not hasattr(A, 'C')) fails, then it checks if it is a module (i.e. __import__('A.C')) probably to address this exact case.  Any chance __Pyx_ImportFrom could do this too to mimic CPython behavior?  Obviously the above scenario is silly, but the pattern shows up in an existing library that I'm trying to embed into a static Python interpreter and it seems like it is supported by the language.

Below is a list of commands that will generate the source files and test the above scenario on CentOS/RedHat 7, presumably minimal changes for other Linux variants.  Thanks for your help!

mkdir circular_import

cd circular_import
mkdir -p src/A
mkdir -p install/A
echo "from . import C" > src/A/B.py
echo "from . import D" > src/A/C.py
echo "from . import C" > src/A/D.py
for i in A/{B,C,D} ; do cython src/"${i}".py ; gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python3.6m -o install/"${i}".so src/"${i}".c ; done

(cd src && echo "import A.B" | python3 )  # succeeds

(cd install && echo "import A.B" | python3 )  # fails

[da-woods]

I'm created a bug report on the Cython issue tracker based on this: https://github.com/cython/cython/issues/4390

It looks like a reasonable report to me, and I think it should be possible to get it to work with the multi-phase import system (i.e. Python 3.5+).