Issues with compiling several files

[Blue]

Hello, I am having issues trying to setup a good setup.py file, I have read Source Files and Compilation and my file currently looks like this:

from setuptools import Extension, setup

from Cython.Build import cythonize

setup(

    name='test',

    version='0.0.1',

    packages=['test'],

    install_requires=['aiohttp'],

    ext_modules=cythonize([

        Extension('test', [

            'test/models/base.py',

        ]),

    ]),

)

But when I run this (by doing python -m build) I get the following error:

Traceback (most recent call last):

  File "C:\Users\%username%\Projects\wumpy\venv\lib\site-packages\pep517\in_process\_in_process.py", line 349, in <module>

    main()

  File "C:\Users\%username%\Projects\wumpy\venv\lib\site-packages\pep517\in_process\_in_process.py", line 331, in main

    json_out['return_val'] = hook(**hook_input['kwargs'])

  File "C:\Users\%username%\Projects\wumpy\venv\lib\site-packages\pep517\in_process\_in_process.py", line 117, in get_requires_for_build_wheel

    return hook(config_settings)

  File "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\setuptools\build_meta.py", line 154, in get_requires_for_build_wheel

    return self._get_build_requires(

  File "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\setuptools\build_meta.py", line 135, in _get_build_requires

    self.run_setup()

  File "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\setuptools\build_meta.py", line 150, in run_setup

    exec(compile(code, __file__, 'exec'), locals())

  File "setup.py", line 10, in <module>

    ext_modules=cythonize([

  File "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\Cython\Build\Dependencies.py", line 965, in cythonize

    module_list, module_metadata = create_extension_list(

  File "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\Cython\Build\Dependencies.py", line 815, in create_extension_list

    for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern):

  File "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\Cython\Build\Dependencies.py", line 114, in nonempty

    raise ValueError(error_msg)

ValueError: 'test/models/base.py' doesn't match any files

I do see a file called `test/models/base.c` popup as expected, so the file has been cythonized. If I change the extension to:

Extension('test', [

    'test/models/base.c','

]),

It works and I get a .whl file. To reiterate, my installation steps are: run setup with .py extension so that they're cythonized, then run setup with .c extension so that the .whl file can be built and setup be successful.

------

Blue

[Blue]

Ah sorry, this didn't end up as clear as I thought.

The issue I am having is that I don't get a .whl file unless I first setup with .py which makes Cython cythonize them. Then I need to change the files to .c, so that setuptools pick them up and I can get a .whl file.

[Blue]

I have since fixed this, no idea why it failed in the first place. It seemed like `cythonize` didn't change the Extension classes returned (so that they were the correct `C` files).

My setup.py now looks as follow:

from Cython.Build import build_ext

from setuptools import Extension, setup

setup(

    packages=['test'],

    extensions = [

        Extension('test', [

            'test/models/base.py',

        ]),

    ],

    cmd_data = {'build_ext': build_ext},

)

[Stefan Behnel]

Blue schrieb am 03.08.21 um 14:16:

> Hello, I am having issues trying to setup a good setup.py file, I have read Source
> Files and Compilation

> <https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html> and

> my file currently looks like this:
>
> from setuptools import Extension, setup
> from Cython.Build import cythonize
>
> setup(
> name='test',
> version='0.0.1',
> packages=['test'],
>
> install_requires=['aiohttp'],
> ext_modules=cythonize([
> Extension('test', [
> 'test/models/base.py',
> ]),
> ]),
> )

I understand that this is probably just stripped-down code, but if you name
everything "test", both the package and the extension module, then things
are bound to go wrong … somewhere.

Is there a reason why you don't name the extension "test.models.base", as
its package path suggests?

> But when I run this (by doing python -m build) I get the following error:
>
> Traceback (most recent call last):
> File

> "C:\Users\%username%\AppData\Local\Temp\build-env-d_iy8lu7\lib\site-packages\Cython\Build\Dependencies.py",
> line 114, in nonempty
> raise ValueError(error_msg)
> ValueError: 'test/models/base.py' doesn't match any files

Which directory are you running this from? It seems to be unable to find
the file given by the relative path.

Stefan

[Blue]

Hello Stefan!

Because of conflicting sources and variables used (probably with some lazy reading sprinkled in there),

I was a bit confused about the keyword-arguments I could pass to `setup()`.

Even the post I made commenting about it working, did in fact not.

It appeared that having "packages=['test']" along with both keyword-arguments being wrong, the files were not cythonized but simply included.

> Is there a reason why you don't name the extension "test.models.base", as

> its package path suggests?

This acts as __init__.py, though I could just use such a file and do ``Extension('*', ['**/*.py'])``.

> Which directory are you running this from? It seems to be unable to find

> the file given by the relative path.

I was running this in the project directly. The relative paths should've been correct.

In fact Cython was able to find them, but it didn't seem like they were changed before being passed to ext_modules.

I have now actually figured it out though, I was missing two warnings raised. If you notice, there's actually no kwarg called ``cmd_data``.

The two kwargs should be: ``ext_modules`` and ``cmdclass`` (instead of ``extensions`` and ``cmd_data``).

That way my setup script now looks like this and get compiled .so/.pyd files successfully.

from Cython.Build import build_ext

from setuptools import Extension, setup

setup(

    ext_modules=[

        Extension('test', [

            'test/models/base.py',

        ]),

    ],

    cmdclass={'build_ext': build_ext},

)

[Stefan Behnel]

Blue schrieb am 07.08.21 um 18:17:

> It appeared that having "packages=['test']" along with both
> keyword-arguments being wrong, the files were not cythonized but simply
> included.
>
>> Is there a reason why you don't name the extension "test.models.base", as
>> its package path suggests?
>
> This acts as __init__.py, though I could just use such a file and do
> ``Extension('*', ['**/*.py'])``.

I still don't understand what you're doing (or trying to say here). What is
the relation between the package, the package name, and the extension
module? Is there a package at all?

> I have now actually figured it out though, I was missing two warnings
> raised. If you notice, there's actually no kwarg called ``cmd_data``.
> The two kwargs should be: ``ext_modules`` and ``cmdclass`` (instead of
> ``extensions`` and ``cmd_data``).
>
> That way my setup script now looks like this and get compiled .so/.pyd
> files successfully.
>
> from Cython.Build import build_ext
> from setuptools import Extension, setup
>
> setup(
> ext_modules=[
> Extension('test', [
> 'test/models/base.py',
> ]),
> ],
> cmdclass={'build_ext': build_ext},
> )

That's why I was asking – you shouldn't need to do this. Calling
cythonize() should be enough.

Stefan

[Blue]

>  I still don't understand what you're doing (or trying to say here). What is
> the relation between the package, the package name, and the extension
> module? Is there a package at all?

Nevermind, this is bad for users trying to use my package.

> That's why I was asking – you shouldn't need to do this. Calling

> cythonize() should be enough.

Are you talking about the usage of ``cmdclass`` or ``Extension``?

Either way I think I'll just go ahead and pass those files directly to ``cythonize()`` yeah thank you.

-------

Blue