Handling conditional .pxi includes post-deprecation

[shakfu]

I'd like to ask how to address a use case where the `IF/DEF` combination seems to be the most straightforward solution:

My project wraps an external c++ project which has a minimal core api and two optional apis. I have currently provided for this distinction by conditional `include` statements for the optional apis in the single main `.pyx` file:

```python
# ... main wrapper code

# conditional parts of main wrapper code which use optional apis

IF FOO_API:
    def func_uses_main_and_foo(..)

IF BAR_API:
    def func_uses_main_and_bar(..)

# optional api includes

IF FOO_API:
    include "foo.pxi"

IF BAR_API:
    include "bar.pxi"
```
In this scenario:

- It is not clear if the user will want to have one or even both of the optional apis

- A regular python `if` is not possible and while I think it is possible to use a macro to define the constants, it is not clear how one would exclude the `include` so to speak.

- While it is possible to separate the functionality into separate imports (which I have indeed tested as a variant), the issue is that the optional code makes the core extension 16x to 20x larger, so it would be not make sense to package the optional imports with the core extension. The other option is to create separate subprojects which are pip installable, but (1) I prefer to only distribute the wrapper code in source form so as to allow the user to compile their own version as required as 'one' tailored extension which fits their needs and (2) the code above is also intended be used in another "embedded" project which requires a single-source approach.

So in summary, the current `IF/DEF` solution seems (at least for me) to be the simplest and easiest to maintain in this particular case.

I checked out the migration page linked to above, but it is very brief and doesn't cover cases such as the one outlined above. What is the recommended way to resolve this post-deprecation?

[da-woods]

On 28/05/2024 18:24, shakfu wrote:
> While it is possible to separate the functionality into separate
> imports (which I have indeed tested as a variant), the issue is that
> the optional code makes the core extension 16x to 20x larger, so it
> would be not make sense to package the optional imports with the core
> extension.

I really don't understand this point. Surely the optional code makes the
module larger whether it's included by IF or in separate modules?

So either:

* you're distributing source files for the user to build (in which case
both options have a similar cost)
* you're always distributing a full binary module with the optional
code, in which case distributing the optional imports is the same cost,
* you're always distributing a cut-down binary module with the optional
code, in which case you just don't distribute the compiled optional modules.

However: `IF/DEF` isn't going away any time soon, even if it is
deprecated. Therefore if you find it the best solution, then keep using
it for now.

[shakfu]

Thanks for your reply,

 D Woods wrote:

I really don't understand this point. Surely the optional code makes the
module larger whether it's included by IF or in separate modules?

 

Sorry, agree that what I wrote could have been expressed more clearly: If I used the 'IF/include' idiom, I tended to compile a fit-for-purpose single module statically which suited me fine, but then due to the deprecation I considered that I would have to split the module (and include all optional code) as separate imports within a package. When I tried this, these separate modules of course became huge if these were all compiled statically due to the duplication, but were of a reasonable size if linked them dynamically to dylibs which were packaged in the wheel (and fixed with something like 'delocate') but were still larger and more complex to build than the static version which is also more suitable for embedded contexts.

In any case, I was concerned that I would lose the benefit of IF/include due to the deprecation, and I still don't see how it's possible to conditionally include other python pxi module without IF.