Issue using CYTHON_LIMITED_API with very simple Cython code

[Thomas Robitaille]

Hi all,

I'm currently experimenting with CYTHON_LIMITED_API and am having issues getting things to work even for simple Cython code. I've set up a minimal example package here:

  https://github.com/astrofrog/limited-api-experiments

and running:

  pip wheel .

at the root of the repository results in many warnings and errors (I am using Python 3.10 in case that matters), such as:

      test_cython/example.c:1442:12: error: implicit declaration of function 'PyMethod_New' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          return PyMethod_New(func, self);
                 ^
      test_cython/example.c:1442:12: note: did you mean 'PySet_New'?
      /Library/Frameworks/Python.framework/Versions/3.10/include/python3.10/setobject.h:80:24: note: 'PySet_New' declared here
      PyAPI_FUNC(PyObject *) PySet_New(PyObject *);
                             ^
      test_cython/example.c:1442:12: warning: incompatible integer to pointer conversion returning 'int' from a function with result type 'PyObject *' (aka 'struct _object *') [-Wint-conversion]
          return PyMethod_New(func, self);
                 ^~~~~~~~~~~~~~~~~~~~~~~~
      test_cython/example.c:1485:5: error: unknown type name 'PyCMethodObject'; did you mean 'PyCodeObject'?
          PyCMethodObject func;
          ^~~~~~~~~~~~~~~
          PyCodeObject

 

I know that support for CYTHON_LIMITED_API is at an early stage, but I was curious about whether it would be possible to get a simple example working. Can anyone see what I might be doing wrong above?

Thanks!

Tom

[da-woods]

One of the things we haven't tested is what happens when you restrict Python to the actual limited API. It doesn't look like PyMethod_New or PyCMethodObject are part of the limited API.

One option for replacing it would be to get `PyMethod_Type` (possibly by importing it from `types`) and to just call that with `func` and `self`.

I'm not sure how we can do without PyCMethodObject. Possibly by turning off the `cython.binding` directive?

But I honestly doubt you'll be able to get your simple example working while enforcing the limited api.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/8a9395df-dc8c-4711-8f6e-058a946c26a9n%40googlegroups.com.

[Thomas Robitaille]

Hi everyone,

A little while back, I posted below about experiments I was doing with CYTHON_LIMITED_API (see messages below). I now see that Cython 3.0.0 has been released (congratulations!) and the release docs mention that there should be initial support for setting both CYTHON_LIMITED_API and Py_LIMITED_API (which I assume means this has now been tested successfully at least for a minimal example).

The experimental code I have can be found at

https://github.com/astrofrog/limited-api-experiments

In short, the Cython code I am compiling is:

def fib(int n):
    cdef int a, b
    a, b = 0, 1
    while b < n:
        a, b = b, a + b
    return b

and I am defining the following macros in setup.py:

define_macros = [('CYTHON_LIMITED_API', '1'),
                 ('Py_LIMITED_API', '0x030700f0')]

However, when I try and build the package (pip install .), I still get errors related to e.g. PyCFunction_New not being defined for instance.

I guess I am curious how my minimal example could be adjusted to work? Or alternatively whether anyone else could provide a minimal example of how to use CYTHON_LIMITED_API in practice?

Thanks!

Tom

[da-woods]

The explanation in the docs is slightly unhelpful and probably implies more support than actually exists. Essentially:

* defining CYTHON_LIMITED_API makes Cython try to generate C code that should work with the limited API (as far as it can).

* Defining Py_LIMITED_API stops Python exposing any functions that aren't in the limited API. Without it a module that you build may not actually be "limited API" since it might link to functions it shouldn't use.

I don't think the combination is actually tested since we know there's a few places we use functions that we shouldn't. Therefore it was probably a mistake to advise it at this point. On the other hand, bug reports like this are helpful since they remind us to chase it.

I'll try to have a look at that example some time in the next week and work out where the PyCFunction_New is coming from (I can see find two examples in the Cython code-base; one is Py2 only so I don't think you should be hitting it; and the other I don't think you should be generating either).

David

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/5ca7e9d3-053e-478a-9595-95e268f803e5n%40googlegroups.com.

[da-woods]

So I've made some changes and I believe I've got your example working (almost). The one change you have to make is @cython.binding(False) to decorate your function - the CyFunction type we use for more introspectable functions is a long way off working so it's necessary to turn that off for the forseeable future.

See https://github.com/cython/cython/pull/5550 for the PR if you're interested. I've not gone beyond that exact function though, so I'd imagine plenty of other bugs and omissions remain...

Davod

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/697e3379-9e8e-350f-dd95-9acf05f64882%40d-woods.co.uk.