Cython Failed to Build other Packages with Extra Fields in PyObject

[Yuze Li]

I was hacking through CPython and added 1 extra field within PyObject (ie., _object struct), and built CPython successfully. Then I built Cython from source, against this Python, which also works.

However, when I try to build other packages (eg., Numpy or Pandas) from source, which requires Cython. It shows error message (e.g., in Pandas directory):

-----------------

$ rm -rf build; ~/workspace/cpython/python -m pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/lyuze/workspace/pandas
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
 
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      + meson setup /home/lyuze/workspace/pandas /home/lyuze/workspace/pandas/.mesonpy-w_u_sho4/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/home/lyuze/workspace/pandas/.mesonpy-w_u_sho4/build/meson-python-native-file.ini
      The Meson build system
      Version: 1.2.1
      Source dir: /home/lyuze/workspace/pandas
      Build dir: /home/lyuze/workspace/pandas/.mesonpy-w_u_sho4/build
      Build type: native build
      Project name: pandas
      Project version: 2.2.0.dev0+749.gae6d27937b
      C compiler for the host machine: ccache cc (gcc 11.4.0 "cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
      C linker for the host machine: cc ld.bfd 2.38
      C++ compiler for the host machine: ccache c++ (gcc 11.4.0 "c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
      C++ linker for the host machine: c++ ld.bfd 2.38
     
      ../../meson.build:2:0: ERROR: Unknown compiler(s): [['cython'], ['cython3']]
      The following exception(s) were encountered:
      Running `cython3 -V` gave "[Errno 2] No such file or directory: 'cython3'"
     
      A full log can be found at /home/lyuze/workspace/pandas/.mesonpy-w_u_sho4/build/meson-logs/meson-log.txt
      Py_FinalizeEx called
      Py_FinalizeEx called
      [end of output]
 
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

-----------------

The same error shows when I try to build Numpy. However, both work when I discard that extra field in PyObject struct. Is it because Cython checks the ABI compatibility in case it violates the PyObject memory layout? Thanks!

[da-woods]

I don't think adding an extra field should break Cython. However it will break compatibility with the pre-built Cython wheels available on Pip. Although it sounds like this isn't what you're using.

Cython checks some struct sizes for ABI compatibility, but those are checked against `sizeof` so if you change the Python headers and then rebuild Cython it should be happy.

I don't know where your error comes from - it just looks like it isn't installed properly.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/6ba4aa41-9c88-4136-bdc1-021f80f7d628n%40googlegroups.com.

[Yuze Li]

Thanks I got it fixed. Turns out I have to use `<path-to-my-python> -m pip install -e . --no-build-isolation` to build it inplace. Not sure why but it did the trick. 

I agree that any packages that rely on cython probably shouldn't fail (if build from source) if cython can be built against my modified python.