Parsing Python code to C code with Cython

[Viktor Depunkt]

Hello there!

I am an mechanical engineer with a passion for programming and I am currently working on my final thesis. The topic includes writing a program for an on-satellite experiment in Python. Since the on-board computer (OBC) of the satellite will most likely not support Python directly, the idea was to use Cython to translate my CPython code to a code the OBC can work with.

However, I am as of yet unsure whether or not this is actually possible and, if it is possible, how to do it.

What I have done so far is following Mr. Behnel's example in one of his presentations (page 23) and this actually resulted in an executable that prints "Hello World" to the console on a RaspberryPi with Python 2.7. However, repeating the same steps on my Windows 10 PC while replacing all the "2.7" with "3.9" does not work and I am anyways unsure if having an executable is what I need.

I have spent many hours on this problem and I am only going in cirlces at this point so any help to find the right path would be greatly appreciated.

Thanks in advance

Viktor

[Golden Rockefeller]

That looks like an older version of cython, and an older version of Python, so I don't know if you can find up-to-date information that will help you make an executable in this way. However, you can make an executable with py2exe or PyInstaller.

I have compiled an ".exe" before using py2exe for code that used Cython-compiled code.  Py2exe only works for making Windows-compatible executables, but I doubt your on-board computer will use Windows. If your cython code is already compiled to a .so or a .pyd file (e.g. compiled using cythonize+setuptool), then Python and other packages will treat it as a Python C extension. Under that assumption I think you can try PyInstaller for compiling to OS X, GNU/Linux, Windows and (maybe?) others.

Let me know what works.

- Golden

[da-woods]

The "build an executable" feature from Cython does still exist and it does still work. Have a look at the "Embed demo" that's shipped as part of Cython: https://github.com/cython/cython/tree/master/Demos/embed

However, that executable still depends on the Python runtime (libpython) and any Python modules that you use. Therefore if your on-board computer doesn't support Python then it doesn't actually solve your problem. (Py2exe and PyInstaller and similar also don't solve this problem).

You might look at MicroPython supports the on-board computer you're trying to use?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/533c5de2-d355-4061-ba1a-dae6fcf408f7n%40googlegroups.com.

[Viktor Depunkt]

Thank you for your replies!

So if I get it right, then you are not agreeing on whether or not creating an executable will enable me to run my program on a Real-Time Operating System (that is what it will most probably come down to, we're planning on using FreeRTOS)?

@Golden, you seem to be positive that it could work but @da-woods, you say there is the problem that the executable still needs to work with the Python interpreter (did I get that right?). But isn't the idea of an executable that it contains machine code that does not need any interpretation anymore? Or is it that there are only some parts that are compiled into machine code and those then link to parts "outside", Python code for example? This might be a basic question, but my ME courses at uni didn't teach me much (or anything tbh) about computer science unfortunately...

Regards

Viktor

[John Ehresman]

I’m fairly sure that any executable created from Cython generated .c code will have the Python runtime linked in. The Python runtime is a library of C implemented functions that is used to run Python programs. It might be possible to avoid using Python bytecode, but you would need to avoid using most libraries and the benefits may not be that great.

Cython was developed to be used with Python, not to replace it.

The questions you probably need to answer are: can the Python runtime be used? And, is the performance of a Cython based program acceptable?

John

> To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/8a94eeae-21af-4d66-a4c6-95b777db4c45n%40googlegroups.com.

[da-woods]

Yes, to clarify there's a lot to the Python runtime beyond "the Python
interpreter":
* If you create a Python object then it's the Python runtime that
handles its reference counting.
* If you access an attribute of that object then it uses the internal
Python function "PyObject_GetAttrString".
* If you use a dict or a list or a tuple or a str or anything  else like
that then you're using the implementations built into the Python runtime.
* If you use any standard library modules then you're using their
implementation from the Python runtime (or you're using the interpreter,
depending on whether they're written in C or not).
Cython only translates Python(ish) code into C code that calls the
Python runtime.

So Cython doesn't solve your problem. It may even make your problem
worse - Cython puts a big emphasis on code being fast rather than code
being small.

However - to repeat: MicroPython is an implementation of Python
specifically written to run on this kind of small computer. It probably
has some limitations but it is probably your most likely solution. I
have never used it so I could not comment more.

[Golden Rockefeller]

""" @Golden, you seem to be positive that it could work"""

On FreeRTOS? No, not confident that it will work there. Maybe if the onboard computer was like a Rasberrypi that could run a version of linux (doesn't necessary need to have Python installed on the computer if you can deploy an executable compiled with PyInstaller).

""" But isn't the idea of an executable that it contains machine code that does not need any interpretation anymore?"""

Yes, but one of the problems is whether you can compile the executable down to machine code for your target operating system, and Pyinstaller can compile to Windows, Mac, Linux and some others but I don't know about FreeRTOS. If you are looking for something that is not interpreted, or perhaps allows for real-time execution, then I don't know whether you can do that with Python. Micropython, maybe, but I don't know. I doubt that most things compiled by Cython would conform to real-time constraints due to automatic memory management.

[Viktor Depunkt]

Thank you all for your replies and sorry that it took so long for me to reply but I am nearing my deadline and it's been very busy :)

So if I get this right (please correct me if I don't), the problem is that code created by Cython needs to access the Python runtime (which is, as John Ehresman pointed out, written in C) and there is no way around it but (possibly) to strip down my code to a point where I only rely on the functions that have been implemented in C anyways.

Under this premise, my question now is: can't I just install the Python runtime on my OBC and go from there? This should be possible, since the runtime has been written in C, right? Or would that just mean that I install Python itself and then I have basically no need for Cython anyways? Would this, then, mean that there would have to be a Python distribution that supports FreeRTOS?

Again, very basic questions but this is very interesting to me and very far from what I have come in contact with up to now, so thanks for your patience!