Unable to Integrate Cython and Pyinstaller

[tyson...@gmail.com]

I am attempting to convert Python to Cython to run my pyinstaller executable to speed up the job and for obfuscation purposes. I have noticed however, that when I do this, it can be easily decompiled, the C files do not seem to be loaded correctly. Here’s the steps I have undertaken. I am doing so on windows.

HA.Py to Pyx with:

driver = webdriver.Chrome(chrome_options = options)
driver.get('https://google.com')
driver.close()
I then Convert to pyx (rename Py -> Pyx)
Set up.py
from distutils.core import setup
from Cython.Build import cythonize
setup(
    ext_modules = cythonize("HA.pyx")
)

python setup.py build_ext –inplace  (pyd, I got a so file in Unix)
C:\Python31\python.exe C:\Python31\Scripts\cython.py smalltest1.py --embed
CMD:
python setup.py develop

pyinstaller -r file_a.so,dll,file_a.so -r file_b.so,dll,file_b.so -F /C:/ANDAR

In the folder it contains the pyx, as well as the setup file.

It seems that when I compile this, it tends not to run or if I put the python file, it runs with python only and can be easily decompiled. I have tried creating a c file from Python though this did not seem to work through visual Studios.

[Robert Bradshaw]

Have you tried creating a main.py that does nothing but import your
pyx modules and invoke them? I don't think --embed is compatible with
pyinstaller.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[tyson...@gmail.com]

If you can get it working, you're a legend.  I've tried just about every variation under the sun with lots of error messages.  There doesn't seem to be a lot of documentation out there and lots of questions in relation to cython go unanswered.  I think most people with Python just accept that obfuscating your code with cython is hard as heck.. or at least in my experience.  I've spent days bug fixing this... and its not that popular so unless you have C++ team on your back hand this is a nightmare.

[Stefan Behnel]

Hi!

tyson...@gmail.com schrieb am 16.11.2017 um 01:36:
> I am attempting to convert Python to Cython to run my pyinstaller
> executable to speed up the job and for obfuscation purposes.

Let me drop a quick reminder that "obfuscation" is different from
"security". Just in case.

> I have noticed
> however, that when I do this, it can be easily decompiled, the C files do
> not seem to be loaded correctly. Here’s the steps I have undertaken. I am
> doing so on windows.
>
> HA.Py to Pyx with:
>
> driver = webdriver.Chrome(chrome_options = options)
> driver.get('https://google.com')
> driver.close()
> I then Convert to pyx (rename Py -> Pyx)

You don't need to do that. Cython can compile .py files just fine.

> Set up.py
> from distutils.core import setup
> from Cython.Build import cythonize
> setup(
> ext_modules = cythonize("HA.pyx")
> )
>
> python setup.py build_ext –inplace (pyd, I got a so file in Unix)
> C:\Python31\python.exe C:\Python31\Scripts\cython.py smalltest1.py --embed

"--embed" generates a C main() function into the module file, which can
make it executable if you compile it into an executable. Which you don't.

> CMD:
> python setup.py develop
>
> pyinstaller -r file_a.so,dll,file_a.so -r file_b.so,dll,file_b.so -F
> /C:/ANDAR

PyInstaller assumes a standard Python setup, i.e. it wants to start the
Python interpreter and then execute some Python code in it. Give it a
simple Python file as entry point, as Robert suggested. All the rest can
then be imported as compiled modules.

> In the folder it contains the pyx, as well as the setup file.

You don't need the .pyx files after compilation anymore and shouldn't
include them in the installed files if your goal is to prevent people from
seeing your code.

> It seems that when I compile this, it tends not to run or if I put the
> python file, it runs with python only and can be easily decompiled.

Allowing the Python entry point to be visible shouldn't harm the
obfuscation, if it really just imports compiled modules and calls into them.

> I have
> tried creating a c file from Python though this did not seem to work
> through visual Studios.

No idea what you mean here.

Note that PyInstaller is only one way to do these things. You can also
directly link together multiple compiled modules, and have one of them use
"--embed". That gives you one big executable file. See the "cython_freeze"
tool and the docs on compilation, especially:

http://docs.cython.org/en/latest/src/reference/compilation.html#integrating-multiple-modules

And, yes, someone should finally automate this build setup. It's really not
difficult, but it's a lot of manual work.

Stefan

[Chris Barker]

On Thu, Nov 16, 2017 at 9:57 PM, <tyson...@gmail.com> wrote:

If you can get it working, you're a legend.  I've tried just about every variation under the sun with lots of error messages.  There doesn't seem to be a lot of documentation out there and lots of questions in relation to cython go unanswered. 

we've done Cython extensions just fine with PyINstaller -- as far as PyInstaller is concerned, it's just another compiled extension.

I think the "trick" is to get your modules all to build with cython the way you want, and then worry about PyInstaller.

And maybe not try to build the WHOLE THING with Cython -- that's really not what it was designed for. At the very leas it will be easier to have a "main" module to launch everything else.

 

I think most people with Python just accept that obfuscating your code with cython is hard as heck..

Cython is not an obfuscation tool -- that is correct. use ust like it was intended, and it will work, and you'll get (a tiny bit of) obfuscation out of the deal.

-CHB

 

To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Martin Bammer]

What is important to consider is that PyInstaller does not recognize imported modules in the compiled modules. In the spec file "hidden_imports" has to be used to inform PyInstaller about all dependencies.

Just for clarification. If I run "cython --embed main.py" then main.py is not compiled, but just "linked" with the Python interpreter to an exe file?

Martin

[Stefan Behnel]

Chris Barker schrieb am 17.11.2017 um 19:02:
> Cython is not an obfuscation tool -- that is correct. use ust like it was
> intended, and it will work, and you'll get (a tiny bit of) obfuscation out
> of the deal.

I still think it would be helpful to simplify the general use case of
compiling a whole application. Nuitka is reportedly quite good at that, so
there is no reason why Cython shouldn't be. As much as I agree that "just
compile the whole thing" is usually the wrong approach and suggests that
"your problem is probably elsewhere", I don't think it helps anyone to keep
this use case difficult for people.

https://github.com/cython/cython/issues/2011

Stefan

[Stefan Behnel]

"--embed" generates a C main function into the module file. That's all it
does, no building, no linking. The user can then link against libpython
(statically or dynamically) and get an executable program out of it.

Also see

https://github.com/cython/cython/blob/0.27.3/Cython/Build/BuildExecutable.py

Help is welcome to simplify this whole use case further:

https://github.com/cython/cython/issues/2011

Stefan

[Daπid]

One stupid question:

On 16 November 2017 at 01:36, <tyson...@gmail.com> wrote:

C:\Python31\python.exe C:\Python31\Scripts\cython.py smalltest1.py --embed

Are you using Python 3.1? Cython supports 3.3+.

[Martin Bammer]

Don't use Python <3.3. As far as I've heard these old versions have serious bugs.

[Stefan Behnel]

Daπid schrieb am 18.11.2017 um 13:42:

> On 16 November 2017 at 01:36, <tysondogerz@...l.com> wrote:
>> C:\Python31\python.exe C:\Python31\Scripts\cython.py smalltest1.py --embed
>
> Are you using Python 3.1? Cython supports 3.3+.

Older releases supported 3.1, so the OP Is probably also using a rather old
Cython version.

Stefan

[Chris Barker]

On Sat, Nov 18, 2017 at 12:00 AM, Stefan Behnel <stef...@behnel.de> wrote:

Chris Barker schrieb am 17.11.2017 um 19:02:
> Cython is not an obfuscation tool -- that is correct. use ust like it was
> intended, and it will work, and you'll get (a tiny bit of) obfuscation out
> of the deal.

I still think it would be helpful to simplify the general use case of
compiling a whole application.

sure -- though it still wouldn't be an obfuscation tool.... :-)

 

Nuitka is reportedly quite good at that, so
there is no reason why Cython shouldn't be. As much as I agree that "just
compile the whole thing" is usually the wrong approach and suggests that
"your problem is probably elsewhere", I don't think it helps anyone to keep
this use case difficult for people.

Sure -- that's a fine (and shouldn't be that difficult to do) use case, if anyone want to work on better supporting it. But the OP should know that iot's not currently the case that that will "just work"

 

Also -- I've always thought that  ALL the "make an executable" tools approach or examining the source for imports was the wrong way to go :-)

So, yeah, if you Cythonize all your modules, it wont' find the imports. But you can tell it to specifically include certain things -- it'll be a long list in this use case, but not hard to build by hand, or you could:

run your app (maybe ideally the test cases) and dump out sys.modules, and you'd have a pretty good list.

-CHB