Include of .pyd files inside the .exe when --onedir option used
Czarek
so they are included inside .exe along with other .py files (they
are compiled to .pyc files right? the .py sources are not included
in the exe created by pyinstaller?).
I know I could create one .exe using --onfile, but that does not
work all the time, when creating onefile I encounter some dependencies
problems, the solution I've found is to use --onedir and copy the
missing dlls and others.
I've come across Cython recently, it allows me to compile .py
files to .pyd files, I would like to increase the security of my app
a little, by compiling my all .py files to .pyd ones (when compiling
to pyd, I also am creating an application-wrapper.py that imports
these
pyd files and runs main() from the application.py). From what I've
read .pyd files should be harder to decompile than .pyc files, is
that right?
The problem that I come across now, after compiling all .py to .pyd
files,
is that when I run pyinstaller to create exe with --onedir option, the
dist directory contains all the structure of my .py files, I don't
want
user to see all names of these files, as it tells some extra info
about
the flow of my program.
I see some solutions:
1(*). Allow me to pack my .pyd files along with the main application-
wrapper.py
that resides in the exe. (probably inside some zip - btw. how does it
work?
I tried viewing the exe using Resource Editor and it didn't show any
resources,
except the icon and manifest - I like it that it is hidden and not so
easy to view,
but also I'm curious how it's done).
2. Allow me to create a onefile exe, but by providing a dist directory
created
using --onedir option, so first I create the onedir, copy missing
dlls, then I create
final onefile exe from the contents of this directory.
3. There is a way to pack all the .pyd files into single .pyd file,
I've found it here:
http://groups.google.com/group/cython-users/msg/4587051829c63d15
But still, don't know what to do with that code provided, are these
functions
part of Cython sources? Or maybe CPython sources? Anyway I have a
feeling
that it won't be that easy.
(*) The most I would like to use solution no 1.
I tried editing .spec file and adding .pyd files to COLLECT() as an
array using
different typecodes from here:
http://www.pyinstaller.org/export/latest/tags/1.4/doc/Manual.html#toc-class-table-of-contents
But it didn't help, they were not being included inside the exe.
Czarek.
Martin Zibricky
- copy your source code directory to a tmp directory
- in this tmp directory you would replace your .py files by .pyd files.
- then you would supply this tmp directory to pyinstaller instead your
source code
- run pyinstaller with --onefile mode
- create script to automate this process
I think this way pyinstaller would automatically include your .pyd files
into one .exe executable.
Czarek píše v Ne 20. 11. 2011 v 10:54 -0800:
Czarek
I am already doing this and have this automated.
> - run pyinstaller with --onefile mode
Like I said, onefile mode does not always work, some of the
dependencies
are missing and I need to copy them manually.
Czarek.
Martin Zibricky
> Like I said, onefile mode does not always work, some of the
> dependencies
> are missing and I need to copy them manually.
Then you could try create import hook for the python package of your app
and specify there variable 'hiddenimports' where you would set names of
your missing modules in .pyd. This variable should tell pyinstaller to
include these modules with your app.
Hartmut Goebel
> I know I could create one .exe using --onfile, but that does not
> work all the time, when creating onefile I encounter some dependencies
> problems, the solution I've found is to use --onedir and copy the
> missing dlls and others.
So your .spec files does not collect all required files (dlls and
others)?! Then it's no wonder, why the created --fileone package does
not work. You should use COLLECT for all "missing dlls and others".
As Martin wrote: A hook may help.
--
Schönen Gruß - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP
Goebel Consult
Spezialist für IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de
Monatliche Kolumne: http://www.cissp-gefluester.de/
Goebel Consult ist Mitglied bei http://www.7-it.de
Czarek
mind which would require to include 50 MB or more in dll dependencies,
I'm not sure whether putting them all into one exe is a good idea
(performance reasons, but not only, there are also other reasons that
I would like not to OT here).
Would it be hard to implement an option in pyinstaller to treat .pyd
files the same as .py files and include them (hide them) into an exe?
Or an option to pack all .pyd files into one big pyd?
Czarek.
Florian Höch
> I've come across Cython recently, it allows me to compile .py
> files to .pyd files
Cython != Python. It compiles Cython code to C code. That's an important
detail.
> Would it be hard to implement an option in pyinstaller to treat .pyd
> files the same as .py files and include them (hide them) into an exe?
> Or an option to pack all .pyd files into one big pyd?
But why should .pyd files be handled differently than DLLs? They are
basically the same thing. On Linux, there is not even such a distinction
(file extension wise), it uses .so files.
And even if they were part of the executable, they would need to be
extracted at some point because (afaik, I may be wrong) you can't load
DLLs from within a packed archive (it could maybe work if the DLL was
embedded as resource in the exe, not within the archive, but that is not
currently supported).
Regards
--
Florian Höch
Hartmut Goebel
> Still, I would like to be able to use --onedir. I have one project in
> mind which would require to include 50 MB or more in dll dependencies,
> I'm not sure whether putting them all into one exe is a good idea
> (performance reasons, but not only, there are also other reasons that
> I would like not to OT here).
Parton?
In your original posting you wrote:
a) --onefile does not work at all
b) for --onedir you need to copy "required dlls"
For me this means: You packaging is not complete. So you need to fix
this first. And the first question is: Does your packages program run,
when *not* using Cython?
> Would it be hard to implement an option in pyinstaller to treat .pyd
> files the same as .py files and include them (hide them) into an exe?
We are happily including your enhancements. I personally did not worked
with Cython yet, so I do not know what hast to be done here.
--
Schönen Gruß - Regards
Hartmut Goebel
Czarek
Okay, I understand. But let's look at it from another perspective.
Cython allows me to convery each of my .py source files into .pyd
file, which are harder to decode and that's why I'm doing it. Each of
these .pyd files is treated the same as .py file, the application
works exactly the same, I only have to create a wrapper.py which makes
all imports extracted from all .py sources (so pyinstaller includes
all needed dependencies) and then I call main from the .pyd file.
> DLLs from within a packed archive (it could maybe work if the DLL was
> embedded as resource in the exe, not within the archive, but that is not
> currently supported).
That might solve the problem.
I don't know how to do that, but I know how to compile .py files
into .pyd using Cython, and I've created a tool that automatically
extracts dependencies from .py files and creates a wrapper that
imports all required modules and calls main function on main pyd file.
I would be happy to implement such tool for pyinstaller, that would
allow our applications sources to be more secure. It could be added as
an option in pyinstaller (as it requires installing Cython).
Czarek.
Czarek
> > files the same as .py files and include them (hide them) into an exe?
>
> We are happily including your enhancements. I personally did not worked
> with Cython yet, so I do not know what hast to be done here.
I can provide a script that automates the whole process of
compiling .py to .pyd files, if you would like to include it in
pyinstaller, as an option to have source better protected.
Czarek.
Hartmut Goebel
> I can provide a script that automates the whole process of
> compiling .py to .pyd files, if you would like to include it in
> pyinstaller, as an option to have source better protected.
Please submit a ticket and attach the script. Then we can discuss
integration on the issue tracker.
--
Schönen Gruß - Regards
Hartmut Goebel
Dipl.-Informatiker (univ.), CISSP, CSSLP
Goebel Consult
Spezialist für IT-Sicherheit in komplexen Umgebungen
http://www.goebel-consult.de
Monatliche Kolumne: http://www.cissp-gefluester.de/
Czarek
> integration on the issue tracker.
I've created an issue:
http://www.pyinstaller.org/ticket/459
A working script is also included.
Czarek.
Hartmut Goebel
Thanks. Can you please upload the files separately instead of a
zip-file?! This way they can be reviews much easier.
Czarek
into one big .py file. Then this file is compiled into .pyd file.
Structure of source files is well hidden. I've also added obfuscation
for function names, classes and global variables, as these symbols are
exposed via pyd interface.
Czarek.