Python executables?
Catalin
I found some utilities that claim they can do something like that like
Installer and py2exe but they actualy pack the code in a huge arhive!
This solves the problem of giving python programs to users who don't
have python but doesn't solve the problem of the source "secrecy"
(copyright).
And the programs also run much slower and become extremely big compared
to a normal C++ program for example. I made a test and a 2 programs
doing the same thing where 400 KB with C Builder (static linked) and
2.80 MB with python+installer in an arhive packed with upx and 6.9 MB
with py2exe(unpacked). And the speed difference was huge.
So can a python program become a "real" executable(I am refering both to
windows and unix platforms)?
If this is imposible with python is it possible with jpython?
Bruno Desthuilliers
Here you expose 3 different problems :
1/ source "secrecy" (copyright) :
It's the wrong problem. *Any* binary code can be subject to
reverse-engineering. There are even tools to do this quite easily for
Java. The right way to protect your property is via copyright and licence.
2/ Size of "packed" programs :
Realize that the pack must include the whole Python interpreter and
librairies. BTW, I personnaly never used such tools, but I think I
remember that some of them allow you to specify which parts you really need.
3/ 'Slowness' :
I don't believe that 'packing' the program makes it slower.
Are you sure your Python code is really Pythonic ? There are tips and
tricks in how to 'optimize' Python code, and it can be very different
from low-level (C/C++ etc) languages techniques. You may want to have a
look at :
http://manatee.mojam.com/~skip/python/fastpython.html
Now if you really need smallest possible footprint and blazing-fast
execution speed (which are antagonist needs anyway), and your program is
about low-level stuff, you may not have choosen the right tool !-)
Bruno
Aurélien Géron
> > How can I make executables with python?
> > I found some utilities that claim they can do something like that like
> > Installer and py2exe but they actualy pack the code in a huge arhive!
> > This solves the problem of giving python programs to users who don't
> > have python but doesn't solve the problem of the source "secrecy"
> > (copyright).
> > And the programs also run much slower and become extremely big compared
> > to a normal C++ program for example. I made a test and a 2 programs
> > doing the same thing where 400 KB with C Builder (static linked) and
> > 2.80 MB with python+installer in an arhive packed with upx and 6.9 MB
> > with py2exe(unpacked). And the speed difference was huge.
> > So can a python program become a "real" executable(I am refering both to
> > windows and unix platforms)?
> > If this is imposible with python is it possible with jpython?
Bruno wrote:
> Here you expose 3 different problems :
>
> 1/ source "secrecy" (copyright) :
> It's the wrong problem. *Any* binary code can be subject to
> reverse-engineering. There are even tools to do this quite easily for
> Java. The right way to protect your property is via copyright and licence.
IMHO, Catalin has a good point here. I'm no legal expert, but I believe that
copyrights and licences are not quite enough to protect your code. They just
mean that if someone uses your code without your authorisation, you *could*
theoretically sue them, but :
1- Would it be worth it to go and hire a lawyer and everything?
2- How would you prove it (or even know about it) if they just stole pieces
of your code? Or even algorithms?
3- Moreover, you may never know who hacked your code. Look at all the games
and excellent software cracked everyday: do you know who dunnit? Who would
you sue?
So why not simply compile your code and make it *harder* (although not
impossible) to decypher: it'll stop most of the potential hackers. It's
like the lock on your door: however weak it is, it'll stop most burglars
because they won't bother fighting it at all: they'll just go and look for
an unlocked house! Well... unless everyone knows there's a treasure inside
it, that is. In which case there's not much you can do against determined
hackers except to make the task difficult for them.
I agree with Bruno about Java decompilers, though : I used them many times
and I am still amazed at the quality of the decompilation process. In one
instance it even helped me recover my own code when all I had left was the
compiled result! The recovered code was neatly indented and perhaps clearer
than the original code! But there are also free "obfuscators" that make your
compiled bytecode (a lot) harder to decompile.
Python bytecode has some pretty good decompilers too.
But I don't know about any decent C decompiler. If anyone does, though, I'd
be greatly interested.
> 2/ Size of "packed" programs :
> Realize that the pack must include the whole Python interpreter and
> librairies. BTW, I personnaly never used such tools, but I think I
> remember that some of them allow you to specify which parts you really
need.
Yes, some do.
> 3/ 'Slowness' :
> I don't believe that 'packing' the program makes it slower.
>
> Are you sure your Python code is really Pythonic ? There are tips and
> tricks in how to 'optimize' Python code, and it can be very different
> from low-level (C/C++ etc) languages techniques. You may want to have a
> look at :
> http://manatee.mojam.com/~skip/python/fastpython.html
>
> Now if you really need smallest possible footprint and blazing-fast
> execution speed (which are antagonist needs anyway), and your program is
> about low-level stuff, you may not have choosen the right tool !-)
>
> Bruno
>
I don't see small footprint and fast execution speed as antagonist at all,
quite the contrary. In fact, assembly code produces the fastest and
smallest programs.
But Bruno is right, IMHO, about choosing the right tool: if you need a 50k
program calculating Pi to the 5000th decimal in 0.1 seconds... python is
definitely *not* the way to go.
Aurélien
Ben Finney
> How can I make executables with python?
Python scripts are executable, so long as they can be fed through the
Python interpreter.
> I found some utilities that claim they can do something like that like
> Installer and py2exe but they actualy pack the code in a huge arhive!
Yes. Python scripts are always interpreted (more accurately, compiled
at run-time to a Pytohn-specific bytecode, then interpreted). You can
bundle the scripts and interpreter into a huge bolus, but that only
eases transmission and installation, it doesn't prevent Python being
interpreted.
> This solves the problem of giving python programs to users who don't
> have python but doesn't solve the problem of the source "secrecy"
> (copyright).
Wrong problem. If you want to hide your source code from your users,
don't expect help from free software programmers.
> And the programs also run much slower and become extremely big compared
> to a normal C++ program for example.
Yes. Python emphasises programmer time (both the writer and the reader)
as being more valuable than CPU time. If you want something as fast as
platform-bound machine-code, use something that generates platform-bound
machine-code, like C.
(That said, much of PYthon's CPU-intensive tasks are actually
implemented as C modules, but only where the performance gain outweighs
the loss of Python code.)
> So can a python program become a "real" executable(I am refering both
> to windows and unix platforms)?
No executable bound to a particular execution model (i.e. compiled to
interface with one OS and CPU architecture) can execute on any other
architecture directly.
> If this is imposible with python is it possible with jpython?
With Jython, you still have an interpreted lagnuage, being compiled at
run-time to a bytecode and then interpreted. This brings all the
performance bottlenecks you complained of. (More experienced Jython
people will probably correct me on details, but it doesn't change the
fact that Jython isn't going to be as fast as platform-bound
machine-code.)
--
\ "It is well to remember that the entire universe, with one |
`\ trifling exception, is composed of others." -- John Andrew |
_o__) Holmes |
http://bignose.squidly.org/ 9CFE12B0 791A4267 887F520C B7AC2E51 BD41714B
ach...@easystreet.com
>
> And the programs also run much slower and become extremely big
> compared to a normal C++ program for example.
>
This is a funny thing. IBM has done some studies that showed that
users do prefer faster programs, but only 25% faster. If a program
runs more than 25% faster than what they are used to, users don't
like it. I find that despite the interpreter and whatnot, my Python
code runs about 25% faster than what users are used to when loading
and running the humongous applications of the humongous vendors.
Thus, Python is just perfect, if it was any faster, it would be
annoying.
Since you are obviously having some interest in commercial pursuits,
consider that a similar heuristic probably works now wrt program size,
another one of your concerns, as well. With big disks and CD-ROMs,
etc, program size is just about irrelevant as a practical matter
to everyone except the shareware developer who has to pay for 100
downloads per sale, but it is more important psychologically. (Even
the shareware guy doesn't really have much of a complaint, actually,
anymore. You can buy bandwidth for about $2/GB. My programs, built
with the McMillan installer including lots and lots of screens and
libraries and all kinds of run-on code, only come out 3.5 MB when
compressed. That's way under $0.01 per download at $2/GB.)
Let's say that there are two boxes on the shelf at the software store.
They both have the same price, $100. They both have the same feature
set. They both look ok. One says, "Disk space required, 25 MB." The
other one says, "Disk space required, 25 KB." Which one would you buy?
You've got to give them something to make them think that they are
getting something. It's a status thing, like silicone. How else can
the individual code writer keep up with the humongous applications
from the humongous software companies? Just like George Bush, jr, said
that the $1.6 trillion 2001 tax cut was 'just right', I think that the
3.5 MB program is 'just right'.
Do whatever is 'just right' for you.
Al
Ben Finney
> This is a funny thing. IBM has done some studies
Cite, please.
> that showed that users do prefer faster programs, but only 25% faster.
> If a program runs more than 25% faster than what they are used to,
> users don't like it.
I suspect the data would show they were studying a particular class of
program, and a particular class of user; the answer would only be
relevant to that scope.
We surely couldn't conclude that, if a web server became 60% faster in
completing its tasks, or if an interactive video game's refresh speed or
startup time became 60% faster, that "users [wouldn't] like it".
So, it's necessary to see the study to know what kind of software, and
what kind of users, were being studied; only then can we make
intelligent generalisations.
--
\ "I went to a fancy french restaurant called 'Deja Vu.' The |
`\ headwaiter said, 'Don't I know you?'" -- Steven Wright |
_o__) |
W Isaac Carroll
> I don't see small footprint and fast execution speed as antagonist at all,
> quite the contrary. In fact, assembly code produces the fastest and
> smallest programs.
When choosing between programming languages there is little correlation
between size of code and speed of execution. However, within a given
language size and speed are usually opposing considerations. For
example, to speed up a loop you often unroll it thus decreasing
execution time at the expense of increasing code size.
TTFN
Roy Smith
> When choosing between programming languages there is little correlation
> between size of code and speed of execution. However, within a given
> language size and speed are usually opposing considerations. For
> example, to speed up a loop you often unroll it thus decreasing
> execution time at the expense of increasing code size.
With the kinds of I-cache architectures common today, you can get some
unexpected effects. It's possible that the increase in code size from
unrolling a loop blows your cache and ends up running slower. If you
ever get to the point where you're worrying about loop unrolling, you'd
better really understand the hardware you're running on, and do a lot of
benchmarking to make sure you're getting the improvement you expect.
Benchmarking non-deterministic things like cache behavior is an
adventure in itself.
My general philosophy these days is to write it how it makes sense and
let the compiler take care of the optimizing. But then again, I don't
do the kind of stuff where grubbing a CPU cycle here or there makes a
difference.
Bruno Desthuilliers
> Catalin wrote:
>
>>>How can I make executables with python?
>>>I found some utilities that claim they can do something like that like
>>>Installer and py2exe but they actualy pack the code in a huge arhive!
>>>This solves the problem of giving python programs to users who don't
>>>have python but doesn't solve the problem of the source "secrecy"
>>>(copyright).
(snip)
>
> Bruno wrote:
(snip)
>>1/ source "secrecy" (copyright) :
>>It's the wrong problem. *Any* binary code can be subject to
>>reverse-engineering. There are even tools to do this quite easily for
>>Java. The right way to protect your property is via copyright and licence.
>
>
> IMHO, Catalin has a good point here. I'm no legal expert, but I believe that
> copyrights and licences are not quite enough to protect your code. They just
> mean that if someone uses your code without your authorisation, you *could*
> theoretically sue them, but :
> 1- Would it be worth it to go and hire a lawyer and everything?
> 2- How would you prove it (or even know about it) if they just stole pieces
> of your code? Or even algorithms?
> 3- Moreover, you may never know who hacked your code. Look at all the games
> and excellent software cracked everyday: do you know who dunnit? Who would
> you sue?
This point out that binarie compilation does not enforce licence and
copyright...
<troll>A simple solution is to make the code open-source !-)</troll>
> So why not simply compile your code and make it *harder* (although not
> impossible) to decypher:
Compiling also means loosing quite a great part of interpreted languages
power.
> it'll stop most of the potential hackers. It's
> like the lock on your door: however weak it is, it'll stop most burglars
> because they won't bother fighting it at all: they'll just go and look for
> an unlocked house! Well... unless everyone knows there's a treasure inside
> it, that is. In which case there's not much you can do against determined
> hackers except to make the task difficult for them.
You can choose to only deliver bytecode (.pyc) files. But you have to be
sure the user has the right interpreter version... And like Java, there
is a simple way to decompyle...
> I agree with Bruno about Java decompilers, though : I used them many times
> and I am still amazed at the quality of the decompilation process. In one
> instance it even helped me recover my own code when all I had left was the
> compiled result! The recovered code was neatly indented and perhaps clearer
> than the original code! But there are also free "obfuscators" that make your
> compiled bytecode (a lot) harder to decompile.
Doesn't this affect bytecode quality ?
> Python bytecode has some pretty good decompilers too.
>
> But I don't know about any decent C decompiler. If anyone does, though, I'd
> be greatly interested.
I guess there is none. All you can do is hack the binary code. But some
people are pretty good at this.
(snip)
>>3/ 'Slowness' :
>>I don't believe that 'packing' the program makes it slower.
>>
>>Are you sure your Python code is really Pythonic ? There are tips and
>>tricks in how to 'optimize' Python code, and it can be very different
>>from low-level (C/C++ etc) languages techniques. You may want to have a
>>look at :
>>http://manatee.mojam.com/~skip/python/fastpython.html
>>
>>Now if you really need smallest possible footprint and blazing-fast
>>execution speed (which are antagonist needs anyway), and your program is
>>about low-level stuff, you may not have choosen the right tool !-)
>
> I don't see small footprint and fast execution speed as antagonist at all,
> quite the contrary.
This seems to be a quite common observation that programs can be
optimized for speed or for size, but not both (the program being
correctly written, of course...).
> In fact, assembly code produces the fastest and
> smallest programs.
I was talking about the same program written with the same language...
BTW, I'm not sure a human programmer can beat a good compiler, unless
the code is to be 'optimized' for one specific processor, which I don't
call optimization !-)
Bruno
SFBayling
news:mailman.1056619905...@python.org:
> And the programs also run much slower and become extremely big compared
> to a normal C++ program for example.
What are you doing that the speed decrease matters?
What are you doing that is so secret?
If it really is too slow (i.e. 100 times slower than C++ is one thing, but
if it still only takes 0.1s is that really a problem?), consider writing
the slow part in C or C++ as a compiled extension. That could get you the
speed and mildly-increased-code-security you desire for the delicate parts
of your program, plus the advantages of Python for all the rest.
sfb.
Mark 'Kamikaze' Hughes
> How can I make executables with python?
> I found some utilities that claim they can do something like that like
> Installer and py2exe but they actualy pack the code in a huge arhive!
> This solves the problem of giving python programs to users who don't
> have python but doesn't solve the problem of the source "secrecy"
> (copyright).
You can use python -OO to produce .pyo "optimized" files, and then
just ship those. .pyo isn't truly optimized or completely binary, but
it's about as secure as Python normally gets. .pyo classes are tied to
specific versions of Python, but if you're using py2exe, that's not a
problem.
Anyone with a decompiler will still be able to get at them, and there
are AFAIK no obfuscators for Python, but at least it closes the door,
even if it doesn't lock it.
Ultimately, the only way to have full secrecy is to keep all your code
on a trusted server (and hope nobody ever breaks in), and only give the
users a minimal dumb client.
--
<a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>
"We remain convinced that this is the best defensive posture to adopt in
order to minimize casualties when the Great Old Ones return from beyond
the stars to eat our brains." -Charlie Stross, _The Concrete Jungle_
Geoff Howland
<bignose-h...@and-zip-does-too.com.au> wrote:
>> This solves the problem of giving python programs to users who don't
>> have python but doesn't solve the problem of the source "secrecy"
>> (copyright).
>
>Wrong problem. If you want to hide your source code from your users,
>don't expect help from free software programmers.
I thought about this for a few days before responding (and Im sure I
did a few other things too ;) ), but I wanted to comment on this.
I think everyone that uses Python wants it to gain acceptance for the
great language that it is. I believe that stating an attitude in this
way is pretty counter productive in gaining any kind of wide-spread
acceptance.
Most of the replies to this request didn't mention the concept of NOT
protecting the software, but 2 did to different degrees. As someone
who uses and like open source software, and is slowly starting to
release some things as open source, and ALSO someone who sells
software and will continue in the future to sell software, I can say
that nothing turns me off more to a community than being told what my
goals should be.
I can understand wanting everything to be open, but thats not reality
and it never will be. Some people will always want things
proprietary, and they will only work within systems that allow that.
I think to be truly successful, systems will have to allow for this
and make it easy to do.
Currently Python does not make this REALLY easy to do, and in the
privacy portion, I believe its not even possible. This was a big
concern for me when I released my last for-sale software, but I just
decided I didn't care that much, and I love working in Python.
Some people will care enough, and will avoid Python because the
ability to protect their end results aren't there.
So far, the only semi-workable way to do this would be something like:
- Build a C program that embeds Python.
- Encrypt all the Python script/bytecode files.
- On runtime, decrypt the files and then execute.
Optional:
- Change the bytecode values in the Python source, and include your
new Python with different bytecode values.
I tried this last thing just to see if it would work, and I got some
problems compiling initially, so just gave up, but I think in theory
it should work.
Ignoring the Optional portion, this semi-solution is not actually very
secure. It does however move the problem into having to decompile a C
program, and then get it to decrypt the Python files. Then the normal
Python bytecode -> source. It means any cracker will have to know
something about both C and Python to do it, so a bit more barrier to
entry. It also means that in the US, the (vile and despicable, but
present) DMCA laws will make it a much more severe crime because
"cryptography reverse engineering" needed to be applied, and may at
least reduce corporations from doing this for fear of
lawsuits/criminal charges if they are exposed.
Anyway, this is a good bit of work and not a great solution, but is
there anything else? If Python is to have avenues of support in all
walks of the software industry, will something like this be required?
From what I understand, there are also very good Java decompilers, but
no one seems to complain about Java's lack of security. Perhaps it is
because it is seen as something that is really "compiled" while
Python's loose compilation seems more whimsicle.
I think Python faces a lot of different public relations problems, and
just thought I'd pipe up about one that I have looked at myself, and
that I think most people coming into the Python world are faced with
and have to decide whether to ignore or not.
-Geoff Howland
http://ludumdare.com/