I've spent a few evenings working on a project to be able to write
Android applications in Python by compiling Python source into Java
source. I thought I'd share what I have in case there is interest in
this (maybe even outside of Android development). I see this project
having two parts, a Python to Java converter and a very basic
simulated/stubbed out Java/Android API (so that one could write unit
tests in python and run them in python, before compiling everything
into Java).
Of course 100% Python to Java is not possible or practical to
implement in a reasonable amount of time but I think enough could be
implemented so that we could have 100% Python syntax with maybe 50%
Python semantics and 50% Java semantics. I think just getting to that
point would be a huge productivity gain on working with any java
frameworks/APIs. Over time I think there can be work arounds for
things that Java can't do like multiple inheritance, dynamic class
creation, duck typing, etc.
So, how does it work? I use Pythons ast and symtable modules to walk
the source code and try to infer the possible types that a variable
can have. This gives me the type information that Java will need. Then
another pass generates Java source using this type information. There
is nothing particularly magical especially since duck typing isn't
implemented and you can't change the type of a variable after you've
assigned to it the first time. So you have to be very careful with the
Python code you write, think of it as statically typed Python where
you don't have to declare types since they are inferred. I haven't
gotten to lists and dictionaries yet but the principles will be the
same, you can't have heterogeneous lists so the first thing you put in
a list will provide that lists type.
Why not use PyPy? It's possible to achieve some of my goals with
PyPy and in some cases it will achieve a lot more than I will ever
attempt to implement. One advantage of PyPy is that it supports a lot
more of Pythons dynamism (for one, PyPy generates code from a loaded
Python module while py2j uses static python source files). There are
two major problems with PyPy for my purposes: 1) it generates
specialized jvm byte code that the dalvik vm cannot convert 2) because
PyPy is more pythonic it will be much harder to compile code in a way
that will just plugin into the Android APIs, with PyPy you can't
import Java packages or inherit from Java classes as you would have to
do to write Android apps.
On Thu, Sep 1, 2011 at 12:08 PM, eukreign <eukre...@gmail.com> wrote: > Hello,
> I've spent a few evenings working on a project to be able to write > Android applications in Python by compiling Python source into Java > source. I thought I'd share what I have in case there is interest in > this (maybe even outside of Android development). I see this project > having two parts, a Python to Java converter and a very basic > simulated/stubbed out Java/Android API (so that one could write unit > tests in python and run them in python, before compiling everything > into Java).
> Of course 100% Python to Java is not possible or practical to > implement in a reasonable amount of time but I think enough could be > implemented so that we could have 100% Python syntax with maybe 50% > Python semantics and 50% Java semantics. I think just getting to that > point would be a huge productivity gain on working with any java > frameworks/APIs. Over time I think there can be work arounds for > things that Java can't do like multiple inheritance, dynamic class > creation, duck typing, etc.
> So, how does it work? I use Pythons ast and symtable modules to walk > the source code and try to infer the possible types that a variable > can have. This gives me the type information that Java will need. Then > another pass generates Java source using this type information. There > is nothing particularly magical especially since duck typing isn't > implemented and you can't change the type of a variable after you've > assigned to it the first time. So you have to be very careful with the > Python code you write, think of it as statically typed Python where > you don't have to declare types since they are inferred. I haven't > gotten to lists and dictionaries yet but the principles will be the > same, you can't have heterogeneous lists so the first thing you put in > a list will provide that lists type.
> Why not use PyPy? It's possible to achieve some of my goals with > PyPy and in some cases it will achieve a lot more than I will ever > attempt to implement. One advantage of PyPy is that it supports a lot > more of Pythons dynamism (for one, PyPy generates code from a loaded > Python module while py2j uses static python source files). There are > two major problems with PyPy for my purposes: 1) it generates > specialized jvm byte code that the dalvik vm cannot convert 2) because > PyPy is more pythonic it will be much harder to compile code in a way > that will just plugin into the Android APIs, with PyPy you can't > import Java packages or inherit from Java classes as you would have to > do to write Android apps.
> I would say it's still more of a prototype than even alpha level > code. You can see an example of input file and output in the samples > directory.
> Also, the bottom of both analyze.py and compiler.py have a ton of > unit tests you can look at to see what is possible so far.
> If you have questions or are interested in helping me please send me > an email.
> - lex
> -- > You received this message because you are subscribed to the Google > Groups "Android Developers" group. > To post to this group, send email to android-developers@googlegroups.com > To unsubscribe from this group, send email to > android-developers+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/android-developers?hl=en
Lex,
Sounds like a fun hobby project, but also a lot of work if you *actually* want it to work well ;-). If you want to just get a quick and dirty solution, perhaps you should look here:
You can't write an Android application using android-scripting. Even
if you did mange to do it somehow you'd still need to bundle Python
with it.
My goal is to be able to write a full blown app using Activities,
Services, etc in Python and then translate that to Java so that it
could be packaged like any other Android app with no dependencies on
Python.
- lex
On Sep 1, 12:17 pm, Kristopher Micinski <krismicin...@gmail.com>
wrote:
> On Thu, Sep 1, 2011 at 12:08 PM, eukreign <eukre...@gmail.com> wrote:
> Lex,
> Sounds like a fun hobby project, but also a lot of work if you
> *actually* want it to work well ;-). If you want to just get a quick
> and dirty solution, perhaps you should look here:
On Thu, Sep 1, 2011 at 12:22 PM, eukreign <eukre...@gmail.com> wrote: > You can't write an Android application using android-scripting. Even > if you did mange to do it somehow you'd still need to bundle Python > with it.
> My goal is to be able to write a full blown app using Activities, > Services, etc in Python and then translate that to Java so that it > could be packaged like any other Android app with no dependencies on > Python.
> - lex
> On Sep 1, 12:17 pm, Kristopher Micinski <krismicin...@gmail.com> > wrote: >> On Thu, Sep 1, 2011 at 12:08 PM, eukreign <eukre...@gmail.com> wrote: >> Lex,
>> Sounds like a fun hobby project, but also a lot of work if you >> *actually* want it to work well ;-). If you want to just get a quick >> and dirty solution, perhaps you should look here:
> -- > You received this message because you are subscribed to the Google > Groups "Android Developers" group. > To post to this group, send email to android-developers@googlegroups.com > To unsubscribe from this group, send email to > android-developers+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/android-developers?hl=en
<krismicin...@gmail.com> wrote: > On Thu, Sep 1, 2011 at 12:08 PM, eukreign <eukre...@gmail.com> wrote: >> Hello,
>> I've spent a few evenings working on a project to be able to write >> Android applications in Python by compiling Python source into Java >> source. I thought I'd share what I have in case there is interest in >> this (maybe even outside of Android development). I see this project >> having two parts, a Python to Java converter and a very basic >> simulated/stubbed out Java/Android API (so that one could write unit >> tests in python and run them in python, before compiling everything >> into Java).
>> Of course 100% Python to Java is not possible or practical to >> implement in a reasonable amount of time but I think enough could be >> implemented so that we could have 100% Python syntax with maybe 50% >> Python semantics and 50% Java semantics. I think just getting to that >> point would be a huge productivity gain on working with any java >> frameworks/APIs. Over time I think there can be work arounds for >> things that Java can't do like multiple inheritance, dynamic class >> creation, duck typing, etc.
>> So, how does it work? I use Pythons ast and symtable modules to walk >> the source code and try to infer the possible types that a variable >> can have. This gives me the type information that Java will need. Then >> another pass generates Java source using this type information. There >> is nothing particularly magical especially since duck typing isn't >> implemented and you can't change the type of a variable after you've >> assigned to it the first time. So you have to be very careful with the >> Python code you write, think of it as statically typed Python where >> you don't have to declare types since they are inferred. I haven't >> gotten to lists and dictionaries yet but the principles will be the >> same, you can't have heterogeneous lists so the first thing you put in >> a list will provide that lists type.
>> Why not use PyPy? It's possible to achieve some of my goals with >> PyPy and in some cases it will achieve a lot more than I will ever >> attempt to implement. One advantage of PyPy is that it supports a lot >> more of Pythons dynamism (for one, PyPy generates code from a loaded >> Python module while py2j uses static python source files). There are >> two major problems with PyPy for my purposes: 1) it generates >> specialized jvm byte code that the dalvik vm cannot convert 2) because >> PyPy is more pythonic it will be much harder to compile code in a way >> that will just plugin into the Android APIs, with PyPy you can't >> import Java packages or inherit from Java classes as you would have to >> do to write Android apps.
>> I would say it's still more of a prototype than even alpha level >> code. You can see an example of input file and output in the samples >> directory.
>> Also, the bottom of both analyze.py and compiler.py have a ton of >> unit tests you can look at to see what is possible so far.
>> If you have questions or are interested in helping me please send me >> an email.
>> - lex
>> -- >> You received this message because you are subscribed to the Google >> Groups "Android Developers" group. >> To post to this group, send email to android-developers@googlegroups.com >> To unsubscribe from this group, send email to >> android-developers+unsubscribe@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/android-developers?hl=en
> Lex,
> Sounds like a fun hobby project, but also a lot of work if you > *actually* want it to work well ;-). If you want to just get a quick > and dirty solution, perhaps you should look here:
> -- > You received this message because you are subscribed to the Google > Groups "Android Developers" group. > To post to this group, send email to android-developers@googlegroups.com > To unsubscribe from this group, send email to > android-developers+unsubscribe@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/android-developers?hl=en
On Thu, Sep 1, 2011 at 12:38 PM, Michael Banzon <mich...@banzon.dk> wrote: > Is there any particular reason to translate the python code to Java?
> Couldn't that step be omitted by making a python to Dalvik bytecode compiler?
I don't see any reason to translate to down to dalvik if you can target java instead, the java compiler does all the nasty work, and the toolchain already works. The extra work of constructing the dex file, doing the compilation, analysis, etc... is quite a bit...
On Sep 1, 12:29 pm, Kristopher Micinski <krismicin...@gmail.com>
wrote:
> Fair enough, sorry, I didn't mean to sound brush-offy. Is there not
> another python source to source compiler.
> Kris
I have searched and have not found any other Python source to Java
source compiler. My guess is that it's not practical to implement
something like this as a general solution so nobody has bothered to. I
do not intend to make a general purpose Python to Java converter, it
will be only for a very restricted set of Python semantics and will
require that in Python code you import from java namespace. I do not
intend to make any Python standard library available.
I will do the same sort of thing for any Java API that I would want
unit testable from Python. When py2j runs on a .py file and it starts
importing packages, if it sees that something is annotated with
@java.Class() it will assume that this exists in the Java world and
thus will leave the import statement "junit.framework.TestCase" but
will no convert any classes.
In other words, from Python you would have to access the Java API and
a layer would exist to mimick the java functionality in Python (it
would go in that lib directory). I will write some scripts to generate
the entire Android API and actually implement any methods that would
be useful for unit testing.
Built-in python methods like string.replace() or len(), etc I would re-
implement in Java so that the semantics would translate over directly.
My other goal is to make the generated Java code actually readable and
so that you could pass it off as hand written to some extent if you
wanted :-)
On Sep 1, 12:37 pm, Alessandro Pellizzari <a...@amiran.it> wrote:
> Being completely ignorant on the matter, I would like to ask you if you
> had a look at Jython, and, if yes, why you discarded it.
Last time I checked this was not possible. There have been several
projects to try and make this work and they have either been abandoned
or never really got going. If/when someone does get this working you'd
be running one virtual machine inside another virtual machine ... on a
cellphone.
With py2j you end up with Java source which is then compiled into
simple java byte code by the java compiler of your choice and then
translated to dalvik byte code by the android tools that were designed
and optimized to do this sort of thing.
Also, by targeting Java source you can tie into some of the existing
Android tools that google developed for Eclipse.
Python then becomes like a macro language for Java boosting
productivity significantly. You'd write a quarter of the code, unit
tests would run lightning fast and you could, if you wanted to, write
your entire android app in a single concise .py file. :-D
I believe that the javac compiler and then the java byte code to
dalvik byte code converter apply all sorts of optimizations. To build
a Python to dalvik byte code compiler would require a lot of intimate
knowledge of how this process works. I don't really want to go that
far down the rabbit hole, I just want to write Android apps in Python
and py2j is the quickest path between point A and point B.
The other advantage of py2j vs. generating byte code is that debugging
is easier. After you generate the Java Source you can use Eclipse to
debug the Java code directly. With a Python to dalvik byte code
compiler someone would have to implement a debugger that can match up
a line of Python to a particular dalvik byte code, etc.
By going source to source you can continue to use all of the existing
Android and Java source tools.
My goal is not to eliminate knowledge of Java, just to make
development much more productive.
This is kind of a niche project for people who know both Python and
Java relatively well and would prefer to express their application in
Python but still build & deploy Android apps using the native
toolchain.
- lex
On Sep 1, 12:38 pm, Michael Banzon <mich...@banzon.dk> wrote:
> Is there any particular reason to translate the python code to Java?
> Couldn't that step be omitted by making a python to Dalvik bytecode compiler?
> On Thu, Sep 1, 2011 at 6:17 PM, Kristopher Micinski
> <krismicin...@gmail.com> wrote:
> > On Thu, Sep 1, 2011 at 12:08 PM, eukreign <eukre...@gmail.com> wrote:
> >> Hello,
> >> I've spent a few evenings working on a project to be able to write
> >> Android applications in Python by compiling Python source into Java
> >> source. I thought I'd share what I have in case there is interest in
> >> this (maybe even outside of Android development). I see this project
> >> having two parts, a Python to Java converter and a very basic
> >> simulated/stubbed out Java/Android API (so that one could write unit
> >> tests in python and run them in python, before compiling everything
> >> into Java).
> >> Of course 100% Python to Java is not possible or practical to
> >> implement in a reasonable amount of time but I think enough could be
> >> implemented so that we could have 100% Python syntax with maybe 50%
> >> Python semantics and 50% Java semantics. I think just getting to that
> >> point would be a huge productivity gain on working with any java
> >> frameworks/APIs. Over time I think there can be work arounds for
> >> things that Java can't do like multiple inheritance, dynamic class
> >> creation, duck typing, etc.
> >> So, how does it work? I use Pythons ast and symtable modules to walk
> >> the source code and try to infer the possible types that a variable
> >> can have. This gives me the type information that Java will need. Then
> >> another pass generates Java source using this type information. There
> >> is nothing particularly magical especially since duck typing isn't
> >> implemented and you can't change the type of a variable after you've
> >> assigned to it the first time. So you have to be very careful with the
> >> Python code you write, think of it as statically typed Python where
> >> you don't have to declare types since they are inferred. I haven't
> >> gotten to lists and dictionaries yet but the principles will be the
> >> same, you can't have heterogeneous lists so the first thing you put in
> >> a list will provide that lists type.
> >> Why not use PyPy? It's possible to achieve some of my goals with
> >> PyPy and in some cases it will achieve a lot more than I will ever
> >> attempt to implement. One advantage of PyPy is that it supports a lot
> >> more of Pythons dynamism (for one, PyPy generates code from a loaded
> >> Python module while py2j uses static python source files). There are
> >> two major problems with PyPy for my purposes: 1) it generates
> >> specialized jvm byte code that the dalvik vm cannot convert 2) because
> >> PyPy is more pythonic it will be much harder to compile code in a way
> >> that will just plugin into the Android APIs, with PyPy you can't
> >> import Java packages or inherit from Java classes as you would have to
> >> do to write Android apps.
> >> I would say it's still more of a prototype than even alpha level
> >> code. You can see an example of input file and output in the samples
> >> directory.
> >> Also, the bottom of both analyze.py and compiler.py have a ton of
> >> unit tests you can look at to see what is possible so far.
> >> If you have questions or are interested in helping me please send me
> >> an email.
> >> - lex
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Android Developers" group.
> >> To post to this group, send email to android-developers@googlegroups.com
> >> To unsubscribe from this group, send email to
> >> android-developers+unsubscribe@googlegroups.com
> >> For more options, visit this group at
> >>http://groups.google.com/group/android-developers?hl=en
> > Lex,
> > Sounds like a fun hobby project, but also a lot of work if you
> > *actually* want it to work well ;-). If you want to just get a quick
> > and dirty solution, perhaps you should look here:
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Android Developers" group.
> > To post to this group, send email to android-developers@googlegroups.com
> > To unsubscribe from this group, send email to
> > android-developers+unsubscribe@googlegroups.com
> > For more options, visit this group at
> >http://groups.google.com/group/android-developers?hl=en
On Thu, Sep 1, 2011 at 7:10 PM, Kristopher Micinski
<krismicin...@gmail.com> wrote: > On Thu, Sep 1, 2011 at 12:38 PM, Michael Banzon <mich...@banzon.dk> wrote: >> Is there any particular reason to translate the python code to Java?
>> Couldn't that step be omitted by making a python to Dalvik bytecode compiler?
> I don't see any reason to translate to down to dalvik if you can > target java instead, the java compiler does all the nasty work, and > the toolchain already works. The extra work of constructing the dex > file, doing the compilation, analysis, etc... is quite a bit...
But... But... Thats the _fun_ part :o)
I get the point - as mentioned in another mail - the purpose is to write working stuff in python not re-invent the wheel :-P
