Re: Python in the browser: Merge efforts into one project?

71 views
Skip to first unread message

Amirouche Boubekki

unread,
Nov 20, 2013, 7:54:05 AM11/20/13
to pythoni...@googlegroups.com, pyjs-...@googlegroups.com


On Wednesday, November 20, 2013 1:25:06 PM UTC+1, Amirouche Boubekki wrote:
Héllo,


Pythonium [0] creator here. I write this email to ask the community what they think about merging the different efforts to have Python in the browser.


What do you think about merging Python in the browser effort into one unique project?


What can prevent you from contributing to such a projects? Mind the fact, that I'm willing to change the license to a more liberal license (LGPLv3) if people come and contribute to the project.


Frankly, I think pythonium approach is superior to every other solutions. Anything other solution does, Pythonium can do, except it's faster.

If you don't know Pythonium, basicly current version use Python 3 ast parser to generate, fast javascript code unlike Brython and Skulpt and like pyjaco, pyjs pyjeon which is written in javascript. Unlike Brython and Skulpt which are self hosted.

Like I said, it can be self-hosted, but with proper tooling [1] we don't need that.

Also, I don't care about the name and we can take whatever name the community think is best.


So what's do you think?


Make sure to have pythonium-users@googlegroups.com in the reponse header, so that other people can follow the conversation.


Best regards,


Amirouche



[0] https://github.com/pythonium/pythonium
[1] Integration with frameworks, compile server with inotify, backend side compilation

Lex Berezhny

unread,
Nov 20, 2013, 9:39:10 AM11/20/13
to pyjs-...@googlegroups.com, pythoni...@googlegroups.com

Could you change it to apache license? That is what pyjs uses.

I could not find the Pythonium Veloce source mentioned in the readme, that seems closest to the problem that pyjs is trying to solve.

--
 
---
You received this message because you are subscribed to the Google Groups "Pyjs.org Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyjs-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Amirouche Boubekki

unread,
Nov 20, 2013, 12:43:55 PM11/20/13
to Lex Berezhny, pyjs-...@googlegroups.com, pythoni...@googlegroups.com



2013/11/20 Lex Berezhny <l...@damoti.com>

Could you change it to apache license? That is what pyjs uses.

I could not find the Pythonium Veloce source mentioned in the readme, that seems closest to the problem that pyjs is trying to solve.

It's not written yet.

Lex Berezhny

unread,
Nov 20, 2013, 2:06:57 PM11/20/13
to pyjs-...@googlegroups.com, pythoni...@googlegroups.com
Hi Amirouche,

Can you provide some more details on what you see being merged?

If you are asking whether you can become a contributor on pyjs then I would say of course!

I reviewed some of the Pythonium Core code and I'm not sure if any of that would be worth merging with pyjs. It seems like very preliminary experimental implementation.

The difference between pyjs and literally every other Python to JavaScript implementation out there is that pyjs actually supports 95% of Python semantics... including multiple inheritance, etc.

If you are looking to just translate Python to JavaScript source there are a lot of solutions that do this already (you've mentioned many of them yourself).

Personally, unless the eventual goal is 100% Python support I don't see any point in these semantic-free translators. I think CoffeeScript fills that void very well especially since most IDE now-a-days provide excellent support for it (which you wouldn't have with a JavaScript semantic+Python syntax hybrid language).

If you just want an alternative to writing JS syntax I would highly recommend CoffeeScript. And if you want to re-use code between CPython and a web browser than you can't really avoid implementing as much of Python semantics as possible. Pyjs gets enough of the semantics implemented to be productive but once you start trying to build frameworks or more complex code the remaining 5% of missing semantics really gets in the way. That's where we need the most help and effort.

 - lex

Amirouche Boubekki

unread,
Nov 20, 2013, 4:01:57 PM11/20/13
to Lex Berezhny, pyjs-...@googlegroups.com, pythoni...@googlegroups.com

2013/11/20 Lex Berezhny <l...@damoti.com>

Hi Amirouche,

Can you provide some more details on what you see being merged?

We could share demos, compatibility test suite, documentation, etc...
 
If you are asking whether you can become a contributor on pyjs then I would say of course!

The thing is in Pythonium we get the extra feature that the three flavors of the compiler (Core, Veloce & Compliant) will be somewhat compatible so that people can downgrade easly from one version to another. This is the extra feature I really like, since it allows to create in pure Python and then have the extra flexibility to optimise critical paths.
 
I reviewed some of the Pythonium Core code and I'm not sure if any of that would be worth merging with pyjs. It seems like very preliminary experimental implementation.

Actually it's the 4th complete rewrite I do. Pythonium Core flavor, the only code available yet, only aims at supporting JavaScript semantic with a Python syntax.
 
The difference between pyjs and literally every other Python to JavaScript implementation out there is that pyjs actually supports 95% of Python semantics... including multiple inheritance, etc.

Personally, unless the eventual goal is 100% Python support I don't see any point in these semantic-free translators.

It is aiming 100% support.
 
If you just want an alternative to writing JS syntax I would highly recommend CoffeeScript. And if you want to re-use code between CPython and a web browser than you can't really avoid implementing as much of Python semantics as possible. Pyjs gets enough of the semantics implemented to be productive but once you start trying to build frameworks or more complex code the remaining 5% of missing semantics really gets in the way. That's where we need the most help and effort.

Thanks for pointing me in the right direction.

Lex Berezhny

unread,
Nov 20, 2013, 5:07:04 PM11/20/13
to pyjs-...@googlegroups.com, pythoni...@googlegroups.com
On Wed, Nov 20, 2013 at 4:01 PM, Amirouche Boubekki <amirouche...@gmail.com> wrote:

2013/11/20 Lex Berezhny <l...@damoti.com>
Hi Amirouche,

Can you provide some more details on what you see being merged?

We could share demos, compatibility test suite, documentation, etc...


I'm not sure about demos and documentation (given our APIs/goals might be different) but I can see there being a benefit to a shared compatibility test suite.

I think it would be cool if there was a "test suite" project that you could then pass in your python->js compiler function and it would run a whole bunch of tests and return the results. There is definitely lots of merit there.

I can't imagine it would be hard to convince other projects to also use and contribute to a shared test suite given that testing Python compatibility is going to be the same regardless of your particular translator implementation.

For each test the suite would use the supplied translator function to convert Python->JavaScript then execute it in a JS engine and check state of output.

Someone just has to start such a project.


If you are asking whether you can become a contributor on pyjs then I would say of course!

The thing is in Pythonium we get the extra feature that the three flavors of the compiler (Core, Veloce & Compliant) will be somewhat compatible so that people can downgrade easly from one version to another. This is the extra feature I really like, since it allows to create in pure Python and then have the extra flexibility to optimise critical paths.


In Pyjs you can get dozens of flavors by passing different arguments to the translator command. You can achieve Pythonium Core by using the --enable-stupid-mode translator flag. Take a look at some of the available translator flags:


  -h, --help            show this help message and exit
  -o OUTPUT, --output=OUTPUT
                        Place the output into <output>
  -m MODULE_NAME, --module-name=MODULE_NAME
                        Module name of output
  -i, --list-imports    List import dependencies (without compiling)
  -D, --enable-default  (group) enable DEFAULT options
  -d, --enable-debug    (group) enable DEBUG options
  -O, --enable-speed    (group) enable SPEED options, degrade STRICT
  -S, --enable-strict   (group) enable STRICT options, degrade SPEED
  --enable-wrap-calls   enable call site debugging [False]
  --enable-print-statements
                        enable printing to console [True]
  --enable-check-args   enable function argument validation [False]
  --enable-check-attrs  enable attribute validation [False]
  --enable-accessor-proto
                        enable __get/set/delattr__() accessor protocol [True]
  --enable-bound-methods
                        enable proper method binding [True]
  --enable-descriptor-proto
                        enable __get/set/del__ descriptor protocol [False]
  --enable-track-sources
                        enable tracking original sources [False]
  --enable-track-lines  enable tracking original sources: every line [False]
  --enable-store-sources
                        enable storing original sources in javascript [False]
  --enable-inline-code  enable bool/eq/len inlining [False]
  --enable-operator-funcs
                        enable operators-as-functions [True]
  --enable-number-classes
                        enable float/int/long as classes [False]
  --enable-locals       enable locals() [False]
  --enable-stupid-mode  enable minimalism by relying on javascript-isms
                        [False]
  --use-translator=TRANSLATOR
                        override translator [proto]


I reviewed some of the Pythonium Core code and I'm not sure if any of that would be worth merging with pyjs. It seems like very preliminary experimental implementation.

Actually it's the 4th complete rewrite I do. Pythonium Core flavor, the only code available yet, only aims at supporting JavaScript semantic with a Python syntax.


I don't mean to diminish your work in anyway but I'm just not sure I understand the point of Pythonium Core given all of the existing solutions out there. There are dozens of Python->JavaScript translators that ignore semantics.

  
The difference between pyjs and literally every other Python to JavaScript implementation out there is that pyjs actually supports 95% of Python semantics... including multiple inheritance, etc.

Personally, unless the eventual goal is 100% Python support I don't see any point in these semantic-free translators.

It is aiming 100% support.


It took many years for Pyjs to get to where it is today. I hope you are successful in your efforts. I think I speak for more than just myself when I say it would be awesome to have a new clean implementation of the Python semantics. Pyjs works but it is rather crufty and difficult to maintain.

Have you come up with a speedy solution for true multiple inheritance and attribute lookups (part of the 5% missing from Pyjs)? I have a prototype that I've thrown together that uses JavaScript prototypes to create an inheritance chain, it even supports super classes modifying their own bases and this propagates to sub-classes (including instances of subclasses) and because it uses prototypes there isn't a huge performance penalty. If you are interested I can send you the code.

  
If you just want an alternative to writing JS syntax I would highly recommend CoffeeScript. And if you want to re-use code between CPython and a web browser than you can't really avoid implementing as much of Python semantics as possible. Pyjs gets enough of the semantics implemented to be productive but once you start trying to build frameworks or more complex code the remaining 5% of missing semantics really gets in the way. That's where we need the most help and effort.

Thanks for pointing me in the right direction.

If you decide to continue with your separate project I can see two different paths of collaboration for Pythonium and Pyjs:

1. test suite as discussed earlier
2. a runtime engine, you call this "pythonium.veloce.js" in your readme

I think both of those could be independent projects that both Pyjs and Pythonium could depend on and thus allow some diversity and at the same time some re-use of resources on the really hard stuff.

 - lex

Sarvi Shanmugham

unread,
Nov 30, 2013, 12:50:56 AM11/30/13
to pyjs-...@googlegroups.com, pythoni...@googlegroups.com
When I read this a couple of things struck me. May thats the intent of core, veloce and compliant i.e. quick python compliant development vs less compliant but more performance code generation/
   1. The general approach is to develop quickly and optimize the 10% of the code that might actually matter for performance.
   2. This means that I would like to develop most of the code with full python compatibility.
   3. But then it would nice to be able to compile sections of code with less python compatibility but more optimized code generation. This could be because there is some way to mark some specific python functions(say with a decorator) OR some X.py file to be compiled less python compatibility and generation of more performant code. Yes. I know i can always drop down to javascript. But wouldn't it be better to write a more restricted python syntax for performance that drop down java script,
   4. Yes there is a whole list of optimization flags. But ideally wouldnt it be better to have just 2 options full python compliance Vs high performance code generation. Do we really need this many options?

Sarvi 

Amirouche Boubekki

unread,
Dec 2, 2013, 6:10:35 AM12/2/13
to pythoni...@googlegroups.com, pyjs-...@googlegroups.com, pythoni...@googlegroups.com, bry...@googlegroups.com
If you decide to continue with your separate project I can see two different paths of collaboration for Pythonium and Pyjs:

1. test suite as discussed earlier
2. a runtime engine, you call this "pythonium.veloce.js" in your readme

I think both of those could be independent projects that both Pyjs and Pythonium could depend on and thus allow some diversity and at the same time some re-use of resources on the really hard stuff.

I made good progress on Pythonium compliant mode, it now runs pystone.py [1], I beg you to try it to at least compare the generated code. I evaluate support of Python language between 70% and 80% as python exception are still not completly handled, there is no support for yield and yield from, nor with statment, I did not update list comprehension support, args & kwargs in calling and receiving code. But all this is just about doing it. There is no major road blocker ahead, I can see.

As far as merging efforts, I gave more look at pyjs and here are my results:

- I took me a while to understand which command to use
- I couldn't run it, it's missing the runtime library I couldn't figure out how to generate it

I think what would be best for the community is to create a brythoniumjs which will be based on all three projects:

- Pythonium translation scheme and offline compiler
- Brython on the fly Praser/Javascript emitter
- PYJS builtins and framework

What must be done to have a complete solution is the following:

- Update Bryhon parser/emitter to emit Pythonium code
- Update PYJS framework to be compatible with Pythonium API (new special function) and implement missing libraries (or take them from PYSJ or Brython)
- Add the missing builtins/stdlib that makes sens
- Implement a sourcemap generator for both on-the-fly generated code (if possible) and compiled code.

What do you think?

I'm not interested in borrowing on my own all the missing pieces from the other compatible project for Pythonium, what I would like is really to merge the different efforts into one project. As far as I'm concerned I'm more interested in the Veloce mode of Pythonium (Python syntax with Javascript semantic) and I'm only further working on the compliant mode to show that the design is ok and that it's possible to have 100% or 99% support of Python.

Regards,


Amirouche

[1] https://groups.google.com/d/msg/pythonium-users/wv264TWdsTI/co5lJjjkvzYJ
Reply all
Reply to author
Forward
0 new messages