A Python implementation of JavaScript objects

[Jonathan Fine]

Hello

I've just pushed something I've wanted to do for a year or so, namely a toy implementation of JavaScript objects in Python.  (It took about 5 hours to program, including some thinking time.  Why didn't I do it earlier?)

I've put it in a new 'toys' branch on BitBucket, and here's a URL for the changeset.
    http://bitbucket.org/PeterRust/py2js/changeset/481c7822f188

The py-builtins.js code implements Python builtins in Javascript.  My new file starts to implement the Javascript builtins in Python.

A major difficulty in the py-to-js translation is that Javascript and Python objects have markedly different semantics.  If, on the Python side, we use Javascript object then the translation is much easier.

Our current translator sends
    def plus(a, b):
        return a + b
to
    function plus(a, b) {
        return (a + b);
    }

Of course this is right!  But if it is correct on the Javascript side then all admissable arguments to plus must, on the Python side, have the semantics of Javascript objects.

Here is an example:
    js> a = {a:3, b:4}
    [object Object]
    js> b = 4
    4
    js> a + b
    [object Object]4

Javascript likes to convert things into strings.
    js> typeof (a + b)
    string

So my idea is this.  We translate functions written in Python to functions written in JavaScript in a way that preserves the semantics provided the arguments on both sides are Javascript objects.  This translation will translate plus to plus and minus to minus.

We don't need Javascript objects in Python to write the translator, but we do to test that the semantics are preserved.

--
Jonathan

--
You received this message because you are subscribed to the Google Groups "JavaScript for Python programmers" group.
To post to this group, send an email to js...@googlegroups.com.
To unsubscribe from this group, send email to js4py+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/js4py?hl=en-GB.

[Ondrej Certik]

In my py-builtins.js file, I have implemented a "js" function, that
takes any Python object and converts it to a javascript object. This
is necessary, so that I can write code like this in Python:

items = [
{ "icon":
'http://www.extjs.com/deploy/dev/examples/menu/list-items.gif', "cls":
'x-btn-icon',
"handler": toolbar_mesh1,
"tooltip": '<b>Draw Mesh I</b><br/>Show an example mesh' },
{ "icon":
'http://www.extjs.com/deploy/dev/examples/menu/list-items.gif', "cls":
'x-btn-icon',
"handler": toolbar_mesh2,
"tooltip": '<b>Draw Mesh II</b><br/>Show an example mesh' },
{ "icon":
'http://www.extjs.com/deploy/dev/examples/menu/list-items.gif', "cls":
'x-btn-icon',
"handler": toolbar_mesh3,
"tooltip": '<b>Draw Mesh III</b><br/>Show an example mesh' },
]
_new(Ext.Toolbar, js({"renderTo": 'mesh-editor', "items": items}))

where items is a tuple() instance, which contains dict() instance as
it's elements. The js() function converts this into a javascript
array, so that ExtJS is happy.


Is this what your js() functions is doing?

Ondrej

[Jonathan Fine]

On Sat, May 1, 2010 at 10:42 PM, Ondrej Certik <ond...@certik.cz> wrote:

 

In my py-builtins.js file, I have implemented a "js" function, that
takes any Python object and converts it to a javascript object. This
is necessary, so that I can write code like this in Python:

   items = [
           { "icon":
'http://www.extjs.com/deploy/dev/examples/menu/list-items.gif', "cls":
'x-btn-icon',
               "handler": toolbar_mesh1,
           "tooltip": '<b>Draw Mesh I</b><br/>Show an example mesh' },
           { "icon":
'http://www.extjs.com/deploy/dev/examples/menu/list-items.gif', "cls":
'x-btn-icon',
               "handler": toolbar_mesh2,
           "tooltip": '<b>Draw Mesh II</b><br/>Show an example mesh' },
           { "icon":
'http://www.extjs.com/deploy/dev/examples/menu/list-items.gif', "cls":
'x-btn-icon',
               "handler": toolbar_mesh3,
           "tooltip": '<b>Draw Mesh III</b><br/>Show an example mesh' },
           ]

So what we have here is a JSON object, assigned to 'items'.  The above code works in both Python and JavaScript.  Such objects, modulo 'var' and the like (such as the trailing comma in the sequence), can often reasonably be translated to themselves, so to speak.  (Indeed, that is the whole point of JSON.)
 

   _new(Ext.Toolbar, js({"renderTo": 'mesh-editor', "items": items}))

This line of code is in examples/ui.py (I wasn't sure at first, so I grep'ed for Toolbar).

It seems to me that this is translated to
    _new(Ext.Toolbar,js(dict(tuple([tuple([str("renderTo"), str("mesh-editor")]), tuple([str("items"), items])]))));
which is, forgive me for saying, not as readable as before.

 

where items is a tuple() instance, which contains dict() instance as
it's elements. The js() function converts this into a javascript
array, so that ExtJS is happy.

Yes.  Good point.  We want on the JS side to be able to pass ordinary JavaScript objects to third party code.

Is this what your js() functions is doing?

Sort of.  Let's suppose we have two functions, js(obj) and py(obj) in both JavaScript and Python.

In Python, js(obj) returns an object that has the special JavaScript semantics.
In Python, py(obj) returns an object that does not have the special JavaScript semantics.

Similarly
In JavaScript py(obj) returns an object that has the special Python Semantics.
In JavaScript js(obj) returns an object that does not have the special JavaScript semantics.

I've written a js(obj) function for Python.  You've written a js(obj) function for JavaScript.

Here's a question for you.  In the function defintions in examples/ui.py there are many references to a function called 'js', but this non-builtin function in not imported from anywhere.  This puzzled me when I read it recently.

In fact, if you do
    py2js-github$ python -i examples/ui.py
and then
     # (after output scrolls by)
    </head>
    <body></body>
    </html>
    >>> get_toolbar
    <py2js.JavaScript object at 0x1e6fb50>
    >>> get_toolbar()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/jfine/2010/py2js/py2js-github/py2js.py", line 711, in __call__
        return self._obj(*args, **kwargs)
      File "examples/ui.py", line 64, in get_toolbar

        _new(Ext.Toolbar, js({"renderTo": 'mesh-editor', "items": items}))

     NameError: global name '_new' is not defined
    >>>
you get an exception.

Am I right in thinking that 'js' and '_new' are so to speak dummy functions whose purpose is to influence the operation of your 'JavaScript' decorator?

--
Jonathan

[Ondrej Certik]

As long as you only use strings, yes. See below.

>
>>
>>    _new(Ext.Toolbar, js({"renderTo": 'mesh-editor', "items": items}))
>
> This line of code is in examples/ui.py (I wasn't sure at first, so I grep'ed
> for Toolbar).
>
> It seems to me that this is translated to
>     _new(Ext.Toolbar,js(dict(tuple([tuple([str("renderTo"),
> str("mesh-editor")]), tuple([str("items"), items])]))));
> which is, forgive me for saying, not as readable as before.

Yes, it's messy. However, it's super simple to change this in the
py2js.py, essentially just change how strings, tuples/lists and dicts
are converted (one liner changes usually).

The big problem is that javascript {} syntax is not a dictionary. It
only assigns properties to an object, which means that it only accept
strings as keys. However, in Python, I need to use other keys than
just strings. In our py-builtins.js we currently don't support it yet,
but we will eventually.

However, our py-builtins.js support a syntax, that you probably prefer:

dict({1: 3, "ok": 6})

which is equivalent to:

dict(tuple([tuple([1, 3]), tuple(["ok": 6])]))

just like in Python (try the dict() in ipython). The former syntax is
easy to read/write by hand, but it doesn't support other objects
besides numbers/strings. In particular, if you want to use something
like this:

dict({str("x"): 5})

it will fail in JS. So I don't see any other way than we currently
did. We should at least use "\n" more often, so that it looks more
readable, but it will never look as easy as Python.

Another issue is that I have introduces a str() class and made
py2js.py use it for all strings. Again, this is trivial to change, but
I think that we need this.

>
>>
>> where items is a tuple() instance, which contains dict() instance as
>> it's elements. The js() function converts this into a javascript
>> array, so that ExtJS is happy.
>
> Yes.  Good point.  We want on the JS side to be able to pass ordinary
> JavaScript objects to third party code.

I agree, that is absolutely necessary.

The js() and _new() are implemented in py-builtins.js, among all other
functions, like dict(), tuple(), list(), iter(), str(), exceptions,
and many other things. So you need to include it somehow, for example
by using:

py2js.py --include-builtins somefile.py

We don't support modules yet, which I would like to just use your
code, after we merge it. I spent last couple days to make more tests
work, so I have implemented str() and except formatting (%), all
things with strings work fine now. I also need to implement class
inheritance and couple other things, but then it will be hopefully
ready for you to seriously look at it and see what's the best way to
merge it.

I really appreciate the feedback and questions that you ask on the
way. I really like the rich py-builtins.js library approach (that we
all need, and thus maintain) and then all of us can maintain the exact
translator, that one needs (e.g. more or less python compatibility, vs
javascript readability).

While reading and gettting familiar with your py-builtins.js, I
noticed that it is a bit more lowlevel than our approach, e.g. no
list(), no str(), no iter() and so on. What is your view on it?

Ondrej

[Jonathan Fine]

On Mon, May 3, 2010 at 5:38 AM, Ondrej Certik <ond...@certik.cz> wrote:

The big problem is that javascript {} syntax is not a dictionary. It
only assigns properties to an object, which means that it only accept
strings as keys. However, in Python, I need to use other keys than
just strings.

We require object that have the same semantics on both sides, for the inputs we are dealing with. There are benefits in choosing Python semantics (we know and like Python) and in JavaScript semantics (better performance, interfacing with other libraries).

We can also have quite distinct implementations of the same semantics (such as converting an object to a JSON string).
 

> Yes.  Good point.  We want on the JS side to be able to pass ordinary
> JavaScript objects to third party code.

I agree, that is absolutely necessary.

 
Thank you for discovering this requirement by using py2js.

We don't support modules yet, which I would like to just use your

code, after we merge it. I spent last couple days to make more tests
work, so I have implemented str() and except formatting (%), all
things with strings work fine now. I also need to implement class
inheritance and couple other things, but then it will be hopefully
ready for you to seriously look at it and see what's the best way to
merge it.

I'm pushed a test for inheritance translation to Bitbucket.

I really appreciate the feedback and questions that you ask on the
way. I really like the rich py-builtins.js library approach (that we
all need, and thus maintain) and then all of us can maintain the exact
translator, that one needs (e.g. more or less python compatibility, vs
javascript readability).

While reading and gettting familiar with your py-builtins.js, I
noticed that it is a bit more lowlevel than our approach, e.g. no
list(), no str(), no iter() and so on. What is your view on it?

I've had very little to do with the code that was part of the original push onto Bitbucket, and so it's not my approach you are talking about.  I might have a view on the difference later.

--
Jonathan

[Peter Rust]

Ondrej, Jonathan,

> While reading and gettting familiar with your py-builtins.js, I
> noticed that it is a bit more lowlevel than our approach, e.g. no
> list(), no str(), no iter() and so on. What is your view on it?

The original js4py code was a different approach -- it was entirely done in
a functional style with no wrapper objects. In the bitbucket repository,
I've been converting this to the same approach you're using -- list(),
str(), iter(), dict() and def() wrappers.

If you look at the latest
http://www.bitbucket.org/PeterRust/py2js/src/tip/py-builtins.js, you'll see
that I've created a list wrapper (list() on line 530), a dict wrapper
(dict() on line 578) and a function wrapper (def() on line 341). I'm not
sure why you didn't see the list() wrapper, it should be there.

I haven't yet converted the original string functions (lines 101 to 143)
into a str() wrapper and I haven't created an iter() wrapper yet. I also
realized recently that my splice, slice and pop methods, which I had tried
to cleverly derive from Array.prototype wouldn't work, so I just fixed them
and pushed the fix this morning -- that may make a few more tests pass.

I had hoped to get a chance to do some additional work on this over the
weekend, getting more tests to pass, but ended up having to work on a
project in php (yuck!). Hopefully I'll get a chance to work on it today or
tomorrow.

Thanks for the thinking and the discussion about the differences between JS
objects and Python objects. I'm thinking we'll want to support 3 basic modes
in py2js:

(1) Syntax Only. This mode wouldn't wrap [] with list() or {} with dict()
and would therefore create the cleanest Javascript, but you would be writing
code against the Javascript built-ins (Array, Object, etc), not the Python
ones. Basically the only thing it would give you is the clean Python syntax.
In this mode, I could see it being useful to have a class in Python that has
simulates the behavior of Javascript objects.

(2) Almost Everything (everything but operator overloading). In this mode,
you would be writing against the Python built-ins and would depend on
py-builtins.js to provide those same constructs and functionality in
Javascript. It would throw an error if it encountered a class that tried to
overload the operators.

(3) Everything. This would translate "=", "+", "*", etc into __eq__(),
__mul__(), etc and be fairly verbose and not as pretty, but could
potentially translate any arbitrary Python code into Javascript.

-- Peter Rust
Developer, Cornerstone Systems

[Ondrej Certik]

On Mon, May 3, 2010 at 10:33 AM, Peter Rust <pe...@cornerstonenw.com> wrote:
> Ondrej, Jonathan,
>
>> While reading and gettting familiar with your py-builtins.js, I
>> noticed that it is a bit more lowlevel than our approach, e.g. no
>> list(), no str(), no iter() and so on. What is your view on it?
> The original js4py code was a different approach -- it was entirely done in
> a functional style  with no wrapper objects. In the bitbucket repository,
> I've been converting this to the same approach you're using -- list(),
> str(), iter(), dict() and def() wrappers.
>
> If you look at the latest
> http://www.bitbucket.org/PeterRust/py2js/src/tip/py-builtins.js, you'll see
> that I've created a list wrapper (list() on line 530), a dict wrapper
> (dict() on line 578) and a function wrapper (def() on line 341). I'm not
> sure why you didn't see the list() wrapper, it should be there.

I apologize. It is there of course. My fault.

>
> I haven't yet converted the original string functions (lines 101 to 143)
> into a str() wrapper and I haven't created an iter() wrapper yet. I also
> realized recently that my splice, slice and pop methods, which I had tried
> to cleverly derive from Array.prototype wouldn't work, so I just fixed them
> and pushed the fix this morning -- that may make a few more tests pass.
>
> I had hoped to get a chance to do some additional work on this over the
> weekend, getting more tests to pass, but ended up having to work on a
> project in php (yuck!). Hopefully I'll get a chance to work on it today or
> tomorrow.
>
> Thanks for the thinking and the discussion about the differences between JS
> objects and Python objects. I'm thinking we'll want to support 3 basic modes
> in py2js:
>
> (1) Syntax Only. This mode wouldn't wrap [] with list() or {} with dict()
> and would therefore create the cleanest Javascript, but you would be writing
> code against the Javascript built-ins (Array, Object, etc), not the Python
> ones. Basically the only thing it would give you is the clean Python syntax.
> In this mode, I could see it being useful to have a class in Python that has
> simulates the behavior of Javascript objects.

And potentially we can use this mode to write the py-builtins.js
itself. Later on.

>
> (2) Almost Everything (everything but operator overloading). In this mode,
> you would be writing against the Python built-ins and would depend on
> py-builtins.js to provide those same constructs and functionality in
> Javascript. It would throw an error if it encountered a class that tried to
> overload the operators.
>
> (3) Everything. This would translate "=", "+", "*", etc into __eq__(),
> __mul__(), etc and be fairly verbose and not as pretty, but could
> potentially translate any arbitrary Python code into Javascript.

Yes. In fact, I already translate == into __eq__ and "if x:" into "if
(py.bool(x)) {", it just makes things easier to work with and it
allowed me to fix some problems with strings. As you'll see yourself,
after you implement the str(), you run into all kinds of problems, for
example:

if "":

evaluates to False (both in python and javascript), but if it's in
javascript and you use:

if str(""):

it evaluates to True in JS but False in Python. So that's a big
problem, and so I am now translating:

if str(""):

into

if (py.bool(str(""))) {

and that works fine. So I think for me it makes sense to just support
1) and 3). After a bit of thinking, I am sure we can find out quite a
clean syntax, so that the generated JS can look good. And we can later
on add there some optimizations, for example that the translator would
be clever enough to figure out that

if "":

can also be translated directly into:

if ("") {

in javascript and so on.

[Jonathan Fine]

On Mon, May 3, 2010 at 9:34 PM, Ondrej Certik <ond...@certik.cz> wrote:

On Mon, May 3, 2010 at 10:33 AM, Peter Rust <pe...@cornerstonenw.com> wrote:

 

> Thanks for the thinking and the discussion about the differences between JS
> objects and Python objects. I'm thinking we'll want to support 3 basic modes
> in py2js:
>
> (1) Syntax Only. This mode wouldn't wrap [] with list() or {} with dict()
> and would therefore create the cleanest Javascript, but you would be writing
> code against the Javascript built-ins (Array, Object, etc), not the Python
> ones. Basically the only thing it would give you is the clean Python syntax.
> In this mode, I could see it being useful to have a class in Python that has
> simulates the behavior of Javascript objects.

And potentially we can use this mode to write the py-builtins.js
itself. Later on.

Yes, there are different modes of translation. At this level the programmer has to know Python (for the syntax and logic) and JavaScript (for the objects). Roughly speaking, one is writing Python code for manipulating JavaScript objects.

I like the idea of programming py-builtins.js using this mode.  It will give the system a real test, and maps well to the situation of many JavaScript programmers.
 

> (2) Almost Everything (everything but operator overloading). In this mode,
> you would be writing against the Python built-ins and would depend on
> py-builtins.js to provide those same constructs and functionality in
> Javascript. It would throw an error if it encountered a class that tried to
> overload the operators.

I'm not exactly sure what this means.  We certainly want to have 'builtins' available when we are doing JavaScript programming, even if we're 'writing the JavaScript in Python'.
 

> (3) Everything. This would translate "=", "+", "*", etc into __eq__(),
> __mul__(), etc and be fairly verbose and not as pretty, but could
> potentially translate any arbitrary Python code into Javascript.

Yes. In fact, I already translate == into __eq__ and "if x:" into "if
(py.bool(x)) {", it just makes things easier to work with and it
allowed me to fix some problems with strings. As you'll see yourself,
after you implement the str(), you run into all kinds of problems, for
example:

I've learnt from Cockford's little book (JS: The Good Parts) that '==' is broken, and that one should use '==='.

Python and JavaScript have very different ideas regarding equality, and I don't know how that should be handled.

--
Jonathan

[Peter Rust]

>> (2) Almost Everything (everything but operator overloading). In this mode,
>> you would be writing against the Python built-ins and would depend on
>> py-builtins.js to provide those same constructs and functionality in
>> Javascript. It would throw an error if it encountered a class that tried to
>> overload the operators.

>I'm not exactly sure what this means.  We certainly want to have 'builtins' available when we are doing JavaScript >programming, even if we're 'writing the JavaScript in Python'.

This is my preferred mode – the one I anticipate using most of the time and the one I would like to “market” on the wiki. By “writing against the Python built-ins”, I simply mean that you’d be writing code like “my_list.append(…)” in your python source file as opposed to mode 1, in which you’d write code like “my_array.push(…)” in your python source file.

 

By not attempting to handle 100% of the issues, we can generate much more readable code. Things start getting bloated and unrecognizable when “if protocol + domain == 'http://www.basecamphq.com'” becomes “if (protocol.__add__(domain).__eq__('http://www.basecamphq.com'))”.

 

Mode 2 will really focus on generating readable code even when that is at the expense of being able to translate 100% of the python files that exist (it would throw an exception if, for instance, the python source was a class that overloaded an operator).

 

As Ondrej has pointed out, this gets really tricky when it gets to the falseness of empty strings and using the plus operator for string concatenation. In order the support these, I will need to either (a) decorate each individual string instance with additional methods when it’s passed to str() and return the string itself, rather than a wrapper object or (b) add properties to String.prototype, which I’ve been warned against. At this point, I think I’ll go with (a).

 

Again, this is just for Mode 2 (Peter’s mode) – I think Ondrej has already started down the path of converting “+” to __add__() and “==” to “__eq__” – this is what Mode 3 would do.

 

My advice at this point is to focus hard on Mode 2 and Mode 3 – these are the ones we have immediate needs for and can easily figure out. I think it’ll be real hard to have decent visibility on our needs for Mode 1 before we have Modes 2 and 3 working, plus it’ll be distracting. I’d rather not muddy the waters with Mode 1 discussions until we’ve got Modes 2 and 3 nailed down…

 

-- peter

 

From: js...@googlegroups.com [mailto:js...@googlegroups.com] On Behalf Of Jonathan Fine
Sent: Monday, May 03, 2010 2:03 PM
To: js...@googlegroups.com
Subject: Re: A Python implementation of JavaScript objects

 

On Mon, May 3, 2010 at 9:34 PM, Ondrej Certik <ond...@certik.cz> wrote:

[Ondrej Certik]

Yes, I agree. I would also concentrate on mode 2 and mode 3 and
especially on keeping the translator as simple as possible, e.g. doing
most of the stuff in the py-builtins.js library (that would support
both mode 2 and mode 3).

Then it will be trivial to have a simple switch in the translator, if
it should generate mode 2 or mode 3 code.

Ondrej

[Jonathan Fine]

On Tue, May 4, 2010 at 12:32 AM, Ondrej Certik <ond...@certik.cz> wrote:

Yes, I agree. I would also concentrate on mode 2 and mode 3 and
especially on keeping the translator as simple as possible, e.g. doing
most of the stuff in the py-builtins.js library (that would support
both mode 2 and mode 3).

Then it will be trivial to have a simple switch in the translator, if
it should generate mode 2 or mode 3 code.

Well, I would like to have mode 1 working as well, and there seems to be a dependency, namely we use mode 1 to write py-builtins.js.

--
Jonathan

[Ondrej Certik]

I have improved the printing of dicts a bit, so the above line now
get's translated into:

_new(Ext.Toolbar,js(dict(tuple([
tuple([str("renderTo"), str("mesh-editor")]),
tuple([str("items"), items])
]))));

it's still not perfect, because if this is used in some nested tuples,
the multiple indentation still looks a bit ugly, but at least it's
some improvement.

After we satisfy all the js4py tests and expand our test suite to
cover almost all python, and have more tests for dictionaries of
arbitrary objects, we can come back to this and optimize the syntax
for simple cases (numbers and strings).

Ondrej