Starting work on dual-mode compiler

[Christian Iversen]

Hey guys

I just wanted to let you know that I'm starting work on the dual-mode
compiler in my development branch.

I discussed the design with Sam, and now I have a model that I can try
to implement. Therefore, if you want to get involved in the core
compiler work, please let me know so we can coordinate. If would be a
shame if we do the same work in parallel :)

--
Med venlig hilsen / Best regards
Christian Iversen

Sikkerhed.org ApS
Fuglebakkevej 88 E-mail: sup...@sikkerhed.org
1. sal Web: www.sikkerhed.org
DK-2000 Frederiksberg Direkte: c...@sikkerhed.org

[Ondrej Certik]

On Mon, Mar 7, 2011 at 2:33 AM, Christian Iversen <c...@sikkerhed.org> wrote:
> Hey guys
>
> I just wanted to let you know that I'm starting work on the dual-mode
> compiler in my development branch.

Which branch? I didn't find any here:

https://github.com/chrivers/py2js

>
> I discussed the design with Sam, and now I have a model that I can try to
> implement. Therefore, if you want to get involved in the core compiler work,
> please let me know so we can coordinate. If would be a shame if we do the
> same work in parallel :)

Cool.

Btw, I would suggest you to develop in a different branch than your
"master", otherwise your master will diverge from the official master,
if for example Sam's patch gets in first, and it is conflicting with
your code. Just do:

git checkout -b dual_mode
<develop>
git push g...@github.com:chrivers/py2js dual_mode

and send a pull request from this dual_mode branch against
qsnake/py2js. Feel free to ask if you have any questions about the git
workflow.

Ondrej

[Christian Iversen]

On 2011-03-07 18:35, Ondrej Certik wrote:
> On Mon, Mar 7, 2011 at 2:33 AM, Christian Iversen<c...@sikkerhed.org> wrote:
>> Hey guys
>>
>> I just wanted to let you know that I'm starting work on the dual-mode
>> compiler in my development branch.
>
> Which branch? I didn't find any here:
>
> https://github.com/chrivers/py2js

With some help from Sam, I got my development branch uploaded to github.
You should be able to see it now. It's not quite ready to be pulled yet.

> Btw, I would suggest you to develop in a different branch than your
> "master", otherwise your master will diverge from the official master,
> if for example Sam's patch gets in first, and it is conflicting with
> your code. Just do:
>
> git checkout -b dual_mode
> <develop>
> git push g...@github.com:chrivers/py2js dual_mode
>
> and send a pull request from this dual_mode branch against
> qsnake/py2js. Feel free to ask if you have any questions about the git
> workflow.

Thanks, I got it figured out with both your and Sams help :)

I can report that the dual-mode compiler has had its first success.
Consider this simple python example function:

def func(a, b, c):
return jQuery("#button").click(
lambda: [jQuery("#dialog-confirm").dialog('open'),
jQuery(this).button("option", "disabled", foo.bar.baz)])

Now, the naïve python translation results in this:

return jQuery(str('#button')).__getattr__("click")(function()
{list([jQuery(str('#dialog-confirm')).__getattr__("dialog")(str('open')), jQuery(this).__getattr__("button")(str('option'),str('disabled'),foo.__getattr__("bar").__getattr__("baz"))])});
});

Clearly, a lot of boxing and __foo__ magic. With the multiplexing
compiler, I name "jQuery" as a javascript var, and behold:

var func = $def({}, function(a, b, c) {
return jQuery('#button').click(function()
{[jQuery('#dialog-confirm').dialog('open'),
jQuery(this).button('option', 'disabled',
foo.__getattr__("bar").__getattr__("baz"))]});
});

Notice that the arguments to jQuery() as well as jQuery.click() are all
javascript vars, while the "foo" variable is still being treated as
being python.

I think we have a working model on our hands, but let's not celebrate
too much before we have seen the unit tests and examples run ;-)

[Ondrej Certik]

Hi Christian!

Definitely! Wow, this is really exciting. Once tests are running and
these two modes are tested, I wonder if there is any way to send you a
beer. :)

Ondrej

[Luke Stanley]

Hey guys! Great work.
I think this is really where it starts to come together and this
project vastly increases in value from elegant code interchange.
I am in the red but in such an event, I would also like to send beer
or, a glucose drink of your choice, green tea or favourite caffeinated
beverage.
I'm working on a web.py and Py2JS based Pythonic MVC framework,
allowing heavy use of jQuery and it's vast sea of plugins.

I see the recent changes here:
https://github.com/chrivers/py2js/commit/31d6901313c155f6e9e1104e7f42c9e01ef7d9e6
https://github.com/chrivers/py2js/commit/59e3523b0c0d9b8772ca0d7750d67aa965aa72a8

So you're currently using a list of regular JS typed object names,
manually entered.
The browser environment is JS already, not Py2JS so that could be a
lot of variables to list.
What if Py2JS tracked assignment of it's Pythonic types instead - and
the rest were assumed to be regular JS types?
What do you think about that?
Since I'm working on an MVC framework, I think I can narrow the domain
down to Py2JS objects in expected places - parts of the model, and
some working out done in the controller, and there rendering in the
view. I'm not sure this inverted approach would be best suited to
other approaches, I wonder what you guys think about this.
I won't be surprised if this is what you're planning anyway.

Amazing work guys, thank you for making Python and Javascript
development nicer!

--Luke Stanley

[Ondrej Certik]

Hi Luke,

On Sat, May 7, 2011 at 8:07 AM, Luke Stanley <luke.s...@gmail.com> wrote:
> Hey guys! Great work.

Thanks, and welcome!

> I think this is really where it starts to come together and this
> project vastly increases in value from elegant code interchange.
> I am in the red but in such an event, I would also like to send beer
> or, a glucose drink of your choice, green tea or favourite caffeinated
> beverage.
> I'm working on a web.py and Py2JS based Pythonic MVC framework,
> allowing heavy use of jQuery and it's vast sea of plugins.
>
> I see the recent changes here:
> https://github.com/chrivers/py2js/commit/31d6901313c155f6e9e1104e7f42c9e01ef7d9e6
> https://github.com/chrivers/py2js/commit/59e3523b0c0d9b8772ca0d7750d67aa965aa72a8
>
> So you're currently using a list of regular JS typed object names,
> manually entered.
> The browser environment is JS already, not Py2JS so that could be a
> lot of variables to list.
> What if Py2JS tracked assignment of it's Pythonic types instead - and
> the rest were assumed to be regular JS types?
> What do you think about that?

I think both can work. I myself don't have an opinion on this yet,
until I can see it working. I would stick with whatever way is more
robust.

One disadvantage of having all types as JS by default is that errors
might go undetected? (If you mistype a name.)

> Since I'm working on an MVC framework, I think I can narrow the domain
> down to Py2JS objects in expected places - parts of the model, and
> some working out done in the controller, and there rendering in the
> view. I'm not sure this inverted approach would be best suited to
> other approaches, I wonder what you guys think about this.
> I won't be surprised if this is what you're planning anyway.
>
> Amazing work guys, thank you for making Python and Javascript
> development nicer!

Send us any improvements, if you have any.

Ondrej

[Samuel Ytterbrink]

2011/5/7 Ondrej Certik <ond...@certik.cz>

What i have understood of what Christian have explained to me. The decorator is only used when you want to "creatre" JavaScript vars from python context. That means that using JavaScript objects from outside of py2js is okay without using any JSVar decerator.

> Since I'm working on an MVC framework, I think I can narrow the domain
> down to Py2JS objects in expected places - parts of the model, and
> some working out done in the controller, and there rendering in the
> view. I'm not sure this inverted approach would be best suited to
> other approaches, I wonder what you guys think about this.
> I won't be surprised if this is what you're planning anyway.
>
> Amazing work guys, thank you for making Python and Javascript
> development nicer!

Send us any improvements, if you have any.

Ondrej

--
//Samuel Ytterbrink

[Luke Stanley]

Thanks for the welcome.

The latest code doesn't yet support typical JSON access where attributes and index access are both possible.

@JSVar("m")

def test():

m = {}

m.clientOnly = {}

m.clientOnly['test'] = 42

produces:

var m = {};

m.clientOnly = {};

m.clientOnly.__setitem__(str('test'), 42); 

rather than
m.clientOnly['test']=42;  

You guys are much more proficient at AST hacking than me but if I coble anything together on Py2JS that isn't a terrible hack I'll share it. I'm keen to share my Pythonic MVC code soon which builds on your work also.

[Christian Iversen]

Ah! This is a pretty simple error. Type searches should resolve upwards
in the heirarchy, which it doesn't right now. I'll fix this when I get
back to hacking py2js again, which I hope will be very soon :)

You can see the problem for yourself by doing only 1 level of access:

m['test'] = 42

Should produce

m['test'] = 42

Otherwise, that's a very bad bug. But I don't think there's any chance
you're going to get a different output :)

If you want a workaround, try this hack:

@JSVar("m", "q"):
def test():
m = {}
q = {}
m.clientOnly = q
q['test'] = 42

It's nasty, but it should work until we get a real solution going :)

[Luke Stanley]

Thanks Christian, might use that instead of a dirty search and replace!

[Christian Iversen]

On 2011-05-15 17:51, Luke Stanley wrote:
> Thanks Christian, might use that instead of a dirty search and replace!

You'll be pleased to know that the bug you reported, along with many
many others, are now fixed :)

Try the newest git from my github to see the improvements.

[Luke Stanley]

You'll be pleased to know that the bug you reported, along with many many others, are now fixed :)

Try the newest git from my github to see the improvements.

Awesome!

I have a few more bugs. ;D

Maybe I should login to Github and file them there?

Adding to a dict with a string key didn't seem to work for me, something to do with the new __call__ approach?

E.g: 

m = {}

m.clientOnly = {}

m.clientOnly['test'] = 42

Produces: Uncaught TypeError: Object function (x){return x} has no method '__call__'

It might be good to find some pure Python modules and use them as part of the testing process in case some of these bugs exist in hard to reach places which prevent them from being guessed beforehand. PyPy does this too I think, perhaps for speed tests mostly.

I will keep my eyes peeled on your branch. Btw what is the difference between your Devel branch and the Master branch?

Is there a function or simple way to convert JSON to Py2JS objects?

I made a Py2JS to JSON converter by the way, with an old branch, don't know it you'd want it.

Best,

--Luke

[Christian Iversen]

On 2011-05-20 14:43, Luke Stanley wrote:
> You'll be pleased to know that the bug you reported, along with many
> many others, are now fixed :)
>
> Try the newest git from my github to see the improvements.
>
> Awesome!
>
> I have a few more bugs. ;D
> Maybe I should login to Github and file them there?

Yes, please do :)

> Adding to a dict with a string key didn't seem to work for me, something
> to do with the new __call__ approach?
> E.g:
> m = {}
> m.clientOnly = {}
> m.clientOnly['test'] = 42
>
> Produces: Uncaught TypeError: Object function (x){return x} has no
> method '__call__'

Can you give me some more context? Is "m" a JSVar() or normal python
var? There are no python calls in this example, so I don't see why it
complains about __call__.

> It might be good to find some pure Python modules and use them as part
> of the testing process in case some of these bugs exist in hard to reach
> places which prevent them from being guessed beforehand. PyPy does this
> too I think, perhaps for speed tests mostly.

Good idea, I'll look into that. It would have to be purely
data-manipulation libraries though, as we lack major parts of the
standard library (file access, for example)

> I will keep my eyes peeled on your branch. Btw what is the difference
> between your Devel branch and the Master branch?

I try to keep my master branch stable, so it only merges complete
features from my devel branch. The devel branch is pushed to regularly,
so it might not work all the time.

> Is there a function or simple way to convert JSON to Py2JS objects?
> I made a Py2JS to JSON converter by the way, with an old branch, don't
> know it you'd want it.

Please send it! There are some additions I have planned, to make it
easier to create web applications with py2js, which involve these kinds
of conversions. I'd be happy to see what you have made for that.

Look forward to hearing from you, and please report any bugs you might
find. That's how we improve :)

[Luke Stanley]

On 25 May 2011 07:20, Christian Iversen <c...@sikkerhed.org> wrote:

On 2011-05-20 14:43, Luke Stanley wrote:

   You'll be pleased to know that the bug you reported, along with many
   many others, are now fixed :)

   Try the newest git from my github to see the improvements.

Awesome!

I have a few more bugs. ;D
Maybe I should login to Github and file them there?

Yes, please do :)

Adding to a dict with a string key didn't seem to work for me, something
to do with the new __call__ approach?
E.g:
m = {}
m.clientOnly = {}
m.clientOnly['test'] = 42

Produces: Uncaught TypeError: Object function (x){return x} has no
method '__call__'

Can you give me some more context? Is "m" a JSVar() or normal python var? There are no python calls in this example, so I don't see why it complains about __call__.

Sorry I should have been clear, that was on a Pythonic version, not JSVarified. I suspect you've fixed it in your recent updates already, I have been following your commits.

It might be good to find some pure Python modules and use them as part
of the testing process in case some of these bugs exist in hard to reach
places which prevent them from being guessed beforehand. PyPy does this
too I think, perhaps for speed tests mostly.

Good idea, I'll look into that. It would have to be purely data-manipulation libraries though, as we lack major parts of the standard library (file access, for example)

I will keep my eyes peeled on your branch. Btw what is the difference
between your Devel branch and the Master branch?

I try to keep my master branch stable, so it only merges complete features from my devel branch. The devel branch is pushed to regularly, so it might not work all the time.

ok makes sense, thanks. 

Is there a function or simple way to convert JSON to Py2JS objects?
I made a Py2JS to JSON converter by the way, with an old branch, don't
know it you'd want it.

Please send it! There are some additions I have planned, to make it easier to create web applications with py2js, which involve these kinds of conversions. I'd be happy to see what you have made for that. 

Look forward to hearing from you, and please report any bugs you might find. That's how we improve :) 

ok not sure where best to but this on github at the moment so for now it's here:

http://paste2.org/p/1438311

--Luke