Trying to make it installable with pip

[Sarvi Shanmugham]

Hi

    I was trying to make this installable with pip and have a few questions for the experts who understand how the code is organized.

1. Is there a description of how the code is organized. What parts are the compiler. What parts are standard libraries/stuff that should be part of the compiler. What parts are pyjamas widget sets separate from the compiler. 

2. What is difference between pyjampiler, pyjscompile and pyjsbuild. 

3. I also notice that wihin pyjs/pyjs there is a setup.py

    3.1 Doesn't seem to work for me. How is that different from the bootstrap.py.

    3.2 that also has a few entry points, translate and sm. what are they?

4. In general what is the history behind bootstrap.py and not having a setup.py and not installing it before running. 

Sarvi

[Lex Berezhny]

This is exactly why I think it needs to be broken up.

I think just splitting up the compiler from the widgets would make the source much clearer.

Anthony, any thoughts on this? I know you said you agree with this plan.

Is this something you'd prefer to do yourself or would you like to delegate this very important task?

 - lex

[Steve Waterbury]

Sarvi,

INSTALL.txt in the pyjs distribution will answer a number
of your questions -- in particular, why it doesn't make
sense to 'pip install' pyjs.

Steve

> --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "Pyjs.org Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pyjs-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Sarvi Shanmugham]

Thanks for the pointer. I had gone through that doc a couple of times and still had some questions and hence i asked.

1. sand boxing

   I use virtualenv for sandboxing and has worked great. infact when experimenting with making this work with pip, 

thats what I am doing trying to make install work into a virtualenv sandboxed python installation. So I suspect I can make this work

with virtualenv

2. pyjamas libraries and its replacements for sys and time and stuff.

   If this gets installed to override the standard libraries it will be a problem. But from what I can tell, these can be installed

as pyjs.stdlib.sys/time/etc and when the compile/build script could still do their little path magic to work with such an install.

Atleast thats what I am thinking of doing. Would love to know if this has been tried or if there are any pitfalls I should be aware off.

3. I see pyjs/pyjs/setup.py but I can't seem to get it to work and is not documented in INSTALL.txt I only see bootstrap.py

approaches described.

4. I see this setup.py has entry points for sm and translate. i.e. it will install commands called sm and translate into /usr/bin

Can't seem to figure out what their purpose would or if they should be eliminated or ignored.

5. I've tried both pyjscompile, which seems to be for a single file and pyjsbuild for a whole suite of files. But not sure what pyjampiler is?

6. Not sure what the role of pgen/ and library/ directories are. Are they part of the compiler or are they part of the widget library.

7. The last thing I am trying to do is to see how I can translate a simple helloworld.py program into java script and run using the V8 command line interpreter.

Can't seem to figure out if I should use pyjampiler, pyjscompile or pyjsbuild. I've tried them they seem to generating incomplete code as is the case of 

pyjscompile or too much more code relating to windows and stuff if I used pyjsbuild. Has any one tried compiling a simple helloworld.py program, translated 

it into simple javascript and run it inside V8 interpreter as opposed to the browser?

Sarvi

[Kees Bos]

On Wed, 2013-10-09 at 11:54 -0700, Sarvi Shanmugham wrote:
>
> Hi
> I was trying to make this installable with pip and have a few
> questions for the experts who understand how the code is organized.


> 1. Is there a description of how the code is organized. What parts are
> the compiler. What parts are standard libraries/stuff that should be
> part of the compiler. What parts are pyjamas widget sets separate from
> the compiler.

examples/
What says
examples/libtest/
The very important 'unittest'
doc/
Documentation
pyjs/
translator and parts of python standard libs
library/
gwt and pyjamas libraries
pgen/
external library used in translator
pyjd/
pyjd stuff (i.e. use of gwt and pyjamas lib in plain python)
pyv8/
v8 engine stuff
contrib/
some general (un)related scripts etc
pysm/
spidermonkey stuf
pygtkweb/
gtk lookalike in gwt widgets

> 2. What is difference between pyjampiler, pyjscompile and pyjsbuild.

pyjampiler and pyjscompile just translate code

pyjsbuild 'compiles and links'

[Steve Waterbury]

Sarvi,

Kees knows *way* more about this than I do -- I am only a user --
and I see that he's answered your questions about what the
stuff in the pyjs subdirs is and the roles of pyjampiler,
pyjscompile, and pyjsbuild are. I have provided some more
data based on my experience using pyjs, below ...

On 10/09/2013 04:16 PM, Sarvi Shanmugham wrote:
> Thanks for the pointer. I had gone through that doc a couple of times
> and still had some questions and hence i asked.
>
> 1. sand boxing
> I use virtualenv for sandboxing and has worked great. infact when
> experimenting with making this work with pip,
> thats what I am doing trying to make install work into a virtualenv
> sandboxed python installation. So I suspect I can make this work
> with virtualenv

There is no need for sandboxing if you are running pyjsbuild
on the python code you write for translation to javascript -- the
python code you write will never be run by a python interpreter,
so it should never be in any system.path.

> 2. pyjamas libraries and its replacements for sys and time and stuff.
> If this gets installed to override the standard libraries it will be
> a problem. But from what I can tell, these can be installed
> as pyjs.stdlib.sys/time/etc and when the compile/build script could
> still do their little path magic to work with such an install.
> Atleast thats what I am thinking of doing. Would love to know if this
> has been tried or if there are any pitfalls I should be aware off.

Again, it makes no sense to install *any* pyjamas libraries -- they are
not intended to be run in a python interpreter. All the "path magic"
is done when bootstrap.py creates 'pyjsbuild' -- after that, all you do
is run pyjsbuild whenever you change something in your app and it
re-generates the javascript for the app.

> 3. I see pyjs/pyjs/setup.py but I can't seem to get it to work and is
> not documented in INSTALL.txt I only see bootstrap.py
> approaches described.

To be honest, I have no idea what that setup.py is for --
I have used pyjs for almost 3 years now and have created
several applications, never having touched that setup.py.

> 4. I see this setup.py has entry points for sm and translate. i.e. it
> will install commands called sm and translate into /usr/bin
> Can't seem to figure out what their purpose would or if they should be
> eliminated or ignored.

Maybe one of the developers can explain -- I have no idea.

> 5. I've tried both pyjscompile, which seems to be for a single file and
> pyjsbuild for a whole suite of files. But not sure what pyjampiler is?

(Kees explained this.)
I have only used pyjsbuild and contrib/pyjscompressor, which calls
google's javascript "compiler" to compress the pyjs-generated
javascript. It works great -- gets around 50% compression.

> 6. Not sure what the role of pgen/ and library/ directories are. Are
> they part of the compiler or are they part of the widget library.
>
> 7. The last thing I am trying to do is to see how I can translate a
> simple helloworld.py program into java script and run using the V8
> command line interpreter.
> Can't seem to figure out if I should use pyjampiler, pyjscompile or
> pyjsbuild. I've tried them they seem to generating incomplete code as is
> the case of
> pyjscompile or too much more code relating to windows and stuff if I
> used pyjsbuild. Has any one tried compiling a simple helloworld.py
> program, translated
> it into simple javascript and run it inside V8 interpreter as opposed to
> the browser?

I have never done that. As I say, I have always used pyjsbuild. Its
output directory contains an html file named for the "main" module of
your app, which creates an iframe into which the results of the
"app.onModuleLoad()" call goes. I don't know how to run the generated
javascript in the V8 interpreter but I would be interested in knowing,
if you get that to work.

HTH,
Steve

[Sarvi Shanmugham]

What I am trying to do is to change enough to be able to do a pip install into virtualenv, organize where things get put to point where things work

Once this is done, it should be relatively easy to split the code into 2 halves.

Sarvi

[Glyph]

On Oct 9, 2013, at 12:16 PM, Steve Waterbury <wate...@pangalactic.us> wrote:

... in particular, why it doesn't make sense to 'pip install' pyjs.

Personally I'd love to be able to 'pip install' it simply so that I can do 'pip install pyjs' and then get pyjampiler and pyjsbuild on my $PATH automatically, since pip provides nice tools and automation for that.  Plus that would make it easier for build systems that already support setuptools (debian, fedora, et. al.) to build ready-made PyJS packages.

This would also make it a lot easier for PyJS to depend on other Python projects for its own use, so that they could be automatically installed.  If PyJS is to be split up into multiple pieces, it should be able to use *itself* as a Python library :-).

I'd have no expectation that after 'pip install' it would be usable as a regular platform Python library though.

-glyph

[Steve Waterbury]

On 10/09/2013 07:49 PM, Glyph wrote:
>
> On Oct 9, 2013, at 12:16 PM, Steve Waterbury <wate...@pangalactic.us

> <mailto:wate...@pangalactic.us>> wrote:
>
>> ... in particular, why it doesn't make sense to 'pip install' pyjs.
>
> Personally I'd love to be able to 'pip install' it simply so that I can
> do 'pip install pyjs' and then get pyjampiler and pyjsbuild on my $PATH
> automatically, since pip provides nice tools and automation for that.
> Plus that would make it easier for build systems that already support
> setuptools (debian, fedora, et. al.) to build ready-made PyJS packages.

I think it would be easy enough to create a pip install that does
what you suggest. I suppose for some uses having that stuff on your
path might be good; however, some configuration is always needed to
use it for development, so just having that stuff in your $PATH is not
a big win, IMO. For example, for each app that you are creating you
will need to decide what parameters and paths to use when you run
pyjsbuild, which you will do every time you make a change to
your app code. The suggested way to do that is a build.sh script --
I've attached one of mine so you can see an example. Since build.sh
is simply a shell wrapper for pyjsbuild, there's really no need
to have pyjsbuild on your path.

NOTE: as you can infer from the path names in my build.sh, I am
using pyjs in a directory that is located inside a virtualenv, but
the virtualenv is for the backend [Django] of the app, not for pyjs --
i.e., the pyjs libraries are not "installed" in the virtualenv.

I keep my own library of widgets derived from pyjs widgets in a
separate repository, but I manage my app-specific pyjs code in the
same repository with the Django backend code for the app, in a
directory called "customjammies", which is purposely not a python
package so it's invisible to the Django app -- pyjsbuild includes
it in the "build" because of the '-I' flag.

> This would also make it a lot easier for PyJS to depend on other Python
> projects for its own use, so that they could be automatically installed.
> If PyJS is to be split up into multiple pieces, it should be able to
> use *itself* as a Python library :-).

It already does that ... or else I don't understand what you're saying.

> I'd have no expectation that after 'pip install' it would be usable as a
> regular platform Python library though.

No, that would make absolutely no sense at all.

Steve

[Kees Bos]

I hadn't noticed it. It has been there for a long time, but is never
really used. Forget about this file.

>
> > 4. I see this setup.py has entry points for sm and translate. i.e. it
> > will install commands called sm and translate into /usr/bin
> > Can't seem to figure out what their purpose would or if they should be
> > eliminated or ignored.
>
> Maybe one of the developers can explain -- I have no idea.

I _think_ spidermonkey is not used anymore (in general). The pysm could
probably go to a legacy/ directory (same for pygtkweb).

>
> > 5. I've tried both pyjscompile, which seems to be for a single file and
> > pyjsbuild for a whole suite of files. But not sure what pyjampiler is?
>
> (Kees explained this.)
> I have only used pyjsbuild and contrib/pyjscompressor, which calls
> google's javascript "compiler" to compress the pyjs-generated
> javascript. It works great -- gets around 50% compression.

The pyjampiler comes from someone who wanted to have pyjs without the
gwt stuff. It has its own boilerplate and should build stripped down
code (no library initiation afaik). I've never used it.

>
> > 6. Not sure what the role of pgen/ and library/ directories are. Are
> > they part of the compiler or are they part of the widget library.
> >
> > 7. The last thing I am trying to do is to see how I can translate a
> > simple helloworld.py program into java script and run using the V8
> > command line interpreter.
> > Can't seem to figure out if I should use pyjampiler, pyjscompile or
> > pyjsbuild. I've tried them they seem to generating incomplete code as is
> > the case of
> > pyjscompile or too much more code relating to windows and stuff if I
> > used pyjsbuild. Has any one tried compiling a simple helloworld.py
> > program, translated
> > it into simple javascript and run it inside V8 interpreter as opposed to
> > the browser?
>
> I have never done that. As I say, I have always used pyjsbuild. Its
> output directory contains an html file named for the "main" module of
> your app, which creates an iframe into which the results of the
> "app.onModuleLoad()" call goes. I don't know how to run the generated
> javascript in the V8 interpreter but I would be interested in knowing,
> if you get that to work.
>

Have a look at examples/libtest/pyv8test.sh That shows how the pyv8
should be invoked. You might have to use pyjsbuild first, but that could
just be a procedure that's stuck in my mind.

[Sarvi Shanmugham]

Thanks for the description.

Question on pgen. Is it still being used?

I just renamed the pgen -> pgen.old to see what happens if it was removed

And then used the bin/pyjsbuild to build a simple test.py file and the build seems to match the regular build before renaming pgen

Makes me wonder if pgen is still used?

Sarv

PS: I have been able to get create setup.py to to pip instal pyjs compiler. Next step do the same for pyjamas/wdgetset as well

[Kees Bos]

On Thu, 2013-10-10 at 00:52 -0700, Sarvi Shanmugham wrote:
> Thanks for the description.
>
>
> Question on pgen. Is it still being used?

I think it's optionally used (when lib2to3 is not available)

>
> I just renamed the pgen -> pgen.old to see what happens if it was
> removed
> And then used the bin/pyjsbuild to build a simple test.py file and the
> build seems to match the regular build before renaming pgen
>
>
> Makes me wonder if pgen is still used?
>
>
> Sarv
>
>
> PS: I have been able to get create setup.py to to pip instal pyjs
> compiler. Next step do the same for pyjamas/wdgetset as well

Good work!

[Kees Bos]

On Thu, 2013-10-10 at 00:52 -0700, Sarvi Shanmugham wrote:

> Thanks for the description.
>
>
> Question on pgen. Is it still being used?

I think it's optionally used (when lib2to3 is not available)
>

> I just renamed the pgen -> pgen.old to see what happens if it was
> removed
> And then used the bin/pyjsbuild to build a simple test.py file and the
> build seems to match the regular build before renaming pgen
>
>
> Makes me wonder if pgen is still used?
>
>
> Sarv
>
>
> PS: I have been able to get create setup.py to to pip instal pyjs
> compiler. Next step do the same for pyjamas/wdgetset as well

Good work!

[Kees Bos]

On Thu, 2013-10-10 at 10:12 +0200, Kees Bos wrote:
> On Thu, 2013-10-10 at 00:52 -0700, Sarvi Shanmugham wrote:
> > Thanks for the description.
> >
> >
> > Question on pgen. Is it still being used?
>
> I think it's optionally used (when lib2to3 is not available)

It will definitely be needed when implementing things like 'eval'

[Łukasz Mach]

W dniu 10.10.2013 03:43, Steve Waterbury pisze:

>> I'd have no expectation that after 'pip install' it would be usable as a
>> regular platform Python library though.
>
> No, that would make absolutely no sense at all.

I cannot agree with your anti-pip-install-ism :).

It should be fairly easy to avoid conflicting with core libraries and
pyjs core libraries.

setup.py can have some trigger which won't allow to install if not in
virtualenv.

Also, some part of pyjs/pyjamas library can be used by native python
code. Eg. in pyjd, when is fixed, or in new style of desktop pyjamas
(that one which uses qt/wxgtk...)

--
pozdrawiam

Łukasz Mach - lukas...@pagema.net

[Sarvi Shanmugham]

You are right I do see that traslator_dict.py seems to be importing stuff from lib2to3 which is in python

from lib2to3 import pygram, pytree

But this seems to work for regular python which means its part of the standard python. 

I am using python2.7 and lib2to3 exists in there

So I am guessing its needed by older python 2.6 and likes. 

So I thought I'd package it with pyjs anyway but ran into a issue I don't understand, or rather don't see how its working code.

Trying to package and install pgen causes the following error

processing pyjsc-0.8.1-py2.7.egg

creating /Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/pyjsc-0.8.1-py2.7.egg

Extracting pyjsc-0.8.1-py2.7.egg to /Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages

  File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/pyjsc-0.8.1-py2.7.egg/pyjs/pgen/lib2to3/pygram.py", line 10

    from lib2to3/pgen2 import token

                ^

SyntaxError: invalid syntax

Adding pyjsc 0.8.1 to easy-install.pth file

From what I understand "lib2to3/pgen2" should be "lib2to3.pgen2" and slashes are not permitted in python import statements.

Can't find any documentation that explains the use of "/" and python2.7 doesn't like it which tells me it is not a supported syntax.

What am I missing here. How can this be working code?

Sarvi

[Kees Bos]

On Thu, 2013-10-10 at 10:08 -0700, Sarvi Shanmugham wrote:
> You are right I do see that traslator_dict.py seems to be importing
> stuff from lib2to3 which is in python
> from lib2to3 import pygram, pytree
>
> But this seems to work for regular python which means its part of the
> standard python.
> I am using python2.7 and lib2to3 exists in there
>
>
> So I am guessing its needed by older python 2.6 and likes.

2.6 has also lib2to3 afaik

> So I thought I'd package it with pyjs anyway but ran into a issue I
> don't understand, or rather don't see how its working code.
>
>
> Trying to package and install pgen causes the following error
> processing pyjsc-0.8.1-py2.7.egg
> creating /Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/pyjsc-0.8.1-py2.7.egg
> Extracting pyjsc-0.8.1-py2.7.egg
> to /Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages
> File
> "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/pyjsc-0.8.1-py2.7.egg/pyjs/pgen/lib2to3/pygram.py", line 10
> from lib2to3/pgen2 import token
> ^
> SyntaxError: invalid syntax

Yeah, that should definitely be 'lib2to3.pgen2'.

>
> Adding pyjsc 0.8.1 to easy-install.pth file
>
>
> From what I understand "lib2to3/pgen2" should be "lib2to3.pgen2" and
> slashes are not permitted in python import statements.
> Can't find any documentation that explains the use of "/" and
> python2.7 doesn't like it which tells me it is not a supported syntax.
>
>
> What am I missing here. How can this be working code?
>

Not ;-)

Probably, pgen/lib2to3/pygram.py is not used.

Maybe we can skip pgen in pip. We can always add this when needed.

[Steve Waterbury]

On 10/10/2013 04:54 AM, Łukasz Mach wrote:
> W dniu 10.10.2013 03:43, Steve Waterbury pisze:
>>> I'd have no expectation that after 'pip install' it would be usable as a
>>> regular platform Python library though.
>>
>> No, that would make absolutely no sense at all.
>
> I cannot agree with your anti-pip-install-ism :).

I'm not anti-it, I just don't see any advantage it gives.
Using pyjs in a directory has always worked for me,
no installation necessary.

In what you quoted above, what I was saying was that
using pyjs as a regular Python library would make no
sense at all, not that developing a pip install capability
makes no sense at all -- it makes some sense, just not much
ROI that I can see ... although I admit it would
have psychological attractiveness, which shouldn't be
underrated. ;)

> It should be fairly easy to avoid conflicting with core libraries and
> pyjs core libraries.
>
> setup.py can have some trigger which won't allow to install if not in
> virtualenv.
>
> Also, some part of pyjs/pyjamas library can be used by native python
> code. Eg. in pyjd, when is fixed, or in new style of desktop pyjamas
> (that one which uses qt/wxgtk...)

I don't understand this idea -- there are way more mature and
powerful python desktop gui technologies that use wx/qt,
such as pyface / enaml. I don't see how pyjd or any new style of
desktop pyjamas could possibly offer any benefits over those ...
if anything, it would make sense to work on adding to pyjs a
capability for translating code from *them* into javascript ...

Also, I think pyjd is a distraction from the unique strength of
pyjs: using Python to generate javascript for web applications.
IMO, if pyjs simply focuses on that alone, it will be an
excellent library.

Steve

[C Anthony Risinger]

the GI backend is 95% feature complete (save addEventListener, which i must still implement via ctypes/cffi), and the MS one is pretty much a non-issue.

the GI backend supersedes all other linux backends (hulahop/pythonwebkit/etc), works fantastic,  and once feature complete should work without issue for a very long time... it *should* even work with webkit2 variants, though i have not confirmed.

while i don't think pyjd is necessary for success, it's absolutely wonderful for development... and that alone is enough to motivate me at least.

as for pip, my goal is to consolidate all pyjs* commands into a single `pyjs [sub] [--arg]` style (eg. like git), which would encompass everything from translation to switchable pyjd backends, so being able to install via pip is a good thing IMO. the fact that all libraries are not useable by python is not an issue, because they can just be put outside PYTHONPATH, and only included as needed (the meta_path importer mentioned later will be able to do this transparently).

additionally, and this is probably one of the better reasons, once i pull out the importlib stuff in favor of a meta_path impl, we'll be able to cache BOTH translated JS AND compiled python files for packaging... this means we can properly pre-translate/compile the entire library (with a couple common variants, eg. with/without --debug, etc), resulting in a SIGNIFICANT speed boost, due to the fact that the translator need only translate your code, and can include the stdlib verbatim (additionally, the waf builder properly caches your code locally, so only changed code is re-translated).

...this combined with the waf builder should result in rebuild that literally complete in less than 5-10 seconds, if that; i've already done massive amounts of development in distutils/waf/meta_path and all the other pieces needed to implement these things for related projects at $DAYJOB, i mainly just need to port them over to pyjs.

SIDENOTE...

i know i've been pretty silent for much longer than i'd like... since last year I've switched employers, moved twice (just last month!), got married (again last month!), my kid started school, my [now] wife completed grad school, and she just landed her job less than 2 weeks ago... needless to say i've barely found time whatsoever for devel outside work (which frankly, depresses me), but all along i've been trying to make pyjs relevant at $DAYJOB, which would allow me to do some devel on the clock... i very much think this is a real possibility rather soon, but as always, it's just a matter of time.

do note, i don't offer any of this as an excuse/justification of anything (or even expect anyone to care for that matter ;), i'm simply relaying my situation which hopefully highlights some reasons for my lack of activity and/or presence.

the good news is that, after 7+ years of schooling for my wife, moving around 2 states, and in general, dealing with non-stop upheaval, i am literally a month or 2 from finally being on a path to financial (and general) stability... to the point that i am now looking into buying the house i recently moved to.

tl;dr... i'll brb :)

--

C Anthony

[Sarvi Shanmugham]

Congratulations on your wedding :-)

Just wanted to let you know I've just gotten pip install working for all the pyjsbuild stuff and in progress to do the same for the pyjamas widget library separately.

Should I continue to finish this?

Sarvi

PS: My changes are very minimal outside of the one setup.py with multiple setup() calls I have implemented. No other restructuring or splitting of code.

[Łukasz Mach]

W dniu 10.10.2013 20:09, C Anthony Risinger pisze:
[...]

> last year I've switched employers, moved twice (just last month!), got
> married (again last month!),

Congrats!!!

[...]

> the good news is that, after 7+ years of schooling for my wife, moving
> around 2 states, and in general, dealing with non-stop upheaval, i am
> literally a month or 2 from finally being on a path to financial (and
> general) stability... to the point that i am now looking into buying the
> house i recently moved to.

Be careful with your new house, it can eat your financial stability :)

[Łukasz Mach]

W dniu 10.10.2013 19:37, Steve Waterbury pisze:
[...]

>>
>> Also, some part of pyjs/pyjamas library can be used by native python
>> code. Eg. in pyjd, when is fixed, or in new style of desktop pyjamas
>> (that one which uses qt/wxgtk...)
>
> I don't understand this idea -- there are way more mature and
> powerful python desktop gui technologies that use wx/qt,
> such as pyface / enaml.

But do they give you the same python code working on both: desktop and web?

[Steve Waterbury]

No, but way better desktop capabilities -- pyjd isn't in
the same league. That's why I suggested using pyjs to
translate apps from those frameworks to the web, rather
than the other way around.

Full disclosure: these are comments from one who *uses*
pyjs/pyjd, rather than a developer of pyjs/pyjd. Obviously,
if you are developing pyjd, you will (and should) make decisions
based on your own judgment -- I'm just providing my opinion as a
user's perspective.

I believe that having a web app and/or UI and a desktop
UI generated from the same code is potentially a worthy
goal and possibly even a killer app, but I think starting with
one of the existing qt/wx frameworks or metaframeworks would be
the best approach -- for example, think of the fans that pyjs
would have if it could generate a web interface from some
*existing* qt or wx apps ... the mind boggles.

OTOH, I think it's a much more difficult case to make to a
developer who wants to create a good desktop app to say
"use pyjd because it will give you a web interface too",
unless there are minimal compromises in actual capabilities.

For example, for scientific apps that involve graphing, the
Bokeh js framework being developed by continuum.io is very
promising as a web UI with similar capabilities to existing
desktop UIs developed using (e.g.) scipy's "chaco". It would
make a *huge* impact on that community if a Bokeh app could be
*generated* from the code for a chaco or matplotlib app ...
and I daresay pyjs/pyjd would then attract a large, competent,
and well-funded/supported community of potential new
contributors.

But first pyjs needs to be quite solid, because for example
the tool in which Bokeh is currently being developed,
CoffeeScript, is quite powerful and its syntax is already
quite similar in some ways to python's ... check out your
competition: <http://coffeescript.org/>. Yes, pyjs could
offer them a huge win if they could write a desktop app in
chaco or enaml and generate a nearly identical web UI, but
the generated javascript would need to be very efficient,
too, to make the trade-off worthwhile.

Steve

[C Anthony Risinger]

On Oct 10, 2013 4:39 PM, "Steve Waterbury" <wate...@pangalactic.us> wrote:

This has already been done actually; probably in the old ML of pyjamas-dev, but at one point pyjd was backed by GTK/QT... There were however, major deficiencies in that the widgets provided simply cannot achieve the same richness/flexibility as HTML/CSS... The specifics are detailed in the archives.

Now, I know that pyjs-on-QT-or-GTK is actually the reverse of what you said! alas, GTK-on-pyjs has *also* been done -- see the pygtkweb directory... It implements the GTK API thus allowing a GTK app to be translated to pyjs counterparts.  This code however has been unmaintained/unused for some time, so its state/usefulness is unknown (I've been tempted to kill it at least 3 times that I recall).

Ultimately, I think that path leads to a can of worms, with an explosion of APIs that need support -- we'd then need to track and maintain compat with an ever growing list of versions and libraries -- which do we support? And which versions? Only the latest? What about foreign bugs? ...do we faithfully duplicate them? By focusing on the DOM (which by itself is a lot to chew) that scope is clearly defined and contained.

--

C Anthony [mobile]

[Glyph]

On Oct 9, 2013, at 6:43 PM, Steve Waterbury <wate...@pangalactic.us> wrote:

> On 10/09/2013 07:49 PM, Glyph wrote:
>>
>> On Oct 9, 2013, at 12:16 PM, Steve Waterbury <wate...@pangalactic.us
>> <mailto:wate...@pangalactic.us>> wrote:
>>
>>> ... in particular, why it doesn't make sense to 'pip install' pyjs.
>>
>> Personally I'd love to be able to 'pip install' it simply so that I can
>> do 'pip install pyjs' and then get pyjampiler and pyjsbuild on my $PATH
>> automatically, since pip provides nice tools and automation for that.
>> Plus that would make it easier for build systems that already support
>> setuptools (debian, fedora, et. al.) to build ready-made PyJS packages.
>
> I think it would be easy enough to create a pip install that does
> what you suggest. I suppose for some uses having that stuff on your
> path might be good; however, some configuration is always needed to
> use it for development, so just having that stuff in your $PATH is not
> a big win, IMO. For example, for each app that you are creating you
> will need to decide what parameters and paths to use when you run
> pyjsbuild, which you will do every time you make a change to
> your app code. The suggested way to do that is a build.sh script --
> I've attached one of mine so you can see an example. Since build.sh
> is simply a shell wrapper for pyjsbuild, there's really no need
> to have pyjsbuild on your path.

A build.sh script means a tool that only works on UNIX. This is an antipattern, in my opinion. pyjsbuild ought to read a configuration file from the current directory if this is so common that everyone has to do it. (And yes, for every toy pyjs project I built, I had to do it too!)

> NOTE: as you can infer from the path names in my build.sh, I am
> using pyjs in a directory that is located inside a virtualenv, but
> the virtualenv is for the backend [Django] of the app, not for pyjs --
> i.e., the pyjs libraries are not "installed" in the virtualenv.

Why not, though?

> I keep my own library of widgets derived from pyjs widgets in a
> separate repository, but I manage my app-specific pyjs code in the
> same repository with the Django backend code for the app, in a
> directory called "customjammies", which is purposely not a python
> package so it's invisible to the Django app -- pyjsbuild includes
> it in the "build" because of the '-I' flag.

I understand that for the most part this code is not going to be invoked by the same interpreter, but it seems like it would be better for pyjs to just respect existing python conventions than to invent a parallel toolchain. Just look at $PYTHONPATH to locate dependencies, rather than having -I flags to various tools, for example. It's just less stuff to teach people. Right now it's quite hard to write a library designed for use with PyJS, because you have to explain where to put everything manually, rather than just saying 'pip install <mylib>'.

This goes hand-in-hand with PyJS supporting more of the Python language, of course. If regular libraries could reasonably be made portable to the browser environment, then this would become _hugely_ powerful :).

>> This would also make it a lot easier for PyJS to depend on other Python
>> projects for its own use, so that they could be automatically installed.
>> If PyJS is to be split up into multiple pieces, it should be able to
>> use *itself* as a Python library :-).
>
> It already does that ... or else I don't understand what you're saying.

PyJS is one monolithic piece right now. It can get away with a crummy dependency-management strategy because it doesn't have any dependencies beyond Python: you just 'git clone' PyJS and run the tools. (Well, except you still need to run the 'bootstrap' script, so there is *already* some manual setup which pip could automate away for you.) If it wanted to use Twisted to have a built-in demo server, for example, it would be 'git clone' plus 'pip install'; rather than 'pip install pyjs' and it fetches everything it needs.

But, more importantly, if PyJS were broken into five separate components (as people have been discussing with respect to the Great Schism), then it would be "git clone *five* repositories" and then set up your PYTHONPATH and who knows what else. I believe that this manual setup would be very offputting to new users. This is exactly why tools like pip and virtualenv exist; to automate all these boring code-aggregation tasks.

>> I'd have no expectation that after 'pip install' it would be usable as a
>> regular platform Python library though.
>
> No, that would make absolutely no sense at all.

I don't think it would make *no* sense. Since it's not designed for use as a library, it's not a high priority, but it would still be beneficial if it could work. For example, perhaps someone could write a higher-level framework that depended on PyJS to do JavaScript translation on the fly by monitoring filesystem events, to make the development cycle more Python-ish and less like building a big C++ app ;-).

-glyph

[Steve Waterbury]

On 10/10/2013 10:35 PM, Glyph wrote:
>
> On Oct 9, 2013, at 6:43 PM, Steve Waterbury <wate...@pangalactic.us> wrote:
>
>> On 10/09/2013 07:49 PM, Glyph wrote:
> A build.sh script means a tool that only works on UNIX. This
> is an antipattern, in my opinion. pyjsbuild ought to read a
> configuration file from the current directory if this is so
> common that everyone has to do it. (And yes, for every toy
> pyjs project I built, I had to do it too!)

Agreed. I don't care how the configuration is done -- a config
file would be fine, and then it would make sense to have pyjsbuild
in your path ... and as in your later comment, the "build" process
could even be automated so you wouldn't even need pyjsbuild in your
path ... ;)

>> NOTE: as you can infer from the path names in my build.sh, I am
>> using pyjs in a directory that is located inside a virtualenv, but
>> the virtualenv is for the backend [Django] of the app, not for pyjs --
>> i.e., the pyjs libraries are not "installed" in the virtualenv.
>
> Why not, though?

Because it's not necessary and doesn't buy anything. Actually
it wouldn't hurt anything, they would just be ignored. But
currently there is no reason to install them in the virtualenv.

>> I keep my own library of widgets derived from pyjs widgets in a
>> separate repository, but I manage my app-specific pyjs code in the
>> same repository with the Django backend code for the app, in a
>> directory called "customjammies", which is purposely not a python
>> package so it's invisible to the Django app -- pyjsbuild includes
>> it in the "build" because of the '-I' flag.
>
> I understand that for the most part this code is not going to
> be invoked by the same interpreter, but it seems like it would
> be better for pyjs to just respect existing python conventions
> than to invent a parallel toolchain. Just look at $PYTHONPATH
> to locate dependencies, rather than having -I flags to various
> tools, for example. It's just less stuff to teach people.
> Right now it's quite hard to write a library designed for use
> with PyJS, because you have to explain where to put everything
> manually, rather than just saying 'pip install <mylib>'.

That would be a good idea *if* pyjs were going to translate
code that can actually run as python code -- e.g., my idea
of having pyjs translate qt/wx applications. Otherwise, it
would make more sense for pyjs to have its *own* virtualenv ...
I agree that *that* would be a good idea.

> This goes hand-in-hand with PyJS supporting more of the Python
> language, of course. If regular libraries could reasonably be
> made portable to the browser environment, then this would
> become _hugely_ powerful :).

Agreed.

> PyJS is one monolithic piece right now. It can get away with a
> crummy dependency-management strategy because it doesn't have
> any dependencies beyond Python: you just 'git clone' PyJS and
> run the tools. (Well, except you still need to run the
> 'bootstrap' script, so there is *already* some manual setup

> which pip could automate away for you.) ...

True enough.

> If it wanted to use
> Twisted to have a built-in demo server, for example, it would
> be 'git clone' plus 'pip install'; rather than 'pip install
> pyjs' and it fetches everything it needs.

Now *that* is a selling point. Some of the old pyjs demos
had a php app on the back-end ... yuck! :p

> But, more importantly, if PyJS were broken into five separate
> components (as people have been discussing with respect to the
> Great Schism), then it would be "git clone *five* repositories"
> and then set up your PYTHONPATH and who knows what else. I
> believe that this manual setup would be very offputting to new

> users. ...

Agreed.

> This is exactly why tools like pip and virtualenv
> exist; to automate all these boring code-aggregation tasks.

Actually that's only *one* of the reasons pip and virtualenv
exist ... and IMO not the most important: allowing multiple
versions of python and/or libraries/apps to coexist peacefully on
one machine (and in the unix case, avoidance of the pesky "system
python" -- to be sure, the latter is a stupid problem that the
Linux distros should have solved themselves).

>>> I'd have no expectation that after 'pip install' it would be
>>> usable as a regular platform Python library though.
>>
>> No, that would make absolutely no sense at all.
>
> I don't think it would make *no* sense. Since it's not
> designed for use as a library, it's not a high priority, but it
> would still be beneficial if it could work. For example,
> perhaps someone could write a higher-level framework that
> depended on PyJS to do JavaScript translation on the fly by
> monitoring filesystem events, to make the development cycle
> more Python-ish and less like building a big C++ app ;-).

Interesting idea. *Then* it might make sense, though you had not
proposed that. In fairness, using pyjs as it is currently isn't
*that* much like building a C++ app, or you wouldn't catch me
doing it! ;)

> -glyph

Peace,
Steve

[Sarvi Shanmugham]

The following 2 files had syntax errors and looks like some sort of a cross between pythong and C++ code.

Since pip install does try to compile them all to .pyc files.

library/pyjamas/selection/HtmlBBox.py

ibrary/pyjamas/media/NetworkState.py.needsfixing

So have temporarily renamed them as folllows

library/pyjamas/selection/HtmlBBox.py

ibrary/pyjamas/media/NetworkState.py.needsfixing

Is that ok?

Sarvi

[Kees Bos]

On Thu, 2013-10-10 at 22:11 -0700, Sarvi Shanmugham wrote:
> The following 2 files had syntax errors and looks like some sort of a
> cross between pythong and C++ code.
> Since pip install does try to compile them all to .pyc files.
>
>
> library/pyjamas/selection/HtmlBBox.py
> ibrary/pyjamas/media/NetworkState.py.needsfixing
>
>
> So have temporarily renamed them as folllows
>
>
>
> library/pyjamas/selection/HtmlBBox.py
> ibrary/pyjamas/media/NetworkState.py.needsfixing
>
>
> Is that ok?

I don't see the difference (maybe it's my eyes), but renaming them is OK

[Sarvi Shanmugham]

Sorry. I meant I am renaming 

library/pyjamas/selection/HtmlBBox.py

library/pyjamas/media/NetworkState.py

as they seem to contain nonpython code that causes pip install to fail.

[Kees Bos]

On Thu, 2013-10-10 at 22:54 -0700, Sarvi Shanmugham wrote:
> Sorry. I meant I am renaming
> library/pyjamas/selection/HtmlBBox.py
> library/pyjamas/media/NetworkState.py
>
>
>
> as they seem to contain nonpython code that causes pip install to
> fail.

Yeah. Go ahead. These files are in a kind of intermediate state
(java->python). They don't work, so they shouldn't be distributed with
pip.

[Sarvi Shanmugham]

Ok. I have things building comparable to the bootstrap stuff.

with a single setup.py to install everything but organized as 3 separate setup.py underneath so that its easy to break things into components at some point.

I've tried creating a virtualenv, installing into it and then using pyjsbuild from that installation without using the bootstrap.py stuff and it works

Compared with the bootstrap build process and seems comparable.

There is probably more work need to get pyjd complete.

I now have a git repo I pulled from pyjs, made the changes and commited it into my local repo.

So the changes are here.

What is the best I can give it to you guys for review and commit?

Sarvi

[Sarvi Shanmugham]

Just want to make sure what i have done is ok. Havent done got commits before

I hadn't create a branch when I started and had commited by changes

So then I went ahead and create a branch

git branch pip-install

git checkout pip-install

tried pushing the named branch as follows and got an error.

git push origin pip-install

localhost:pyjs sarvi$ git push origin pip-install

Identity added: /Users/sarvi/.ssh/id_rsa (/Users/sarvi/.ssh/id_rsa)

ERROR: Permission to pyjs/pyjs.git denied to sarvi.

fatal: Could not read from remote repository.

Please make sure you have the correct access rights

and the repository exists.

I guess I need to be added

Sarvi

[Łukasz Mach]

Right way is:

- login to github
- make fork of pyjs
- make changes there
- create pull request


W dniu 11.10.2013 10:42, Sarvi Shanmugham pisze:

> --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "Pyjs.org Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pyjs-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Kees Bos]

On Fri, 2013-10-11 at 01:42 -0700, Sarvi Shanmugham wrote:
> Just want to make sure what i have done is ok. Havent done got commits
> before
> I hadn't create a branch when I started and had commited by changes
>
>
> So then I went ahead and create a branch
> git branch pip-install
> git checkout pip-install
>
>
> tried pushing the named branch as follows and got an error.
> git push origin pip-install
>
>
>
> localhost:pyjs sarvi$ git push origin pip-install
> Identity added: /Users/sarvi/.ssh/id_rsa (/Users/sarvi/.ssh/id_rsa)
> ERROR: Permission to pyjs/pyjs.git denied to sarvi.
> fatal: Could not read from remote repository.
>
>
> Please make sure you have the correct access rights
> and the repository exists.
>
>
> I guess I need to be added
>

Yep. An admin will have to add you (Anthony? Lex?). Or you could fork
and create a pull request.

[Sarvi Shanmugham]

Ok. Have forked and commited and sent a pull request.

pip install -e (development mode) does not work due to the non-standard directory structure

Sarvi

[Duly]

We are attempting to fix `pip install pyjs` on GitHub.

https://github.com/pypa/pip/issues/1649
https://github.com/pyjs/pyjs/issues/742

Also, please note that the main website, http://pyjs.org , does not work over HTTP Secure, https://pyjs.org .

Sincerely,
Jose Torres
CEO
Duly Corporation
Secure Offline Sharing
http://duly.co

[Clavier]

On Friday, October 11, 2013 4:59:39 PM UTC+9, Sarvi Shanmugham wrote:

There is probably more work need to get pyjd complete.

I tested pyjd from the latest GitHub version. It didn't work.  I found that pyjd and other related directories were not copied during installation. I modified the setup file so that they were copied to the target directory as following.

    in setup_pyjd.py (created):

    packages=['pyjd', 'pyjs.runners']

    package_data={}

    entry_points = {}

pyjd/ and pyjs.runners/ directories were copied correctly (to C:\Python27\Lib\site-packages\pyjs-0.8.1-py2.7.egg\pyjs), but still an error appears when importing a class in pyjamas.ui. I tried to trace down this error further, but it was too complicated because pyjd intercepted "import" using its own library. 

I think that this problem appeared due to the modification for pip installation. I hope that pyjd installation by pip is implemented soon. I am using pyjs installed from GitHub version, and pyjd installed from a released version. It makes really difficult to fix other bugs in pyjd in the current GitHub version. 

[ming]

Hi, Clavier, What's the error message? Which runner and host os?

Is it " 'module' object has no attribute 'threading' "? I got that on windows only when using Timer, but no problem on fedora 21, or 20 with updated gobject-introspection, pygobject3, webkitgtk3. In my case it makes more sense if it is the runner engine part.