Using web2py with jython

[Nik]

Hi folks,

For testing web2py with jython I have started it
by executing "jython web2py.py" and got the following
error message:

==================================================================
saturn:web2py_1.71.2 nik$ jython web2py.py
WARNING:root:no file locking
WARNING:root:unable to import dbhash
WARNING:root:GUI not available because Tk library is not installed
default applications appear to be installed already
web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2009
Version 1.71.2 (2009-11-04 15:48:54)
Database drivers available:
Starting cron...
choose a password:********
please visit:
http://127.0.0.1:8000
use "kill -SIGTERM 949" to shutdown the web2py server
Traceback (most recent call last):
File "web2py.py", line 20, in <module>
gluon.widget.start(cron=True)
File "/Users/nik/work/web/web2py_1.71.2/gluon/widget.py", line 852,
in start
server.start()
File "/Users/nik/work/web/web2py_1.71.2/gluon/widget.py", line 852,
in start
server.start()
File "/Users/nik/work/web/web2py_1.71.2/gluon/main.py", line 710, in
start
self.server.start()
File "/Users/nik/work/web/web2py_1.71.2/gluon/wsgiserver.py", line
1884, in start
self.socket.listen(self.request_queue_size)
File "<string>", line 1, in listen
File "/Users/nik/Applications/jython2.5.1/Lib/socket.py", line 824,
in listen
self._config()
File "/Users/nik/Applications/jython2.5.1/Lib/socket.py", line 824,
in listen
self._config()
File "/Users/nik/Applications/jython2.5.1/Lib/socket.py", line 763,
in _config
self.sock_impl.setsockopt(level, optname, self.pending_options
[ (level, optname) ])
File "/Users/nik/Applications/jython2.5.1/Lib/socket.py", line 282,
in setsockopt
raise error(errno.ENOPROTOOPT, "Socket option '%s' (level '%s')
not supported on socket(%s)" % (_constant_to_name(option),
_constant_to_name(level), str(self.jsocket)))
socket.error: (42, "Socket option 'TCP_NODELAY' (level 'IPPROTO_TCP')
not supported on socket(ServerSocket[addr=/
127.0.0.1,localport=8000])")
==================================================================

Am I missing something here?

Regards
Nik

[mdipierro]

jython does not seem to support socket option no_delay. In gluon/
wsgiserver.py replace

nodelay = True

with

nodelay = False


Massimo

[mdipierro]

I changed it in trunk so that it is automatic and now (again) web2py
works with Jython out of the box. I have email the author of
wsgiserver hoping this is fixed in the official distribution.

[Vasile Ermicioi]

I have to convince someone that python is better than Java (and even can work with Java) - 

Jython+Web2py is my best argument :)

[Timbo]

Jython does support TCP_NODELAY, but it does so in the Java
fashion...that is to say only on a client socket. The short story is
this: most platforms allow you to set socket options to server
(accepting) sockets and the accepted (client) sockets that come from
them inherit the options from the server socket. This varies based on
platform. Some platforms inherit all options, some just TCP_NODELAY.
Java is a little different in that it inherits none and consequently
it does not allow setting TCP_NODELAY to a server socket.

The good news is that it does support it on the client socket it just
that you have to socket.setsockopt() on the client connection after
you accept it. This behavior is currently up for debate right now
with Jython. See http://bugs.jython.org/issue1309

Now the blocker that I ran into with Jython + web2py is the template
module. web2py templates rely heavily on regexps. This is great
because it makes them fast...but Jython does something different with
cPython such that everytime I try to render a template, I get a
"Recursion limit exceeded error". Maybe Regexps are not the same in
Java as in Python (Perl). This is one I haven't been able to track
down.

Java (and hence Jython) is great from a concurrency perspective since
it is fully multi-threaded whereas cPython has the GIL to deal with.
It would be awesome if we could find a happy marriage between web2py
templates and Jython.

-tim

On Nov 9, 3:04 am, Vasile Ermicioi <elff...@gmail.com> wrote:
> I have to convince someone that python is better than Java (and even can
> work with Java) -
> Jython+Web2py is my best argument :)
>

[mdipierro]

Can you send me the view that creates the regex problem in Jython. I
have seen it before and I thought it was fixed.

On Nov 9, 7:53 am, Timbo <tfarr...@swgen.com> wrote:
> Jython does support TCP_NODELAY, but it does so in the Java
> fashion...that is to say only on a client socket.  The short story is
> this: most platforms allow you to set socket options to server
> (accepting) sockets and the accepted (client) sockets that come from
> them inherit the options from the server socket.  This varies based on
> platform.  Some platforms inherit all options, some just TCP_NODELAY.
> Java is a little different in that it inherits none and consequently
> it does not allow setting TCP_NODELAY to a server socket.
>
> The good news is that it does support it on the client socket it just
> that you have to socket.setsockopt() on the client connection after
> you accept it.  This behavior is currently up for debate right now

> with Jython.  Seehttp://bugs.jython.org/issue1309

[Timbo]

No need, it blows up for me on the admin interface. I have no special
customizations. I'm trying this on Sun JRE6/Jython2.5.1 on WinXP.

[mdipierro]

I cannot reproduce this with jython2.5rc3 on mac.

[Timbo]

OK so I downloaded fresh versions of both. If put in the above
workaroudn for the TCP_NODELAY issue. And upon running I get:

C:\jython2.5.1>jython.bat \web2py2\web2py.py

WARNING:root:no file locking
WARNING:root:unable to import dbhash
WARNING:root:GUI not available because Tk library is not installed

unpacking apps; this may take a few seconds...

Traceback (most recent call last):

File "\web2py2\web2py.py", line 20, in <module>
gluon.widget.start(cron=True)
File "C:\web2py2\gluon\widget.py", line 726, in start
(options, args) = console()
File "C:\web2py2\gluon\widget.py", line 699, in console
w2p_unpack('admin.w2p', 'applications/admin/')
File "C:\web2py2\gluon\fileutils.py", line 189, in w2p_unpack
os.unlink(tarname)
File "C:\jython2.5.1\Lib\os.py", line 342, in remove
raise OSError(0, "couldn't delete file", path)
OSError: [Errno 0] couldn't delete file: 'admin.tar'

So I wrapped that in a try-block. The next issue to deal with is a
lack of sqlite (Welcome app). That's not difficult but annoying.
Just delete the db.py model (for now). And that gets us to the root
issue I can't get past with any template. Here's the traceback for
the Welcome app main page:

Traceback (most recent call last):

File "C:\web2py2\gluon\main.py", line 459, in wsgibase
serve_controller(request, response, session)
File "C:\web2py2\gluon\main.py", line 459, in wsgibase
serve_controller(request, response, session)
File "C:\web2py2\gluon\main.py", line 206, in serve_controller
run_view_in(response._view_environment)
File "C:\web2py2\gluon\compileapp.py", line 492, in run_view_in
ccode = parse_template(response.view,
File "C:\web2py2\gluon\template.py", line 149, in parse_template
return parse(text)
File "C:\web2py2\gluon\template.py", line 65, in parse
text = replace(re_strings, text, lambda x: x.replace('\n', '\\n'))
File "C:\web2py2\gluon\template.py", line 77, in replace
for item in regex.finditer(text, i):
RuntimeError: maximum recursion depth exceeded

This is Jython 2.5.1 fresh install, 1.71.2 fresh install with only the
changes I've mentioned in this post. There are no bugs for this in
Jython as the answer has always been "just increase the recursion
depth". I've tried setting the recursion depth to something
ridiculously high and that made no difference.

Can anyone else confirm/deny this? I'm testing on Windows, can we get
a Linux user to test this?

-tim

[mdipierro]

The first problem is because you did not upgraded admin in a while ;-)

cd applications/admin
tar zxvf ../../admin.w2p

The second error. Can you help me fix it?

Try each one of these models?

<html><body>{{='hello world'}</body></html>

<html><body>{{='''hello world'''}</body></html>

<html><body>{{="hello world"}</body></html>

<html><body>{{="""hello world"""}</body></html>

which ones break? Any, none? Please email me privately.

Massimo

[mdipierro]

another test. in gluon/templates.py

can you replace

re_strings = re.compile(r'(?P<name>'
+ r"[uU]?[rR]?'''([^']+|'{1,2}(?!'))*'''|"
+ r"'([^'\\]|\\.)*'|"
+ r'"""([^"]|"{1,2}(?!"))*"""|'
+ r'"([^"\\]|\\.)*")', re.DOTALL)


with

re_strings = re.compile(r'(?P<name>'
+ r"[uU]?[rR]?'''([^']|'{1,2}(?!'))*'''|"
+ r"'([^'\\]|\\.)*'|"
+ r'"""([^"]|"{1,2}(?!"))*"""|'
+ r'"([^"\\]|\\.)*")', re.DOTALL)

Does it make things better?

Massimo

[Timbo]

No difference.

I think this is the bug we're dealing with:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5050507

Since it's a wont-fix, our only option is to work around it by relying
less on regexps. Is that an acceptable solution?

> ...
>
> read more »

[mdipierro]

This is clearly a Java bug. A major one. I love the "won't fix"

> ...
>
> read more »

[Timbo]

As it turns out, Java has a fair number of nasty WONT-FIX bugs.

In my above documented attempt where it couldn't delete the
file...there's a WONT-FIX bug for that:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4715154

No wonder ppl hate Java.

Massimo, are you going to re-work template.py to work around the bug?
I know that template.py is kinda touchy when it comes to backward
compatibility. I've looked into this a little more and I'm
considering creating a Jython branch as I could really use it.
However the two bugs above might cause problems with my application
code. I'll have to look into that.

-tim

> ...
>
> read more »

[Timbo]

Massimo,

I'm trying to rewrite template.py for Jython and I can't seem to
figure out what the re_strings regex is doing. I wrote a routine for
what I thought it was doing and that puts "\n" all over my pages. So
I took it out entirely and things seem OK.

What is re_strings for?

> ...
>
> read more »

[mdipierro]

It finds all strings in the code, single quote, double quote, triple
single and triple double. The complication is that single quotes for
example, allow escaped single quotes inside, etc.

> ...
>
> read more »

[Jonathan Lundell]

On Nov 10, 2009, at 12:55 PM, Timbo wrote:

>
> Massimo,
>
> I'm trying to rewrite template.py for Jython and I can't seem to
> figure out what the re_strings regex is doing. I wrote a routine for
> what I thought it was doing and that puts "\n" all over my pages. So
> I took it out entirely and things seem OK.
>
> What is re_strings for?

I wouldn't mind taking a shot at commenting all of those re's, but it'd be useful to have a top-level description first.

A specific question: what's <name> doing in re_strings? (I know what the syntax means; I don't see 'name' being used.)

[hcvst]

Hi Massimo,

I ran web2py 16.7.2 on jython2.5.1/Sun java 1.6.0_11/Win XP without a
problem after changing nodelay to False. Now suddenly I get this
recursion error. I haven't changed anything but then again one never
knows what changes the admins push to one's machine in a corporate
environment. Any idea why it should suddenly stop working? Are there
any files generated perhaps, that could cause the problem and that I
could delete? This comes at a bad time of course as I need to demo our
web2py app on Friday and we use some Java for SOAP/WSDL stuff as
SOAPpy is not up for it. Stress :-)

Thanks,
HC

> > > > > This isJython2.5.1 fresh install, 1.71.2 fresh install with only the

> > > > > changes I've mentioned in this post.  There are no bugs for this in

> > > > >Jythonas the answer has always been "just increase therecursion
> > > > >depth". I've tried setting therecursiondepthto something

> > > > > ridiculously high and that made no difference.
>
> > > > > Can anyone else confirm/deny this?  I'm testing on Windows, can we get
> > > > > a Linux user to test this?
>
> > > > > -tim
>
> > > > > On Nov 9, 10:00 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > I cannot reproduce this with jython2.5rc3 on mac.
>
> > > > > > On Nov 9, 9:52 am, Timbo <tfarr...@swgen.com> wrote:
>
> > > > > > > No need, it blows up for me on the admin interface.  I have no special
> > > > > > > customizations.  I'm trying this on Sun JRE6/Jython2.5.1 on WinXP.
>
> > > > > > > On Nov 9, 8:34 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>

> > > > > > > > Can you send me the view that creates the regex problem inJython. I

> > > > > > > > have seen it before and I thought it was fixed.
>
> > > > > > > > On Nov 9, 7:53 am, Timbo <tfarr...@swgen.com> wrote:
>

> > > > > > > > >Jythondoes support TCP_NODELAY, but it does so in the Java

> > > > > > > > > fashion...that is to say only on a client socket.  The short story is
> > > > > > > > > this: most platforms allow you to set socket options to server
> > > > > > > > > (accepting) sockets and the accepted (client) sockets that come from
> > > > > > > > > them inherit the options from the server socket.  This varies based on
> > > > > > > > > platform.  Some platforms inherit all options, some just TCP_NODELAY.
> > > > > > > > > Java is a little different in that it inherits none and consequently
> > > > > > > > > it does not allow setting TCP_NODELAY to a server socket.
>
> > > > > > > > > The good news is that it does support it on the client socket it just
> > > > > > > > > that you have to socket.setsockopt() on the client connection after
> > > > > > > > > you accept it.  This behavior is currently up for debate right now

> > > > > > > > > withJython.  Seehttp://bugs.jython.org/issue1309
>
> > > > > > > > > Now the blocker that I ran into withJython+ web2py is the template

> > > > > > > > > module.  web2py templates rely heavily on regexps.  This is great

> > > > > > > > > because it makes them fast...butJythondoes something different with

> > > > > > > > > cPython such that everytime I try to render a template, I get a

> > > > > > > > > "Recursionlimitexceedederror".  Maybe Regexps are not the same in

> > > > > > > > > Java as in Python (Perl).  This is one I haven't been able to track
> > > > > > > > > down.
>

> > > > > > > > > Java (and henceJython) is great from a concurrency perspective since

> > > > > > > > > it is fully multi-threaded whereas cPython has the GIL to deal with.
> > > > > > > > > It would be awesome if we could find a happy marriage between web2py
> > > > > > > > > templates andJython.
>
> > > > > > > > > -tim
>
> > > > > > > > > On Nov 9, 3:04 am, Vasile Ermicioi <elff...@gmail.com> wrote:
>
> > > > > > > > > > I have to convince someone that python is better than Java (and even can
> > > > > > > > > > work with Java) -
> > > > > > > > > >Jython+Web2py is my best argument :)
>
> > > > > > > > > > On Mon, Nov 9, 2009 at 8:38 AM, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > > > > > I changed it in trunk so that it is automatic and now (again) web2py

> > > > > > > > > > > works withJythonout of the box. I have email the author of

> > > > > > > > > > > wsgiserver hoping this is fixed in the official distribution.
>
> > > > > > > > > > > On Nov 8, 1:56 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:

> > > > > > > > > > > >jythondoes not seem to support socket option no_delay. In gluon/

> > > > > > > > > > > > wsgiserver.py replace
>
> > > > > > > > > > > >  nodelay = True
>
> > > > > > > > > > > > with
>
> > > > > > > > > > > >  nodelay = False
>
> > > > > > > > > > > > Massimo
>
> > > > > > > > > > > > On Nov 8, 1:48 pm, Nik <nitral...@googlemail.com> wrote:
>
> > > > > > > > > > > > > Hi folks,
>

> > > > > > > > > > > > > For testing web2py withjythonI have started it
> > > > > > > > > > > > > by executing  "jythonweb2py.py" and got the following

> > > > > > > > > > > > > error message:
>
> > > > > > > > > > > > > ==================================================================
> > > > > > > > > > > > > saturn:web2py_1.71.2 nik$jythonweb2py.py
> > > > > > > > > > > > > WARNING:root:no file locking
> > > > > > > > > > > > > WARNING:root:unable to import dbhash
> > > > > > > > > > > > > WARNING:root:GUI not available because Tk library is not installed
> > > > > > > > > > > > > default applications appear to be installed already
> > > > > > > > > > > > > web2py Enterprise Web Framework
> > > > > > > > > > > > > Created by Massimo Di Pierro, Copyright 2007-2009
> > > > > > > > > > > > > Version 1.71.2 (2009-11-04 15:48:54)
> > > > > > > > > > > > > Database drivers available:
> > > > > > > > > > > > > Starting cron...
> > > > > > > > > > > > > choose a password:********
> > > > > > > > > > > > > please visit:
> > > > > > > > > > > > >        http://127.0.0.1:8000
> > > > > > > > > > > > > use "kill -SIGTERM 949" to shutdown the web2py server
> > > > > > > > > > > > > Traceback (most recent call last):
> > > > > > > > > > > > >   File "web2py.py", line 20, in <module>
> > > > > > > > > > > > >     gluon.widget.start(cron=True)
> > > > > > > > > > > > >   File
>

> ...
>
> Erfahren Sie mehr »

[Timbo]

hvcst,

This is an issue that we addressed recently but I don't think there
has been a new release since this patch.

Go here and hit the "Download File" linke:

http://bazaar.launchpad.net/~mdipierro/web2py/devel/annotate/head%3A/gluon/template.py

Copy that file to /web2py/gluon/template.py (overwrite the old one).
See if that fixes your problem.

-tim

> ...
>
> read more »

[Dmitri Zagidulin]

I don't know if this'll help you, but I've recently wrestled with SOAP/
WSDL stuff on python with SOAPpy, and also ran into problems.
I ended up using SUDS, and the problems went away -- it's an excellent
SOAP library for python.
Hopefully you can fix the jython issue in time for your demo, but if
not, consider running the whole thing on regular python and using suds
for the soap part.

> ...
>
> read more »