"Last Saved On" field displays "communication error" using HTTPS on Apache2, OpenSSL 0.98k, mod_wsgi, python 2.6.5

[Salvor Hardin]

I'm new to web2py and python (currently evaluating python frameworks
to replace Ruby on Rails on production servers by July 4th weekend.)

PROBLEM:

* Saving a file using the admin interface displays a red
"communication error" in the "Last Saved On" field. However, when I
click Reload in the browser, a new date and time appears correctly so
the file is successfully saved.

I'm using HTTPS (web2py is not running on 127.0.0.1.)

SYSTEM:
* Software: web2py 1.79.1, Python 2.6.5, Apache 2.2.14, mod_wsgi 2.8,
OpenSSL 0.98k, sqlite 3.6.22
* Browsers: Latest stable versions of Chrome and Firefox on Vista 32-
bit
* OS: Ubuntu Server 10.04 LTS guest running in Virtual PC 2007 SP1 on
Vista 32-bit host
(because Virtual PC 2007 only uses ~16 (sixteen) MB RAM for
actively used Ubuntu Server 10.04 guest setup with 512 MB RAM,
and because my work desktop is Vista, and I need to develop
web apps on identical OS & packages as the production server due to
policy)

CONFIGURATION:

I'm using HTTPS, so here's the SSL portion of the Apache2 config
(ubuntu.private.lan is the dev machine mapped to a LAN address):

<VirtualHost *:80>
ServerName ubuntu.private.lan
WSGIDaemonProcess web2py user=www-data group=www-data display-
name=web2py
...
</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName ubuntu.private.lan

SSLEngine on
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-6]" nokeepalive ssl-unclean-shutdown
downgrade-1.0 force-response-1.0
# MSIE 7 and newer should be able to use keepalive
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

# WSGIDaemonProcess defined is under <VirtualHost *:80> as:
WSGIDaemonProcess web2py user=www-data group=www-data display-
name=web2py
WSGIProcessGroup web2py
WSGIScriptAlias / /srv/web2py/wsgihandler.py

<Directory /srv/web2py>
AllowOverride None
Order Allow,Deny
Deny from all
<Files wsgihandler.py>
Allow from all
</Files>
</Directory>

AliasMatch ^/([^/]+)/static/(.*) \
/srv/web2py/applications/$1/static/$2

<Directory /srv/web2py/applications/*/static/>
Order Allow,Deny
Allow from all
</Directory>

CustomLog /var/log/apache2/access.log common
ErrorLog /var/log/apache2/error.log

</VirtualHost>
</IfModule>

[mdipierro]

Does it work without ssl? Which browser are you using? Have you tried
other browsers?

Massimo

[Salvor Hardin]

I tried latest stable Firefox and Chrome browsers, both running on
Vista.

I'll try using SSH tunnel next and access using plain HTTP, to see if
problem goes away when not using SSL.

[Salvor Hardin]

Here's what I tried:

* I setup SSH tunnel from my Vista desktop's localhost:8080 to
ubuntu.private.lan:80.
* I modified Apache2's <virtualhost *:80> to enable access to /admin

The new non-HTTPS, ssh-tunneled, remote URL to ubuntu.private.lan is:

http://localhost:8080/admin/default/edit/myblog/views/default/index.html

I got same problem again on Firefox and Chrome. Files get saved
successfully, but "Last Saved On" field shows "communication error" in
red background color every time I click Save button.

Next, I'll try using Rocket on Ubuntu, just for debugging this
problem. Rocket won't be approved for production servers here (this
summer), so I hope we can find a solution using Apache 2.2.14.

On Jun 7, 11:58 am, mdipierro <mdipie...@cs.depaul.edu> wrote:

[mdipierro]

try the following. delete all your tickets app/errors/*
reproduce the problem and see if you find a new ticket file. If so,
please email it to me.

[Salvor Hardin]

I also get the same problem using Rocket, using plain HTTP from Chrome
5.0.375.55 and Firefox 3.6.3.

Files get saved, but "Last Saved On" field shows "communication error"
in red background color every time "Save" button is clicked.

Here's what I tried:

* I setup SSH tunnel from my Vista desktop's localhost:8888 to
ubuntu.private.lan:8888.

* I ran using:
sudo -u www-data python web2py -p 8888

* tunneled URL served by Rocket at ubuntu.private.lan, accessed from
my desktop as:

http://127.0.0.1:8888/admin/default/edit/myblog/views/default/index.html

On Jun 7, 11:58 am, mdipierro <mdipie...@cs.depaul.edu> wrote:

[Salvor Hardin]

I'm sorry, there are no error tickets generated from this. And I
didn't see any errors in javascript consoles in Chrome and Firefox
+Firebug.

I only got 1 ticket yesterday while setting up apache due to file
permission error. Deleted that ticket. No errors today at all.

When using Apache 2 with Python 2.6.5, I also checked /var/log/apache2/
error.log which only had one "harmless" error (which I'm told to
ignore based on Googling)

[Mon Jun 07 12:47:45 2010] [error] Exception KeyError:
KeyError(-1216764160,) in <module 'threading' from '/usr/lib/python2.6/
threading.pyc'> ignored

Other than that, no web2py error tickets and no Apache2 errors
logged. I'm new to Rocket, so I'll see if I can find the log file(s)
for that.

[Lisandro Rostagno]

Hi.
First, sorry about my english, I'm from Argentina. I'm a user from the
group of spanish users. I've written in that group, but I did't get
any answer.

My problem is "exactly" the same that the one reported by Salvor
Hardin.

I'm using Ubuntu 9.10, with the last version of web2py (Version 1.79.1
(2010-06-05 19:48:42)).
I've downloaded web2py, unzip, run development server, create
application, edit file, there I receive the "communication error". The
changes that were made before de first error, are saved correctly, but
any changes I do after that it's not saved.

I tried with Firefox and Chrome, and the same thing happens in both
cases.
I'm working with the user I always worked in ubuntu, and I unzipped
web2py in my home folder. No directory permission conflict.

In fact, the error started to "appear" (i don't know how to say that)
just after the installation of the last stable version of web2py.

I also tried to delete all "app/errors/" files, but no error file is
generated.

¿Could someone help me? ¿Or at least guide me "in the look of a"
solution? Thanks very much!

[Salvor Hardin]

More info:

This problem does not occur on web2py 1.79.1 running on Vista.
All is well using web2py.exe, and web2py src using both Python 2.5.4
and Python 2.6.5 in Vista (32-bit .MSI installer versions at
python.org.)

So, the problem happens when web2py is running in Ubuntu Server 10.04
LTS (Apache2 and Rocket.) I didn't see option to install Python 2.5
on Ubuntu Server 10.04, yet.

I will see if the problem can be reproduced on Ubuntu Server 10.04
running on a production server (not Virtual PC 2007.)

Thanks so much for your help.

On Jun 7, 1:04 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:

[Salvor Hardin]

I just tried web2py 1.78.3 on Ubuntu Server 10.04 using Rocket and
there is no "communication error" message.

This problem appears to be introduced in 1.79.1 for some operating
systems. It works fine in Windows, but not in Ubuntu Server 10.04 for
me, and Ubuntu 9.10 for Lisandro.

On Jun 7, 1:41 pm, Lisandro Rostagno <rostagnolisan...@gmail.com>
wrote:

[Thadeus Burgess]

*leaves gedit / netbeans world to test said bug*

It does not work for me either. No tickets are generated however a 500
internal server error is caused when looking at firebug.

Ubuntu 10.4, firefox 3.6.3 or Epiphany

Quite strange, I can't figure out what is causing the error =/

*goes back to gedit / netbeans world*

--
Thadeus

[mdipierro]

can you add some print statements in the admin/controller/default.py
edit action check whether the

POST http://127.0.0.1:8000/admin/default/edit/<file>

is calling the action or not?

On Jun 7, 2:15 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> *leaves gedit / netbeans world to test said bug*
>
> It does not work for me either. No tickets are generated however a 500
> internal server error is caused when looking at firebug.
>
> Ubuntu 10.4, firefox 3.6.3 or Epiphany
>
> Quite strange, I can't figure out what is causing the error =/
>
> *goes back to gedit / netbeans world*
>
> --
> Thadeus
>

[mdipierro]

correction....

of course it is calling it since it saves the file. Can you figure out
what it returns?

On Jun 7, 2:34 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
> can you add some print statements in the admin/controller/default.py
> edit action check whether the
>

> POSThttp://127.0.0.1:8000/admin/default/edit/<file>

[Álvaro Justen]

On Mon, Jun 7, 2010 at 15:10, Salvor Hardin <salvor...@gmail.com> wrote:
>  * I ran using:
>   sudo -u www-data python web2py -p 8888

You are running web2py with user www-data. Please check if this user
have *all permissions* in the whole web2py directory. To be fast:
chown -R www-data /path/to/web2py/

--
Álvaro Justen - Turicas
http://blog.justen.eng.br/
21 9898-0141

[Salvor Hardin]

Good suggestion. I tried this, restarted web2py, and the
"communication error" in "Last Saved On" field still appears when
clicking Save button.

So far, this is what I know about this problem:

Problem: web2py 1.79.1 on Ubuntu 10.04 (also Ubuntu 9.10 reported by
someone else)

No problem: web2py 1.79.1 on Windows Vista (both web2py.exe and src,
using both Python 2.5.4 and Python 2.6.5)

No problem: web2py 1.78.3 on both Ubuntu 10.04 and Windows Vista

DISCLAIMER: I'm very sleep deprived today, so it is possible I made
mistakes during testing, etc. But others confirmed same problem on
Ubuntu using 1.79.1.

On Jun 7, 5:07 pm, Álvaro Justen <alv...@justen.eng.br> wrote:

[Lisandro Rostagno]

I'll add the fact that the version 1.78.3 works fine in Ubuntu 9.10.

2010/6/7 Salvor Hardin <salvor...@gmail.com>

[Salvor Hardin]

I'm sorry, I looked in the wrong directory and made a mistake with
Firebug.

I found this in admin/errors/ and hope you find it useful:

Traceback (most recent call last):
File "gluon/restricted.py", line 178, in restricted
exec ccode in environment
File "/srv/web2py/applications/admin/controllers/default.py", line
1060, in <module>
File "gluon/globals.py", line 96, in <lambda>
self._caller = lambda f: f()
File "/srv/web2py/applications/admin/controllers/default.py", line
404, in edit
return response.json({'file_hash': file_hash, 'saved_on':
saved_on, 'functions':functions, 'controller': controller,
'application': request.args[0] })
File "gluon/globals.py", line 203, in json
return json(data)
File "gluon/serializers.py", line 29, in json
return simplejson.dumps(value)
File "gluon/contrib/simplejson/__init__.py", line 228, in dumps
return _default_encoder.encode(obj)
File "gluon/contrib/simplejson/encoder.py", line 229, in encode
chunks = self.iterencode(o, _one_shot=True)
File "gluon/contrib/simplejson/encoder.py", line 290, in iterencode
self.skipkeys, self.allow_nan, key_memo, self.use_decimal)
TypeError: make_encoder() takes at most 9 arguments (11 given)

On Jun 7, 1:04 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:

[mdipierro]

aha. json problem. We upgraded to the latest simplejson. Yo may have
an older version installed in your system that is conflicting with
it.

user easy_install to upgrade simplejson

[ron_m]

I just started using web2py and am on Ubuntu 10.04 386 Desktop.
Browser is Firefox and I get this error. Looking for the error tickets
I found them under the admin user and was making the first mods to the
db.py file for a connection string to MySQL.

Here is the traceback from the ticket

Error ticket for "admin"
Ticket
127.0.0.1.2010-06-07.16-45-34.99c56d94-2936-4f38-81b4-799f483e2ee6


Error traceback

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.

Traceback (most recent call last):

File "/home/ronm/Dev/web2py/gluon/restricted.py", line 178, in

restricted
exec ccode in environment

File "/home/ronm/Dev/web2py/applications/admin/controllers/

default.py", line 1060, in <module>

File "/home/ronm/Dev/web2py/gluon/globals.py", line 96, in <lambda>

self._caller = lambda f: f()

File "/home/ronm/Dev/web2py/applications/admin/controllers/
default.py", line 343, in edit
return response.json({'error': T('file changed on disk'),
'redirect': URL(r=request, f='resolve', args=request.args)})
File "/home/ronm/Dev/web2py/gluon/globals.py", line 203, in json
return json(data)
File "/home/ronm/Dev/web2py/gluon/serializers.py", line 29, in json
return simplejson.dumps(value)
File "/home/ronm/Dev/web2py/gluon/contrib/simplejson/__init__.py",

line 228, in dumps
return _default_encoder.encode(obj)

File "/home/ronm/Dev/web2py/gluon/contrib/simplejson/encoder.py",

line 229, in encode
chunks = self.iterencode(o, _one_shot=True)

File "/home/ronm/Dev/web2py/gluon/contrib/simplejson/encoder.py",

line 290, in iterencode
self.skipkeys, self.allow_nan, key_memo, self.use_decimal)
TypeError: make_encoder() takes at most 9 arguments (11 given)

In file: /home/ronm/Dev/web2py/applications/admin/controllers/
default.py

I presume you don't need the applications/admin/controllers/default.py
file listing.

I installed as myself in a Dev directory, Python is the standard 2.6.5
install.

Thanks,

Ron

[mdipierro]

yes. 2.6 ships with an older version of simplejson than the 2.1.1 that
ships with web2py and they seem to conflict. I will look into a
solution.

[mdipierro]

Here is a fix.

in gluon/contrib/simplejson/encoder.py

def _import_speedups():
try:
+ raise ImportError
from simplejson import _speedups
return _speedups.encode_basestring_ascii,
_speedups.make_encoder
except ImportError:
return None, None

and in gluon/contrib/simplejson/decoder.py

def _import_c_scanstring():
try:
+ raise ImportError
from simplejson._speedups import scanstring
return scanstring
except ImportError:
return None

and in gluon/contrib/simplejson/scanner.py

def _import_c_make_scanner():
try:
+ raise ImportError
from simplejson._speedups import make_scanner
return make_scanner
except ImportError:
return None
c_make_scanner = _import_c_make_scanner()

(add the line starting with +). If this solves the problem I could
push the fix temporarily until a better one is found.

massimo

On Jun 7, 6:52 pm, ron_m <ron.mco...@gmail.com> wrote:

[Salvor Hardin]

Thank you!

Your suggestion fixed the "communication error" on Ubuntu Server 10.04
when saving files using ajax editor.

Here are the 3 steps in case you want to cut/paste to FAQ:

1. sudo aptitude install python-setuptools
2. sudo easy_install simplejson
3. sudo /etc/init.d/apache2 restart

Speaking of 3rd-party libraries, I noticed jquery 1.4.2 is
available :)

[mdipierro]

On Jun 7, 11:30 pm, Salvor Hardin <salvor.pub...@gmail.com> wrote:
> Thank you!
>
> Your suggestion fixed the "communication error" on Ubuntu Server 10.04
> when saving files using ajax editor.
>
> Here are the 3 steps in case you want to cut/paste to FAQ:
>
> 1. sudo aptitude install python-setuptools
> 2. sudo easy_install simplejson
> 3. sudo /etc/init.d/apache2 restart
>
> Speaking of 3rd-party libraries, I noticed jquery 1.4.2 is
> available :)

In trunk now. please give it a try.

[ron_m]

Thanks for getting back so quickly.

I tried the edits you suggested on a clean Ubuntu 10.04 install with
web2py 1.79.1 and adding the three raise ImportError statements works
- no more communication error notifications. The file always saved
correctly looking at the target file from a different editor when the
error was occurring. Using Synaptic Package Manager the version of
python-simplejson installed on Ubuntu 10.04 is 2.0.9-1build1

I also tried the other suggestion of

sudo aptitude install python-setuptools
sudo easy_install simplejson

on another Ubuntu 10.04 system without the suggested file edits and
that works as well. I am running with the rocket web server after
initial install so apache2 doesn't figure into the equation in my
case. Of course Synaptic still thinks version 2.0.9-1 is installed.

Thanks,

Ron

On Jun 7, 8:58 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
> Here is a fix.
>
> in gluon/contrib/simplejson/encoder.py
>
>   def _import_speedups():
>       try:
> +        raise ImportError
>           from simplejson import _speedups
>           return _speedups.encode_basestring_ascii,
> _speedups.make_encoder
>       except ImportError:

>           return None, Noneggested on a

>
> and in gluon/contrib/simplejson/decoder.py
> default database name for

[Lisandro Rostagno]

Thanks!
I tried installing python-setuptools and simplejson, and it worked! (I
was having the problem on Ubuntu 9.10, with the development server).

I only executed this commands:

sudo aptitude install python-setuptools
sudo easy_install simplejson

And now everything works just fine. Thanks again!

[Salvor Hardin]

The lastest trunk, revision 689fb804f2 (2010-06-08), fixed the
"communication error" on Ubuntu 10.04.

Here's what I did:
1. Uninstalled simplejson 2.1.1 in Ubuntu using:
"sudo easy_install -m simplejson"
2. Confirmed that "communication error" comes back after uninstall.
3. Cloned web2py trunk using:
"sudo -u www-data hg clone https://web2py.googlecode.com/hg/
web2py"
4. Confirmed web2py works without "communication error" on Ubuntu
10.04.

Thank you for amazing support. I find it interesting that web2py
1.79.1 worked without problems on Windows (even with Python 2.6.5 and
web2py 1.79.1 src code version.)

[Thadeus Burgess]

The simplejson shipped with web2py and the os version of simplejson
conflicted is a really bad problem.

I *need* simplejson installed on the system so that other python
programs can use it. However, I can't use web2py then!

Very annoying situation.

--
Thadeus

[mdipierro]

I modified the one that ships in web2py in 1.79.1 and they should no
longer conflict. Can you confirm?

On Jun 8, 8:21 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> The simplejson shipped with web2py and the os version of simplejson
> conflicted is a really bad problem.
>
> I *need* simplejson installed on the system so that other python
> programs can use it. However, I can't use web2py then!
>
> Very annoying situation.
>
> --
> Thadeus
>

> On Tue, Jun 8, 2010 at 12:17 PM, Salvor Hardin <salvor.pub...@gmail.com> wrote:
> > The lastest trunk, revision 689fb804f2 (2010-06-08), fixed the
> > "communication error" on Ubuntu 10.04.
>
> > Here's what I did:
> > 1. Uninstalled simplejson 2.1.1 in Ubuntu using:
> >   "sudo easy_install -m simplejson"
> > 2. Confirmed that "communication error" comes back after uninstall.
> > 3. Cloned web2py trunk using:

> >    "sudo -u www-data hg clonehttps://web2py.googlecode.com/hg/

[Salvor Hardin]

I uninstalled simplejson 2.1.1 which I installed earlier by
easy_install.

web2py 1.79.2 is released and should work fine on Ubuntu 10.04 with os
version of simplejson, too.

On Jun 8, 8:21 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:

> The simplejson shipped with web2py and the os version of simplejson
> conflicted is a really bad problem.
>
> I *need* simplejson installed on the system so that other python
> programs can use it. However, I can't use web2py then!
>
> Very annoying situation.
>
> --
> Thadeus
>
>
>
> On Tue, Jun 8, 2010 at 12:17 PM, Salvor Hardin <salvor.pub...@gmail.com> wrote:
> > The lastest trunk, revision 689fb804f2 (2010-06-08), fixed the
> > "communication error" on Ubuntu 10.04.
>
> > Here's what I did:
> > 1. Uninstalled simplejson 2.1.1 in Ubuntu using:
> >   "sudo easy_install -m simplejson"
> > 2. Confirmed that "communication error" comes back after uninstall.
> > 3. Cloned web2py trunk using:

> >    "sudo -u www-data hg clonehttps://web2py.googlecode.com/hg/