web2py 1.90.1 is OUT

[mdipierro]

This is a major - major release mostly because it includes the new
DAL.

It is supposed to be 100% backward compatible but cleaner, and more
readable. We have fixed every known issue. If 1.90.1 breaks you DAL
queries, please report it asap so that it can be addressed sooner.

It also includes a lot of new features as described below.

Changelog 1.90.1:

- new DAL (complete rewrite of the web2py DAL is more modular)
- rewrite has fail safe reload, thanks Jonathan
- better CAS with v2 support, thanks Olivier ROCH VILATO
- better markmin2latex
- session.connect(separate=True) to handle many session files, thanks
huaiyu wang
- changed bahvior of impersonate (more secure, can generate form or
used as API)
- new rocket, thanks Tim
- new pyfpdf
- no more old style classes (we did some cleanup in the past sbut we
had missed a few).
- experimental couchdb support in new dal (only insert, select, update
by id)
- mysql support out of the box via pymysql
- SQLITABLE(...,headers='labels') thanks Bruno
- optional: digitally signed URLs, thanks Brian Meredyk
- minor bug fixes

[Luis Díaz]

+1
excellent teamwork

2010/12/18 mdipierro <mdip...@cs.depaul.edu>

--
Díaz Luis
TSU Analisis de Sistemas
Universidad de Carabobo

http://web2pyfacil.blogspot.com/

Facultad de Odontología

[tomt]

I upgraded to Version 1.90.1 and all of my applications that use mysql
stopped working. The traceback shows:

Traceback (most recent call last):
File "/home/xa21/workspace/webpy4/gluon/restricted.py", line 188, in
restricted
exec ccode in environment
File "/home/xa21/workspace/webpy4/applications/myimages/models/
db.py", line 15, in <module>
db = DAL('mysql://myimages:myimages@localhost/myimages') #
if not, use SQLite or other DB
File "/home/xa21/workspace/webpy4/gluon/dal.py", line 3237, in
__init__
raise RuntimeError, "Failure to connect, tried 5 times:\n%s" %
error
RuntimeError: Failure to connect, tried 5 times:
No module named ssl

- startup shows:
Version 1.90.1 (2010-12-18 19:40:48)
Database drivers available: pysqlite2, pymysql
- earlier version showed:
Version 1.89.5 (2010-11-21 22:12:54)
Database drivers available: pysqlite2, MySQL

Do I need to install a new module to correct this problem or can I
correct this with a modification to my database connection string?

- Tom

On Dec 18, 1:38 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
> This is a major - major release mostly because it includes the new
> DAL.
>
> It is supposed to be 100% backward compatible but cleaner, and more
> readable. We have fixed every known issue.  If 1.90.1 breaks you DAL
> queries, please report it asap so that it can be addressed sooner.
>
> It also includes a lot of new features as described below.
>
> Changelog 1.90.1:
>
> - new DAL (complete rewrite of the web2py DAL is more modular)
> - rewrite has fail safe reload, thanks Jonathan
> - better CAS with v2 support, thanks Olivier ROCH VILATO
> - better markmin2latex

> - session.connect(separate=True) to handle many session files, thanksVersion 1.89.5 supports pysqlite2, MySQL

[czamb]

Congratulations, great work!
For your information: an older version of an app that I'm working on
crashes immediately as shown below. Maybe it helps you fixing
something in the new DAL... in this app I used authorization as in the
reddish example app... the newer version with the auth module runs
fine.
Cheers
czamb

Traceback (most recent call last):

File "c:\web2py\gluon\main.py", line 453, in wsgibase
session._try_store_on_disk(request, response)
File "c:\web2py\gluon\globals.py", line 402, in _try_store_on_disk
cPickle.dump(dict(self), response.session_file)
File "C:\Python26\lib\copy_reg.py", line 74, in _reduce_ex
getstate = self.__getstate__
File "c:\web2py\gluon\dal.py", line 3702, in __getattr__
return dict.__getitem__(self,key)
KeyError: '__getstate__'

[Bruno Rocha]

Crashes in setvirtualfields for Rows object

sqlrows.setvirtualfields(virtual=virtualFields())

 File "/Users/brunomac/web2py_new_dal/web2py/gluon/dal.py", line 4471, in setvirtualfields

    if callable(method) and method.im_func.func_code.co_argcount:

  File "/Users/brunomac/web2py_new_dal/web2py/gluon/dal.py", line 3060, in __getattr__

    return dict.__getitem__(self,key)

KeyError: 'im_func'

--

Bruno Rocha

http://about.me/rochacbruno/bio

[mdipierro]

yes you need to

easy_install ssl

sorry about this. This is actually a problem with pymysql and they
should make the dependency optional. I will contact them.

Massimo

[Bruno Rocha]

Crashes in shell too:

>>> rows = db(db.products.id>0).select()

>>> class virtual(object):

...     def test(self):

...         return 'ok'

... 

>>> rows.setvirtualfields(virtual=virtual())

Traceback (most recent call last):

  File "<console>", line 1, in <module>

  File "/Users/brunomac/web2py_new_dal/web2py/gluon/dal.py", line 4471, in setvirtualfields

    if callable(method) and method.im_func.func_code.co_argcount:

  File "/Users/brunomac/web2py_new_dal/web2py/gluon/dal.py", line 3060, in __getattr__

    return dict.__getitem__(self,key)

KeyError: 'im_func'

>>> 

But using the other way, the virtual field is appended only for the first row:

>>> db.category.virtualfields.append(virtual())

>>> db.category.virtualfields

[<virtual object at 0x101a27410>]

>>> rows = db(db.category.id>0).select()

>>> 

>>> print rows[0]

<Row {'picture': 'category.picture.9f4f39811d410b54.3634362d4c5f7468756d622e6a7067.jpg', 'description': None, 'update_record': <function <lambda> at 0x101d9f668>, 'products': <gluon.dal.Set object at 0x101d93bd0>, 'test': 'ok', 'id': 1, 'delete_record': <function <lambda> at 0x101d9f6e0>, 'name': 'Vegan Food'}>

>>> print rows[1]

<Row {'picture': 'category.picture.b1974a4805e47745.393733432d4c5f7468756d622e6a7067.jpg', 'description': None, 'update_record': <function <lambda> at 0x101d9f758>, 'products': <gluon.dal.Set object at 0x101d93c50>, 'id': 2, 'delete_record': <function <lambda> at 0x101d9f7d0>, 'name': 'Dairy Free Candy'}>

>>> 

2010/12/19 Bruno Rocha <rocha...@gmail.com>

[mdipierro]

All problems (virtualfields and missing ssl) should be fixed in
1.90.2. out .... now!

On Dec 18, 9:09 pm, Bruno Rocha <rochacbr...@gmail.com> wrote:
> Crashes in shell too:
>
> >>> rows = db(db.products.id>0).select()
> >>> class virtual(object):
>
> ...     def test(self):
> ...         return 'ok'
> ...>>> rows.setvirtualfields(virtual=virtual())
>
> Traceback (most recent call last):
>   File "<console>", line 1, in <module>
>   File "/Users/brunomac/web2py_new_dal/web2py/gluon/dal.py", line 4471, in
> setvirtualfields
>     if callable(method) and method.im_func.func_code.co_argcount:
>   File "/Users/brunomac/web2py_new_dal/web2py/gluon/dal.py", line 3060, in
> __getattr__
>     return dict.__getitem__(self,key)
> KeyError: 'im_func'
>
>
>

> *But using the other way, the virtual field is appended only for the first
> row:*

>
> >>> db.category.virtualfields.append(virtual())
> >>> db.category.virtualfields
>
> [<virtual object at 0x101a27410>]>>> rows = db(db.category.id>0).select()
>
> >>> print rows[0]
>
> <Row {'picture':
> 'category.picture.9f4f39811d410b54.3634362d4c5f7468756d622e6a7067.jpg',
> 'description': None, 'update_record': <function <lambda> at 0x101d9f668>,

> 'products': <gluon.dal.Set object at 0x101d93bd0>, *'test': 'ok'*, 'id': 1,

> 'delete_record': <function <lambda> at 0x101d9f6e0>, 'name': 'Vegan Food'}>>>> print rows[1]
>
> <Row {'picture':
> 'category.picture.b1974a4805e47745.393733432d4c5f7468756d622e6a7067.jpg',
> 'description': None, 'update_record': <function <lambda> at 0x101d9f758>,
> 'products': <gluon.dal.Set object at 0x101d93c50>, 'id': 2, 'delete_record':
> <function <lambda> at 0x101d9f7d0>, 'name': 'Dairy Free Candy'}>
>
>
>

> 2010/12/19 Bruno Rocha <rochacbr...@gmail.com>
>
>
>
>
>
> > Crashes in *setvirtualfields *for Rows object

[sushanth]

web2py team rocks :)

[tomt]

Updating to 1.90.2 fixed all the mysql problems I listed and all my
apps are working again.

Thanks alot - Tom

On Dec 18, 7:13 pm, tomt <tom_tren...@yahoo.com> wrote:

[Bruno Rocha]

from trunk VirtualFields is working well! Thanks!

But, 1.90.2 still has this line: if callable(method).......  and virtual fields not working.

I'll wait next update to upgrade my working sites.

Thank you, new dal seen to be much faster!

2010/12/19 tomt <tom_t...@yahoo.com>

[berubejd]

I started receiving this traceback on the console after the upgrade:

Traceback (most recent call last):

File "web2py.py", line 20, in <module>
File "gluon/widget.py", line 781, in start
File "gluon/shell.py", line 194, in run
File "<string>", line 1, in <module>
File "applications\Jellybean_Kingdom\controllers\internal.py", line
43, in lottery
db.lottery.truncate()
File "gluon/dal.py", line 3764, in truncate
File "gluon/dal.py", line 830, in truncate
File "gluon/dal.py", line 3702, in __getattr__
KeyError: '_db_adapter'

This is received running web2py 1.90.2 from a cron job on Windows.

[selecta]

self reference with IS_IN_DB pointing to self

Field('parent', 'reference mytable', requires =
IS_EMPTY_OR(IS_IN_DB(db,'mytable.id','%(name)s'))),

this does not work any more

[Martin.Mulone]

Great!, I will test tomorrow!.

[mdipierro]

fixing this in trunk.

[mdipierro]

fixing in trunk.

[mdipierro]

All the issues raised should now be fixed in 1.90.3

[czamb]

Thanks for the fast reply.
Unfortunately, after upgrade to 1.90.3 and restart, the same result as
before:

Traceback (most recent call last):

File "c:\web2py\gluon\main.py", line 453, in wsgibase
session._try_store_on_disk(request, response)
File "c:\web2py\gluon\globals.py", line 402, in _try_store_on_disk
cPickle.dump(dict(self), response.session_file)
File "C:\Python26\lib\copy_reg.py", line 74, in _reduce_ex
getstate = self.__getstate__

File "c:\web2py\gluon\dal.py", line 3702, in __getattr__
return dict.__getitem__(self,key)
KeyError: '__getstate__'

I tried to "easy_install ssl" before and it refused to install with:
ValueError: This extension should not be used with Python 2.6 or later
(already
built in), and has not been tested with Python 2.3.4 or earlier.

Regards
czamb

[mdipierro]

Looks like you are trying store on a session something that is not
pickable.

Can you show us what objects are your storing in the session?
It will help me figure out what is wrong.

Massimo

[Gary Herron]

In  1.90.3, my SQLFORM calls are failing with

Traceback (most recent call last

):
  File "/home/gherron/Dropbox/projects/web2py/gluon/restricted.py", line 188, in restricted
    exec ccode in environment
  File "/home/gherron/Dropbox/projects/web2py/applications/ServeITS/controllers/default.py", line 233, in <module>
  File "/home/gherron/Dropbox/projects/web2py/gluon/globals.py", line 95, in <lambda>
    self._caller = lambda f: f()
  File "/home/gherron/Dropbox/projects/web2py/applications/ServeITS/controllers/default.py", line 209, in AJAXmember
    form = SQLFORM(db.Member, request.vars.record, _class='formMember')
  File "/home/gherron/Dropbox/projects/web2py/gluon/sqlhtml.py", line 792, in __init__
    inp = self.widgets.options.widget(field, default)
  File "/home/gherron/Dropbox/projects/web2py/gluon/sqlhtml.py", line 205, in widget
    options = requires[0].options()
  File "/home/gherron/Dropbox/projects/web2py/gluon/validators.py", line 424, in options
    self.build_set()
  File "/home/gherron/Dropbox/projects/web2py/gluon/validators.py", line 407, in build_set
    fields = [self.dbset.db[ktable][k] for k in self.fields]
NameError: global name 'ktable' is not defined


I (think I) fixed it by replacing two occurrences (lines 405 and 407)
of "ktable" with "self.ktable".

My app is working with those fixes, but since I have not attempted to
figure out what that code is actually doing, I'm hoping that someone
who knows the code can look.

Gary Herron

-- 
Gary Herron, PhD.
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

[mdipierro]

Thanks Gary, just fixed it in 1.90.4

On Dec 19, 3:06 pm, Gary Herron <gher...@digipen.edu> wrote:
> In  1.90.3, my SQLFORM calls are failing with
>

> Traceback(most recent call last):

[berubejd]

This issue is resolved in 1.90.3.

Thanks!

[sushanth]

Found mysql two issues in new dal.py

First issues


mysql VARCHAR length should be 0 to 255,but web2py default varchar value was set to length = 512,if some forgot to declare varchar length he will get error message key too long.

Second issue

After creating FK in the model

if we declare
db.define_table('dogs',
       Field('teamname'),primarykey=['teamname'],migrate=False
     )

db.define_table('users',
    Field('name'),
    Field('team',db.dogs)
    )


db.users.name.requires = IS_NOT_EMPTY()
db.users.team.requires = IS_IN_DB(db,'dogs.teamname','dogs.teamname')


sql :
CREATE TABLE users(
    id INT AUTO_INCREMENT NOT NULL,
    name VARCHAR(100),
    team id, INDEX team__idx (team), FOREIGN KEY (team) REFERENCES dogs(teamname),
    PRIMARY KEY(id)
) ENGINE=InnoDB CHARACTER SET utf8;

Here teamname was set to string and team in the user table goes interger ,then it throws error 1005 can't create table users

        'reference': 'INT, INDEX %(field_name)s__idx (%(field_name)s), FOREIGN KEY (%(field_name)s) REFERENCES %(foreign_key)s ON DELETE %(on_delete_action)s',

as wrokaround i have changed the dal file to

        'reference': 'VARCHAR(100), INDEX %(field_name)s__idx (%(field_name)s), FOREIGN KEY (%(field_name)s) REFERENCES %(foreign_key)s ON DELETE %(on_delete_action)s',

then it works.

[mdipierro]

I think I have fixed the former issue length now defaults to 255 for
mysql.

I am not sure how to fix the second issue and I will wait to form a
suggestion from Denes.

Massimo

[mdipierro]

What should the 'reference' string to be in the case the they is more
than one field?

On Dec 19, 11:55 pm, sushanth <sushant...@gmail.com> wrote:

[iiijjjiii]

I get an error if I try to copy a row that has a date field.

>>> import copy
>>> db.define_table('test_table', db.Field('test_field', 'date'), migrate=True)
>>> db.test_table.insert(test_field='2010-12-20')
1
>>> row = db(db.test_table.id==1).select()[0]
>>> row_copy = copy.copy(row)

Traceback (most recent call last):

File "<console>", line 1, in <module>
File "/usr/lib/python2.7/copy.py", line 84, in copy
rv = reductor(x)
File "/srv/http/igeejo/web2py/gluon/dal.py", line 3106, in
Row_pickler
return Row_unpickler, (marshal.dumps(data.as_dict()),)
ValueError: unmarshallable object

The data field looks like this:

data: <Row {'update_record': <function <lambda> at 0x90501ec>,
'adate': datetime.date(2010, 12, 20), 'id': 1L, 'delete_record':
<function <lambda> at 0x9109c34>}>

The marshal.dumps method doesn't appear to handle datetime instances.

>>> import marshal
>>> import datetime
>>> marshal.dumps({'field': datetime.date(2010,12,20)})

Traceback (most recent call last):

File "<console>", line 1, in <module>
ValueError: unmarshallable object

Other field types, for example, datetime.datetime and decimal.Decimal,
will cause the same problem.

This worked in version 1.89.5. Is there another recommended way to
copy a row?

Jim Karsten

[iiijjjiii]

I was able to get around the problem by converting the row to a
dictionary.

>>> import copy
>>> db.define_table('test_table', db.Field('test_field', 'date'), migrate=True)
>>> db.test_table.insert(test_field='2010-12-20')
1
>>> row = db(db.test_table.id==1).select()[0]

>>> row_copy = copy.copy(row.as_dict())

Regards,
Jim Karsten

[mdipierro]

Thanks Jim,

I just fixed it in trunk and posting 1.90.6. Thanks for reporting the
issue.

Massimo

[sushanth]

db=DAL('mysql://root:test1234123@localhost/new')


db.define_table('dogs',
       Field('id'),
       Field('teamname'),primarykey=['teamname'],migrate=True

     )

db.define_table('users',
    Field('name'),
    Field('team',db.dogs)
    )


db.users.name.requires = IS_NOT_EMPTY()

db.users.team.requires = IS_IN_DB(db, 'dogs.teamname', '%(dogs.teamname)')



sql.log

timestamp: 2010-12-20T22:39:05.866199
CREATE TABLE dogs(
    teamname VARCHAR(100) NOT NULL,
    PRIMARY KEY(teamname))  ENGINE=InnoDB CHARACTER SET utf8;
success!
timestamp: 2010-12-20T22:39:05.968109

CREATE TABLE users(
    id INT AUTO_INCREMENT NOT NULL,
    name VARCHAR(100),

    team VARCHAR(100), INDEX team__idx (team), FOREIGN KEY (team) REFERENCES dogs(teamname),

    PRIMARY KEY(id)
) ENGINE=InnoDB CHARACTER SET utf8;

success!


def index():
    form=SQLFORM(db.users)
    if form.accepts(request,session):
            response.flash='new record inserted'
    records=SQLTABLE(db().select(db.users.ALL))
   
  
    return dict(form=form,records=records)


I am able to create fk now with string,but if i insert any values into dogs table and refersh the page it throws error

error :

Traceback (most recent call last

):
  File "/home/sushanth/Desktop/web2py/gluon/restricted.py", line 188, in restricted
    exec ccode in environment
  File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py", line 64, in <module>
  File "/home/sushanth/Desktop/web2py/gluon/globals.py", line 95, in <lambda>
    

self._caller = lambda f: f

()
  File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py", line 12, in index
    form=SQLFORM(db.users)
  File "/home/sushanth/Desktop/web2py/gluon/sqlhtml.py", line 792, in __init__
    inp = self.widgets.options.widget(field, default)
  File "/home/sushanth/Desktop/web2py/gluon/sqlhtml.py", line 205, in widget

    options = requires[0].options

()
  File "/home/sushanth/Desktop/web2py/gluon/validators.py", line 424, in options
    self.build_set()
  File "/home/sushanth/Desktop/web2py/gluon/validators.py", line 419, in build_set
    self.labels = [self.label % dict(r) for r in records]
KeyError: 'dogs.teamname'

I think there is an issue with new dal.py with pymsql.

can you please help me on this.

Thanks
sushanth

[mdipierro]

db.users.team.requires = IS_IN_DB(db, 'dogs.teamname', '%
(dogs.teamname)')

should be

db.users.team.requires = IS_IN_DB(db, 'dogs.teamname', '%

(teamname)s')

>   File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py" <http://127.0.0.1:8000/admin/default/edit/welcome/controllers/default.py>, line 64, in <module>

>   File "/home/sushanth/Desktop/web2py/gluon/globals.py", line 95, in <lambda>
>     self._caller = lambda f: f()

>   File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py" <http://127.0.0.1:8000/admin/default/edit/welcome/controllers/default.py>, line 12, in index

[sushanth]

thanks thats got fix,if i nesrt any value ,i am getting below error

RACEBACK

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.

Traceback (most recent call last): File "/home/sushanth/Desktop/web2py/gluon/restricted.py", line 188, in restricted exec ccode in environment

File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py", line 53, in <module>

File "/home/sushanth/Desktop/web2py/gluon/globals.py", line 95, in <lambda> self._caller = lambda f: f

() File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py", line 13, in index if form.accepts(request,session): File "/home/sushanth/Desktop/web2py/gluon/sqlhtml.py", line 1159, in accepts self.vars.id = self.table.insert(**fields) File "/home/sushanth/Desktop/web2py/gluon/dal.py", line 3759, in insert return self._db._adapter.insert(self,self._listify(fields)) File "/home/sushanth/Desktop/web2py/gluon/dal.py", line 703, in insert raise e IntegrityError: (1452, u'Cannot add or update a child row: a foreign key constraint fails (`new`.`users`, CONSTRAINT `users_ibfk_1` FOREIGN KEY (`team`) REFERENCES `dogs` (`teamname`))')

Can you please look into it.

[mdipierro]

I strongly suggest you move this discussion to a private discussion
with Denes.

The DAL has limited support for legacy tables and all the work on this
topic was done by Denes.

I doubt anybody else can help you and it is a complex matter.

Massimo

On Dec 20, 12:09 pm, sushanth <sushant...@gmail.com> wrote:
> thanks thats got fix,if i nesrt any value ,i am getting below error
>
> RACEBACK
>
> 1.
> 2.
> 3.
> 4.
> 5.
> 6.
> 7.
> 8.
> 9.
> 10.
> 11.
> 12.
> 13.
> 14.
> 15.
> 16.
>
> Traceback (most recent call last):
>   File "/home/sushanth/Desktop/web2py/gluon/restricted.py", line 188, in restricted
>     exec ccode in environment

>   File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py" <http://127.0.0.1:8000/admin/default/edit/welcome/controllers/default.py>, line 53, in <module>

>   File "/home/sushanth/Desktop/web2py/gluon/globals.py", line 95, in <lambda>
>     self._caller = lambda f: f()

>   File "/home/sushanth/Desktop/web2py/applications/welcome/controllers/default.py" <http://127.0.0.1:8000/admin/default/edit/welcome/controllers/default.py>, line 13, in index