py4web and record versioning

[Manuel Vio]

Hi, I'm testing py4web and I'm stuck with record versioning.

If in models.py I define a trivial table with versioning enabled like this:

### Define you table below

#

# db.define_table('thing', Field('name'))

#

## always commit your models to avoid problems later

#

# db.commit()

#

db.define_table('thing', Field('name'))

db.thing._enable_record_versioning()

db.commit()

I get this error:

FAILED] loading biolab    
Traceback (most recent call last):
  File "/opt/anaconda/envs/py4web_biolab/lib/python3.7/site-packages/py4web/core.py", line 668, in import_apps
    module_name, init).load_module()
  File "<frozen importlib._bootstrap_external>", line 407, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 907, in load_module
  File "<frozen importlib._bootstrap_external>", line 732, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/anaconda/apps/biolab/__init__.py", line 6, in <module>
    from . models import db
  File "/home/anaconda/apps/biolab/models.py", line 21, in <module>
    db.thing._enable_record_versioning()
  File "/opt/anaconda/envs/py4web_biolab/lib/python3.7/site-packages/pydal/objects.py", line 436, in _enable_record_versioning
    *clones, **d)
  File "/opt/anaconda/envs/py4web_biolab/lib/python3.7/site-packages/pydal/base.py", line 592, in define_table
    table = self.lazy_define_table(tablename, *fields, **kwargs)
  File "/opt/anaconda/envs/py4web_biolab/lib/python3.7/site-packages/pydal/base.py", line 610, in lazy_define_table
    if field.requires is DEFAULT:
  File "/opt/anaconda/envs/py4web_biolab/lib/python3.7/site-packages/threadsafevariable/__init__.py", line 16, in __get__
    return getattr(self.storage, "%s.%s" % (id(instance), id(self)))
  File "src/gevent/local.py", line 408, in gevent._local.local.__getattribute__
AttributeError: 'gevent._local.local' object has no attribute '140473993792312.140473994751056'

I tried to follow pydal source to find the cause, the odd thing is that the creation of "thing" table seems to go well, but its archive definition apparently clashes with threadsafe variables (or their lack).

What am I missing?

[Manuel Vio]

Ok, I narrowed the problem a bit:

In py4web/core.py, around line 207, there is this adjustment in order to make some variables threadsafe:

# make sure some variables in pydal are thread safe

for _ in ['readable', 'writable', 'default', 'update', 'requires']:

setattr(pydal.DAL.Field, _, threadsafevariable.ThreadSafeVariable())

Turns out that PyDAL's _enable_record_versioning(), in order to create the archive table, clones all original fields with their properties. Or, at least, it tries to: it seems that copy() fails to correctly transfer threadsafevariable informations when dealing with those variables.

I guess that when cloned field's requires attribute is accessed those information are not valid anymore and py4web throws the aforementioned exception.

I was able to circumvent this error adding an explicit copy of the attribute in pydal, but still I'm not sure if this could eventually lead to thread problems:

for field in self:

nfk = same_db or not field.type.startswith('reference')

clone = field.clone(unique=False, type=field.type if nfk else 'bigint')

clone.requires = copy.copy(field.requires)

clones.append(

clone

)

Any thougths?

[Massimo Di Pierro]

Sorry for the late response. I just run into this myself. Do not have a good solution at this time, unless we make pydal aware of ThreadSafeVariable(s). Maybe we have to.

[Carlos Hanson]

Greetings,

I just started using py4web, and I'm very excited to do so. 

Has there been any more thought on this issue? I was partially into a new web2py app in which I use _enable_record_versioning(), and I am migrating it to py4web.

Thanks.

[Carlos Hanson]

It looks like table inheritance is affected by the same issue:

monster = db.Table(db, 'monster', Field('color'), Field('size'))
db.define_table('thing', Field('name'), monster)

The following traceback is the same as the error using _enable_record_versioning() starting at table = self.lazy_define_table:

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/home/chanson/py/py4web/apps/thing/__init__.py", line 7, in <module>

    from . models import db

  File "/home/chanson/py/py4web/apps/thing/models.py", line 9, in <module>

    db.define_table('thing', Field('name'), monster)

  File "/home/chanson/.local/share/virtualenvs/py4web-recc_4-8/lib/python3.6/site-packages/pydal/base.py", line 656, in define_table

    table = self.lazy_define_table(tablename, *fields, **kwargs)

  File "/home/chanson/.local/share/virtualenvs/py4web-recc_4-8/lib/python3.6/site-packages/pydal/base.py", line 676, in lazy_define_table

    if field.requires is DEFAULT:

  File "/home/chanson/.local/share/virtualenvs/py4web-recc_4-8/lib/python3.6/site-packages/threadsafevariable/__init__.py", line 16, in __get__

    return getattr(self.storage, "%s.%s" % (id(instance), id(self)))

  File "src/gevent/local.py", line 408, in gevent._local.local.__getattribute__

AttributeError: 'gevent._local.local' object has no attribute '139976984026752.139976984930736'

[Massimo Di Pierro]

No obvious solution yet but will prioritize this. Would you mind opening an issue on github about this?

[Carlos Hanson]

Consider it done.