strange dal issue

[Massimo Di Pierro]

I am running into a potentially serious problem with Dal. I am using the latest stable and not the latest dal.

I am suing sqlite and create a simple table. Then I remove the content of the database folder, without rebooting the web server/web2py.

when trying to access the app:

ERROR:web2py:Traceback (most recent call last):

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/restricted.py", line 227, in restricted

    exec ccode in environment

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/applications/friends/models/db.py", line 34, in <module>

    check_reserved=['all'])

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/base.py", line 174, in __call__

    obj = super(MetaDAL, cls).__call__(*args, **kwargs)

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/base.py", line 473, in __init__

    "Failure to connect, tried %d times:\n%s" % (attempts, tb)

RuntimeError: Failure to connect, tried 5 times:

Traceback (most recent call last):

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/base.py", line 446, in __init__

    self._adapter = ADAPTERS[self._dbname](**kwargs)

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/adapters/base.py", line 60, in __call__

    obj = super(AdapterMeta, cls).__call__(*args, **kwargs)

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/adapters/sqlite.py", line 79, in __init__

    if do_connect: self.reconnect()

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/connection.py", line 104, in reconnect

    self.connection = f()

  File "/Users/massimodipierro/Desktop/DePy2016/web2py/gluon/packages/dal/pydal/adapters/sqlite.py", line 77, in connector

    return self.driver.Connection(dbpath, **driver_args)

OperationalError: unable to open database file

this is not the correct behavior. If DAL does not find anything in databases, it should recreate the table. In fact that is what it does if I reboot the server. Somehow DAL is caching something that causes the connection to remain open.

Anybody has seen this before and can reproduce it?

I have the DePy conference tomorrow but I will try debug more on Sunday.

Massimo

[Michele Comitini]

cannot test this right now.

did you check it is not  pool_size causing that?

--
-- mail from:GoogleGroups "web2py-developers" mailing list
make speech: web2py-d...@googlegroups.com
unsubscribe: web2py-develop...@googlegroups.com
details : http://groups.google.com/group/web2py-developers
the project: http://code.google.com/p/web2py/
official : http://www.web2py.com/
---
You received this message because you are subscribed to the Google Groups "web2py-developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py-develop...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Massimo DiPierro]

Did not check but sqlite should not be using pool

[LightDot]

I've seen a similar error just yesterday, didn't have time to investigate, I'm on recent trunk (details below).

It took several attempts for the DAL to create an initial sqlite db, but it did eventually. Before that i've got errors claiming that the tables already exist although the database directory was completely empty (apart from sql.log after the first attempt).

Ok, I recreated the issue (clean welcome app):

OperationalError: table auth_user already exists

Traceback (most recent call last

):
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/restricted.py", line 227, in restricted
    exec ccode in environment
  File "/var/www/vhosts/lab.modronebo.net/passenger/applications/welcome/models/db.py", line 93, in <module>
    auth.define_tables(username=False, signature=False)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/tools.py", line 2376, in define_tables
    format='%(first_name)s %(last_name)s (%(id)s)'))
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/base.py", line 576, in define_table
    table = self.lazy_define_table(tablename,*fields,**args)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/base.py", line 615, in lazy_define_table
    polymodel=polymodel)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/adapters/base.py", line 768, in create_table
    return self.migrator.create_table(*args, **kwargs)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/migrator.py", line 269, in create_table
    self.adapter.create_sequence_and_triggers(query, table)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/adapters/base.py", line 820, in create_sequence_and_triggers
    self.execute(query)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/adapters/__init__.py", line 68, in wrap
    return f(*args, **kwargs)
  File "/var/www/vhosts/lab.modronebo.net/passenger/gluon/packages/dal/pydal/adapters/base.py", line 417, in execute
    rv = self.cursor.execute(command, *args[1:], **kwargs)
OperationalError: table auth_user already exists

Environment:

web2py 2.14.5-stable+timestamp.2016.04.13.22.22.13
python-2.6.6-64.el6.x86_64
sqlite-3.6.20-1.el6_7.2.x86_64
using Apache + Passenger with mod_passenger

Regards

[LightDot]

Ok, probably a false alarm. Can't recreate anymore with the latest master (commit 82d79e7 from 4th of May). Was using an older checkout from 2016-04-26 before.

Sorry for the noise.

[Mathieu Clabaut]

I regularly see something similar (web2py post R-2.12.3) when doing some unit test on my application that may erase the database. And then the running server complains, and doesn't recreate the database.

I thought  it was the expected behaviour.

On Fri, May 6, 2016 at 1:03 PM LightDot <ligh...@gmail.com> wrote:

Ok, probably a false alarm. Can't recreate anymore with the latest master (commit 82d79e7 from 2nd of May). Was using an older checkout from 2016-04-26 before.

Sorry for the noise.

[Massimo DiPierro]

My guess is that at some point we started not inoreing the connection pool for sqlite as we should.