warning: new pydal

[Massimo Di Pierro]

I made some major changes to pydal:

- internal refactoring in the db connection logic

- dropped pg8000 support (never worked well)

- reformatting

The refactoring is a simplification of some pre-existing convoluted logic but it also makes it easy to use pydal with multi-threaded (non web2py) applications and other frameworks. Basically it automatically re-establish a connection when it detects a DB objects is been used in a different thread.

Before you had to call db._adapter.reconnect() This is no no longer necessary.

One side effect is that db = DAL(...) no longer establishes a connection, until the connection is actually used to run a query.

Please report any problem.

Massimo

[JSalvat]

Hi Massimo, still getting errors:

c:\wwwroot\py4web>python -m pip install -r requirements.txt --user

....     Successfully installed pydal-20191227.1


c:\wwwroot\py4web>python ./py4web-start.py apps

.....
Is still experimental...


Dashboard is at: http://127.0.0.1:8000/_dashboard
[ ] loading examples ...
[A[X] loaded examples
[ ] loading todo ...
[A[X] loaded todo
[ ] loading _dashboard ...
[A[FAILED] loading _dashboard
Traceback (most recent call last):
  File "c:\wwwroot\py4web\py4web\core.py", line 826, in import_app
    module_name, init
  File "<frozen importlib._bootstrap_external>", line 407, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 907, in load_module
  File "<frozen importlib._bootstrap_external>", line 732, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "apps\_dashboard\__init__.py", line 24, in <module>
    error_storage = ErrorStorage()
  File "c:\wwwroot\py4web\py4web\core.py", line 701, in __init__
    Field("snapshot", "json"),
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\base.py", line 656, in define_table
    table = self.lazy_define_table(tablename, *fields, **kwargs)
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\base.py", line 697, in lazy_define_table
    polymodel=polymodel,
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\adapters\base.py", line 913, in create_table
    return self.migrator.create_table(*args, **kwargs)
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\migrator.py", line 302, in create_table
    table,
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\migrator.py", line 540, in log
    logfile = self.file_open(table._loggername, "ab")
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\migrator.py", line 548, in file_open
    fileobj = portalocker.LockedFile(filename, mode)
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\contrib\portalocker.py", line 186, in __init__
    self.file = open_file(filename, mode.replace("w", "a"))
  File "C:\Users\salva\AppData\Roaming\Python\Python37\site-packages\pydal\contrib\portalocker.py", line 171, in open_file
    f = open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'apps\\.service\\sql.log'


[ ] loading _default ...

[A[X] loaded _default
[ ] loading _documentation ...
[A[X] loaded _documentation
[ ] loading _minimal ...
[A[X] loaded _minimal
[ ] loading _scaffold ...
[A[X] loaded _scaffold
Bottle v0.12.18 server starting up (using WSGIRefServer())...
Listening on http://127.0.0.1:8000/
Hit Ctrl-C to quit.

Thanks for all your prompt updatings ...

[Massimo Di Pierro]

wrong thread. This is about pydal.

Anyway. It looks for a folder apps/.service that do not exist. Try create that folder "mkdir apps/.service"

[Yael Lador]

Hi Massimo,

I am seeing: 'Mongo' object has no attribute 'server_version_major'"

S'Traceback (most recent call last):\n  File "/opt/python/current/app/gluon/restricted.py", line 219, in restricted\n    exec(ccode, environment)\n  File "/opt/python/bundle/2/app/applications/beengo/models/db.py", line 42, in <module>\n    session.connect(request, response, db, masterapp=None)\n  File "/opt/python/current/app/gluon/globals.py", line 965, in connect\n    row = table(record_id, unique_key=unique_key)\n  File "/opt/python/run/venv/local/lib/python2.7/site-packages/pydal/objects.py", line 703, in __call__\n    orderby_on_limitby=False,\n  File "/opt/python/run/venv/local/lib/python2.7/site-packages/pydal/objects.py", line 2634, in select\n    return adapter.select(self.query, fields, attributes)\n  File "/opt/python/run/venv/local/lib/python2.7/site-packages/pydal/adapters/mongo.py", line 242, in select\n    return self.__select(query, fields, **attributes)\n  File "/opt/python/run/venv/local/lib/python2.7/site-packages/pydal/adapters/mongo.py", line 383, in __select\n    if self.server_version_major < 2.6:\nAttributeError: \'Mongo\' object has no attribute \'server_version_major\'\n'

any help would be greatly appreciated.

Thanks

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/28987515-0e77-43c2-901a-2aa54f0dccf8%40googlegroups.com.

[Leonel Câmara]

It seems good, the only problem I've had until now was that I had to update my Validators to have a record_id argument in their validate method. Make sure that you warn people about that in the release notes.

[Massimo Di Pierro]

I just pushed a new version of web2py linking the new pydal. Can you check if this error is still present. I have been unable to test it but the error is not present in travis-ci

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscribe@googlegroups.com.

[Leonel Câmara]

Actually the problem kind of is mine (oopsies) because I updated my app validators to the new style using the validate method instead of __call__, so it's not a web2py backwards compatibility problem I think, since the old way to do validators should work fine.

[Massimo Di Pierro]

I am curious about the mongo issue. I have seen it before and I thought I had fixed it.

[Leonel Câmara]

I seem to be having quite a few issues with json and jsonb fields.

[Kevin Keller]

In postresql? 

What issues do you have? 

On Mon, Jan 6, 2020 at 7:24 PM Leonel Câmara <leonel...@gmail.com> wrote:

I seem to be having quite a few issues with json and jsonb fields.

--

Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/1d405d5f-6f7c-497c-b6e9-560955ade81d%40googlegroups.com.

[Leonel Câmara]

Kevin, you need to be using jsonb fields, so yes in postgres.

First I needed to fix the jsonb parser:

https://github.com/web2py/pydal/pull/606/files

Buf after I did that I'm getting a lot of errors like this:

  File "C:\Users\Leonel\Desktop\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line 82, in __call__
    return self.call(value, field_type)
  File "C:\Users\Leonel\Desktop\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line 79, in _call
    return self.f(self.parser, value)
  File "C:\Users\Leonel\Desktop\web2py\gluon\packages\dal\pydal\parsers\base.py", line 133, in _json
    raise RuntimeError("json data not a string")
RuntimeError: json data not a string

Where it seems like it's trying to parse the json a second time, I'm not sure how that happens. This was working fine before the reformatting.

[Leonel Câmara]

I'm still investigating but in the middle of the request the parser in the postgres adapter seems to be changing:

<pydal.parsers.postgre.PostgreAutoJSONParser object at 0x0B5D53B0>
<pydal.parsers.postgre.PostgreParser object at 0x0C748590>

This is the cause. Now I need to investigate why it happens. Because this doesn't happen if I'm using the shell in my app and everything seems to be fine in the shell.

[Leonel Câmara]

Found it, the adapter was only setting the parser after the first connection. For some reason the parser was being changed back to the one registered in parsers at a later moment.

I've fixed all my json issues here:

https://github.com/web2py/pydal/pull/606