MongoDb issues?

[Gertjan Martens]

I am investigating if py4Web is easy to use for us to set settings in a MongoDB.

My experiences so far,  are mixed. The examples and documentation look very good. Bu when I started, the problems arrived and I keep getting stuck. Not knowing if it is a bug or wrong use from my side. 

In the documentation, it shows that MongoDB is not fully supported, but people have been working with it (I also checked the web2py usergroup), so it must be something I do wrong in combination with some problems in the library.

• I got problems like explained in https://www.mongodb.com/community/forums/t/djongo-notimplementederror-database-objects-do-not-implement-truth-value-testing-or-bool/188912/4
  After moving back to pymongo v3.12.3 (from v4.3.3.), this problem was solved
• With DB_MIGRATE=False,  an error occurs after an insert: 'Table' object has no attribute '_notnulls'. This was already mention in the web2py group
  
• The correct amount of records are retreived. I see also that the db connection works. But the values shown (and also retreived with select() are the same as the fieldname. Also the id integers are all the same. That should also be a problem, because an objectID is larger than an integer.
• db.Books.insert(Name="Test", Price=0, Category="Fiction", Author="GMA") works, probably because it does not need the objectID. But for e.g the update function does not work (without exception)
  

I can share my  very simple DB and code, but I wonder if someone else can confirm this behaviour, or knows a simple solution that I missed. 

Thanks in advance, best regars,

Gert-jan Martens

[Gertjan Martens]

To make it clear, this is what  it looks like:

And it is not an error of the Ajax Grid, because I could confirm the (pydal) data was the same:

And the data is simple:

And models.py shows :

from .common import db, Field
from pydal.validators import *

db.define_table(
    "Books",
    Field("Name"),
    Field("Category"),
    Field("Price", type='double'),
    Field("Author"),
    format='%(name)s')

db.commit()

Any help is appreciated,

Best regards,

Gertjan Martens

Op zaterdag 27 mei 2023 om 11:19:25 UTC+2 schreef Gertjan Martens:

[Gertjan Martens]

And another addition... It seems I was wrong about the same ID. I could have sworn it was the same when I tested with two records, but looking again, they are different.
But when I look somewhat deeper, I find the response information that shows the correct objectID but returns the fieldnames instead of the values:

I hace checked with some other settings (like ignore_field_case to False),  but no result so far...

Op zaterdag 27 mei 2023 om 11:39:59 UTC+2 schreef Gertjan Martens:

[Rob Dekkers]

I think I found the place where the issue occurs, although I do not know why that specific code is build that way. So I have no direct solution :-(
The problem starts at pydal/adapters/mongo.py at line 337:

            mongo_list_dicts = ctable.find(
                expanded.query_dict,
                expanded.field_dicts,
                skip=limitby_skip,
                limit=limitby_limit,
                sort=mongosort_list,
                modifiers=modifiers,
            )

ctable.find without the field_dicts variable returns the correct values. But with the field_dicts (that contains SON keys), it goes wrong:

So I tried it again without pydal and just via pymongo:

mydb = db._adapter.connection.client.BookstoreDb
test = mydb.Books.find({}, {'id': '_id', 'Category': 'Category'})[0]
#This was the original:  expanded.field_dicts SON([('id', '_id'), ('Name', 'Name'), ('Category', 'Category'), ('Price', 'Price'), ('Author', 'Author')])

Same issue, the projection parameters makes it problematic. No idea btw why they are there anyway, but I did not check other parts or code.

Now I have to rethink if and how to continue..

Best regards,

Gertjan

Op zaterdag 27 mei 2023 om 11:19:25 UTC+2 schreef Gertjan Martens:

I am investigating if py4Web is easy to use for us to set settings in a MongoDB.

[Rob Dekkers]

This works (adding the '$' ):

test = mydb.Books.find({}, {'id': '$_id', 'Category': '$Category', 'Name':'$Name', 'Price':'$Price', 'Author':'$Author'})[0]

Op dinsdag 30 mei 2023 om 22:10:58 UTC+2 schreef Rob Dekkers:

[Alan Etkin]

Can anyone tell what is the current state of the MongoDB experimental pydal adapter?

Is there anyone in the group using it? I tried to connect to a mongodb cloud project form a py4web dev server

My connection URI looks something like

DB_URI = "mongodb://spametki:<password>@mibddnosql.0zlqjzb.mongodb.net/mibddnosql"

When pydal tries to set the DAL online connection, it throws an error (significative part below)

[2025-11-08T12:37:32.061209]: Traceback (most recent call last):
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/py4web/core.py", line 1447, in import_app
    module = load_module(module_name, init)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/py4web/core.py", line 164, in load_module
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/alan/Proyectos/py4web/apps/mymongo/__init__.py", line 7, in <module>
    from . import controllers
ImportError: cannot import name 'controllers' from 'apps.mymongo' (/home/alan/Proyectos/py4web/apps/mymongo/__init__.py)

I'm working in a mongodb tutorial and I'm not sure about using DAL or simply using the Python library connection without DAL API

Regards

Alan

[Alan Etkin]

The complete error traceback as shown in dashboard

Traceback (most recent call last):
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/py4web/core.py", line 1447, in import_app
    module = load_module(module_name, init)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/py4web/core.py", line 164, in load_module
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/alan/Proyectos/py4web/apps/mymongo/__init__.py", line 7, in <module>
    from . import controllers

  File "/home/alan/Proyectos/py4web/apps/mymongo/controllers.py", line 32, in <module>
    from .common import (
  File "/home/alan/Proyectos/py4web/apps/mymongo/common.py", line 43, in <module>
    db = DAL(
         ^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/base.py", line 215, in __call__
    obj = super(MetaDAL, cls).__call__(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/base.py", line 535, in __init__
    raise RuntimeError(
RuntimeError: Failure to connect, tried 5 times:

Traceback (most recent call last):

  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/base.py", line 514, in __init__
    self._adapter = adapter(**kwargs)
                    ^^^^^^^^^^^^^^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/adapters/__init__.py", line 41, in __call__
    obj = super(AdapterMeta, cls).__call__(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/adapters/base.py", line 86, in __init__
    self._initialize_()
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/adapters/mongo.py", line 72, in _initialize_
    self.get_connection()
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/connection.py", line 67, in get_connection
    self.set_connection(connection, run_hooks=True)
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pydal/connection.py", line 74, in set_connection
    if connection:
       ^^^^^^^^^^
  File "/home/alan/py4web_venv/lib/python3.12/site-packages/pymongo/synchronous/database.py", line 342, in __bool__
    raise NotImplementedError(
NotImplementedError: Database objects do not implement truth value testing or bool(). Please compare with None instead: database is not None