Temporarily disable/intercept ORM events on mutation

[Luna Lucadou]

When customers call our JSON:API API, they can use an "include" parameter to specify related objects to be appended to the response.

However, requests to include are not always satisfiable (e.g. if job.supervisor=null, include=supervisor is ignored).

In order to prevent Marshmallow from trying to load nonexistent related objects to append to our API responses, we need to tell it when to ignore a relationship attribute, such as via setting job.supervisor=marshmallow.missing (if it sees job.supervisor=None, it will attempt (and fail) to load the null supervisor object, so we cannot just leave it as-is).

Unfortunately, this causes problems as SQLAlchemy attempts to handle the new value:

Error Traceback (most recent call last): File "/Users/lucadou/IdeaProjects/person-api/api/unit_tests/test_person_service.py", line 601, in test_get_people_include_job response = self.person_service.get_people(QueryParameters({"include": "jobs"})) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/src/person_service.py", line 61, in get_people response = person_schema.dump(people, many=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 557, in dump result = self._serialize(processed_obj, many=many) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 519, in _serialize return [ ^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 520, in <listcomp> self._serialize(d, many=False) File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 525, in _serialize value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow_jsonapi/fields.py", line 248, in serialize return super().serialize(attr, obj, accessor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/fields.py", line 344, in serialize return self._serialize(value, attr, obj, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow_jsonapi/fields.py", line 274, in _serialize self._serialize_included(item) File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow_jsonapi/fields.py", line 280, in _serialize_included result = self.schema.dump(value) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 551, in dump processed_obj = self._invoke_dump_processors( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 1068, in _invoke_dump_processors data = self._invoke_processors( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/marshmallow/schema.py", line 1225, in _invoke_processors data = processor(data, many=many, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/src/model/schema/job_schema.py", line 62, in set_null_supervisor job.supervisor = missing ^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 536, in __set__ self.impl.set( File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 1466, in set value = self.fire_replace_event(state, dict_, value, old, initiator) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 1505, in fire_replace_event value = fn( ^^^ File "/Users/lucadou/IdeaProjects/person-api/api/venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 2167, in emit_backref_from_scalar_set_event instance_state(child), ^^^^^^^^^^^^^^^^^^^^^ AttributeError: '_Missing' object has no attribute '_sa_instance_state'

Is there any way to temporarily disable ORM event listeners when we mutate objects and have no intention of saving the changes/do not intend for the ORM to act on them?

[Mike Bayer]

unfortunately no, that's a backref event handler, that's within the class instrumentation and has no mechanism to be disabled on a per-class basis, not to mention the backref handler is not the only aspect of things that expects a certain kind of value to be present.  Marshmallow should have alternate APIs that allow the attributes in question to be named by their string name somewhere, rather than being assigned some out-of-band object; that's not very good design in today's pep-484 typed Python.

--

SQLAlchemy -

The Python SQL Toolkit and Object Relational Mapper

 

http://www.sqlalchemy.org/

 

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.

---

You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/34ee3121-e284-4fcb-9c1d-6191557e89bcn%40googlegroups.com.

[Mike Bayer]

On Fri, Sep 15, 2023, at 8:59 PM, Mike Bayer wrote:

unfortunately no, that's a backref event handler, that's within the class instrumentation and has no mechanism to be disabled on a per-class basis, not to mention the backref handler is not the only aspect of things that expects a certain kind of value to be present.  Marshmallow should have alternate APIs that allow the attributes in question to be named by their string name somewhere, rather than being assigned some out-of-band object; that's not very good design in today's pep-484 typed Python.

sorry, I meant "no mechanism to be disabled on a per-instance basis".