Help needed troubleshooting error on session commit after deletion

[Hugo Heyman]

Hello,

I get an error I don't really understand. When I do session commit after a deletion like this:

App: app.bootstrap_new [debug]
Instance: /Users/hugo/Dropbox/lahey/api/instance
>>> from app.extensions import db
>>> from app.models.user import User
>>> user = User.query.all()[0]
>>> db.session.delete(user)
>>> db.session.commit()

 (this is in a shell with flask app context but I get the same error when trying to delete in app that is being served)

 

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 153, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 937, in commit
    self.transaction.commit()
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 461, in commit
    self._prepare_impl()
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
    self.session.flush()
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2237, in flush
    self._flush(objects)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2363, in _flush
    transaction.rollback(_capture_exception=True)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2327, in _flush
    flush_context.execute()
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 370, in execute
    postsort_actions = self._generate_actions()
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 329, in _generate_actions
    if action.execute(self):
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 454, in execute
    self.dependency_processor.presort_deletes(uow, delete_states)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 419, in presort_deletes
    self._passive_delete_flag)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 225, in get_attribute_history
    attributes.LOAD_AGAINST_COMMITTED)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line 928, in get_history
    current = self.get(state, dict_, passive=passive)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line 603, in get
    value = self.callable_(state, passive)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 623, in _load_for_state
    return self._emit_lazyload(session, state, ident_key, passive)
  File "<string>", line 1, in <lambda>
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 747, in _emit_lazyload
    result = q(session).params(**params).all()
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/ext/baked.py", line 399, in all
    return list(self)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/ext/baked.py", line 296, in __iter__
    baked_context = bq._bake(self.session)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/ext/baked.py", line 198, in _bake
    query = self._as_query(session)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/ext/baked.py", line 221, in _as_query
    query = step(query)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 727, in <lambda>
    strategy_options.Load.for_existing_path(
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 977, in __getattr__
    attr = getattr(self.module, key)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 767, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 962, in module
    % (self._il_path, self._il_addtl))
ImportError: importlater.resolve_all() hasn't been called (this is sqlalchemy.orm strategy_options)

My model declaration for this object looks like this:

import datetime
from sqlalchemy_utils.types.password import PasswordType
from sqlalchemy_utils import force_auto_coercion

from app.extensions import db

# Setup coercion of passwords
force_auto_coercion()


class User(db.Model):
 id = db.Column(db.Integer, primary_key=True)
 email = db.Column(db.String(120), unique=True, nullable=False)
 password = db.Column(PasswordType(schemes=['pbkdf2_sha512']), nullable=False)
 name = db.Column(db.String(256))
 created_at = db.Column(db.DateTime, default=datetime.datetime.now)
 updated_at = db.Column(db.DateTime, onupdate=datetime.datetime.now)

I can delete objects of other models without trouble. I've tried to follow the code in the traceback, but can't really make sense of it.

Could this have something to do with the PasswordType column from sqlalchemy_utils?

Help is much appreciated!

/Hugo

[Mike Bayer]

That's probably our fault, what is the exact version of SQLAlchemy in use ?  Did you try upgrading to the latest release ?

--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
http://www.sqlalchemy.org/
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscribe@googlegroups.com.
To post to this group, send email to sqlal...@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

[Hugo Heyman]

1.2.0b3 but I also tried 1.1.15 now with the same result.

/Hugo

You received this message because you are subscribed to a topic in the Google Groups "sqlalchemy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sqlalchemy/kz47tCpbf-A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sqlalchemy+unsubscribe@googlegroups.com.

[Mike Bayer]

Ok, see if for the moment some imports at the top help:

import sqlalchemy

import sqlalchemy.orm

Make sure those run first. 

[Hugo Heyman]

Unfortunately it didn't help. I tried adding those imports both in the models file and run them first in shell.

[Mike Bayer]

Can you show me the stack trace you get with 1.1.15?   Thanks 

[Hugo Heyman]

Traceback (most recent call last):

  File "<console>", line 1, in <module>

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 157, in do

    return getattr(self.registry(), name)(*args, **kwargs)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 921, in commit

    self.transaction.commit()

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 461, in commit

    self._prepare_impl()

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl

    self.session.flush()

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2192, in flush

    self._flush(objects)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2312, in _flush

    transaction.rollback(_capture_exception=True)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__

    compat.reraise(exc_type, exc_value, exc_tb)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise

    raise value

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2276, in _flush

    flush_context.execute()

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 368, in execute

    postsort_actions = self._generate_actions()

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 327, in _generate_actions

    if action.execute(self):

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 447, in execute

    self.dependency_processor.presort_deletes(uow, delete_states)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 430, in presort_deletes

    self._passive_delete_flag)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 223, in get_attribute_history

    attributes.LOAD_AGAINST_COMMITTED)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line 909, in get_history

    current = self.get(state, dict_, passive=passive)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line 584, in get

    value = self.callable_(state, passive)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 557, in _load_for_state

    return self._emit_lazyload(session, state, ident_key, passive)

  File "<string>", line 1, in <lambda>

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 615, in _emit_lazyload

    strategy_options.Load.for_existing_path(

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 977, in __getattr__

    attr = getattr(self.module, key)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 767, in __get__

    obj.__dict__[self.__name__] = result = self.fget(obj)

  File "/Users/hugo/Dropbox/lahey/api/.venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 962, in module

    % (self._il_path, self._il_addtl))

ImportError: importlater.resolve_all() hasn't been called (this is sqlalchemy.orm strategy_options)

[Mike Bayer]

Are you able to reproduce this error in other environments or only this one particular virtual environment?   If it doesn't reproduce elsewheere can you try recreating your environment?   Otherwise if you can reproduce it in other environments can you show me the complete steps I'd need to do to recreate ?

[Hugo Heyman]

Hi again,

I've been trying to isolate the problem and I think I have found what's causing this for me. A while back I added this package https://github.com/operator/sqlalchemy_bulk_lazy_loader

(it simplified things when using graphene-sqlalchemy)

The error seem to happen even without using the custom 'bulk' lazy option on any relation, as long as BulkLazyLoader.register_loader()runs. 

I'll take this issue to bulk lazy loader, since the origin of this error is probably there.

Thanks for all your effort trying to help me!

/Hugo

[Mike Bayer]

On Wed, Nov 8, 2017 at 7:43 AM, Hugo Heyman <hugo....@gmail.com> wrote:
> Hi again,
>
> I've been trying to isolate the problem and I think I have found what's
> causing this for me. A while back I added this package
> https://github.com/operator/sqlalchemy_bulk_lazy_loader
> (it simplified things when using graphene-sqlalchemy)

OK, yeah this line:

https://github.com/operator/sqlalchemy_bulk_lazy_loader/blob/master/lib/sqlalchemy_bulk_lazy_loader.py#L141

should be removed and a simple "from sqlalchemy.orm import
strategy_options" should be placed at the top of the module.

otherwise, this line is adding a "dependency" to the "importlater"
thing (which is something that resolves import cycles in SQLAlchemy in
a thread-safe way) that isn't getting resolved.

>>>>>>>> send an email to sqlalchemy+...@googlegroups.com.

>>>>>>>> To post to this group, send email to sqlal...@googlegroups.com.
>>>>>>>> Visit this group at https://groups.google.com/group/sqlalchemy.
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>> --
>>>>>>> SQLAlchemy -
>>>>>>> The Python SQL Toolkit and Object Relational Mapper
>>>>>>>
>>>>>>> http://www.sqlalchemy.org/
>>>>>>>
>>>>>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>>>>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>>>>>> description.
>>>>>>> ---
>>>>>>> You received this message because you are subscribed to a topic in
>>>>>>> the Google Groups "sqlalchemy" group.
>>>>>>> To unsubscribe from this topic, visit
>>>>>>> https://groups.google.com/d/topic/sqlalchemy/kz47tCpbf-A/unsubscribe.
>>>>>>> To unsubscribe from this group and all its topics, send an email to

>>>>>>> sqlalchemy+...@googlegroups.com.

>>>>>>> To post to this group, send email to sqlal...@googlegroups.com.
>>>>>>> Visit this group at https://groups.google.com/group/sqlalchemy.
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> SQLAlchemy -
>>>>>> The Python SQL Toolkit and Object Relational Mapper
>>>>>>
>>>>>> http://www.sqlalchemy.org/
>>>>>>
>>>>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>>>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>>>>> description.
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "sqlalchemy" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send

>>>>>> an email to sqlalchemy+...@googlegroups.com.

>>>>>> To post to this group, send email to sqlal...@googlegroups.com.
>>>>>> Visit this group at https://groups.google.com/group/sqlalchemy.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>> --
>>>>> SQLAlchemy -
>>>>> The Python SQL Toolkit and Object Relational Mapper
>>>>>
>>>>> http://www.sqlalchemy.org/
>>>>>
>>>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>>>> description.
>>>>> ---
>>>>> You received this message because you are subscribed to a topic in the
>>>>> Google Groups "sqlalchemy" group.
>>>>> To unsubscribe from this topic, visit
>>>>> https://groups.google.com/d/topic/sqlalchemy/kz47tCpbf-A/unsubscribe.
>>>>> To unsubscribe from this group and all its topics, send an email to

>>>>> sqlalchemy+...@googlegroups.com.

>>>>> To post to this group, send email to sqlal...@googlegroups.com.
>>>>> Visit this group at https://groups.google.com/group/sqlalchemy.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>> --
>>>> SQLAlchemy -
>>>> The Python SQL Toolkit and Object Relational Mapper
>>>>
>>>> http://www.sqlalchemy.org/
>>>>
>>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>>> description.
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "sqlalchemy" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send

>>>> an email to sqlalchemy+...@googlegroups.com.

>>>> To post to this group, send email to sqlal...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/sqlalchemy.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> SQLAlchemy -
>>> The Python SQL Toolkit and Object Relational Mapper
>>>
>>> http://www.sqlalchemy.org/
>>>
>>> To post example code, please provide an MCVE: Minimal, Complete, and
>>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>>> description.
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "sqlalchemy" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/sqlalchemy/kz47tCpbf-A/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to

>>> sqlalchemy+...@googlegroups.com.

>>> To post to this group, send email to sqlal...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/sqlalchemy.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> description.
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to sqlalchemy+...@googlegroups.com.

>> To post to this group, send email to sqlal...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> description.
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "sqlalchemy" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/sqlalchemy/kz47tCpbf-A/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> sqlalchemy+...@googlegroups.com.

>> To post to this group, send email to sqlal...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sqlalchemy+...@googlegroups.com.