automap multithreading

[mdob]

Hi SQLAlchemy Group, 

I've been experimenting with automap (in SQLAlchemy 1.0.20) a bit and I wonder how safe is to automap multiple databases in threads because I run into random errors of classes not being mapped:

One or more mappers failed to initialize - can't proceed with initialization of other mappers.  Original exception was: Class 'sqlalchemy.ext.automap.Manufacturer' is not mapped

I was a bit surprised the class was added to automap module which could cause conflicts when automapping databases simultaneously in threads. Is this intentional that the automapped class is added to automap module or I messed something up? 

I was digging in sqlalchem/ext/automap.py and I found that a class method AutomapBase.prepare uses class sqlalchemy.ext.declarative.base._DeferredMapperConfig to store configs. This class would be shared between threads. Is it possible one thread flashes configs and other breaks automap process on other thread?

I can't reproduce the problem when secluding automap from my application so it's hard to give code example. I'm sorry that question is not very precise but maybe you could give me any hint where to look?

Kind regards, 

Michal

[Mike Bayer]

On Wed, Nov 22, 2017 at 10:56 AM, mdob <mike.do...@gmail.com> wrote:
> Hi SQLAlchemy Group,
>
> I've been experimenting with automap (in SQLAlchemy 1.0.20) a bit and I
> wonder how safe is to automap multiple databases in threads because I run
> into random errors of classes not being mapped:
>
> One or more mappers failed to initialize - can't proceed with initialization
> of other mappers. Original exception was: Class
> 'sqlalchemy.ext.automap.Manufacturer' is not mapped
>
> I was a bit surprised the class was added to automap module which could
> cause conflicts when automapping databases simultaneously in threads. Is
> this intentional that the automapped class is added to automap module or I
> messed something up?

are you using individual bases and metadata collections? this should
be OK in theory, however mappers are subject to a global registry as
well as the global configure_mappers() step - this routine is mutexed
for thread-safe operation.

>
> I was digging in sqlalchem/ext/automap.py and I found that a class method
> AutomapBase.prepare uses class
> sqlalchemy.ext.declarative.base._DeferredMapperConfig to store configs. This
> class would be shared between threads. Is it possible one thread flashes
> configs and other breaks automap process on other thread?

shouldn't.

>
> I can't reproduce the problem when secluding automap from my application so
> it's hard to give code example. I'm sorry that question is not very precise
> but maybe you could give me any hint where to look?

stack trace would help


>
> Kind regards,
> Michal
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.