Best Practice for model discovery?

[John Anderson]

In my pyramid apps I have a create script that generates my database for production and a different script that generates my database for my tests.

But if I don't import the module the models are in then they aren't discovered even though they all share the same Base which is the class I get the metadata from to call create_all, like:

 Base.metadata.create_all(engine)

If I import them then they are picked up but I don't want to have to remember to import every new module I create.   Is there a good way to do model discovery for this?

[Michael Bayer]

usually the pattern is you just import one module, like "from myapp import model".   "model/__init__.py" then has imports for everything within, and the pattern repeats as you descend through the directory tree, that is, every top level __init__.py imports the important bits from within that package.

the only other way would be to do a find of .py files within a model directory and then import them with importlib or similar, which is more complicated, non-deterministic as far as ordering and more prone to import resolution issues.    Adding per-package imports as you go along just creates this same traversal as part of the code.

if the Python interpreter is never told of the existence of some .py file, it doesn't exist.  it doesn't matter that "Base" is used in that file, the classes within don't exist until their owning module is imported.   

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/r95QN9vJ_IgJ.
To post to this group, send email to sqlal...@googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.

[John Anderson]

Do you have an example of an app doing this?

For instance, if your models/__init__.py declares your DBSession and your Base:

# __init__.py

DBSession = scoped_session(session_maker())

class BaseModel(object):

    pass

Base = declarative_base(cls=BaseModel)

you would just import everything after those? i.e

from app.module1.models import *

from app.module2.models import *

then then inside app/module1/__init__.py you would do the imports for all of its models?

# app/module1/__init__.py

from app.module1.modules import *

# app/module1/models.py

from app.models import Base

class Model1(Base):

   pass

or would you structure it differently?   Wouldn't you have issues with __init__.py importing the modules but the modules needing Base from __init__.py?

To unsubscribe from this group, send email to sqlalchemy+unsubscribe@googlegroups.com.

[Michael Bayer]

usually you put the DBSession and such somewhere other than models/__init__.py, like models/meta.py.   That way sub-modules and packages of "models" can import the dependencies from meta.py.

structure is like:

models/__init__.py:

from .model1 import Foo, Bar

from .model2 import Bat, Hoho

models/meta.py:

Base = declarative_base()

models/model1/__init__.py:

from .foo import Foo

from .bar import Bar

models/model1/foo.py:

from ..meta import Base

class Foo(Base):

# ...

models/model1/bar.py:

from ..meta import Base

class Bar(Base):

# ...

models/model2/__init__.py:

from .bat import Bat

from .hoho import Hoho

models/model2/bat.py:

from ..meta import Base

class Bat(Base):

# ...

models/model2/hoho.py:

from ..meta import Base

class Hoho(Base):

# ...

To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/7VlbuxrkgBkJ.

To post to this group, send email to sqlal...@googlegroups.com.

To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.

[Sergey V.]

With Pyramid in particular you can use its code scanning feature to find and load all modules with your modes without having to explicitly import them. Code scanning is used in Pyramid to find @view_config decorators, but you can use it for other purposes too.

http://docs.pylonsproject.org/projects/pyramid/en/1.0-branch/narr/configuration.html#configuration-decorations-and-code-scanning

[Chris Withers]

I tend to use a venusian scan (a pyramid config scan does the same
thing) which spiders your package and makes sure everything is imported

Check out my mortar_rdb package for some more fun and games in this
area, although I need to give it a re-write to use alembic instead of
the terrible sqlalchemy-migrate...

cheers,

Chris

>> <mailto:sqlal...@googlegroups.com>.

>> To unsubscribe from this group, send email to
>> sqlalchemy+...@googlegroups.com

>> <mailto:sqlalchemy+...@googlegroups.com>.

>> For more options, visit this group at
>> http://groups.google.com/group/sqlalchemy?hl=en.
>
>

> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________

>
> --
> You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.

> To post to this group, send email to sqlal...@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk