How to determine whether Alembic is active

[gbr]

In some Alembic upgrade scripts, I need to use application sessions to make use of some code that requires these sessions. To avoid deadlocks from blocking transactions I use the following lines to transparently call `get_db_session()` regardless of whether Alembic or the application is executed:


def _alembic_get_db_session():
    from alembic import op
    alembic_connection = op.get_bind()
    return Session(bind=alembic_connection)


def _normal_get_db_session():
    return scoped_session(session_maker)


def get_db_session():
    if is_alembic_active:
        return _alembic_get_db_session()
    else:
        return _normal_get_db_session()


How can I fill out the `is_alembic_active` part? I know that the application is running and op.get_bind() is called, the following exception is raised (I tried to check with op.get_bind() is None which obviously doesn't work).

NameError: Can't invoke function 'get_bind', as the proxy object has not yet been established for the Alembic 'Operations' class. Try placing this code inside a callable.

 
Any idea?

[Michael Bayer]

if possible, I’d modify env.py to make use of your application’s normal engine/session mechanics.  There would be no “alembic_get_session()” method.   Whatever mechanism your app uses to get at the Engine/Session, pull it in within env.py and use the exact same environment for migrations.

  

--
You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alem...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gbr]

True. I should have modified

def run_migrations_online():
    """Run migrations in 'online' mode.

    In this scenario we need to create an Engine
    and associate a connection with the context.

    """
    engine = engine_from_config(
                config.get_section(config.config_ini_section),
                prefix='sqlalchemy.',
                poolclass=pool.NullPool)

    connection = engine.connect()

to use the application's session/connection management instead of creating a new connection.

Thanks.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsub...@googlegroups.com.

[gbr]

I went ahead and made the modification as suggested. Alembic and my application are now using the application's database connection and sessions.

However, the system locks up during a migration when I query for some application data which I need for data migration. I use Flask-SqlAlchemy and follow the way they described it to be used (and postgres):

When I call

> session.query(SomeClass)

the migration process blocks in engine/default.py in cursor.execute (this is as far as I can trace it).

>    def do_execute(self, cursor, statement, parameters, context=None):
>        cursor.execute(statement, parameters)

This is how I create a connection in env.py (Alembic) now:

> engine = get_db().engine
> connection = engine.connect()

`get_db()` is part of the application. It returns an instance of `db = SQLAlchemy(flask_app)`

I'm not sure how to debug this any further (I could try to throw out Flask-SqlAlchemy and see how it goes, but would like to find an easier way first). Any suggestions are appreciated.

[Michael Bayer]

On Jun 3, 2014, at 10:37 PM, gbr <doub...@directbox.com> wrote:

> I went ahead and made the modification as suggested. Alembic and my application are now using the application's database connection and sessions.
>
> However, the system locks up during a migration when I query for some application data which I need for data migration. I use Flask-SqlAlchemy and follow the way they described it to be used (and postgres):
>
> When I call
>
> > session.query(SomeClass)
>
> the migration process blocks in engine/default.py in cursor.execute (this is as far as I can trace it).
>
> > def do_execute(self, cursor, statement, parameters, context=None):
> > cursor.execute(statement, parameters)
>
> This is how I create a connection in env.py (Alembic) now:
>
> > engine = get_db().engine
> > connection = engine.connect()
>
> `get_db()` is part of the application. It returns an instance of `db = SQLAlchemy(flask_app)`
>
> I'm not sure how to debug this any further (I could try to throw out Flask-SqlAlchemy and see how it goes, but would like to find an easier way first). Any suggestions are appreciated.

well again, part of your system is saying:

conn = engine.connect()

and another part is saying:

sess = Session(engine)
sess.<do stuff> , which means, engine.connect(), which means, two connections concurrently.

A true integration would mean that your env.py would use that same Session for connectivity and transactional context:


env.py

from myapp import how_i_get_my_session
session = how_i_get_my_session()
connection = session.connection

[gbr]

Ok. I tried your suggestion:

in env.py:

from myapp import how_i_get_my_session
session = how_i_get_my_session()
connection = session.connection

results in:

   context.configure(connection=connection, target_metadata=target_metadata)
  File "<string>", line 7, in configure
  File "/env/lib/python2.7/site-packages/alembic/environment.py", line 664, in configure
    opts=opts
  File "//env/lib/python2.7/site-packages/alembic/migration.py", line 141, in configure
    dialect = connection.dialect
AttributeError: 'function' object has no attribute 'dialect'


I tried this:

    s = db.get_db_session()
    connection = s.bind

    context.configure(connection=connection, target_metadata=target_metadata)

    try:
        with context.begin_transaction():
            context.run_migrations()
    finally:
        s.remove()
        # connection.close()

This seems to work fine. Any problems you foresee with my solution or any idea why your's throws an error (perhaps I missed something when converting your suggestion into runnable code)?

[Michael Bayer]

On Jun 4, 2014, at 1:18 AM, gbr <doub...@directbox.com> wrote:

Ok. I tried your suggestion:

in env.py:

from myapp import how_i_get_my_session 
session = how_i_get_my_session() 
connection = session.connection 

sorry, session.connection().   “connection” is a method.

I tried this:

    s = db.get_db_session()
    connection = s.bind

    context.configure(connection=connection, target_metadata=target_metadata)

    try:
        with context.begin_transaction():
            context.run_migrations()
    finally:
        s.remove()
        # connection.close()

This seems to work fine. Any problems you foresee with my solution or any idea why your's throws an error (perhaps I missed something when converting your suggestion into runnable code)?

typically “bind” on Session is just the engine again, so it’s a little odd you can pass that right into context.configure() and nothing breaks.    I’m not really sure it depends on what Session.bind is here.

On Wednesday, June 4, 2014 12:55:57 PM UTC+10, Michael Bayer wrote:

On Jun 3, 2014, at 10:37 PM, gbr <doub...@directbox.com> wrote: 

> I went ahead and made the modification as suggested. Alembic and my application are now using the application's database connection and sessions. 
> 
> However, the system locks up during a migration when I query for some application data which I need for data migration. I use Flask-SqlAlchemy and follow the way they described it to be used (and postgres): 
> 
> When I call 
> 
> > session.query(SomeClass) 
> 
> the migration process blocks in engine/default.py in cursor.execute (this is as far as I can trace it). 
> 
> >    def do_execute(self, cursor, statement, parameters, context=None): 
> >        cursor.execute(statement, parameters) 
> 
> This is how I create a connection in env.py (Alembic) now: 
> 
> > engine = get_db().engine 
> > connection = engine.connect() 
> 
> `get_db()` is part of the application. It returns an instance of `db = SQLAlchemy(flask_app)` 
> 
> I'm not sure how to debug this any further (I could try to throw out Flask-SqlAlchemy and see how it goes, but would like to find an easier way first). Any suggestions are appreciated. 

well again, part of your system is saying: 

conn = engine.connect() 

and another part is saying: 

sess = Session(engine) 
sess.<do stuff> , which means, engine.connect(), which means, two connections concurrently. 

A true integration would mean that your env.py would use that same Session for connectivity and transactional context: 


env.py 

from myapp import how_i_get_my_session 
session = how_i_get_my_session() 
connection = session.connection 

-- 
You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alem...@googlegroups.com.