Mock DBAPI connection for testing custom creator

[Sam Lee]

Hi,

I am passing a custom creator to sqlalchemy.create_engine().
And, I want to simulate the following scenario in automated test without using actual database connection:

1. Request a connection from pool: connects to DB.
2. DB goes down.
3. Request a connection from pool: connection is invalid.
4. Request a connection from pool: connects to DB again.

I'm trying the following testcase:

        def mycreator():
                print('called')
                # return psycopg2.connect(....)
                return unittest.mock.MagicMock()

        engine = create_engine('postgresql://', creator=mycreator, pool_size=1)
        c = engine.connect()  # calls mycreator
        c.close()             # connection returned to pool
        c = engine.connect()  # does not call mycreator (reuses connection in pool)
        c.invalidate()       
        c.close()
        c = engine.connect()  # calls mycreator (because connection is invalidated)


But, it throws exception because MagicMock isn't cursor or connection.
What kind of mock or fake connection object should mycreator return in the test
so that above snippet will behave as I expect it to, calling mycreator twice?
(Using actual connection, psycopg2.connect(), works as expected).

Thanks.
Sam

[Mike Bayer]

I'm assuming you are getting this error from psycopg2 itself. To
use create_engine(), you'd need to mock whatever is being called that
is not compatible, which here I am assuming are things like
psycopg2.extensions.register_type,
psycopg2.extras.register_default_json, and others. Patch these with
mocks so that they have no effect.

However, if your test is just looking for the behavior of the pool,
then test against a QueuePool directly.

The tests in test/engine/test_reconnect.py make extensive use of mocks
to simulate disconnects though most of the create_engine() integration
is done using real database connections with a patch to close out the
DBAPI connection to simulate a disconnect.

> What kind of mock or fake connection object should mycreator return in the
> test
> so that above snippet will behave as I expect it to, calling mycreator
> twice?
> (Using actual connection, psycopg2.connect(), works as expected).
>
> Thanks.
> Sam
>

> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

[Sam Lee]

Thank you.

I didn't even post stacktrace and those were the exact errors!!

I'll take a look at  test/engine/test_reconnect.py and probably test against Pool directly.

What I'm testing is a bit silly cause I'm testing sqlalchemy behavior :P

Thanks.

Sam

> email to sqlalchemy+unsubscribe@googlegroups.com.

> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full description.
---

You received this message because you are subscribed to a topic in the Google Groups "sqlalchemy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sqlalchemy/l7wMxoaNsXE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sqlalchemy+unsubscribe@googlegroups.com.

[Mike Bayer]

On Wed, Oct 18, 2017 at 9:50 PM, Sam Lee <sky...@gmail.com> wrote:
> Thank you.
> I didn't even post stacktrace and those were the exact errors!!

because I've gotten them before and I knew from your description what
you were seeing, they are very specific to what psycopg2 is doing.

> I'll take a look at test/engine/test_reconnect.py and probably test against
> Pool directly.
> What I'm testing is a bit silly cause I'm testing sqlalchemy behavior :P

enjoy

>> > email to sqlalchemy+...@googlegroups.com.

>> > To post to this group, send email to sqlal...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sqlalchemy.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> SQLAlchemy -
>> The Python SQL Toolkit and Object Relational Mapper
>>
>> http://www.sqlalchemy.org/
>>
>> To post example code, please provide an MCVE: Minimal, Complete, and
>> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> description.
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "sqlalchemy" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/sqlalchemy/l7wMxoaNsXE/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> sqlalchemy+...@googlegroups.com.

>> To post to this group, send email to sqlal...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sqlalchemy.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---

> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sqlalchemy+...@googlegroups.com.