Testing with a fake read replica

[Colton Allen]

Hi,

Before we talk about the read-replica, let's talk about the test suite as it is.  I have a sessionmaker in my test suite configured to use an external transaction.  Basically identical to this: https://docs.sqlalchemy.org/en/13/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites .  This is great for performance and I have no complaints about it.

To integrate a "read-replica" in my test suite I'm very clever.  I just say "master = master_connection" and "slave = master_connection".  The test suite has no idea there is a read-replica at all.

Unfortunately, the coverage I get is not comprehensive.  Because a "slave" connection in this context could write to the database and cause errors in production.  So I need some way to differentiate the connections without breaking the external transaction.  Any thoughts on how I could do this?  Any argument I can pass to the sessionmaker to make it read-only while still making the data in the transaction available to each?

Thanks!

[Mike Bayer]

IIUC you want an engine to fail a test if a write operation attempts to proceed upon it.   lots of ways to do this:

1. set logging_name on create_engine(), then use a session before_flush() handler to check the engine's logging_name  (e.g. session.connection().engine.logging_name == 'readonly'), or use more connection local events like before_insert, before_update, before_delete  https://docs.sqlalchemy.org/en/13/orm/events.html

2. use a checkout event on the read-only engine (https://docs.sqlalchemy.org/en/13/core/events.html#sqlalchemy.events.PoolEvents.checkout) put a token in the connection's info dictionary, then use the above techniques

3. intercept INSERT/UPDATE/DELETE directly using cursor_execute() events: https://docs.sqlalchemy.org/en/13/core/events.html#sqlalchemy.events.ConnectionEvents.before_cursor_execute

basically use events to intercept operations and check the connection / engine in use.

Thanks!

--

SQLAlchemy -

The Python SQL Toolkit and Object Relational Mapper

 

http://www.sqlalchemy.org/

 

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.

---

You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/a83a28d9-0982-4686-928e-a451f60ee791%40googlegroups.com.

[Colton Allen]

Thank you.  My use case didn't permit me to use your examples but, based on your advice, I ended up using "after_flush_postexec" event.  Any consequences from doing this?  Seems to work fine for now.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlal...@googlegroups.com.

[Mike Bayer]

On Wed, Mar 11, 2020, at 2:17 PM, Colton Allen wrote:

Thank you.  My use case didn't permit me to use your examples but, based on your advice, I ended up using "after_flush_postexec" event.  Any consequences from doing this?  Seems to work fine for now.

this is just the test suite right?  if it works, it works

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/c8399b15-3a40-41d9-b78a-4166f446c097%40googlegroups.com.