Debugging complex SQLAlchemy applications

[Pavel S]

Hi,

I maintain extremely complex application (>100k LOC) which uses SQLAlchemy. Often in the log I find errors like:

sqlalchemy/engine/default.py:450: Warning: Truncated incorrect DOUBLE value: 'Foo'

or

sqlalchemy/sql/sqltypes.py:201: SAWarning: Unicode type received non-unicode bind param value 'Foo'. (this warning may be suppressed after 10 occurrences)
  (util.ellipses_string(value),))

While those are just warnings and even if I turn warning into errors, traceback does NOT point to the place where the problem was caused, which is very frustrating.

What's the best practice finding and solving those problems?

[Mike Bayer]

the "non-unicode bind parameter" warning occurs exactly where a SQL
statement is invoked that contains a plain bytestring. The stack
trace should show you exactly the origin of the query and its
parameters. should be similar for the truncated DOUBLE value.

within SQLAlchemy, the only place that queries like this may be
emitted that doesn't link directly to a line of code in your
application is within the ORM flush process. If the stack trace is
not within "flush" then you should be able to see exactly where in
your application the problematic query is being created.

If the issue is within a flush, you can still see from the stack trace
what method within your application is in effect. The unicode
warning itself even shows the value that's being flushed (which is why
the "10 occurrences thing" happens, so the app isn't flooded) which
should also provide clues where to look.

Turning on SQL logging will show the exact SQL being emitted when
these warnings occur which should also provide strong clues where the
offending values are coming from.

>
> What's the best practice finding and solving those problems?
>

> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.