How to debug ProxySQL-induced query failures?

574 views
Skip to first unread message

Farley

unread,
Oct 10, 2017, 10:38:57 AM10/10/17
to proxysql
What's the best way to go about debugging failures/failed queries that are ProxySQL-specific that do not happen when connecting to MySQL directly?  I haven't found anything at first glance from searching through this group or reading through the docs.

Without implementing a cache or any rewriting just putting ProxySQL in-between our application and the database there were suddenly a significant number of failed queries in our application and (from what I could tell) I couldn't figure out why ProxySQL was failing on these queries.  These queries work perfectly when directly connected to our MySQL server.  So, I'm looking for either a non-daemonized extremely verbose mode that prints out all the internal logic and/or all queries that come through and how they are routed, or some kind of log of this nature.

I would like to utilize this software for a particular application so I can add some query caching because this particular application is a black-box and can not be changed easily except by pointing it to a new database server.  This particular application has some absolutely horrendous queries (query plans with upwards of 50 joins and hundreds of sub-queries, in queries that have more text than most entire codebases).  I assume the radical nature of these queries is why ProxySQL may be choking on them, but I don't know which ones or why.

I did try enabling the query cache for a while and one concerning aspect is when I send some queries through the system (example below) it doesn't show up at all in the stats_mysql_query_digest table, but the query succeeds.  I can only describe it as "magic".  I tried increasing the mysql-query_digests_max_query_length but no go.  I assume ProxySQL saw this ridiculous query and was just like "forget this" and just skipped the cache/digest logic and forwarded this to MySQL directly.  I tried adjusting the max length of the query cache to no avail, this query will absolutely not show up in the digest table.  This was one small sample, pulled from the mysql slow log, that happens to succeed via ProxySQL, but does not let me cache it.  The real problem though, is I do not know what queries are failing though because the application does not report query failures (very well) and the query fails in ProxySQL and not on the MySQL side, and this does not appear to be reporting anywhere. 

My gut says that something in these queries is maybe causing some error in your digest parsing logic, which is causing this query to fail somehow.  I would hope/expect that a software of this nature would catch that error and then simply pass the query onto mysql and not cause query failures (similar to what I mentioned above).  But, I don't know how to debug this situation to know for sure that's the case.  So... any advice for someone who wants to dive deep and debug this situation?  :P

Sample query (this particular one succeeds, but this is a short one compared to some of the whoppers): https://pastebin.com/PqibKi6m

Cheers,

Farley
Reply all
Reply to author
Forward
0 new messages