django-firebird test expressions

[Maximiliano Robaina]

Hi,

Testing expressions test app, the query generated into BasicExpressionsTests.test_annotate_values_filter method:

companies = Company.objects.annotate(
foo=RawSQL('%s', ['value']),
).filter(foo='value').order_by('name')

Generate:

'SELECT  "EXPRESSIONS_COMPANY"."ID", "EXPRESSIONS_COMPANY"."NAME", "EXPRESSIONS_COMPANY"."NUM_EMPLOYEES", "EXPRESSIONS_COMPANY"."NUM_CHAIRS", "EXPRESSIONS_COMPANY"."CEO_ID", "EXPRESSIONS_COMPANY"."POINT_OF_CONTACT_ID", ? AS "FOO" FROM "EXPRESSIONS_COMPANY" WHERE ? = ? ORDER BY "EXPRESSIONS_COMPANY"."NAME" ASC'

This sql command has 3 params (?), two of which are out of where clause.

? AS "FOO" 

WHERE ? = ?

So, the underlying database driver execute this, doing something like this

 cursor.execute(sql, ['value', 'value', 'value'])

Is correct that the 3 params are replaced into the entire sql and not just on where clause ?

It depend of the  implementation on each database driver?

Best regards

Maxi

[Josh Smeaton]

It looks correct to me. RawSQL is defining a parameter to be included in the query. The RawSQL expression itself is used in the select and on the left hand side of the WHERE clause, so the parameter needs to be included there. The right hand side of the WHERE clause is a regular string (which just happens to also be named "value") which Django will always parameterise and has nothing to do with the RawSQL expression itself.

Is there some reason you think there's an issue with the SQL being constructed?

[Maximiliano Robaina]

Hi Josh,

Thanks for response.

El domingo, 15 de abril de 2018, 20:35:54 (UTC-3), Josh Smeaton escribió:

It looks correct to me. RawSQL is defining a parameter to be included in the query. The RawSQL expression itself is used in the select and on the left hand side of the WHERE clause, so the parameter needs to be included there. The right hand side of the WHERE clause is a regular string (which just happens to also be named "value") which Django will always parameterise and has nothing to do with the RawSQL expression itself.

Is there some reason you think there's an issue with the SQL being constructed?

No, I don't think that the sql statement is wrong. I just traying to figured out how the sql is generated by Django and why it fails in Firebird.

I found the answere, of course, there is a Firebird limitation on this type of statement. Running that query raise a "Data type unknown" error.

https://stackoverflow.com/questions/37348807/data-type-unknown-in-case-expression-with-only-parameters-as-values

I will need to found other strategy to resolve this.

Regards.

[Shai Berger]

Hi Maximiliano,

On Sat, 14 Apr 2018 19:25:48 -0700 (PDT)
Maximiliano Robaina <maxir...@gmail.com> wrote:

> Hi,
>
> Testing expressions test app, the query generated into
> BasicExpressionsTests.test_annotate_values_filter method:
>
> companies = Company.objects.annotate(
> foo=RawSQL('%s', ['value']),
> ).filter(foo='value').order_by('name')
>
> Generate:
>
> 'SELECT "EXPRESSIONS_COMPANY"."ID", "EXPRESSIONS_COMPANY"."NAME",
> "EXPRESSIONS_COMPANY"."NUM_EMPLOYEES",
> "EXPRESSIONS_COMPANY"."NUM_CHAIRS", "EXPRESSIONS_COMPANY"."CEO_ID",
> "EXPRESSIONS_COMPANY"."POINT_OF_CONTACT_ID", ? AS "FOO" FROM
> "EXPRESSIONS_COMPANY" WHERE ? = ? ORDER BY
> "EXPRESSIONS_COMPANY"."NAME" ASC'
>
> This sql command has 3 params (?), two of which are out of where
> clause.
>
> ? AS "FOO"
>
> WHERE ? = ?
>

A few years ago something similar was causing issues with the Oracle
backend. To resolve it, Mariusz added a clevar hack, relying on named
parameters -- he made sure that if the same parameter value is used
more than once, then the statement re-uses the parameter name, passing
the value only once. So, in your case, the equivalent would be
something like

...
"EXPRESSIONS_COMPANY"."POINT_OF_CONTACT_ID", %arg1 AS "FOO" FROM
"EXPRESSIONS_COMPANY" WHERE %arg1 = %arg1 ORDER BY
"EXPRESSIONS_COMPANY"."NAME" ASC'

with

cursor.execute(sql, [], arg1='value')

This was relatively easy to do in the Oracle backend, which has always
used named parameters under the hood (though the idea to do it was, in
my opinion, surprising and brilliant). If Firebird supports them, you
may be able to borrow this solution.

Shai

[Maximiliano Robaina]

Hi Shai,

Thank for the tip

Ok, but it is a cx_Oracle implementation.

Unfortunately fdb (the firebird python driver) doesn't support this and, in any case, is a FirebirdSQL limitation. [1]

Regards.

[1]  https://stackoverflow.com/questions/37348807/data-type-unknown-in-case-expression-with-only-parameters-as-values

 

[Josh Smeaton]

Have you looked into using Cast for this particular query to see if it passes?

https://docs.djangoproject.com/en/2.0/ref/models/database-functions/#cast

Cast(RawSQL('%s', ['value']), CharField(max_length=10) ?

If that worked, then perhaps you could find a clever way to wrap problem expressions, or at least document the workaround for your users.

[Maximiliano Robaina]

El martes, 17 de abril de 2018, 9:11:23 (UTC-3), Josh Smeaton escribió:

Have you looked into using Cast for this particular query to see if it passes?

https://docs.djangoproject.com/en/2.0/ref/models/database-functions/#cast

Cast(RawSQL('%s', ['value']), CharField(max_length=10) ?

If that worked, then perhaps you could find a clever way to wrap problem expressions, or at least document the workaround for your users.

Ok, it worked rigth. It is a good started point and solve the problem. 

I will need to dig a little more to found out a better solution.

Thanks.