Removal of DictCursor from raw query.. why??

[Cal Leeming [Simplicity Media Ltd]]

Hi all,

I've had a look through the tickets but can't find anything in relation to why DictCursor is not the default for all cursor() created manually.

I can see the snippet needed was removed here:

https://code.djangoproject.com/changeset/5970

Examples of how to use the dictfetchall() are here:

http://www.revolunet.com/snippets/snippet/dictcursor-for-raw-sql-using-django

http://www.mail-archive.com/django...@googlegroups.com/msg25368.html

In this case, my code is:

cursor = connection.cursor(row_factory = MySQLdb.cursors.DictCursor)

            

            _sql = """

                SELECT

                    SUM(is_spam) as is_spam,

                    SUM(is_image_blocked) as is_image_blocked,

                    SUM(IF(dl_job_state = 2, 1, 0)) as dl_job_success,

                    COUNT(*) as total_rows

                

                FROM

                    fourchan_post

                WHERE

                    id >= 22350944 AND

                    id < 22360944

            """

            cursor.execute(_sql)

            print cursor.fetchall()

((Decimal('0'), Decimal('52'), Decimal('4159'), 9998L),)

However, it should return the field names too..

Could someone indicate if there is a reason for this, or if I should be filing a bug report?

Cal

[Luke Plant]

On 16/06/11 13:34, Cal Leeming [Simplicity Media Ltd] wrote:
> Hi all,
...

> Could someone indicate if there is a reason for this, or if I should be
> filing a bug report?

As the changeset comment indicates, the snippet you wanted was removed
because it was an internal function that was no longer being used in
Django. Our backwards compatibility promise does not extend to
undocumented internals, so there isn't a bug here.

Regards,

Luke

--
The probability of someone watching you is proportional to the
stupidity of your action.

Luke Plant || http://lukeplant.me.uk/

[Cal Leeming [Simplicity Media Ltd]]

Okay, er.

In reference to the original problem (cursor's not default to DictCursor), thus no field names are returned, is there a specific reason for this? If not, I'll probably raise a ticket to have it considered for change.

Cal

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Cal Leeming [Simplicity Media Ltd]]

Okay, let me put it another way.

Are there any plans to give developers an easy way to retrieve values from a cursor.fetchall(), in a DictCursor style?

Default: ((Decimal('0'), Decimal('52'), Decimal('4159'), 9998L),)

What I'm looking for: 

[{

 'f1' : Decimal('0'),

 'f2' : Decimal('52'),

 'f3' : Decimal('4159'),

 'f4' : 9998L

}]

Maybe something like cursor.fetchall(field_names=True), or cursor.dictfetchall() - which is what the removed function did.

Cal

On Thu, Jun 16, 2011 at 2:54 PM, Luke Plant <L.Pla...@cantab.net> wrote:

On 16/06/11 14:10, Cal Leeming [Simplicity Media Ltd] wrote:
> Okay, er.
>
> In reference to the original problem (cursor's not default to
> DictCursor), thus no field names are returned, is there a specific
> reason for this? If not, I'll probably raise a ticket to have it
> considered for change.

I'm not sure exactly what you are asking, because this is about default
behaviour. The choice of a default is usually made according to what is
thought to be the most useful, or according to the way it happens to
have been done in the past.

I also don't know what exactly you are suggesting. Our backwards
compatibility policy means that we aren't going to change the default,
unless other people's code is going to work transparently (which
wouldn't be the case here), so it doesn't really matter what the
original reason was, if there was one. If you are suggesting that we add
some functionality to make use of DictCursor more useful, then certainly
the suggestion is valid.

[Alex Gaynor]

The field names are available in cursor.description, precisely as in a normal PEP249 compliant DB-API adapter.

Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

[Luke Plant]

On 16/06/11 14:10, Cal Leeming [Simplicity Media Ltd] wrote:
> Okay, er.
>
> In reference to the original problem (cursor's not default to
> DictCursor), thus no field names are returned, is there a specific
> reason for this? If not, I'll probably raise a ticket to have it
> considered for change.

I'm not sure exactly what you are asking, because this is about default

behaviour. The choice of a default is usually made according to what is
thought to be the most useful, or according to the way it happens to
have been done in the past.

I also don't know what exactly you are suggesting. Our backwards
compatibility policy means that we aren't going to change the default,
unless other people's code is going to work transparently (which
wouldn't be the case here), so it doesn't really matter what the
original reason was, if there was one. If you are suggesting that we add
some functionality to make use of DictCursor more useful, then certainly
the suggestion is valid.

Regards,

[Andy Dustman]

You can do something like:

for row in cursor:
dictrow = dict( (d[0], c) for d, c in zip(cursor.description, row) )

(izip may be better than zip. Your call.)

Or for the whole result set:

result = [ dict( (d[0],c) for d, c in zip(cursor.description, row) )
for row in cursor ]

--
Question the answers

[Cal Leeming [Simplicity Media Ltd]]

@Andy / @ Alex:

Yup, I know how to get this, but the point is, it took me 30 minutes of searching to find this information out.

What I'm asking, is for consideration of an *EASY* way to get this format, via something like cursor.fetchall(field_names=True), or cursor.dictfetchall(), and for that feature to be well documented.

[Cal Leeming [Simplicity Media Ltd]]

In fact let me extend off this a little further.

If I was to provide a code and documentation patch, which allows for an easy way to return back values with their field names (via a simple field_names=True), would you guys be willing to consider it for the core?

Cal

[Alex Gaynor]

No.  The cursor Django exposes implements PEP 249, which does not suggest any such interface (nor does any other database adapter implement such an interface).

[Cal Leeming [Simplicity Media Ltd]]

Okay, fair enough.

At the very least, would you accept a documentation patch which would guide other users who come up against this same problem? Maybe it's own little space near the raw() stuff, which shows the example code for dictfetchall() but with a disclaimer saying YMMV??

Cal

[bur...@gmail.com]

Hi Cal,

Why not just put your helper to django snippets?

On Fri, Jun 17, 2011 at 12:25 AM, Cal Leeming [Simplicity Media Ltd]

--
Best regards, Yuri V. Baburov, Skype: yuri.baburov, MSN: bu...@live.com

[Cal Leeming [Simplicity Media Ltd]]

Because I feel this is just something that should work (or be available) out of the box. There are plenty of other places where Django docs has included code snippets to give the user a heads up, and I think this is the perfect case for one.

If anyone has any objections to this, please let me know, if not ill put in a ticket for consideration.

[akaariai]

On Jun 17, 2:54 pm, "Cal Leeming [Simplicity Media Ltd]"

<cal.leem...@simplicitymedialtd.co.uk> wrote:
> Because I feel this is just something that should work (or be available) out
> of the box. There are plenty of other places where Django docs has included
> code snippets to give the user a heads up, and I think this is the perfect
> case for one.
>
> If anyone has any objections to this, please let me know, if not ill put in
> a ticket for consideration.

I just wanted to say I support having something documented about this.
Without documentation new users will most likely use index based
cursors. I know I used to do that.

The problem with no documentation is not so much that it would be hard
to find a snippet about dict cursor implementation. It is more that
new users don't know that using index based cursors might not be the
best of ideas.

- Anssi

[Ian Kelly]

On Fri, Jun 17, 2011 at 5:54 AM, Cal Leeming [Simplicity Media Ltd]
<cal.l...@simplicitymedialtd.co.uk> wrote:
> Because I feel this is just something that should work (or be available) out
> of the box. There are plenty of other places where Django docs has included
> code snippets to give the user a heads up, and I think this is the perfect
> case for one.
>
> If anyone has any objections to this, please let me know, if not ill put in
> a ticket for consideration.

The thing is, this is a DB API snippet, not a Django snippet
specifically. If Django were a DB API toolbox, then it might make
sense to include it in some form or other. But it's not, so in the
interest of keeping things relatively tidy I'm a -0 on this.

Cheers,
Ian

[akaariai]

On Jun 17, 8:02 pm, Ian Kelly <ian.g.ke...@gmail.com> wrote:
> The thing is, this is a DB API snippet, not a Django snippet
> specifically.  If Django were a DB API toolbox, then it might make
> sense to include it in some form or other.  But it's not, so in the
> interest of keeping things relatively tidy I'm a -0 on this.

It is often said here that Django ORM is designed to do 80% of the
stuff, the rest can be done using raw SQL. So, giving pointers to
users how to perform the raw SQL as painlessly as possible is
something Django documentation should do.

- Anssi

[Cal Leeming [Simplicity Media Ltd]]

Hi all,

I've created a ticket for this to get a final decision and feedback:

https://code.djangoproject.com/ticket/16293

Cal