auth_user extra fields + row.to_dict()

[Mark Graves]

Hey everyone,

I saw a random bug in an app I'm working on, and I was wondering what the correct approach is.

I have auth.settings.extra_fields["auth_user"] = [LIST_OF_FIELDS]

later, I select that row and get it as a dict:

row = db(db.auth_user.id == user_id).select().first().as_dict()

then I get the extra fields and set them as normal attributes of the table

_f = row["extra"]

row.update(_f)

The problem is, some of these extra fields are booleans, but they are getting returned via JSON as "F" or "T"

I think its happening in the as_dict() method of the row object where it says:

d = dict(self)

where self is a row object

Any thoughts?

[Richard Vézina]

Hello Mark,

Not sure about the bool part, I think web2py treat bool like that to comply with all the different backend and the apdater does the rest...

On the other hand I am curious that you update row with simple update(), you suppose to use update_record()

Ref: http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#update_record

Richard

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony]

On Wednesday, April 6, 2016 at 2:06:11 AM UTC-4, Mark Graves wrote:

Hey everyone,

I saw a random bug in an app I'm working on, and I was wondering what the correct approach is.

I have auth.settings.extra_fields["auth_user"] = [LIST_OF_FIELDS]

later, I select that row and get it as a dict:

row = db(db.auth_user.id == user_id).select().first().as_dict()

then I get the extra fields and set them as normal attributes of the table

_f = row["extra"]

This is confusing. When you add extra fields to the auth_user table, they are just regular database table fields, so their keys/values are added to the Row object just like any other fields. There should be nothing in row['extra'] (unless you have a field named "extra"). Can you show some more code (e.g., the details of the extra fields) and what the Row object looks like before and after .as_dict (without doing any updating/mutating)?

Anthony

[Anthony]

On the other hand I am curious that you update row with simple update(), you suppose to use update_record()

They are both valid methods -- .update updates the Row object itself (like updating a dictionary), whereas .update_record updates the record in the database. You can actually issue several updates to the Row object via .update, and then later call .update_record() with no arguments, and the changes made to the Row object will be propagated to the database.

Anthony

[Anthony]

On Wednesday, April 6, 2016 at 11:20:44 AM UTC-4, Anthony wrote:

On the other hand I am curious that you update row with simple update(), you suppose to use update_record()

They are both valid methods -- .update updates the Row object itself (like updating a dictionary), whereas .update_record updates the record in the database. You can actually issue several updates to the Row object via .update, and then later call .update_record() with no arguments, and the changes made to the Row object will be propagated to the database.

And in this case, I don't think he wants to make any changes in the database -- he is attempting to create a usable JSON object.

Anthony

[Richard Vézina]

You maybe right Anthony abour update_record() it really depend though of what he is trying todo...

:)

Richard

--

[Anthony]

On Wednesday, April 6, 2016 at 11:30:34 AM UTC-4, Richard wrote:

You maybe right Anthony abour update_record() it really depend though of what he is trying todo...

Well at the point where row.update() is called, row is already a dictionary, so there is no .update_record() at that point. Also, the code is simply moving keys/values from the "extra" sub-dictionary into the top-level of the dictionary, so presumably the intention isn't to change anything in the database record.

Anthony

[Richard Vézina]

Yeah you right...

--

[Mark Graves]

Hey Anthony and Richard,

Yes you are both correct.

I am attempting to create a usable JSON object.

The confounding details from what I can tell are:

1.) the offending method is defined in a model file, which returns a dictionary, which is then json serialized via 

response.json(RETURNED_FROM_MODEL)

This shouldn't be a problem.

2.) I am nesting the results of the row.as_dict() into another dictionary, which contains additional information ala.

RETURNED_FROM_MODEL = {'row':row,'other_key':[]}

The reason I moved the "extra" key into the dictionary manually is because when I did not, the row in the above line was returning it to me as a key in the json response.

Its a nested serialization issue from what I can tell, because if I add another key into RETURNED_FROM_MODEL and set it explicitly as a boolean, it returns properly.  When I add a dictionary with a key that has a boolean value, it returns "T" or "F"

I'm coming up with some example code now.

You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/JlAzUE46s_M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.

[Mark Graves]

Offending code from a fresh web2py welcome app:

in db.py

before auth.define_tables()

from gluon.contrib.populate import populate

auth.settings.extra_fields['auth_user'] = [Field('test_field_1','boolean')]

in controller default.py

def test():

    if db(db.auth_user.id>0).isempty():

         populate(db.auth_user,10)

        

    table = db.auth_user

    fields = ['id','first_name']

    fields.append('test_field_1')

    

    query = table.id > 0

    

    row = db(query).select(*fields).first().as_dict()

    

    profile = {'auth_fields':row,

               "other_stuff":{"hello":False},

              "second_bool":False}

    

    return response.json(profile)

This returns the following JSON response:

{"second_bool": false, "auth_fields": {"_extra": {"test_field_1": "T", "first_name": "Corinne", "id": 1}}, "other_stuff": {"hello": false}}

[Anthony]

    table = db.auth_user

    fields = ['id','first_name']

    fields.append('test_field_1')

    

    query = table.id > 0

    

    row = db(query).select(*fields).first().as_dict()

The problem is that you aren't passing the correct arguments to .select(). You must pass Field objects, not simply the string names of the fields. So, it should be:

    db(query).select(*[table[f] for f in fields]).first().as_dict()

Now you will get a Row object with all the usual keys/values at the top level and no "_extra" sub-dictionary. In this case, the "T" and "F" values will be translated properly to True and False because now .select() knows the field types and does the proper parsing. When you simply pass field names to .select(), it doesn't have any information about the Field objects themselves -- it therefore doesn't know if a field is boolean and doesn't make the translation from "T"/"F" to True/False.

Anthony

[Mark Graves]

Ah!

Of course.

Thanks Anthony!

I knew it was something dumb i did =)

-Mark

--

Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---

[Massimo Di Pierro]

Good catch Anthony!