get all columns as a list

[Larry Martell]

This is probably very simple, but I just can't figure out how to do it.

I want to get all the columns in some rows as a list. I know I could
use values_list and flat=True and list all the columns, but is that
the only way?

I want to do something like this:

rows = FOO.objects.filter(bar='baz')

and get a list of lists instead a list of FOO objects.

[Erik Cederstrand]

"MyModel.objects.filter().values_list()" returns a list of tuples with all column values in MyModel._meta.fields order. Does that not suffice?

Erik

[Gergely Polonkai]

Hello,

Django can’t do this out of the box, but see this post[1] for a possible solution with dicts.

You might also want to look at serialization[2]; it might help you a bit, but again, it’s primarily for dicts, not lists.

On the other hand, I started wondering why you need this, do you care to share the use case?

Cheers,
Gergely

[1] http://stackoverflow.com/a/29088221/1305139
[2] https://docs.djangoproject.com/en/1.9/topics/serialization/

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CACwCsY7EZtbmLqLgggZnnBspKbvOucPDki1i28zD3jtjGp%3DWFQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Larry Martell]

Yes, that will work perfectly for me. I didn't know that about an
empty values_list returning all the columns. Thanks so much!

[Larry Martell]

On Fri, May 20, 2016 at 2:26 AM, Gergely Polonkai <ger...@polonkai.eu> wrote:
> Hello,
>
> Django can’t do this out of the box, but see this post[1] for a possible
> solution with dicts.

Well, it seems it can. As pointed out by Erik in another post, an
empty values_list() returns all the columns, which is what I want.

> On the other hand, I started wondering why you need this, do you care to
> share the use case?

When new data comes in I want to compare it to the most recently added
row and only add a new row if it differs.

[Derek]

"When new data comes in I want to ... only add a new row if it differs."

Pardon my curiosity, but isn't that the role of the set of unique keys for each record - to determine if it is "different"?

[Larry Martell]

It's only 2 consecutive rows identical rows I need to exclude.

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-users/e4a06bdf-c9df-46a7-9848-86238ddbf7cd%40googlegroups.com.

[Derek]

OK - I don't really understand that; there should not be any 2 identical records in a database, but anyway, that was not the issue in this thread.

[Larry Martell]

They're not identical - there's a timestamp - that is not one of the
columns compared.

The data is status data from a piece of equipment and we only want to
store changes. If 2 consecutive rows come in that are the same
(excluding the timestamp) I don't want to store the second one.

[Ketan Bhatt]

Can you not do something like

`qs.filter(...info that is coming in...).exists()`
If ^ is True, then update it, otherwise create a new object?

[James Schneider]

On Mon, May 23, 2016 at 9:20 AM, Larry Martell <larry....@gmail.com> wrote:

They're not identical - there's a timestamp - that is not one of the
columns compared.

The data is status data from a piece of equipment and we only want to
store changes. If 2 consecutive rows come in that are the same
(excluding the timestamp) I don't want to store the second one.

If you can coerce your incoming data into a dict using the same structure as your model, you can probably do something like this:

new_data = {'col1': data1, 'col2': data2}

latest_db_record = FOO.objects.filter(bar='baz').order_by('-timestamp').values('col1', 'col2')[0]

if new_data != latest_db_record:

    new_data['bar'] = 'baz'

    FOO.objects.create(**new_data)

Salt to taste as necessary.

You might also be able to work with qs.last() or qs.latest(), but those return the actual objects and you can't take advantage of qs.values() splitting it into a dict for you.

A DB transaction may also be appropriate if you have a lot of data rapidly coming in. 

-James

[Derek]

Interesting. In all the cases I can think of, I would almost always want to keep the most recent check (not the oldest)... that tells me how recently the status of X was checked.  A more pedantic administrator might also want all those times stored, so a history can be created.

[James Schneider]

On May 24, 2016 9:11 AM, "Derek" <game...@gmail.com> wrote:
>
> Interesting. In all the cases I can think of, I would almost always want to keep the most recent check (not the oldest)... that tells me how recently the status of X was checked.  A more pedantic administrator might also want all those times stored, so a history can be created.
>

Not necessarily. You would want record of the initial change, not a record of the last time that value was seen. The assumption would be that the value remained constant until the next change event. What happens when the sensor goes offline and online, and then continues to report the same value on initialization? You wouldn't see anything until the sensor reported a changed value, which could be seconds, or years. That also means you've lost a data point that probably should have been captured.

In doing so, you also potentially save a ton of write operations, since keeping the latest check would require extra logic to delete the old entry and create the new entry, or update the existing entry in place.

The use case for the data would be the driver for what data needs to be retained. And also the use cases that haven't been thought of.

I personally would prefer to keep all of the data points, and summarize the data in a report using logic similar to the OP's storage strategy. People tend to find interesting ways to use data, and you always end up with egg on your face if you are summarizing/tossing  the data on input rather than filtering/analyzing output.

Obviously this is all barring other technical restrictions that may enforce a reduced data set.

-James

[Derek]

We are both speaking as non-users of the OP's system. ;)

I agree that there are some use cases for keeping the first measurement IF the actual date of measurement makes no difference at all.  We have an app like that, where we process real-time data and use a filter that only lets a record through if there has been a "change since previous".  On the other hand, changing the date means that the user has some reassurance that status changes have been, and are, continuing to be made.  I think we saying the same thing though: in general, in the absence of other constraints, try to retain as much data as possible and archive it in preference to simply tossing it.

[Larry Martell]

I am the OP. I was implementing what my customer asked for. But you
make a good point that they might want the latest data not the oldest.
I will suggest that to them. Thanks.

[Larry Martell]

Just FYI, they did not want that change. They did not feel it added
any value. But in reality, they probably just didn't want me spending
time on it so I could do other things.