Remove items from queryset?

[Igor]

Hello,

I need to exclude some items from a QuerySet based on field analysis
with regular expressions. Which means in my understanding means that I
need to first evaluate the QuerySet and then go thru it and use python
to remove some items based on my analysis.

Now, how can I accomplish that? I was not able to find a way to remove
items from a QuerySet.

Thanks in advance

Igor

[Daniel Roseman]

If it's a simple regex comparison on a field, you may be able to do it
directly in the query:

qs = MyModel.objects.exclude(fieldname__regex=r'^regex')

If you really do need to iterate through and exclude things, a
queryset acts just like a list so you can .pop() items and del() them.
Or better, you can append the items you *do* want to a separate list.
--
DR.

[Shawn Milochik]

How about creating a Python list, which you populate from chosen items of your queryset? You might even be able to do it with a list comprehension, depending on how complex your filtering is. The contents of a queryset are just model instances. Nothing magical you can't pull out and manipulate with standard Python.

Shawn

[Jani Tiainen]

Simplest route:

l = list(qs)

Of course, after that is no more queryset but just a plain old Python
list containing all objects from your queryset.

--

Jani Tiainen

[Karen Tracey]

On Fri, Jan 22, 2010 at 9:15 AM, Igor <igor.ru...@gmail.com> wrote:

Hello,

I need to exclude some items from a QuerySet based on field analysis
with regular expressions. Which means in my understanding means that I
need to first evaluate the QuerySet and then go thru it and use python
to remove some items based on my analysis.

Does the regex lookup (http://docs.djangoproject.com/en/dev/ref/models/querysets/#regex) combined with exclude() not meet your needs?

 
Karen

[Igor]

Wow, thanks for all the replies! :)

I would not like to rely on fieldname__regex because it's database-
dependent.
I've thought about lists as well, but not sure whether it's a good
idea if it loses the queryset-ness.

The idea of pop() and del() is the most appealing... del actually, but
I couldn't get it to work. Same with binary set operations - '&' and
'|' work but '-' doesn't and '-' would really help... any more ideas
maybe?

Thanks,
Igor

[Bill Freeman]

On Fri, Jan 22, 2010 at 10:22 AM, Igor <igor.ru...@gmail.com> wrote:
> I would not like to rely on fieldname__regex because it's database-
> dependent.
> I've thought about lists as well, but not sure whether it's a good
> idea if it loses the queryset-ness.

If a thing is still a queryset proper, it has to embody SQL that can fetch what
you want. If you want database independence (good luck) the query must
only use (the commonly correctly implemented subset of) standard SQL.

Once you have examined the data in python, the query has already been
run. If you look at all of the returned objects in python, they have
all arrived
in memory, and making a list of them is nearly free.

I'm guessing that you want this to remain queryset-like because you want
to apply further filtering to it beyond these exclusions. If so, are you sure
that you can't reorganize to do these exclusions last? Or do you need to
include instances (rows) in you calculation of the exclusion based on rows
that will eventually be excluded by the other filters?

Another possibility, if you can express your constraints in standard SQL is
to use the extra() method of querysets to specify additional WHERE clauses
(will be ANDed with other constraints).

Again if you can do it with standard SQL, there is always the custom query.
There is, however, a lot of manual labor involved in generating instances
from the return. This is expected to get much easier, or so I've heard, in
1.2. And, of course, you don't need to generate actual instances if you
don't intend to modify and save them, and can pick and choose the fields
you want to return (this, too, has it's details to work out).

Bill

[Igor]

Bill,

You're correct in all your guesses - I want to be able to chain more
filtering to the query since I'm not sure what and how might use it
later. Even in the current situation it's much better from design
standpoint to chain a .distinct() on the set AFTER the exclusion of
the values I don't like.

I'm not (yet :)) an expert in Django as it's easy to guess, but in my
understanding even after I've accessed the elements (i.e. SQL has been
executed) filterng/sorting etc. should be working.

The solution I've implemented for now is instead of creating a list,
I've created an empty QuerySet and am joining every element I DO want
to that set. The last line is the join, and my concern is that it's
not too efficient. I know those result sets are not going to be more
than 20 elements or even less so the inefficiency is probably
negligible, but I would still be happy to find a way that would be
efficient for any set size.

results = PDirectory.objects.none()
for d in dirs:
splitpath = d.path.split("/")

if len(splitpath) == 3: # that would be root_dir/sub_dir
results = results | dirs.filter(id=d.id) # This is not efficient
at all

Thanks,
Igor

On Jan 22, 5:14 pm, Bill Freeman <ke1g...@gmail.com> wrote:

[Bill Freeman]

A final possibility is to enumerate the id's of the objects that you want to
exclude, and then apply them to an unevaluated copy of the queryset
using exclude, e.g.;

q = base_queryset_construction()
excludes = []

for i in q.all():
if shoud_be_excluded(i):
excludes.append(i.id)

q = q.exclude(id__in=excludes}

Or, if you'll be excluding more than you will be including, you can enumerate
those you want to keep and use them with filter instead of exclude.

> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to django-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
>
>

[Carl Zmola]

I have a similar problem I want to exclude or include records based on the presence of other records.

I want to march through Users, get their profile, and exclude records who

I have 3 tables: auth_user, myapp_userprofile, myapp_resource  (with the userprofile having a one to Many relationship with the resource)

I want a queryset of users who (through the convoluted mess) have access to a specific resource.
I have the sql to do this, but I can't figure out how to do this without a list.  It looks like I could do some magic in Django1.2 with the "raw" method, but I am in Django1.1.

Is there an API cookbook for more advanced querying?  I may be missing something simple about how to chain models together in a query (like I would with Join's in sql).

Carl

-- 
Carl Zmola
301-562-1900 x315
czm...@woti.com

[Igor]

Bill,

This is excellent! The only problem is whether it would be limited
"physically" by the possible maximum number of characters in an SQL
query? Or, maybe to fix it it somehow makes sense to .exclude AFTER
evaluation of the query? Thanks for your ideas!

- Igor

On Jan 22, 6:52 pm, Bill Freeman <ke1g...@gmail.com> wrote:
> A final possibility is to enumerate the id's of the objects that you want to
> exclude, and then apply them to an unevaluated copy of the queryset
> using exclude, e.g.;
>
> q = base_queryset_construction()
> excludes = []
>
> for i in q.all():
>    if shoud_be_excluded(i):
>       excludes.append(i.id)
>
> q = q.exclude(id__in=excludes}
>
> Or, if you'll be excluding more than you will be including, you can enumerate
> those you want to keep and use them with filter instead of exclude.
>

[derek]

Carl

You may be better off starting a new thread for this, rather than
posting in the middle of someone else's (even if related).

I too have a similar issue; trying to filter records for a user based
on which "group/s" (a separate table) they belong to. Was about to
start a thread when I saw your post here.

Derek

> >> For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.