How do you concatenate two querysets?
Ramdas S
How do you concatenate two querysets? Is there any way to create a
single query set out of two or more query sets?
RS
Tim Chase
> you can combine querysets using the | and & operators:
>
> # You can combine queries with & and |.
>>>> s1 = Article.objects.filter(id__exact=1)
>>>> s2 = Article.objects.filter(id__exact=2)
>>>> s1 | s2
> [<Article: Area woman programs in Python>, <Article: Second article>]
>>>> s1 & s2
> []
If that doesn't work for you, you can use itertools.chain()
import itertools
:
:
q1 = Model1.objects.all()
q2 = Model2.objects.all()
for thing in itertools.chain(q1,q2):
do_something(thing)
If your models are the same and the filter conditions aren't too
tough (both querying the same foreign-key value for the same
property with diff. values) you should be able too use PE's solution.
-tim
Ramdas S
While it works, it does not help me
I am writing a small text search to span some 50,000 records from 2 tables. While there are many common fields, the schemas are different, and trying to search using combine of querysets throws up errors.
Ramdas
Tim Chase
>>
>
> I am writing a small text search to span some 50,000 records
> from 2 tables. While there are many common fields, the schemas
> are different, and trying to search using combine of querysets
> throws up errors.
I'm not sure I follow. chain() doesn't care what the contents
are. If you only reference the common fields, Python's
duck-typing takes care of the details. Since you don't give
details about your models or what fields you're interested in or
even what errors are happening, I'll make up some data-models to
demonstrate:
class Person(Model):
name = CharField(...)
birthday = DateField(...)
spouse = ForeignKey('Person')
class Business(Model):
name = CharField(...)
line_of_business = ForeignKey(BusinessField)
You can then do
search = 'joe'
print "People and businesses containing '%s':" % search
for named_thing in itertools.chain(
Person.objects.filter(name__icontains=search),
Business.objects.filter(name__icontains=search),
):
print named_thing.name
which will work fine because both models have a "name" field. If
you want to use disjoint field-names and use their disjoint
names, then you don't really want to combine them. Or you want
to combine them as Malcom suggests via a SQL query that maps the
fields to the same names.
As for any alleged errors, I suspect it's a coding error, not a
problem with Django. Showing code and errors will help.
-tim
Malcolm Tredinnick
On Wed, 2008-03-19 at 12:30 +0000, Chris Hoeppner wrote:
> How'd you handle combining two lists of custom objects such as two
> querysets in python? Do they behave as standard lists to this extend?
Querysets behave like iterators, not lists. Lists can also behave like
iterators, but they're more powerful (you have random access for a list,
but not an iterator; you can take the length of a list, etc).
To combine two iterators, you can use itertool.chain(), as Tim
indicated. That will exhaust the first iterator and then start taking
things from the second.
Or, if both iterators are sorted and you have a way of comparing the
results at the top of each iterator, you could write another iterator
that returned whichever element of the two original iterators was the
smallest. That way you combine two sorted iterators into a single sorted
iterator (one possibility attached to this email).
The options are pretty endless here. It's basically the case of having N
queues of people and you can do whatever you like with the first person
from each queue (process them or ignore them). And you can tell the
queues apart. Now play at being gatekeeper and get all the people
through the gates in various orders. :-)
Regards,
Malcolm
--
If you think nobody cares, try missing a couple of payments.
http://www.pointy-stick.com/blog/
Chris Hoeppner
never got to ask about. I'll be trying this out asap, and posting my
results for the record.
Again, thanks.
~ Chris
El mié, 19-03-2008 a las 09:14 -0500, Tim Chase escribió:
> > if both models have a date field:
> >
> > def by_date(a, b):
> > # asuming .date is a unix timestamp. convert first if not.
> > if a.date>b.date:
> > return 1
> > elif a.date==b.date:
> > return 0
> > elif a.date<b.date:
> > return -1
> >
> > # Is this a valid way to get a list containing
> > # all of the objects from both querysets?
> > a = [i for i in itertools.chain(
> > model1.filter(some__condition),
> > model2.filter(some__other__condition)
> > )]
> >
> > # If so, this should work too?
> > a.sort(by_date)
>
> Rather than explicitly defining by_date function, I find it
> easier to just use the inbuilt cmp() function:
>
> a.sort(lambda a,b: cmp(a.date, b.date))
>
> or
>
> a.sort(key=lambda x: x.date)
>
> or even
>
> a.sort(key=operator.attrgetter('date'))
>
> The key= form is only available in Python2.4 but is more
> efficient on larger data-sets. The cmp form is slower, but
> available on 2.3
>
> -tim
>
>
>
>
>
>
> >
Tim Chase
> objects from queryset2.
>
> Does anyone have any idea how I could accomplish this?
Uh...have you been reading the thread?
itertools.chain()
-tim
Josh
that I want to have them all in one queryset at the end is that I need
to be able to paginate the results. Really, all I want is to be able
to do .order_by('manytomany__isnull'). :/
On Mar 19, 11:07 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
> On Wed, 2008-03-19 at 13:06 -0700, Josh wrote:
> > I'm also attempting to concatenate two querysets, and the | syntax
> > seems to combine them, but it doesn't appear to preserve the order.
> > There are the two querysets I want to put together:
> > queryset1 =
> > Model.objects.filter(manytomany__isnull=False).order_by('name')
> > queryset2 =
> > Model.objects.filter(manytomany__isnull=True).order_by('name')
>
> > If I then do queryset1 | queryset2 I do indeed get a combined
> > queryset, but the entire thing appears to be sorted by name or
> > something so that items from queryset2 are mixed in with items from
> > objects from queryset2.
>
> two querysets *before* making a call to the database and then runs a
> single SQL query. Most of the time that is the natural behaviour. If you
> really want them evaluated separately, do so at the Python level.
>
> Malcolm
>
> --
> Experience is something you don't get until just after you need it.http://www.pointy-stick.com/blog/