Query based on number of embedded documents matching criteria

[SupernovaMike]

I am writing an application which tracks astronimical observations of
targets. I have looked at documentation and it seems that I can use
$elemMatch to find, for example, targets with at least one observation
with (time > 1 week ago, type=photometry). Is there a way to search
for targets with at least 3 observations matching the criteria, or
less than 2 matching observations etc.?

Thanks,
Mike

[Michael Dirolf]

There's not a great way to do that with the normal query language -
could try changing up the schema and doing counts, or using a $where
to do fine tuning after using $elemMatch for the initial filtering.

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[SupernovaMike]

In what way could I change the schema and do counts? Are you
suggesting having an observations collection then using some form of
aggregation (the equivilent of a GROUP BY target_id) to get a count
out of this then using this list for a $in or $nin query on the
targets collection?

Is there a good reson that this can't be done with the normal query
language? Is it particularly difficult to impliment, or just not used
much? Would it be inconcievable to have a play with the source and
make it work?

Thanks,
Mike

On Aug 16, 3:29 pm, Michael Dirolf <m...@10gen.com> wrote:
> There's not a great way to do that with the normal query language -
> could try changing up the schema and doing counts, or using a $where
> to do fine tuning after using $elemMatch for the initial filtering.
>
> On Mon, Aug 16, 2010 at 10:19 AM, SupernovaMike
>

[Michael Dirolf]

On Mon, Aug 16, 2010 at 12:46 PM, SupernovaMike
<mike....@astro.ox.ac.uk> wrote:
> In what way could I change the schema and do counts? Are you
> suggesting having an observations collection then using some form of
> aggregation (the equivilent of a GROUP BY target_id) to get a count
> out of this then using this list for a $in or $nin query on the
> targets collection?

Yeah, was thinking if you had an observations collection you could do
something along those lines.

> Is there a good reson that this can't be done with the normal query
> language? Is it particularly difficult to impliment, or just not used
> much? Would it be inconcievable to have a play with the source and
> make it work?

I just don't think there's been too much demand for this sort of
thing, not sure that it is particularly difficult beyond that. You
could certainly file a request on JIRA or even have a go at
implementing it yourself if it's something you want to see.

[SupernovaMike]

Ah, I'd originally had observations as separate and had started moving
them into embedded as it seemed like it would make things easier; time
to start undoing the changes. I was considering using $where and some
JS to do it, but really don't fancy querying a database of tens of
thaousands of targets with tens or hundreds of observations each
without indexing.

I guess the searching observations to get a list of target_ids will
use indexes and be the fastest approach I'll get without a deep level
of understanding of MongoDB source?

Thanks,
Mike

On Aug 16, 5:58 pm, Michael Dirolf <m...@10gen.com> wrote:
> On Mon, Aug 16, 2010 at 12:46 PM, SupernovaMike
>

[Michael Dirolf]

On Mon, Aug 16, 2010 at 1:08 PM, SupernovaMike
<mike....@astro.ox.ac.uk> wrote:
> Ah, I'd originally had observations as separate and had started moving
> them into embedded as it seemed like it would make things easier; time
> to start undoing the changes. I was considering using $where and some
> JS to do it, but really don't fancy querying a database of tens of
> thaousands of targets with tens or hundreds of observations each
> without indexing.

You could also consider using a combination of $elemMatch and $where -
use $elemMatch to find docs with at least 1 match and then $where to
fine tune. Might perform well enough, depending on the characteristics
of your data set...

> I guess the searching observations to get a list of target_ids will
> use indexes and be the fastest approach I'll get without a deep level
> of understanding of MongoDB source?

Yes, I think this will probably end up being the best bet.

> Thanks,
> Mike
>
> On Aug 16, 5:58 pm, Michael Dirolf <m...@10gen.com> wrote:
>> On Mon, Aug 16, 2010 at 12:46 PM, SupernovaMike
>>
>> <mike.wo...@astro.ox.ac.uk> wrote:
>> > In what way could I change the schema and do counts? Are you
>> > suggesting having an observations collection then using some form of
>> > aggregation (the equivilent of a GROUP BY target_id) to get a count
>> > out of this then using this list for a $in or $nin query on the
>> > targets collection?
>>
>> Yeah, was thinking if you had an observations collection you could do
>> something along those lines.
>>
>> > Is there a good reson that this can't be done with the normal query
>> > language? Is it particularly difficult to impliment, or just not used
>> > much? Would it be inconcievable to have a play with the source and
>> > make it work?
>>
>> I just don't think there's been too much demand for this sort of
>> thing, not sure that it is particularly difficult beyond that. You
>> could certainly file a request on JIRA or even have a go at
>> implementing it yourself if it's something you want to see.
>