Returning Nth row in the form of array of arrays

[Tanya]

Hi,

We have time series data that we'd like to display in Highcharts. The data comes every minute. We found that Highcharts can cope with about 2 days of data (2880 points), but after that performance is unacceptable. In order to display N days worth of data we have been selecting every Nth row. That's can be easily achieved in Oracle and PHP. With Mongo the process seem to be less straightforward. The best I could do was to use the mod of the timestamp (tm). The code is to get every second minute.

db.store.aggregate([
{$match: {createdAt: {$gt: ISODate("2015-08-28T03:29:50.868Z"), $lt: ISODate("2015-09-07T03:17:50.754Z")}, station: 200}},
{$group: {_id: {$subtract: ["$tm", {$mod: ["$tm", 1200000]}]},
    station: {$first: "$station"},
    "t": { $first: "$t" },
    "tm": { $first: "$tm" }
    }
},
{$sort: {"tm": 1}},
{$group: {_id: "$station", t_series: {$push: {tm: "$tm", t: "$t"}}}}
]);

This, however, returns an array of objects ([{"tm": tm1, "t": t1}, {"tm": tm2, "t": t2}, {"tm": tm3, "t": t3}]). Highcharts do not understand this format, it needs an array of arrays ([[tm1, t1], [tm2, t2], [tm3, t3]]). We have quite a few series and looping through each to create the right format on the client side is time consuming and we'd like to avoid it. We use Meteor for the app, so not sure if we could massage the data on the server to return to the client the correct collection (that could be an option). As a preference we'd like to structure our data in Mongo that we can query and get the format that Highcharts can consume.

Can you help with a more intelligent way of solving this problem (in Mongo or Meteor), please.

Thanks a lot!!!

[Asya Kamsky]

Can you provide a sample/typical document and exactly how you want results ordered? 

I'm not sure why you're doing that first group here since you are not summing or averaging here - wouldn't it be easier to use $project and then another $match? 

I guess the real aggregation on station - but that's just to come up with the desired format, right?

In 3.2 (coming later this year) creating an array would be trivial.  With previous versions I'm not even sure there's a way to do it "correctly", there is a way to do it but it would not guarantee the order of the two members of it (it would depend on their values).

In 3.2 the last $group will just become:

{$group: {_id: "$station", t_series: {$push: {$literal:[ "$tm", "$t"]}}}}

and that will give you array of arrays-of-two.

The other option in 3.2 will be to create the array of two in $project and then $push:"$newArrayField" to get the format you want.

It can be "faked" using $setUnion in the current version (2.6 and 3.0) but that won't give you the order you want unless the value of $t is always larger than the value of $tm and that's unlikely from the looks of it (unless the charts can be told which parameter is first).

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/66a9a759-f3e0-444f-8697-7613e4a1c654%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.