$regex within $cond

[Andre]

I'm not sure if its possible but within the aggregation framework, probably in $project, I'll like to use regular expression to match a field. i.e. say I have a boolean field that should be true if content contains the word MongoDB and also another boolean field that will be true if the text does not contain the word MongoDB.

trying this in several patterns doesn't work, as a matter of fact the only time I got $regex to work is within a $match in the pipeline, I need it in $project- is it possible? 

I've tried several combinations and I'm in no luck, it either crashes or return nothing.

"$project" : { 

            ,"hasmongo": { "$regex" : ["$field", /MongoDB/i] },

            ,"hasmongo": { "$regex" : {"$field": /MongoDB/i} },

            ,"hasmongo": { "$regex": 'acme.*corp', $options: 'i' }//,

            ,"hasmongo": { "$eq": { "field": /MongoDB/i } }

            ,"hasmongo": { "field": /MongoDB/i }            

            ,"hasmongo" : { "$cond" : [{ "$and" : [{ "$or" : [{ "$eq" : ["$field", /MongoDB/i] }] }, { "$or" : [{ "$ne" : ["$field", /Mongodb/i] }] }] }, 1, 0]}

I'm looking for true/false, 1/0 - something!

[Andre]

Here is the most common error I get.

uncaught exception: aggregate failed: {

"errmsg" : "exception: invalid operator '$regex'",

"code" : 15999,

"ok" : 0

[Will Berkeley]

It's not possible to use $regex in a projection stage of a pipeline. See the allowed operators here. What are you trying to achieve with $regex? Perhaps there is another way to do it with existing functionality.

-Will

[Andre]

Thanks Will,

So we've been using MR for while now(ages) with 100's of jobs and Asya has been suggesting that we should start using aggrF to solve other problems especially after the $out parameter since that was one of the major reasons why we went MR and secondly the flexibility of using Javascript. Anyway, long story short, we started converting some of these reports and it seems aggrF is just not there yet, we do so many calculations with JS but using basic functions. Regarding this question we allow our customers to search text using simple functions for example, they could say give me all rows where last name contains "Jack" or first name starts with "John" or email ends with ".io" - so this works in match - perfect.

Now, second scenario (Crucial) is were the customer can say for example if referring url contains "doubleclick.com" then column is "Paid Search" otherwise "Organic"

we knocked this out with MR easily using JS functions, however, in aggr when I project out with a cond it errors out, what I need is a projected field that is like field = (fieldX contains doubleclick ? "Paid Search" : "Organic");

we were able to implement all the basics (equals, notequals, gt, le, gte,lte, etc..) but not (startswith, endswith,contains, doesnotcontain) in aggrf.

in MR we have js functions like...that we use in the map phase or use RegEx /doubleclick.com/i - which works perfectly. hope that explains it. I need to set the values of certains fields in a project based on a true/false condition.

    Contains: function (data, str) {
        /* Determines if data contains string */
        if (data == null || str == null) return false;
        return data.length >= str.length && data.toLowerCase().indexOf(str.toLowerCase()) >= 0;
    }

Andre

[Asya Kamsky]

Andre:

Unfortunately, the only one of these that's easy to do with $cond in
$project is "startsWith" - you just use the $substr operator and
equality check.

The others will have to wait till more functions are available in
aggregation framework.

Asya

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user...@googlegroups.com.
> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mongodb-user/ea64ee1c-00db-4411-b820-6e1df921e21f%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Andre]

Thanks Asya, guess i'll just stop trying to use aggregation framework, other than its speed its completely useless to us, I've hit so many bottle necks and pretty much wasted 2 weeks worth of my time trying to convert to aggregation framework, however, it seems I can't do anything with it past $match.

I wrote a simple inline MR to instead process everything in memory but it took 30+ mins to pull and process 7 million rows from MongoDB, 22mins was spent just reading data into the simple app, it took Map/Reduce just 6 mins. It looks like I'm going to stick with MR and squeeze out as much as I can from it or dump Mongo and use something else as we may have fall for the classic fast writes impossible queries since for the past 3 years 95%+ of our issues have been MongoDB related. Granted it has given us the ability to build our start-up in a flexible way, so success there, if only I can allow our customers to query the data in ad-hoc mode fast, we will be golden.

Andre

[Asya Kamsky]

I guess I'm not sure when you say "if only I can allow our customers
to query the data in ad-hoc mode fast" if you are talking about
something different - neither mapReduce nor aggregation (over large
data sets) are really meant to be *fast* for ad hoc queries (like the
kinds you expect back in milliseconds)...

Asya

> https://groups.google.com/d/msgid/mongodb-user/7c079cdc-81f7-491d-afef-ea73800208cd%40googlegroups.com.