Idea: $quote operator

[Will Shulman]

We often find ourselves often wanting to store queries and expressions
that contain update operators in MongoDB itself. These expressions can
have '$' characters as keys, which are not allowed. The only solution
thus far has been to encode / decoding the expression as a string, or
somehow otherwise escaping the '$'' when storing the db.

It would be nice to have a more formal way of doing this that did not
require as much custom string mangling in the app, and that left the
expressions somewhat readable in the DB itself.

One potentially elegant way of dealing with this would be to introduce
a $quote operator (Lispy, I know...). Then one could do this:

{ "myQuery" : { "$quote" : { "x" : { "$ne" : 0 }}}}

Thoughts?

-will

[Dwight Merriman]

yes it would be nice to do something.

need to think more about ideal way to do as it will be like this for 10 years -- this might be the right way.

is there an existing jira?  i half remember there being one but can't find quickly.

--
You received this message because you are subscribed to the Google Groups "mongodb-dev" group.
To post to this group, send email to mongo...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-dev?hl=en.

[Gregg Lind]

Very LISPy indeed!

How awkward is it to serialize as a string? It seems like a
'$quote'd entity would have to be quoted all the way down (i.e, it
can't contain anything else), so how much benefit is adding a new
operator here? Or is $quote smarter such that it doesn't 'eval'
things? Maybe a real example where this is biting you would help with
guiding the decision?

GL

[Dwight Merriman]

the easiest way to serialzie would be into a BinData field.  i.e. take the bytes of the query bson object, stuff them in a bindata bson element.  then it will be happy.  in this situation the data in the quoted object is opaque, so not as elegant, but very doable today with no changes.

[Will Shulman]

Gregg-

That's right, $quote would help the system be smart about what should
be evaluated and when, giving us a way of dealing with the ambiguity
that led to the need to disallow "$" characters in field names.

So, if there is a document, D, like this:

{
"_id" : 1234,
"q" : { "$quote" : { "$lt" : 0 }}
}

The query:

{ "q" : { "$lt" : 0 }} would NOT match D (this query matches when
documents where q < 0)

but this query:

{ "q" : { "$quote" : { "$lt" : 0 }}}} would match D

Escaping and quoting the query documents strings is a pain, since in
our case, humans are the ones who are writing the documents and doing
the escaping. Also the query expressions get ugly to look at with all
the escape chars.

-will

On Aug 29, 12:02 pm, Gregg Lind <gregg.l...@gmail.com> wrote:
> Very LISPy indeed!
>
> How awkward is it to serialize as a string?   It seems like a
> '$quote'd entity would have to be quoted all the way down (i.e, it
> can't contain anything else), so how much benefit is adding a new
> operator here?   Or is $quote smarter such that it doesn't 'eval'
> things?  Maybe a real example where this is biting you would help with
> guiding the decision?
>
> GL
>
>
>
>
>
>
>
> On Mon, Aug 29, 2011 at 11:56 AM, Dwight Merriman <dwi...@10gen.com> wrote:
> > yes it would be nice to do something.
> > need to think more about ideal way to do as it will be like this for 10
> > years -- this might be the right way.
> > is there an existing jira?  i half remember there being one but can't find
> > quickly.
>

[Dwight Merriman]

do you need to reach in and query on it or just store it

[Will Shulman]

We just want to store it. I gave the example of querying just to
illustrate why allowing "$" operators in field names causes ambiguity
today, and how a general solution will need to deal with that.

-will

[Will Shulman]

I finally got around to creating a Jira ticket for this (I was not
able to find another in Jira so please forgive if a dup). After more
thought and talking with some folks at the office it occurred to us we
probably only need the $quote form to be used in query and update
expressions (a simplification of the suggestion in this thread).

Simplified version of the idea is in the ticket:

https://jira.mongodb.org/browse/SERVER-3943

-will

On Aug 29, 12:34 pm, Will Shulman <w...@objectlabs.com> wrote:
> We just want to store it. I gave the example of querying just to
> illustrate why allowing "$" operators in field names causes ambiguity
> today, and how a general solution will need to deal with that.
>
> -will
>
> On Aug 29, 12:30 pm, Dwight Merriman <dwi...@10gen.com> wrote:
>
>
>
>
>
>
>
> > do you need to reach in and query on it or just store it
>
> > On Mon, Aug 29, 2011 at 3:19 PM, Will Shulman <w...@objectlabs.com> wrote:
> > > Gregg-
>

> > > That's right, $quotewould help the system be smart about what should

> > > be evaluated and when, giving us a way of dealing with the ambiguity
> > > that led to the need to disallow "$" characters in field names.
>
> > > So, if there is a document, D, like this:
>
> > > {
> > >  "_id" : 1234,
> > >  "q" : { "$quote" : { "$lt" : 0 }}
> > > }
>
> > > The query:
>
> > > { "q" : { "$lt" : 0 }} would NOT match D (this query matches when
> > > documents where q < 0)
>
> > > but this query:
>
> > > { "q" : { "$quote" : { "$lt" : 0 }}}} would match D
>
> > > Escaping and quoting the query documents strings is a pain, since in
> > > our case, humans are the ones who are writing the documents and doing
> > > the escaping. Also the query expressions get ugly to look at with all
> > > the escape chars.
>
> > > -will
>
> > > On Aug 29, 12:02 pm, Gregg Lind <gregg.l...@gmail.com> wrote:
> > > > Very LISPy indeed!
>
> > > > How awkward is it to serialize as a string?   It seems like a
> > > > '$quote'd entity would have to be quoted all the way down (i.e, it
> > > > can't contain anything else), so how much benefit is adding a new

> > > >operatorhere?   Or is $quotesmarter such that it doesn't 'eval'

> > > > things?  Maybe a real example where this is biting you would help with
> > > > guiding the decision?
>
> > > > GL
>
> > > > On Mon, Aug 29, 2011 at 11:56 AM, Dwight Merriman <dwi...@10gen.com>
> > > wrote:
> > > > > yes it would be nice to do something.
> > > > > need to think more about ideal way to do as it will be like this for 10
> > > > > years -- this might be the right way.
> > > > > is there an existing jira?  i half remember there being one but can't
> > > find
> > > > > quickly.
>
> > > > > On Fri, Aug 26, 2011 at 3:17 PM, Will Shulman <w...@objectlabs.com>
> > > wrote:
>
> > > > >> We often find ourselves often wanting to store queries and expressions
> > > > >> that contain update operators in MongoDB itself. These expressions can
> > > > >> have '$' characters as keys, which are not allowed. The only solution
> > > > >> thus far has been to encode / decoding the expression as a string, or
> > > > >> somehow otherwise escaping the '$'' when storing the db.
>
> > > > >> It would be nice to have a more formal way of doing this that did not
> > > > >> require as much custom string mangling in the app, and that left the
> > > > >> expressions somewhat readable in the DB itself.
>
> > > > >> One potentially elegant way of dealing with this would be to introduce

> > > > >> a $quoteoperator(Lispy, I know...). Then one could do this: