Adding safe function evaluation

[sybla...@anywhichway.com]

Aseem, JSON5 is a very nice piece of work!

All,

I still needed a safe way to pass function calls over the wire and made a small enhancement to support this. For the time being I'll call it JSON5F.

I added one argument to .parse which is an object containing potentially valid function calls. Only these functions will be parsed, or the parser will throw an error, e.g:

JSON5F.parse({test: eq(2,2)},null,{eq: function(a,b) { return a===b; }) will result in {test: true}

whereas

JSON5F.parse({test: neq(2,2)},null,{eq: function(a,b) { return a===b; }) will throw a parsing exception.

This required adding only 40 lines of sparse code to JSON5. The largest of which was just a copy of the array function with the delimiters changed to parentheses.

I'm pretty new to GitHub collaboration, so I'm unsure of how to proceed ... just fork a copy and make it public (see like noise creation) ... collaborate with Aseem to incorporate it (if people indeed think JSON5F is indeed safe).

Thoughts on utility, safety, and collaboration welcome!

[Andrew Eisenberg]

Interesting idea.  If you wanted to contribute your feature, here are the steps you should do:

1. Create an issue in the json5 project for this enhancement
2. Clone the json5 repo
3. Push your local changes to your clone in a new branch
4. Create a pull request for your change. In the description of the PR, make sure to reference the issue.

Doing this makes it easy for us to see what you want to accomplish and what you want to change.  If something doesn't quite make sense to you, I or someone else can help you through it.

But, before we go further down this path, I'd like to know a bit more about your use case, the kinds of thing you'd like to support, and what you don't want to support.  A feature like this could get hairy if you wanted to support nested functions for example.

Also, I'd think that most of what you require could be implemented using a fancy reviver function (even though it would be much less elegant).  It would be good to discuss this further.

Andrew

[sybla...@anywhichway.com]

Thanks for the response. Here is my use case:

Google is deprecating ScriptDb (https://developers.google.com/apps-script/guides/script-db/). We have a complete replacement (actually more powerful and fully indexed called ReasonDB). We would like to be able to pass queries over the wire in "native format" to keep the syntactic dissonance between client and server side programming low.

What we don't want to support is the evaluation of arbitrary Javascript, just the evaluation of named functions that are passed in, including nested calls, e.g.

{test: eq(3,max(3,0))}.

In reality what we are usually doing is creating deferred execution functions by returning other functions that are then evaluated by the database, e.g. (but grossly over simplified):

given:

var functionMap = {eq: function(val) { return function(dbvalue) { return val===dbvalue; } }

var object = JSON5F.parse({age: eq(25),null,functionmap);

object will actually be {age: [function]} which when passed to the database will return all objects with age 25.

Obviously, someone could create an unsafe function and expose it through the functionMap. But, creating unsafe code can be done in many ways. Think of our approach as half way to a sandbox that can be exposed on the wire.

We have tried reviver functions and have found they require going completly declaritive, resulting in cumbersome JSON introducing a high level of dissonance as well as lots of quoted values and operators that have to be parsed and validated later anyway. There is actually another Javascript database that takes this approach, TaffyDb.

Our approach can't be supported by Javascript sandboxes because it relies heavily on closure values that disappear when the sandbox is exited. Although we could theoretically put the entire database process inside a sandbox other limitations quickly come to the surface given the current state of Javascript sandboxes.

You can see a bit more about ReasonDB here.

[Andrew Eisenberg]

Apologies for not responding earlier. I was traveling.

I see the use case that you are working with. In the case where it is the same process that parses and serializes the JSON, things are relatively safe.  However, my concern is that JSON is often sent across the wire and including a feature like this would require that both endpoints have the same functions defined with the same semantics.  For example, what if my version of eq uses ==, but yours uses ===? This can lead to all sorts of subtle bugs.  (This problem is not avoided if reviver fns are used, but at least this practice is not encouraged).  I am less concerned about safety since the set of functions available and their implementation are provided by the current process, not the process that generated the JSON.

At this point, I see the feature as specific to your use case.  And I can't really see a general use for this that replacer/reviver couldn't do a better job with.  I may be wrong and I am happy to hear any counterpoints.

--
You received this message because you are subscribed to the Google Groups "JSON5" group.
To unsubscribe from this group and stop receiving emails from it, send an email to json5+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simon Blackwell]

Although I understand your concern, it applies to all software that can be built with libraries on different computers, i.e. library versions may not match semantically. I have, in fact, encountered in my career identical code behaving differently because of operating system differences. 

Note, I am not proposing that the JSON5 alow the creation of dynamic functions, only that function calls be expressable. It would be up to the developer to ensure that the same library used on the server were available to the client, if indeed evaluation of the same JSON5 on the client was required.