serialising objects to the system.js DB for server-side execution doesn't do what I'd expect

[dan90]

Hi mongo gurus,

I just opened this as a bug ( http://jira.mongodb.org/browse/SERVER-770
), but then it occurred to me that it might be by design so i should
check in with other users and see what the deal is. I'll reproduce the
bug text here for ease of reference then add a couple of questions.

=====
when an object is serialized to system.js and then later instantiated,
it loses its prototype and all methods defined outside its
constructor.

To see this, consider the following script

//currently this prints outputs and doesn't do any nice asserts
//this si for clarity of bug reporting

var s = db.system.js;

PrototypicalInheritanceObject = function () {
this.constructor_assigned_property = 'this
constructor_assigned_property exists';
}

PrototypicalInheritanceObject.own_property = 'this own_property
exists';

PrototypicalInheritanceObject.own_method = function () {
return "this own_method exists";
}

PrototypicalInheritanceObject.prototype.prototypical_property =
'this prototypical_property exists';

PrototypicalInheritanceObject.prototype.prototypical_method =
function () {
return "this prototypical_method exists";
}

print("======================================");
print("Client side");
print("======================================");

//test that this behaves as normal in the shell interpreter
var p = new PrototypicalInheritanceObject();
print("constructor property - shell interpreter:");
print(p.constructor_assigned_property);
print("");
print("own property - shell interpreter:");
print(p.own_property);
print("");
print("own method - shell interpreter:");
print(tojson(p.own_method));
print("");
print("prototypical property - shell interpreter:");
print(p.prototypical_property);
print("");
print("prototypical method - shell interpreter:");
print(tojson(p.prototypical_method));

print("======================================");
print("server side");
print("======================================");
s.insert( { _id : "PrototypicalInheritanceObject", value :
PrototypicalInheritanceObject});
print("constructor property - server interpreter:");
print(db.eval("var p = new PrototypicalInheritanceObject(); return
p.constructor_assigned_property"));
print("");
print("own property - server interpreter:");
print(db.eval("var p = new PrototypicalInheritanceObject(); return
p.own_property"));
print("");
print("own method - server interpreter:");
print(db.eval("var p = new PrototypicalInheritanceObject(); return
tojson(p.own_method)"));
print("");
print("prototypical property - server interpreter:");
print(db.eval("var p = new PrototypicalInheritanceObject(); return
p.prototypical_property"));
print("");
print("prototypical method - server interpreter:");
print(db.eval("var p = new PrototypicalInheritanceObject(); return
tojson(p.prototypical_property)"));

The output for the client side and server side phases seem like they
should be the same, but they are not.

======================================
Client side
======================================
constructor property - shell interpreter:
this constructor_assigned_property exists

own property - shell interpreter:
undefined

own method - shell interpreter:
undefined

prototypical property - shell interpreter:
this prototypical_property exists

prototypical method - shell interpreter:
function () {
return "this prototypical_method exists";
}
======================================
server side
======================================
constructor property - server interpreter:
this constructor_assigned_property exists

own property - server interpreter:
null

own method - server interpreter:
undefined

prototypical property - server interpreter:
null

prototypical method - server interpreter:
undefined

======

So, what's the go here?

In a separate issue, I've noticed that functions serialised to the db
in system.js also lose their closure, so it seems that the system.js
really only stores the body of functions, and whilst it might look
like it does more, it really doesn't. I mean, the docs don't claim to
store anything other than functions, of course -
http://www.mongodb.org/display/DOCS/Server-side+Code+Execution - but
since objects and functions are so intertwined in JS, and closures,
IMHO are really part of a function definition, the documentation could
possibly use some elaboration here. So on one hand this could be a
mere documentation clarification issue.

On the other hand, this behaviour makes makes writing code that is
portable across client and server very tricky, since they are going to
be executed in substantially different ways - in particular, you're
going to need to include some strange hacks to initialize objects for
server-side execution if you want to use e.g. prototypical
inheritance. if it were possible to serialize a more complete
representation of the object in question that would seem to be a very
much more useful behaviour. Is that a feature request, or a bug
report? I'm not sure.

FYI, we're implementing a javascript full text search for mongo to be
released open source; that's why we're caring about this, since there
are non-trivial objects with inheritance that you have to initialize
to make this happen cleanly.

-dan()

[Eliot Horowitz]

There are a couple of different things going on here.
1) what the shell automatically sends over
2) what is possible

the shell only serializes the function itself

if you are doing something complex, what you probably should do is
write something that sets up your full closure in system.js So you
could create a class, add all the functions, and then write something
that re-builds that in the server.

You should be able to do it in a way that things work the same client
or server side.

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[dan90]

Thanks Eliot. That was fast.

> There are a couple of different things going on here.
> 1) what the shell automatically sends over
> 2) what is possible
>
> the shell only serializes the function itself

yes, in the sense of serialising the body of the function definition
but not the closure and so on. I assumed otherwise, since in other
languages, say python, the serialisation of a function does include
closure - it's still part of the function, if not part of the text fo
the function body. However, I understand of course that no-one around
here is in the business of improved writing javascript serializer, so
I guess this issue is just a documentation one.I just commented on
http://groups.google.com/group/mongodb-user/browse_thread/thread/27aaf8261b5fff46
to that effect. Thanks for clarifying for me!

So, practically:

> if you are doing something complex, what you probably should do is
> write something that sets up your full closure in system.js  So you
> could create a class, add all the functions, and then write something
> that re-builds that in the server.
>
> You should be able to do it in a way that things work the same client
> or server side.

That's easy for bespoke code, but it's hard for 3rd party libraries
which aren't written with system.js's idiosyncrasies in mind. Do you
have recommendations for that? AFAICT the only way of loading a
javascript script is the "load()" function, which isn't great for
telling you //what// you've just loaded up, and requires you to shunt
text fiels around. I suppose you could write whole JS libraries to a
string in a server variable then do a plain old javascript eval() on
the string, but that feels strange, and is itself tricky - AFAICT
there is no way of reading script files in as text from the mongo
shell to allow me to automate that process. I mean, I could fall back
to a language like python and write a deployment script there, but
keeping external dependencies to a minimum would be nice.

Am I on the right track here, or am i missing something obvious?

Thanks!
Dan

>
> <google....@email.possumpalace.org> wrote:
> > Hi mongo gurus,
>

> > I just opened this as a bug (http://jira.mongodb.org/browse/SERVER-770

> > ), but then it occurred to me that it might be by design so i should
> > check in with other users and see what the deal is. I'll reproduce the
> > bug text here for ease of reference then add a couple of questions.
>
> > =====
> > when an object is serialized to system.js and then later instantiated,
> > it loses its prototype and all methods defined outside its
> > constructor.
>

<snip>

[Eliot Horowitz]

the other option is putting in all in a class like
function foo(){
this.bar = function(){ }
}

that might be best.