CouchDB / Spidermonkey: The need for flush
Christopher Lenz
as some here may know, the CouchDB project is using Spidermonkey to
provide a Javascript "view/query server". The way it does this is a
bit unusual, I suppose: instead of embedding it via the C API, the
CouchDB Erlang code spawns the js commandline shell and communicates
with it over a pipe (which is more or less the recommended way to
interface with external programs/libraries from Erlang).
This works pretty well, but there's one thing we'd like to get rid of:
we're keeping a copy of the Spidermonkey code in our SVN repository,
and had to patch it in a rather minor way to make it work reliably in
this scenario:
<http://groups.google.com/group/couchdb-source/browse_thread/thread/
c56979223615ac42/eeae303d3449b7d0>
The changes are:
1) Add an automatic fflush() after every print() from the JS shell.
This is important because CouchDB needs to see the output directly
after its written, and we don't want buffering to get in the way.
2) Removed the printing of GC stats when the gc() function gets called
in the JS shell. CouchDB expects all output from the JS process to be
JSON formatted, and as we also need to invoke gc() every once in a
while, the printing of those stats gets in the way. A simple fix would
be to redirect that output to stderr.
The other change we made was to compile with C_STRINGS_ARE_UTF8 by
default... any chance this will become the default in the future?
We'd love to be able to use an unpatched Spidermonkey in the future
for CouchDB, also because users can than make use of the Spidermonkey
version provided by their packaging system.
What do you think?
Thanks,
--
Christopher Lenz
cmlenz at gmail.com
http://www.cmlenz.net/
bre...@mozilla.org
> The changes are:
>
> 1) Add an automatic fflush() after every print() from the JS shell.
> This is important because CouchDB needs to see the output directly
> after its written, and we don't want buffering to get in the way.
This sounds fine to me (better: set line buffering for gOutFile)
provided it plays nicely with the hacked up old readline (editline) we
use, and doesn't affect testsuite behavior (it shouldn't).
> 2) Removed the printing of GC stats when the gc() function gets called
> in the JS shell. CouchDB expects all output from the JS process to be
> JSON formatted, and as we also need to invoke gc() every once in a
> while, the printing of those stats gets in the way. A simple fix would
> be to redirect that output to stderr.
This is ok with me too; we can always add an optional arg -- perhaps
passing a blank object in could fill it with JSON-ready stats, but
avoid spewing anything to the output file.
> The other change we made was to compile with C_STRINGS_ARE_UTF8 by
> default... any chance this will become the default in the future?
See https://bugzilla.mozilla.org/show_bug.cgi?id=315288 -- not going
to make Mozilla 1.9 (Firefox 3, JS1.8).
> We'd love to be able to use an unpatched Spidermonkey in the future
> for CouchDB, also because users can than make use of the Spidermonkey
> version provided by their packaging system.
>
> What do you think?
Sounds easy -- please file a bug at bugzilla.mozilla.org, product
"Core" component "JavaScript Engine", attach a `cvs diff -pu8 js.c`
patch, and solicit review from any peer (http://www.mozilla.org/
owners.html#javascript).
/be
Wesley W. Garland
Another approach you may wish to consider (that still maintains your
pipelined JS paradigm) is to simply create your own embedding to replace
js.c in its entirety, and become a strict JSAPI consumer.
There is significant code in there you don't need (especially if you're not
wanting to dump GC stats, decompile functions, and so on), and a bare-bones
interpreter which does little more than compile javascript a line-at-a-time
and execute is, honestly, not all that much work.
The real advantage there is that you then depend only on the SpiderMonkey
API, which is quite stable (although the ABI is not), and you only need to
have one file in your SVN repository. You include jsapi.h and link against
libjs.so.
You'll also get better control of when GC is called, as well as the
opportunity to rig the engine to preload javascript files or what have you
on a global basis... and if you allow arbitrary JS to be executed, probably
a slightly safer security profile.
As for compilation defaults, what I personally do is have my project
Makefile build SpiderMonkey using its Makefile.ref. Anything "special" about
my embedding (like JS_THREADSAFE) is encoded in _my_ Makefile. It also
knows to look in the "spidermonkey" directory of my project for the mozilla
sources. If it doesn't find them, it will either untar a release or fetch it
from my local mozilla CVS mirror. This way I can change SpiderMonkey
versions quickly and painlessly, and the only version-specific ties I have
are to specific spidermonkey features (like the availability of gcZeal).
Wes
> _______________________________________________
> dev-tech-js-engine mailing list
> dev-tech-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102