Bringing E4X to NodeJS

560 views
Skip to first unread message

Marcel Laverdet

unread,
Nov 19, 2010, 8:08:03 AM11/19/10
to nod...@googlegroups.com
Hey guys, I wanted to drop a line to everyone about a project I've been playing around with trying to bring E4X support to NodeJS and potentially the web at large.

Basically I feel that some subset of E4X could be highly useful within server-side JavaScript, as a front-end technology. String concatenation and object-based front-end frameworks have been proven cumbersome in my experience, so E4X really fills a big gap here.

What I've put together is a utility to convert JavaScript with E4X into regular JavaScript function calls. Then I have a run-time which aims to implement E4X, from within vanilla JavaScript. The run-time is far from complete but shows promise so far.

Here's a couple of examples of what you will be able to do with E4X in NodeJS:

  marcel@marcel $ cat xss.js 
  var xss = '<script>alert(1)</script>';
  require('util').print(<div>{xss}</div>.toXMLString() + '\n');

  marcel@marcel $ node quick-run xss.js 
  <div>
    &lt;script&gt;alert(1)&lt;/script&gt;
  </div>
  
  marcel@marcel ~/code/e4x-bump $ cat b.js 
  XML.prettyPrinting = false;
  XML.ignoreWhitespace = false;
  var body =
  <body>
    <h1>Hello world!</h1>
    <ul>
      <li><a href="http://nodejs.org">NodeJS</a></li>
      <li><a href="https://github.com/laverdet/e4x-bump">e4x-bump</a></li>
    </ul>
  </body>;  
  var util = require('util');
  util.print(body.toXMLString() + '\n');
  for each (var ii in body..a) {
    util.print('* ' + ii.@href.toXMLString() + '\n');
  }

  marcel@marcel ~/code/e4x-bump $ node quick-run b.js 
  <body>
    <h1>Hello world!</h1>
    <ul>
      <li><a href="http://nodejs.org">NodeJS</a></li>
      <li><a href="https://github.com/laverdet/e4x-bump">e4x-bump</a></li>
    </ul>
  </body>

I think the potential here is huge.

If you want to take a peek at the code it's all up on GitHub:

Any feedback or help with the project is definitely welcome!

Stephen Belanger

unread,
Nov 19, 2010, 12:25:59 PM11/19/10
to nod...@googlegroups.com
Looks pretty neat. It'd be good if this functionality could be added directly to V8, but I've heard they aren't fond of implementing stuff that isn't part of the core standards. >.>

--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

Raffaele Sena

unread,
Nov 19, 2010, 1:12:42 PM11/19/10
to nod...@googlegroups.com
The alternative is to implement it as a "preprocessor" in node.js and run the sources through it before calling v8::Script::Compile.
That shouldn't be too hard to do.

-- Raffaele

Mikeal Rogers

unread,
Nov 19, 2010, 1:15:44 PM11/19/10
to nod...@googlegroups.com
The alternative is not having xml literals in the language.

e4x has pretty much been abandoned and I believe it's not even in the latest Spidermonkey.

We've got sax parsers and pure js dom implementations, we don't need literals and can process xml like normal people.

-Mikeal

Stephen Belanger

unread,
Nov 19, 2010, 1:25:27 PM11/19/10
to nod...@googlegroups.com
E4X is cleaner for fragments though;

var fragment = <div>
    <a href="/somewhere">
        some text
    </a>
</div>;

As opposed to;

var fragment = '<div><a href="/somewhere">some text</a></div>';

or;

var fragment = ['<div>'
    , '<a href="/somewhere">'
        , 'some text'
    , '</a>'
, '</div>'].join('');

Admittedly it's not at all necessary, but it can be a bit more convenient.

Mikeal Rogers

unread,
Nov 19, 2010, 1:29:28 PM11/19/10
to nod...@googlegroups.com
Of all the things we would like to have in JavaScript this has to be at the bottom of the list.

Changes to the language are not going to happen in node. node will take v8, vanilla.

We aren't interested in language features, we're building a platform and not concerning ourselves with the language and vm allows us to build and iterate on this platform significantly faster than if we were building a language.

-Mikeal

Ryan Gahl

unread,
Nov 19, 2010, 1:31:26 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 12:15 PM, Mikeal Rogers <mikeal...@gmail.com> wrote:
The alternative is not having xml literals in the language.

e4x has pretty much been abandoned and I believe it's not even in the latest Spidermonkey.

We've got sax parsers and pure js dom implementations, we don't need literals and can process xml like normal people.


E4X has not been abandoned and nor has it been removed from Spidermonkey. Stop making stuff up and spreading FUD. Also, stop being an ass. If you don't like e4x, don't use it, jeez.

"like normal people"... wow. I know you're normally a nice dude; you must just be a bit off about some other stuff going on in your life right now to have been so rude there. 

Anywho, Marcel, that's awesome. E4X is fantastic for templating (IMHO that was the killer feature in Jaxer when it was still alive). Of course, js isn't the ideal layer to implement it because of the need to then preprocess/etc. 

There is a ton of interest from people nearly begging someone at Google to get on it for V8: http://code.google.com/p/v8/issues/detail?id=235

Keep it up! Maybe you can be the one to augment V8 natively once you've got it all working nicely from js.

Ryan Gahl

unread,
Nov 19, 2010, 1:32:57 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 12:29 PM, Mikeal Rogers <mikeal...@gmail.com> wrote:
Of all the things we would like to have in JavaScript this has to be at the bottom of the list.

Changes to the language are not going to happen in node. node will take v8, vanilla.

We aren't interested in language features, we're building a platform and not concerning ourselves with the language and vm allows us to build and iterate on this platform significantly faster than if we were building a language.


Apparently we are also mounting ourselves upon horses that are very tall.

Mihai Călin Bazon

unread,
Nov 19, 2010, 1:36:49 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 7:29 PM, Mikeal Rogers <mikeal...@gmail.com> wrote:
We aren't interested in language features

That's a pretty dumb thing to say.

But I wouldn't vote for E4X either, that's clear.  What I would vote for is macros (I mean real macros, a-la Lisp, not the lame substitute that C has).  Which unfortunately is not quite possible in the "clean" syntax of JavaScript...

Cheers,
-M.

Ryan Gahl

unread,
Nov 19, 2010, 1:41:08 PM11/19/10
to nod...@googlegroups.com

2010/11/19 Mihai Călin Bazon <mihai...@gmail.com>

But I wouldn't vote for E4X either, that's clear.  What I would vote for is macros (I mean real macros, a-la Lisp, not the lame substitute that C has).  Which unfortunately is not quite possible in the "clean" syntax of JavaScript...


Thankfully, Marcel didn't wait for us to cast our votes before he took the initiative to create something he thought was cool and at least some of us appreciate :)

Dean Landolt

unread,
Nov 19, 2010, 1:52:08 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 1:31 PM, Ryan Gahl <ryan...@gmail.com> wrote:
On Fri, Nov 19, 2010 at 12:15 PM, Mikeal Rogers <mikeal...@gmail.com> wrote:
The alternative is not having xml literals in the language.

e4x has pretty much been abandoned and I believe it's not even in the latest Spidermonkey.

We've got sax parsers and pure js dom implementations, we don't need literals and can process xml like normal people.


E4X has not been abandoned and nor has it been removed from Spidermonkey. Stop making stuff up and spreading FUD.

It has in fact been abandoned as a spec and I have it on good authority (Wes Garland) that while there's no schedule to deprecate it in Spidermonkey it is roundly despised by the devs and will eventually get kicked to the curb. I wouldn't be so quick to call this FUD, it's hyperbole at worst.
 
Also, stop being an ass. If you don't like e4x, don't use it, jeez.
"like normal people"... wow. I know you're normally a nice dude; you must just be a bit off about some other stuff going on in your life right now to have been so rude there. 
 
It was just a simple comment -- a little snarky, sure, but that's no reason to question someone's mental state. Mikeal is a nice dude, but he's not nice at all about language bloat -- he's downright hostile to it. But it's personal :)
 
Anywho, Marcel, that's awesome. E4X is fantastic for templating (IMHO that was the killer feature in Jaxer when it was still alive). Of course, js isn't the ideal layer to implement it because of the need to then preprocess/etc. 

There is a ton of interest from people nearly begging someone at Google to get on it for V8: http://code.google.com/p/v8/issues/detail?id=235

Keep it up! Maybe you can be the one to augment V8 natively once you've got it all working nicely from js.

I doubt you'll ever find e4x in v8 -- the spec itself is irreconcilably flawed and there are countless ways to blow your toes off with the e4x gun. I should know -- I do it often -- I'm like e4x's battered spouse -- still, I just keep coming back for more...

None the less, this is a really neat project and could be really useful to a lot of people -- if for nothing else, to help bridge some e4x-heavy code over to node. Thanks Marcel, very nice work!

Dean Landolt

unread,
Nov 19, 2010, 1:53:23 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 1:52 PM, Dean Landolt <de...@deanlandolt.com> wrote:


On Fri, Nov 19, 2010 at 1:31 PM, Ryan Gahl <ryan...@gmail.com> wrote:
On Fri, Nov 19, 2010 at 12:15 PM, Mikeal Rogers <mikeal...@gmail.com> wrote:
The alternative is not having xml literals in the language.

e4x has pretty much been abandoned and I believe it's not even in the latest Spidermonkey.

We've got sax parsers and pure js dom implementations, we don't need literals and can process xml like normal people.


E4X has not been abandoned and nor has it been removed from Spidermonkey. Stop making stuff up and spreading FUD.

It has in fact been abandoned as a spec and I have it on good authority (Wes Garland) that while there's no schedule to deprecate it in Spidermonkey it is roundly despised by the devs and will eventually get kicked to the curb. I wouldn't be so quick to call this FUD, it's hyperbole at worst.
 
Also, stop being an ass. If you don't like e4x, don't use it, jeez.
"like normal people"... wow. I know you're normally a nice dude; you must just be a bit off about some other stuff going on in your life right now to have been so rude there. 
 
It was just a simple comment -- a little snarky, sure, but that's no reason to question someone's mental state. Mikeal is a nice dude, but he's not nice at all about language bloat -- he's downright hostile to it. But it's personal :)

not personal...whoops 

Isaac Schlueter

unread,
Nov 19, 2010, 1:55:48 PM11/19/10
to nod...@googlegroups.com
If the discussion is: "Hey, guys, I built this userland addon that can
be used by you if you want and like E4X, and ignored safely if you
don't!" than I'm all for it. I'd prefer that it not monkey-patch
node, but instead provide some other mechanism for parsing code that
does the E4X thing, but whatever.

If the discussion is: "Here's a thing that needs to be added to
node-core's Script binding, and we should convince Ryan to accept the
patch!" then please for the love of all that is right and holy in the
world, no.

--i

Isaac Schlueter

unread,
Nov 19, 2010, 1:58:12 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 10:53, Dean Landolt <de...@deanlandolt.com> wrote:
>> Mikeal is a nice dude, but he's
>> not nice at all about language bloat -- he's downright hostile to it. But
>> it's personal :)
>
> not personal...whoops

No, you had it right the first time. Mikeal takes language bloat personally.

We javascripters are lucky to have him for that reason.

--i

Ryan Gahl

unread,
Nov 19, 2010, 2:09:30 PM11/19/10
to nod...@googlegroups.com

On Fri, Nov 19, 2010 at 12:55 PM, Isaac Schlueter <i...@izs.me> wrote:
If the discussion is: "Hey, guys, I built this userland addon that can
be used by you if you want and like E4X, and ignored safely if you
don't!" than I'm all for it.  I'd prefer that it not monkey-patch
node, but instead provide some other mechanism for parsing code that
does the E4X thing, but whatever.

If the discussion is: "Here's a thing that needs to be added to
node-core's Script binding, and we should convince Ryan to accept the
patch!" then please for the love of all that is right and holy in the
world, no.

--i


I could be wrong but I assumed from his post that he made a cool userland thing and wanted to tell people what he did and that anyone who wanted to use it was welcome. 

And then he was all... "...weirdo!", and then I was all "...wait, what?", and then you were like "whatevs".

I get snarky :)

I thought Mikeal's response was a bit beyond snarky, though. Oh well... FWIW, I heart E4X.

Mikeal Rogers

unread,
Nov 19, 2010, 2:10:57 PM11/19/10
to nod...@googlegroups.com
There is a whole committee working on adding features to the javascript language. Agree with them or not, they are smart people and have good reasons for features going one way or another and there are long discussions and arguments about each of them.

Spidermonkey takes damn near every features being considered and adds it to their implementation so that you can all play with it.

There is a community for language additions and implementations and experimentation with those additions, it's just not node.js

If you're interested in language features I suggest you involve yourself with the public side of TC-39, the es-discuss list, https://mail.mozilla.org/listinfo/es-discuss

What I actually take issue with here is that xml parsing in node isn't great right now and not enough people are caring about it to work on fixing it. Changes like adding e4x aren't a viable solution and won't see adoption because they require your code to go through a pre-compiler. It can also never go in to node-core.

What we need are better, probably competing, xml libraries, built on node.js. Python ships with like 6 of them, they even have their own module namespace now (totally ridiculous by the way).

-Mikeal


Stephen Belanger

unread,
Nov 19, 2010, 2:48:52 PM11/19/10
to nod...@googlegroups.com
I think it'd be best for E4X to be made a core feature of a fork, much like that talk of making a node fork that natively supports coffee script. That or having it as a compiler option, much like the option to disable SSL, there could be an option to enable E4X.

I have to agree with Mikeal somewhat in that it would slow the language performance unnecessarily, but I think he's perhaps being a bit too harsh about it. This isn't C, and we aren't using this for kernel programming. Javascript is meant to balance between performance and ease of use--where exactly the balance lies is a matter of opinion.

Personally, I like the idea of E4X, but don't know enough about it to really comment on it's performance and implementation challenges.

Ryan Gahl

unread,
Nov 19, 2010, 2:49:07 PM11/19/10
to nod...@googlegroups.com

On Fri, Nov 19, 2010 at 1:10 PM, Mikeal Rogers <mikeal...@gmail.com> wrote:
What we need are better, probably competing, xml libraries, built on node.js.

I don't argue with that (although some nice ones are emerging: https://github.com/ajaxorg/node-o3-fastxml and https://github.com/ajaxorg/node-o3-xml for example).

But people have their own interests. Lambasting Marcel for not doing "what we need" is discounting his needs/desires that obviously compelled him to write his (very cool) project. As far as I can tell he never once said anything about node-core or anything to suggest he wanted to invade all your code with his(my) beloved XML literals and integrated XML navigation ooey gooey sugary syntaxes... nor invade your mind with bloodsucking techno-leeches of destruction and language bloat :)

Also, CouchDB rocks!


Trent Mick

unread,
Nov 19, 2010, 2:57:19 PM11/19/10
to nod...@googlegroups.com
On Fri, Nov 19, 2010 at 11:10 AM, Mikeal Rogers <mikeal...@gmail.com> wrote:
> What we need are better, probably competing, xml libraries, built on
> node.js. Python ships with like 6 of them, they even have their own module
> namespace now (totally ridiculous by the way).

That is about backwards-compatibility in Python. A separate (but
related) issue. A lot of the XML libs in core Python are a fuster
cluck and that is mostly acknowledged. However the ElementTree
(xml.etree in the stdlib) and lxml (not in the core, ElementTree API
using libxml2) libs for working with XML are awesome... and also what
I gather most Python-land folks use these days.

Trent

--
Trent Mick

Mikeal Rogers

unread,
Nov 19, 2010, 3:28:59 PM11/19/10
to nod...@googlegroups.com
"A lot of the XML libs in core Python are a fuster
cluck and that is mostly acknowledged".replace(' XML'. '');

:)

-Mikeal


--

Marco Rogers

unread,
Nov 19, 2010, 3:34:13 PM11/19/10
to nodejs
I would loooooove for some collaborators to help me make libxmljs more
awesome. It works well for general use right now, but definitely
needs some work.

https://github.com/polotek/libxmljs

I also like the "idea" of e4x but agree that the current spec is
garbage. The only real advantage to it is the syntax is nice when
doing a lot of xml manipulation. But that's only such a big deal
because most of the APIs for dealing with xml are overly verbose
crap. I've come around to the conclusion that xpath is awesome and
you should use it. libxmljs has very good support :)

:Marco


On Nov 19, 2:57 pm, Trent Mick <tre...@gmail.com> wrote:

Mikeal Rogers

unread,
Nov 19, 2010, 3:43:14 PM11/19/10
to nod...@googlegroups.com
Some time in the near future we're going to have to qualify what subset of the JavaScript language node.js supports.

There is already work happening to bind node.js to other JavaScript VMs, those VMs are going to differ from v8 in the features they support. This will cause a compatibility cluster fuck.

Right now, we just support v8 and whatever js features v8 supports. This has been a godsend for compatibility and bike shedding. We haven't had to think about what is going on in the js language and the battles that are happening between VMs and language features being added to Harmony.

But when people start binding to other VMs, and if those get traction in the community, it opens up the very real consequence that two modules written for node will not be interoperable between the same node version on different VMs.

I think once this happens we need to draw a line in the sand and say "this is the JavaScript node supports" and get compile flags in to other VMs that disable features that don't fit that subset. Otherwise we'll have a very fragmented module community.

Languages like Ruby, Python, and Java get to iterate their language and vm along with the platform and library authors can say "we don't support version before x", but we won't have that in node.js because the version of node and the version of the VM might differ in the future (currently, not a problem as the version of v8 is locked to a version of node.js).

This is a lot of hand waving, i know, but there is an effort to get other VMs on node.js primarily because node has traction and it could be used as a vehicle for new language version (see Harmony) and VM adoption. But this will happen at the cost of compatibility within the community which I don't think is a positive thing for us. I don't want to see us fragment all of our effort and devolve in to arguments about language features and VM wars.

-Mikeal

Stephen Belanger

unread,
Nov 19, 2010, 3:48:49 PM11/19/10
to nod...@googlegroups.com
I think if node.js eventually uses multiple VMs, people will just start to refer to them as such, saying "this module works with V8 node" and such. This thread is getting rather derailed though, so if you want to talk about this, you should probably take it to a new thread.

Liam

unread,
Nov 19, 2010, 5:43:52 PM11/19/10
to nodejs
Having used E4X in a FF-only project, I concur that XML literals are a
big advantage.

Allowing preprocessor plugins which run before compile seems pretty
valuable... Embedded SQL anyone? Macros? Is this evil per se?

A minimal XML preprocessor could simply validate and transform XML
literals into objects with a simple internal structure.
var a = <div style="stuff">things</div>
becomes
var a = new XML( { type:'el', tag:'div', attr:{style:'stuff'},
content:[ {type:'txt', text:'things'} ] } )

Any number of modules could then provide the XML type, so you can pick
your fave api. If you desire E4X, you can invoke a second
preprocessor.

Marcel Laverdet

unread,
Nov 19, 2010, 8:50:25 PM11/19/10
to nod...@googlegroups.com
Sorry I seem to have struct a chord with a lot of you, both positively and negatively. Let me take some time to address some of the concerns raised in this thread.

I feel that the useful parts of E4X are a subset of the mammoth ECMA-357 spec. XML software will generally focus on either production or consumption of XML. In this thread I've heard musings of software for consuming XML, i.e. node-o3-xml, libxml2, xpath, and so on. The other family of XML software are libraries which produce XML, i.e. YUI + jsdom, or whatever else exists. The niche I'm looking to fill is the production of XML (or actually HTML). I'm not concerned with using E4X for parsing and inspecting XML. You can't use SASS to create a webpage, and JS DOM implementations are highly cumbersome.

Most people who have worked with XML literals in a web application before will rave about their utility. When combined with a framework which lets you decompose abstract elements into raw HTML you're left with a very powerful and expressive front-end tool. For example, imagine being able to write markup such as:

  <ui:tab-bar>
    <ui:tab href="/home">Home</ui:tab>
    <ui:tab href="/links">Links</ui:tab>
    <ui:tab href="/about">About</ui:tab>
  </ui:tab-bar>

..which would then be decomposed into some HTML before it's sent to the user. Imagine, also, not having to ever worry about XSS holes. By making XML literals a part of the language you are given the ability to perform context-specific escaping automatically. I dare you, go to the NodeJS Knockout entry page (http://nodeknockout.com/teams) and pick a random application. Type "<script>alert(1)</script>" into every text field and watch what happens. XSS will be a problem on this platform, much like it is on other web platforms. With XML literals you can avoid the brunt of these attacks, by making security the default rather than the exception.

I'm well aware that E4X hasn't seen the adoption its creators would have wished for. Outside of ActionScript, there's no ubiquitous platform where using E4X is a possibility. It was clear to me that E4X was on the way to the grave, but when I wanted to add XML literals to JavaScript I opted to build off existing work instead of starting from scratch. If I were to implement XML literals in the absence of E4X I can promise you it wouldn't look much like E4X.

On the topic of whether or not this belongs in node or v8, I think the answer for now is definitely "no". I'd really like to see the E4X spec revisited with less of a focus on consuming XML and more of a focus on producing XML before that happens. Perhaps with e4x-bump I could explore the minimum subset of E4X which is actually reasonable.

Anyway, Mikeal, please at least look at the project for its usefulness instead of trying to tell others what they need or don't need. I'll be continuing to work on it regardless of whether or not you think it's a good idea, because I'm confident that it will be useful to me. If anyone wants to be involved in the development or decision making process please feel free to contact me! Thanks for the support guys!

Preston Guillory

unread,
Nov 19, 2010, 9:52:27 PM11/19/10
to nod...@googlegroups.com
Good luck with the project. HTML/XML generation (somehow) still
doesn't feel like a solved problem, and it's nice to see more options
being put out there.

Jonathan Chetwynd

unread,
Nov 20, 2010, 12:30:31 PM11/20/10
to nod...@googlegroups.com

On 19 Nov 2010, at 18:15, Mikeal Rogers wrote:

> and [I] can process xml like normal people.

am I missing something?

afaict redis processes json natively, but is not currently a native
xml database such as eXist for instance.

ie there is no simple and direct way to parse xml, use XQuery, Xpath...
and xml is effectively stored as a blob.

for the present I'm content to save as json and add an xml wrapper and
use an xslt transform,

but that's not to say that I wouldn't prefer the option of a native
xml database,
even one that stripped and wrapped behind the screen.

regards

Jonathan Chetwynd
peepo.com

Mikeal Rogers

unread,
Nov 21, 2010, 12:41:45 PM11/21/10
to nod...@googlegroups.com
You're using two definitions of "native".

JavaScript, and most languages nowadays, ship with an optimized (usually C based) JSON parser/serializer.

They do not however have "JSON literals" as part of the language.

Lots of languages also ship with native XML parsers, and I'm all for that, but those parsers are libraries you use not literals in the language.

There is a big difference between adding a modules or a new object in the global namespace and adding new syntax to the language that will break the parser when not supported.

-Mikeal

Isaac Schlueter

unread,
Nov 21, 2010, 1:12:42 PM11/21/10
to nod...@googlegroups.com
Mikeal,

({ "json" : "might"
, "not be" : "the best example"
, "of" : ["the", "point", "you're", "trying", "to", "make"] })

We had "JSON Literals" long before we had JSON.parse().

--i

Mikeal Rogers

unread,
Nov 21, 2010, 1:14:09 PM11/21/10
to nod...@googlegroups.com
There are small things that will cause it not to parse.

But yes, this is a side of effect of JSON being developed a subset of javascript.

It also nearly parses as valid Python except for a few gotchas.

-Mikeal
Reply all
Reply to author
Forward
0 new messages