Functionality and separability

[Chris Toomey]

After looking through the documentation on raptorjs.org and downloading it and playing with the samples, let me say it's a VERY impressive body of work for sure -- wow!

I've spent the last couple weeks exploring the landscape of open-source technology for building web apps with rich client functionality, modular web components/widgets, and support for rendering the initial HTML on the server for fast initial loading/rendering in the browser.  I've looked at FlightJS, Rendr, Krakenjs, LinkedIn's work with Dustjs, Backbone Marionette, the early W3C web component implementations polymer and xtags, etc.  I must say that RaptorJS looks like it easily provides the most complete solution for those needs of everything I've come across so far.

Now on to the questions:

* Raptor looks like it's not designed as an MVC (or MV*) framework per se.  It seems that widgets individually encompass all that functionality on the client side (with Raptor being agnostic as to the decomposition/structuring), and that Raptor doesn't provide page/app-level functionality like client MV* frameworks provide.  On the server side it adds support for Raptor packages/modules and UI components on top of Express but is otherwise agnostic as to how controllers get data and pass them into templates.  Is this an accurate characterization?

* The Raptor client-side pub/sub feature is a nice pattern for decoupling widgets that is also used by Marionette (via Backbone.Wreqr library) and by AuraJS amongst others.  Did you look at using either of those (or others) before building Raptor's?  And is the pub/sub mechanism separable from the rest of the widget framework?  E.g., could another pub/sub library be used instead?

* Is it possible to use a different template library instead of Raptor Templates? Nothing necessarily against those, I'm just curious how much work it'd be to use another template library -- say Dust -- with the other pieces of RaptorJS?

* The AMD/packaging/optimizer/loading functionality looks very complete and rich.  Is that usable independently of the rest of Raptor?

* How widely is Raptor deployed within eBay?  What are some of the pages it serves, and is it used for desktop as well as mobile variants of the site?

* From afar it looks like there's a huge overlap between Raptor and your sister company Paypal's Kraken.  Did they know about and explore Raptor before embarking on Kraken?  Are you guys at all trying to collaborate and converge?

* Why isn't there more activity in this group and traction in general with Raptor outside of eBay?  As awesome as Raptor looks to be that's really surprising!

thanks,

Chris

[Phil Gates-Idem]

Chris,

Excellent questions about RaptorJS. I am using this toolkit where I work (OrgSpan) for both a server-side API and web browser application development. I have also been able to contribute to the RaptorJS code base. I am sure Patrick will be able to give you his perspective but I wanted to try to answer your questions about RaptorJS and give you some insight into how we are using it. 

 

* Raptor looks like it's not designed as an MVC (or MV*) framework per se.  It seems that widgets individually encompass all that functionality on the client side (with Raptor being agnostic as to the decomposition/structuring), and that Raptor doesn't provide page/app-level functionality like client MV* frameworks provide.  On the server side it adds support for Raptor packages/modules and UI components on top of Express but is otherwise agnostic as to how controllers get data and pass them into templates.  Is this an accurate characterization?

RaptorJS is not an MVC framework but it could be used as the foundation to build an MVC framework. One of the goals of RaptorJS was to not make it a framework but instead provide a toolkit. By making it a toolkit you could combine it with other libraries that provide higher order functionality. Some good examples of higher order functionality that come to mind are MVC and Data Binding. I am actually toying with the idea of contributing a Data Binding package that might provide functionality that is similar to EmberJS. Backbone+RaptorJS might be a good combination if you really want an MVC framework right out of the gate. One of the beautiful things about RaptorJS is that you can pick and choose which modules/packages you want to use without having to adopt everything and getting the code bloat that might come along with that. The applications that I have built on the client-side have a router that is essentially the controller. I am using https://github.com/philidem/router-js which is admittedly not at all documented but essentially provides a declarative way of defining routes using a simple JSON convention (I will be documenting it later :). On the server-side, I imagine that you'd want to use Express for your routing. So, yes, I think your characterization is accurate.

* The Raptor client-side pub/sub feature is a nice pattern for decoupling widgets that is also used by Marionette (via Backbone.Wreqr library) and by AuraJS amongst others.  Did you look at using either of those (or others) before building Raptor's?  And is the pub/sub mechanism separable from the rest of the widget framework?  E.g., could another pub/sub library be used instead?

I am not sure of the original inspiration for pubsub since Patrick was the one to first write this module. However, I don't think it's a new pattern so I think it has its origins from both Java and JavaScript development and many different libraries/frameworks. The pubsub module is used by the widget framework but you can also use pubsub independently. It's a great module for both server-side and client-side development. It essentially allows you to decouple presentation from business logic (which is also one of the goals of MVC). In my applications I typically use pubsub to emit events that get handled by the controller/router layer. For example, our Single Page Application also has the concept of showing a user's presence so it was very convenient to use the pubsub module to notify various parts of the application when a user's presence changes.

* Is it possible to use a different template library instead of Raptor Templates? Nothing necessarily against those, I'm just curious how much work it'd be to use another template library -- say Dust -- with the other pieces of RaptorJS?

You can use other templates (since the module loader / packaging only pulls in the parts of RaptorJS that you use). I personally wouldn't do this because I am a huge fan of Raptor Templates and I have grown to love and appreciate them the more that I use them :) You can even write your own tags that generate custom JavaScript which is an advanced feature but very powerful. One of the major advantages of Raptor Templates from a performance perspective is that they get compiled to very efficient JavaScript code but provide an excellent mechanism for reusing/including other renderers/templates. Conditional logic and flow statements compile to their native JavaScript counterparts which you can't do with Dust. Furthermore, to even use conditional logic statements with Dust, you have to include Dust Helpers for this functionality -- I think this design decision was a big mistake because simple conditional statements belong in the template many cases. Raptor Templates also support asynchronous rendering which allows the application to immediately rendering the template when not all of the data is available (Dust provides this as well but I haven't seen this feature in other JavaScript template engines). Raptor Templates works extremely well when the output of the template is XML/HTML. Templating engines such EJS might be better if your output is some other format (but I hardly ever encounter this use case).

* The AMD/packaging/optimizer/loading functionality looks very complete and rich.  Is that usable independently of the rest of Raptor?

You can certainly use these features independently. I would also venture to say that you should use the optimizer for any application that you are developing -- especially when want to load bundles asynchronously or you need checksums on all of your files. Even if you are using a heavy framework such as EmberJS then you could utilize package.json files to describe the dependencies within your application's code base. During development, the optimizer can build your index.html file that will automatically include all of the JavaScript and CSS dependencies as individual includes (which is great during development and debugging). During production, you'd change a parameter to the optimizer that would cause it to bundle and minify your applications code.

* How widely is Raptor deployed within eBay?  What are some of the pages it serves, and is it used for desktop as well as mobile variants of the site?

I can't answer the question about adoption within eBay but I can tell you that I am using it at OrgSpan to build desktop and mobile web applications. It also helped me build a very efficient Cordova/PhoneGap application (which we nearly got for free). I can tune the build for my applications so that I create a highly optimized build for the target platform. For example, I can enable certain extensions that only get compiled into my application when targeting Android with Cordova versus iOS.

* From afar it looks like there's a huge overlap between Raptor and your sister company Paypal's Kraken.  Did they know about and explore Raptor before embarking on Kraken?  Are you guys at all trying to collaborate and converge?

I don't work at eBay or PayPal but I do know that the two teams will be collaborating. I had the opportunity to talk to members from both teams at a recent conference.

* Why isn't there more activity in this group and traction in general with Raptor outside of eBay?  As awesome as Raptor looks to be that's really surprising!

Currently, I among the few outside eBay using RaptorJS. However, I will be helping with promoting it and fostering a community. I know that we will be using it regardless but I would love to see it gain wider adoption. I strongly believe that the reason it is not more popular right now is simply because people don't know about it. Patrick has done an excellent job designing and implementing Raptor and we both have battle-tested it at our respective companies so I am confident that others would find it useful. The learning curve might be a little higher than other all-in-one frameworks but once you learn it I think it is very powerful and productive.

Thanks again for your feedback and excellent questions. I would be happy to go into more details about how I am specifically using RaptorJS and the applications I am building if you'd like.

-Phil

[Patrick Steele-Idem]

Hi Chris,

Thanks for taking the time to take a deep-dive into RaptorJS and all of the other open source libraries! Phil did a great job answering many of your questions, but I will also give my perspective (even if it is redundant)...

* Raptor looks like it's not designed as an MVC (or MV*) framework per se.  It seems that widgets individually encompass all that functionality on the client side (with Raptor being agnostic as to the decomposition/structuring), and that Raptor doesn't provide page/app-level functionality like client MV* frameworks provide.  On the server side it adds support for Raptor packages/modules and UI components on top of Express but is otherwise agnostic as to how controllers get data and pass them into templates.  Is this an accurate characterization?

[PATRICK] That is an accurate characterization. RaptorJS provides support for some patterns that we think simplify how complex pages are built (specifically declarative dependency management, dependency loading, pubsub, parent/child widget communication, separating out rendering from client-side behavior and binding behavior to rendered UI components). While RaptorJS offers patterns to solve the problems mentioned, you can still mix and match based on what you think is appropriate for your app. We wanted to leave things open because the technology landscape is changing at a fast pace (e.g. web components, EcmaScript 6, etc.) and we didn't want to place restrictions that would completely invalidate the toolkit in the future.

* The Raptor client-side pub/sub feature is a nice pattern for decoupling widgets that is also used by Marionette (via Backbone.Wreqr library) and by AuraJS amongst others.  Did you look at using either of those (or others) before building Raptor's?  And is the pub/sub mechanism separable from the rest of the widget framework?  E.g., could another pub/sub library be used instead?

[PATRICK] The "raptor/pubsub" module is a tiny module within the RaptorJS toolkit and it is optional and independent. Personally, I wrote "raptor/pubsub" because I think it offered a cleaner way to subscribe and unsubscribe to topics and I also wanted to add support for channels to avoid crosstalk in complex pages. Possibly more importantly, I was only interested in a pubsub solution that was completely independent from the DOM (better performance and simplicity).

* Is it possible to use a different template library instead of Raptor Templates? Nothing necessarily against those, I'm just curious how much work it'd be to use another template library -- say Dust -- with the other pieces of RaptorJS?

[PATRICK] It's possible to use RaptorJS with any popular templating language that supports JavaScript helpers. However, the asynchronous custom tags only work for Raptor Templates and Dust since they are the only asynchronous JavaScript templating languages (NOTE: the optimizer custom tags/helpers are all asynchronous). I introduced a simple open source layer for binding RaptorJS custom tags to Dust by adapting the JavaScript API for Dust (chunks and async chunks) to RaptorJS (writers and async fragments):

https://github.com/cubejs/raptor-dust/

If you want more details on the "raptor-dust" module please let me know.

* The AMD/packaging/optimizer/loading functionality looks very complete and rich.  Is that usable independently of the rest of Raptor?

 

[PATRICK] Absolutely! the RaptorJS optimizer, packaging and loader can all be used independently and with no restrictions on a particular templating language or build/runtime setup. The only caveat is that the client-side loader does require the use of the lightweight RaptorJS AMD implementation (not RequireJS).

* How widely is Raptor deployed within eBay?  What are some of the pages it serves, and is it used for desktop as well as mobile variants of the site?

[PATRICK] The client-side loader, widget framework and packaging functionality of RaptorJS are deployed widely within eBay because they were built first. You will find RaptorJS in pretty much all apps built within the last ~2 years on the latest eBay platforms. Raptor Templates is also extensively used, but less so because it was developed much later and could not be easily used on the server-side of the JVM stack since that requires the Rhino JavaScript engine (slower). However, on eBay's Node.js stack (i.e. CubeJS), Raptor Templates is being used by all of the teams and there are >10 different CubeJS apps being developed. The following pages are being served up by *Node.js* in production and use all of the features of RaptorJS:

• http://www.ebay.com/cln
  
• http://www.ebay.com/cln/dnw555/Great-Board-Games/58950112016
  
• http://www.ebay.com/bit/comics/
  

(there are more pages, but those pages are the easiest to link to)

You'll find RaptorJS being used on about 70% of the eBay pages (homepage, search, view item and including both JVM and Node.js stacks). It's outside eBay where we would like to see more adoption :)

* From afar it looks like there's a huge overlap between Raptor and your sister company Paypal's Kraken.  Did they know about and explore Raptor before embarking on Kraken?  Are you guys at all trying to collaborate and converge?

[PATRICK] Yes, we did have some discussions along the way. I'm not going to speculate as to why exactly the Kraken team decided not to utilize certain features of RaptorJS, but there is an effort to converge and collaborate on the Node.js stacks now that both stacks have proven themselves.

* Why isn't there more activity in this group and traction in general with Raptor outside of eBay?  As awesome as Raptor looks to be that's really surprising!

[PATRICK] I will speculate as to why I believe the adoption outside eBay has not been so great:

• Perception problems:
  
• "RaptorJS adds complexity"
• "RaptorJS is only needed if performance is a top concern"
• "RaptorJS is a monolithic framework"
• NOTE: I don't believe any of those arguments, but that is what I hear :)
• RaptorJS tries to solve "unsexy" problems (e.g. dependency management)
• RaptorJS is not tied to a fancy UI toolkit (such as Bootstrap)
• MV* and Bootstrap has consumed everyone for the last few years
• People use what they are familiar with. For example:
• jQuery selectors!
• Build time tools for resource optimization (e.g. Grunt tasks)
• NOTE: Very few (if any) companies are doing runtime resource optimization. eBay has been doing runtime resource optimization for years and it is really powerful
• For Raptor Templates specifically, there is an aversion to XML (even though HTML is basically XML... I don't get it)
• For the async loader, people just use RequireJS because it doesn't require any server-side piece (until you try to optimize your app...)
• NOTE: Not having a full featured server-side piece is the problem I have with RequireJS... that, and it doesn't use declarative dependencies (dependencies are inside JavaScript code and only JavaScript code). What about non-JS modules with dependencies?
• eBay doesn't have a team dedicated to promoting RaptorJS outside eBay and I tend to focus more on coding than marketing

Eventually I think we will get there. The tides have changed significantly within eBay. Initially developers were very skeptical of developing and using a new JavaScript/UI toolkit (even if it was open source), but now I am seeing a lot of interested in supporting and using RaptorJS. Also, RaptorJS is much more at home on the Node.js stack because it offers a full end-to-end JavaScript platform that we did not have on the JVM stack.

On a related note, eBay has a very active internal discussion forum for all eBay technologies. Most RaptorJS questions end up being posted to the internal discussion forum instead of the public discussion forum.

With that said, if you find RaptorJS interesting we always appreciate when people share it with others.

Thanks again for your interest and please let us know if you have any other questions.

--Patrick

[Chris Toomey]

Thanks a ton Phil and Patrick for your lengthy replies.  I'll be sharing this with the others on our team here and I'm sure will have more questions to come.

cheers,

Chris

On Friday, December 6, 2013 10:48:43 AM UTC-8, Patrick Steele-Idem wrote:

Hi Chris,

Thanks for taking the time to take a deep-dive into RaptorJS and all of the other open source libraries! Phil did a great job answering many of your questions, but I will also give my perspective (even if it is redundant)...

* Raptor looks like it's not designed as an MVC (or MV*) framework per se.  It seems that widgets individually encompass all that functionality on the client side (with Raptor being agnostic as to the decomposition/structuring), and that Raptor doesn't provide page/app-level functionality like client MV* frameworks provide.  On the server side it adds support for Raptor packages/modules and UI components on top of Express but is otherwise agnostic as to how controllers get data and pass them into templates.  Is this an accurate characterization?

[PATRICK] That is an accurate characterization. RaptorJS provides support for some patterns that we think simplify how complex pages are built (specifically declarative dependency management, dependency loading, pubsub, parent/child widget communication, separating out rendering from client-side behavior and binding behavior to rendered UI components). While RaptorJS offers patterns to solve the problems mentioned, you can still mix and match based on what you think is appropriate for your app. We wanted to leave things open because the technology landscape is changing at a fast pace (e.g. web components, EcmaScript 6, etc.) and we didn't want to place restrictions that would completely invalidate the toolkit in the future.

* The Raptor client-side pub/sub feature is a nice pattern for decoupling widgets that is also used by Marionette (via Backbone.Wreqr library) and by AuraJS amongst others.  Did you look at using either of those (or others) before building Raptor's?  And is the pub/sub mechanism separable from the rest of the widget framework?  E.g., could another pub/sub library be used instead?

[PATRICK] The "raptor/pubsub" module is a tiny module within the RaptorJS toolkit and it is optional and independent. Personally, I wrote "raptor/pubsub" because I think it offered a cleaner way to subscribe and unsubscribe to topics and I also wanted to add support for channels to avoid crosstalk in complex pages. Possibly more importantly, I was only interested in a pubsub solution that was completely independent from the DOM (better performance and simplicity).

* Is it possible to use a different template library instead of Raptor Templates? Nothing necessarily against those, I'm just curious how much work it'd be to use another template library -- say Dust -- with the other pieces of RaptorJS?

[PATRICK] It's possible to use RaptorJS with any popular templating language that supports JavaScript helpers. However, the asynchronous custom tags only work for Raptor Templates and Dust since they are the only asynchronous JavaScript templating languages (NOTE: the optimizer custom tags/helpers are all asynchronous). I introduced a simple open source layer for binding RaptorJS custom tags to Dust by adapting the JavaScript API for Dust (chunks and async chunks) to RaptorJS (writers and async fragments):

https://github.com/cubejs/raptor-dust/

If you want more details on the "raptor-dust" module please let me know.

* The AMD/packaging/optimizer/loading functionality looks very complete and rich.  Is that usable independently of the rest of Raptor?

 

[PATRICK] Absolutely! the RaptorJS optimizer, packaging and loader can all be used independently and with no restrictions on a particular templating language or build/runtime setup. The only caveat is that the client-side loader does require the use of the lightweight RaptorJS AMD implementation (not RequireJS).

* How widely is Raptor deployed within eBay?  What are some of the pages it serves, and is it used for desktop as well as mobile variants of the site?

[PATRICK] The client-side loader, widget framework and packaging functionality of RaptorJS are deployed widely within eBay because they were built first. You will find RaptorJS in pretty much all apps built within the last ~2 years on the latest eBay platforms. Raptor Templates is also extensively used, but less so because it was developed much later and could not be easily used on the server-side of the JVM stack since that requires the Rhino JavaScript engine (slower). However, on eBay's Node.js stack (i.e. CubeJS), Raptor Templates is being used by all of the teams and there are >10 different CubeJS apps being developed. The following pages are being served up by *Node.js* in production and use all of the features of RaptorJS:

• http://www.ebay.com/cln
  
• http://www.ebay.com/cln/dnw555/Great-Board-Games/58950112016
  
• http://www.ebay.com/bit/comics/
  

(there are more pages, but those pages are the easiest to link to)

You'll find RaptorJS being used on about 70% of the eBay pages (homepage, search, view item and including both JVM and Node.js stacks). It's outside eBay where we would like to see more adoption :)

* From afar it looks like there's a huge overlap between Raptor and your sister company Paypal's Kraken.  Did they know about and explore Raptor before embarking on Kraken?  Are you guys at all trying to collaborate and converge?

[PATRICK] Yes, we did have some discussions along the way. I'm not going to speculate as to why exactly the Kraken team decided not to utilize certain features of RaptorJS, but there is an effort to converge and collaborate on the Node.js stacks now that both stacks have proven themselves.

* Why isn't there more activity in this group and traction in general with Raptor outside of eBay?  As awesome as Raptor looks to be that's really surprising!

[PATRICK] I will speculate as to why I believe the adoption outside eBay has not been so great:

• Perception problems:
  
• "RaptorJS adds complexity"
• "RaptorJS is only needed if performance is a top concern"
• "RaptorJS is a monolithic framework"
• NOTE: I don't believe any of those arguments, but that is what I hear :)
• RaptorJS tries to solve "unsexy" problems (e.g. dependency management)
• RaptorJS is not tied to a fancy UI toolkit (such as Bootstrap)
• MV* and Bootstrap has consumed everyone for the last few years
• People use what they are familiar with. For example:
• jQuery selectors!
• Build time tools for resource optimization (e.g. Grunt tasks)
• NOTE: Very few (if any) companies are doing runtime resource optimization. eBay has been doing runtime resource optimization for years and it is really powerful
• For Raptor Templates specifically, there is an aversion to XML (even though HTML is basically XML... I don't get it)
• For the async loader, people just use RequireJS because it doesn't require any server-side piece (until you try to optimize your app...)
• NOTE: Not having a full featured server-side piece is the problem I have with RequireJS... that, and it doesn't use declarative dependencies (dependencies are inside JavaScript code and only JavaScript code). What about non-JS modules with dependencies?
• eBay doesn't have a team dedicated to promoting RaptorJS outside eBay and I tend to focus more on coding than marketing

Eventually I think we will get there. The tides have changed significantly within eBay. Initially developers were very skeptical of developing and using a new JavaScript/UI toolkit (even if it was open source), but now I am seeing a lot of interested in supporting and using RaptorJS. Also, RaptorJS is much more at home on the Node.js stack because it offers a full end-to-end JavaScript platform that we did not have on the JVM stack.

On a related note, eBay has a very active internal discussion forum for all eBay technologies. Most RaptorJS questions end up being posted to the internal discussion forum instead of the public discussion forum.

With that said, if you find RaptorJS interesting we always appreciate when people share it with others.

Thanks again for your interest and please let us know if you have any other questions.

--Patrick

On Thursday, December 5, 2013 8:11:14 PM UTC-8, Chris Toomey wrote:

[Senthil Padmanabhan]

Hi Chris,

I am a frontend engineer at eBay and have been using RaptorJS for quiet sometime now. Recently I started using the Raptor Widget framework for one of my projects and was pleasantly surprised by its smart implementation. Mine was a client side heavy application and I was thinking of using Backbone or other MV* frameworks on top of Raptor widgets. But ultimately I ended up using none, as Raptor widgets took care of everything. Given below are some of the points that I want to highlight based on my experience

1. Any complex web application should first be broken into small modules as much as possible and all these module should be assembled to build the final application. Raptor widgets provide a great mechanism to build these independent modules

2. Raptor widgets provide DOM based encapsulation to modules. This is similar to a Backbone View's el attribute, where all event delegation and DOM action happens within the root elements scope. This way all Raptor widget modules can have its own root DOM element and all associated functionalities kept within. Also Raptor widgets provide a much efficient way of accessing the child elements of the root node. Module encapsulation at its best

3. Communication between the modules happen through the Raptor pub/sub mechanism

4. Packaging of these modules are done using the Raptor optimizer's package.json format. This provided a way to make our modules platform independent (Node.js or Java stack) and rendering (Client or server side) independent. Also distribution of modules is painless now, as all you need is a package.json to tell what files make up your module and what other modules it is dependent on

5. We had scenarios to render the page both server (SEO reasons) and client side. Raptor templates provided an easy mechanism to do the same. The same templates were rendered server side if needed without any additional changes

6. Raptor widgets along with jQuery powered up our modules to handle all complex functionality including AJAX requests, model sync-up, event binding etc.

7. Another beauty of Raptor widgets/optimizer is that, when rendering a page server side it aggregates all module initialization code towards the end of the page. This makes the page render fast and progressively enhance with all event binding happen towards the end. On the other hand when doing client side rendering, the widget frameworks renderer module takes care of binding the model to the template, generating the markup, appending the markup to DOM and finally initializing the module to bind events (all above in the correct order). All this happens magically with just one line of code

8. The De-dupe feature of the Raptor optimizer is very helpful, as a lot of modules dependent on the same low level modules and the optimizer makes sure just one is included in the page

-Senthil

[Chris Toomey]

Thanks for sharing, Senthil, it's good to hear how its features worked well for you.

Chris

--
You received this message because you are subscribed to a topic in the Google Groups "RaptorJS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/raptorjs/CdIDHQbuSY8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to raptorjs+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.