Re: [wro4j] Extending Wro4J - Compass (extensions to SASS), JSDoc, QUint and AMD dependency management

[Alex Objelean]

Hi Ed,

sorry for a delayed response. I promise to write a detailed email to
all of your questions by the end of the week as I have limeted time
and intrnet connection during this period.

Thanks
Alex

On 23/07/2012, Ed <ed.sl...@googlemail.com> wrote:
> I've come across Wro4j and am very impressed, thanks to Alex and others!
>
> There are a few things that would be useful additions for
> me... although arguably some of these don't immediately sound like they
> should be Processors, or at least processors that you'd ever want to run at
>
> runtime. I wondered whether an additional annotation to signify a processor
>
> that should only operate at build time might make sense.
>
> The following are all features that'd be useful for me:
>
> Note: I have considered the below processors because I understand that this
>
> is conventional for this library, given that there is a JsLintProcessor
> class. JS lint doesn't really strike me as something that would every be
> usefully run at runtime, again, maybe there's an argument for a build time
> annotation..?
>
> - AMD <https://github.com/amdjs/amdjs-api/wiki/AMD> JavaScript dependency
> management: I've already come across quite a few discussions with Alex and
> others regarding the handling and optimization of AMD defined
> JavaScript modules and their dependencies. I guess really all I want to
> achieve here is expressing my interest in also seeing this feature
> supported soon. Is something already out there? Alex; I think you mention
> that r.js is tricky because it requires Rhino to evaluate which is
> considered a bottleneck, but from scanning the code there are quite a
> few processors that call upon Rhino, e.g. UglifyJsProcessor. It'd only need
>
> to be performed once assuming static files have not changed, or have I
> misunderstood what you were talking about when you said it was a bottleneck
>
> (apologies, I can't remember where I read this).
>
> - Compass <http://compass-style.org>, an extension to SASS: Having had a
> look at the source of the RubySassCssProcessor, I can see that it should be
>
> fairly simple to generate a script that uses the Compass extension
> instead...unfortunately my Ruby skills / Compass understanding is fairly
> minimal so currently I haven't got anywhere. Has anyone else got support
> for Compass? What does the generated Ruby script look like?
>
> - Processor to run QUnit <http://qunitjs.com> tests: As I say above, maybe
> the reason there is no processor for this is because it's not really suited
>
> to being a processor...but for completeness' sake, maybe it should be? What
>
> are people's thoughts on processors that would never make sense to be run
> at runtime? Does a QUint test running processor exist out there? Or are
> there already plans for one?
>
> - JSDoc Toolkit <http://code.google.com/p/jsdoc-toolkit> processor: Again,
> arguably not really suited to a runtime processor but easy to implement,
> are there any plans for a processor? Has this one been avoided because of
> its, inherently build-time use?
>
> Thanks in advance.
>
> --
> You received this message because you are subscribed to the Google Groups
> "wro4j" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/wro4j/-/pdWmFj6OYWAJ.
> To post to this group, send email to wr...@googlegroups.com.
> To unsubscribe from this group, send email to
> wro4j+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/wro4j?hl=en.
>
>

[Ed]

Thanks Alex.

I have an update; I have written a JSDoc Toolkit processor that calls upon code I found licenced under the Apache 2.0 licence (my understanding is that this is fine to include). Hooked that in and it's working nicely. If you're interested it can be found here:

https://github.com/edslocombe/wro4j/commit/223409dbb6821fa9f38513d12fc7b3b5a0ccadca 

See the commit comment for details.

Look forward to hearing back, 
Ed

[Ed]

I've just realised I haven't submitted the latest processor class to the repo linked. I'm away from my computer right now but will add tomorrow-there isnt much to the processor though.

[Alex Objelean]

Hi Ed again,

first of all thank you for a detailed mail, appreciate it. Now back to your questions and suggestions:

1) run-time vs build-time processors - though it is possible, I don't see any benefit of adding this new annotation. Since user can easily specify the processors you want to be applied. Also, it is very unlikely to use the same configuration for run-time and build-time. Still, if you need it, you can easily decorate all of processors and enable or disable them based on some configuration property. I would prefer keep the things as simple as possible. Since there are alternative was to achieve that, I would not vote for adding a build time annotation. 

2) There were many discussions about AMD on this mailing list. You can easily track them by searching this term. Since you also are interested in supporting AMD, I would like to hear your use-case and do you think it can be used. This is important, in order to identify how the AMD support should be implemented. 

   Regarding the rhino being a bottleneck - it is a known fact that rhino is slow. Though there are plenty of processors based on rhino (like lessCss, uglifyJs, coffeeScript, etc), I would keep the rhino usage to a minimum, because it can become unusable for run-time when dealing with a large number of resources. I would rather abstract the AMD support using an interface responsible for analyzing the js dependencies (making it possible to use rhino or just a simple low level parser to identify the dependencies a js resource may have).

3) Compass - I don't have ruby skills either. But I would gladly accept a contribution for Compass support.

4) QUnit - a processor which would run the tests would be great. I'm not sure yet how it would be used, how to define what are the test resources, what should be the behavior in case there are failing tests, etc... The answer to these questions would be helpful to define the feature request and implement it in the future as a dedicated issue.

5) JSDoc Toolkit - this processor would be a nice addition. I also think it would be useful only during build-time. Thanks for starting to work on this one. I was curious if the jsdoc toolkit is available on maven central repo. Also, it would be easier to integrate your changes if you would keep only jsdoc related commits in a separate branch without mixing other features or fixes.

Let me know if you have any other suggestions or questions.

Thanks,

Alex

On Monday, 23 July 2012 12:02:08 UTC+3, Ed wrote:

I've come across Wro4j and am very impressed, thanks to Alex and others!

There are a few things that would be useful additions for me... although arguably some of these don't immediately sound like they should be Processors, or at least processors that you'd ever want to run at runtime. I wondered whether an additional annotation to signify a processor that should only operate at build time might make sense.

The following are all features that'd be useful for me:

Note: I have considered the below processors because I understand that this is conventional for this library, given that there is a JsLintProcessor class. JS lint doesn't really strike me as something that would every be usefully run at runtime, again, maybe there's an argument for a build time annotation..?

- AMD JavaScript dependency management: I've already come across quite a few discussions with Alex and others regarding the handling and optimization of AMD defined JavaScript modules and their dependencies. I guess really all I want to achieve here is expressing my interest in also seeing this feature supported soon. Is something already out there? Alex; I think you mention that r.js is tricky because it requires Rhino to evaluate which is considered a bottleneck, but from scanning the code there are quite a few processors that call upon Rhino, e.g. UglifyJsProcessor. It'd only need to be performed once assuming static files have not changed, or have I misunderstood what you were talking about when you said it was a bottleneck (apologies, I can't remember where I read this).

- Compass, an extension to SASS: Having had a look at the source of the RubySassCssProcessor, I can see that it should be fairly simple to generate a script that uses the Compass extension instead...unfortunately my Ruby skills / Compass understanding is fairly minimal so currently I haven't got anywhere. Has anyone else got support for Compass? What does the generated Ruby script look like?

- Processor to run QUnit tests: As I say above, maybe the reason there is no processor for this is because it's not really suited to being a processor...but for completeness' sake, maybe it should be? What are people's thoughts on processors that would never make sense to be run at runtime? Does a QUint test running processor exist out there? Or are there already plans for one?

- JSDoc Toolkit processor: Again, arguably not really suited to a runtime processor but easy to implement, are there any plans for a processor? Has this one been avoided because of its, inherently build-time use?

Thanks in advance.

[Ed]

Thanks for the reply Alex.

To follow up on your points:

1. Fair enough, not particularly worried
2. AMD is fairly new to me but my team are looking to use it. Because of this I don't feel I'm yet ready to contribute a full answer, however I have been thinking today of a slightly different dependency management approach (see the bottom of this reply)
   
3. I will see what I can do at some point, maybe someone else can help out in the meantime though.
4. Again I will look at implementing, there is already a Maven plugin to run tests, so maybe we can use that as a starting point (see here), in addition through looking at this plugin I discovered phantomjs which is supposedly much quicker than Rhino. Without looking much further I'd presume the downside to it is that it cannot be downloaded as part of Maven, but instead would be a prerequisite on the machine. Perhaps we could check to see if it was installed, if so use it otherwise fallback to Rhino?
5. Ah, good point. I had assumed that JSDoc Toolkit wasn't a in repository based on what the author of the original plugin had done. I will at some point look to depend on that instead (note to myself: remember to lookout for bug 335). I'm new to Git and GitHub so please excuse my ignorance, however I am very familiar with Perforce. I suppose the changes I have submitted to the ejbs-extras branch are all related in effect; I added the foundations of configuration framework that I felt would be helpful for the configuration of processors, which I then used to handle configuration of the JSDoc Toolkit processor. Is there a better way I should be making these changes (that depend on other useful changes)?

Regarding point 2, I have been thinking about an additional wro4j-maven-plugin goal, or two that would allow easy bundling of independent resources (i.e. modularized web resources), that'd then be built as a wro group. My initial thoughts are that I'd configure the directories to resources that'll ultimately be CSS and JS, then the plugin would create a wro-group-fragment.xml file. The module would then be packaged up as a JAR with this kind of structure:

web-button-component.jar
    META-INF/com.example.components/web-button-component/wro-group-fragment.xml
    js/com.example.components/button-model.js
    js/com.example.components/button-view.js
    js/com.example.components/button-dust-template.templ
    css/com.example.components/button.less

With a POM like so:

...
<groupId>com.example.components<groupId>
<artifactId>web-button-component</artifactId>
...
{plugin-config-to-go-here}

And a generated wro-group-fragement.xml like so (all powered off configured Maven resource directories, and what files exist):

<group name='web-button-component'>
    <css>classpath:com.example.components/button.less</css>
    <js>classpath:com.example.components/button-model.js</js>
    <js>classpath:com.example.components/button-view.js</js>
    <js>classpath:com.example.components/button-dust-template.templ</js>
</group>

Once we have our JAR with a wro group model, we can start using it as a dependency to another modularized web resource, and the plugin could then generate a new wro-group-fragment.xml, or complete wro.xml file for the two.

For example, if we then had a table that depends on the button, our POM could look like this:

...
<groupId>com.example.components<groupId>
<artifactId>web-table-component</artifactId>
...
{plugin-config-to-go-here}
...
<dependencies>
    <dependency>
        <groupId>com.example.components</groupId>
        <artifactId>web-button-component<artifactId>
    </dependency>
</dependencies>
...

And, because the plugin finds a JAR with a wro-group-fragment.xml, it references the group in the table component's wro-group-fragment.xml:

<group name='web-table-component'>
    <group-ref>web-button-component</group-ref>
    ...
</group>

Another plugin goal could then be used to collate all of the groups into one wro model. That can then be used as a final complete module build or at runtime.

As I say, this is early days for the idea but I shall be taking a look tomorrow if all goes well. I'd be very interested in your thoughts.

Going back to AMD, I see the above idea as possibly an addition to AMD dependency management, right now I can see them complementing each other. One concern over talks I have read about wro and AMD; there was the mention of AMD usage overriding the need for a wro model, I can't remember where I read it, but think that this couldn't happen because AMD doesn't consider CSS files which I see as a vital part in creating modularized web resources.

Thanks,
Ed

[Ivar]

Ed, It is quite interesting to read your thoughts on how to modularize web-resources using AMD, as this is exactly what I did at my previous customer. 

We ended up defining our plugins as something bigger than just a "button", it can better be described as a widget. The plugin was self responsible for communicating with appropriate data-services. We ended up with a slightly different approach where the plugin itself builds an compressed an optimized version of it JavaScript's (which also includes the HTML-template files). This was done using Dojo's build system (the project was already using dojo). All the plugin files (javascript and CSS) was bundled together as a jar. This made it possible for us to just hot-deploy our plugins inside a "watched" folder. The platform could then, when it discovered a new plugin, unjar the plugin and make it available. Because we was unable to require CSS as "plugin" dependency in a reliable manner (we did experiment a bit with this and it is possible) we ended up using wro4j to just concat and compress all css from the different plugins as plugins.css. Thus we also required the plugins to have correct CSS scoping (but hey, that's something we wanted anyway). For us this was good enough, as we saw that the required CSS for each plugin was very small as we shared a lot of the styling across plugins and the platform. 

The most successful part of our system as I see it was that we was able to break it down to smaller chunks, plugins, that we could load (and unload) based on what the user requested. We still managed to keep the parts big enough so that we only had to load one new file for each plugin/widget. With AMD you easily end up with hundreds of small files. This modularization is good during development, but has a big performance overhead in production (many small files takes longer to load, than a few larger files).

To share smaller pieces of code and CSS between plugins we used a "common.jar" where the different plugins could require common stuff. This bundle was guaranteed to be provided by the platform. The plugins exluded these files from the released plugin bundles. 

And for testing we used JS Test Runner and phantomjs with great success. Phantomjs is a headless webkit browser and made it easy for us to ensure all js-test was excuted as part of the continues integration. The only problem is that it is not a "real" browser, but it us closer than rhino. You could look at Web Driver, cucumber or buster.js if you really need to run tests against real browsers. 

Hope to hear about the final solution!

Regards,

Ivar

--

You received this message because you are subscribed to the Google Groups "wro4j" group.

To view this discussion on the web visit https://groups.google.com/d/msg/wro4j/-/9Xh6IbjMAvMJ.

[Alex Objelean]

Hi Ed, 

I agree with Ivar that it was quite interesting to read your thoughts and suggestions.

Regarding processor handling js unit tests - besides the javascript maven plugin, there are also other projects focusing on this, like jasmine-maven-plugin and other. I'm not sure it is a good idea to "compete" with this kind of plugins. But I do think that it is possible to reuse or integrate some of them to make it possible reuse the model defined with wro4j in order to run js unit tests. 

I have very little experience with phantomjs, but afaik it requires platform dependent libraries, so it is not very easy to create a processor which would run without some sort of extra configuration.. Making it fallback to rhino seems also a very challenging task.

However I wouldn't mind to see an experimental integration of phantomJs as a processor. As always, contributions are welcome.

I am also considering integrating v8 javascript engine as an alternative to rhino (which should boost the performance). But this has the same problem: it is not platform independent and there is no mature java integration which can be reused.

I appreciate the work you've started with integrating JSDoc Toolkit. Don't worry about your git experience, you'll learn it really fast no matter what other VCS you have experience with. If you find it helpful, I've documented the way I do work with git on this project:

http://code.google.com/p/wro4j/wiki/WorkingWithGit . This page describes the used conventions and several useful commands which covers most of use-cases I need. I should make a detailed review of all your changes. Probably I didn't understand everything at the first glance. The foundation of configuration frameworks seems useful, but I would add it as a separate feature (would like to understand it better first).

Your description of modularized web resources with fragments generated by a plugin sounds like a very interesting idea which worth some further prototyping. I would love to see other thoughts like this. This can be one of many other possible approaches for managing web resources of complex projects. I agree that this would be a nice addition complementing AMD. 

You are correct that AMD doesn't consider CSS files to describe dependencies, however this can be emulated using some comments or additional configuration files describing the dependencies of each resource. This wouldn't be a standard though... 

I think the best approach would be to keep it simple, by focusing on a simple use-case and working on a proof-of-concept which could evolve to something really useful. All this requires time and effort. That is why I appreciate any feedback and contribution.

Thanks,

Alex

[Alex Objelean]

Ivar,

your solution also sounds very interesting. Would you mind writing a blog post about it or showing some proof-of-concept project?
I'm pretty sure it would be interesting for a lot of people.

Thanks,

Alex

Ivar

To unsubscribe from this group, send email to wro4j+unsubscribe@googlegroups.com.

[Ivar]

Thanks Alex,

I will present the solution at JavaZone 2012. Based on the preparation
work and the presentation itself, it should be fairly simple to write
a blog-post about it. It's already on my "to-do" list ;)

Ivar

>>> wro4j+un...@googlegroups.com.

>>> For more options, visit this group at
>>> http://groups.google.com/group/wro4j?hl=en.
>>
>>

> --
> You received this message because you are subscribed to the Google Groups
> "wro4j" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/wro4j/-/J4zpjEffINkJ.

>
> To post to this group, send email to wr...@googlegroups.com.
> To unsubscribe from this group, send email to

> wro4j+un...@googlegroups.com.