How to embed generated group names in HTML files programmatically?

Tim Bessie

unread,

Feb 2, 2012, 8:52:46 PM2/2/12

to wro4j

Hi all!

So we're looking for a way to do some cache-busting through naming all
combined resources based on their MD5 hash (or something along those
lines).

Currently, combined resources are run through the YUI compressor and
named myCombinedResource_productReleaseVersion.js (for example).

References in JSPs to these sources are made using JSTL of the form:
<script src="abc_${productReleaseVersion}.js"/> (for example).

The problem with this is, for every release, every combined resource
must be reloaded by users. What we'd like to do is have each resource
be named abc_CHECKSUM.js, which CHECKSUM is the checksum of the file.
This way, only files that have changed since the last release would
have different names, and force the browser to reload them. I'm
assuming here that not all browser/server setups will properly check
to see if the timestamp on the resource has changed, so this name
change will force the reload.

The problem with this is, how to embed the value of CHECKSUM for any
given generated/combined resource (since they will each be different
per file).

Can wro4j do this? The examples all seem to depend on including
resources based on unchanging, known names.

- Tim

Alex Objelean

unread,

Feb 6, 2012, 1:27:42 PM2/6/12

to wr...@googlegroups.com

Are you looking for a runtime or build-time solution?

Currently the cache-busting solution is supported for build-time solution only (maven plugin).

For a runtime solution, you have to do some coding on your own, until this feature is available out of the box (need time or contributions to complete this one).

In any case, I can provide you with some support regarding how you can use it. Just describe your use-case in more details.

Cheers,

Alex

- Tim

--
You received this message because you are subscribed to the Google Groups "wro4j" group.
To post to this group, send email to wr...@googlegroups.com.
To unsubscribe from this group, send email to wro4j+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/wro4j?hl=en.

Tim Bessie

unread,

Feb 17, 2012, 1:32:25 PM2/17/12

to wro4j

Thanks for your help - well, I suppose it depends upon what you mean
by run-time vs build time.

The JavaScript and CSS file combining/crunching/naming is done at
build time, but looking up the name of each of these files is done at
runtime.

Currently, using the Maven plugin we're using (along with the YUI
compressor), we're just having a single variable that creates a
timestamp, and combined/compressed files are named myFile-
timestamp.ext, so all places in the runtime code that needs to
reference the file can reference it as myFile-${timestamp}.ext.

Since what we'd like to do is to have each file (myFile1.ext,
myFile2.ext, etc) be named differently (i.e. myFile1-${myFile1-
checksum}.ext and $myFile2-${myCompressedFile-checksum}.ext), we'd
need to keep a table (or perhaps wro4j does this automatically) that
maps myFile1.ext to its name-with-checksum and myFile2.ext to ITS
name- with-checksum and make that lookup available to our code, in
JSPs, etc.

All file combining, compressing, and naming is done at build time.
All we need to know at runtime is what each new name is for each file
(they will all be different). Currently, we have something like this,
as an example:

.../js/javascriptFile1.js
.../js/javascriptFile2.js
...
.../js/javascriptFileN.js

These are combined and compressed (using YUI compressor maven plugin)
at build time into various combinations for various pages:

.../js/combinedJSFile1-timestamp.js
.../js/combinedJSFile2-timestamp.js
...
.../js/combinedJSFileN-timestamp.js

where 'timestamp' is the same for all files.

We have JSP's that include references like: the following (we're using
JSTL to make data model variables available to our JSP pages):

... src="/js/combinedJSFile1-${timestamp}"

and we have Spring MVC code that adds the munged filenames to list of
JavaScript and CSS files to be included on the next page to be served
(with the value of 'timestamp' available to this code in order to do
this). This timestamp is placed into a properties file to be
accessible to the JSPs and code.

Instead of the above, we'd like to be able to generate files like this
at build time:

.../js/combinedJSFile1-checksum1.js
.../js/combinedJSFile2-checksum2.js
...
.../js/combinedJSFileN-checksumN.js

where 'checksum' is the DIFFERENT for all files. We will need JSP's
that include references like:

... src="/js/combinedJSFile1-${checksums['combinedJSFile1']}"

(or something along those lines) and also have that mapping available
to the Spring MVC code when generating its inclusion lists. This
hints at some properties file that contains this mapping, and which is
generated by your maven plugin, and is read in at server start time so
that it can be referenced by the app.

Sorry for the long description - I just wanted to make it very clear
what I'm trying to do. :-)

- Tim

On Feb 6, 10:27 am, Alex Objelean <alex.objel...@gmail.com> wrote:
> Are you looking for a runtime or build-time solution?
> Currently the cache-busting solution is supported for build-time solution
> only (maven plugin).
> For a runtime solution, you have to do some coding on your own, until this
> feature is available out of the box (need time or contributions to complete
> this one).
>
> In any case, I can provide you with some support regarding how you can use
> it. Just describe your use-case in more details.
>
> Cheers,
> Alex
>

Alex Objelean

unread,

Feb 17, 2012, 2:03:46 PM2/17/12

to wr...@googlegroups.com

You can achieve this with groupNameMappingFile property:

Extracted from mavenPlugin wiki page:

groupNameMappingFile - (available since 1.4.0) is an optional parameter. When provided, it should be the path to the properties file where the mapping between original group names and renamed(by NamingStrategy) are described. The generated file can be used to identify the name of aggregated bundle name.

The file will hold the mapping between original (predictable) bundle name (ex: group1.js) and the name of the file renamed using one of the fingerprint encoder strategy. This allows client code to include the right resource into the final html and use far expiring headers, thus achieving aggressive resource caching.

Example of the mapping file content:

all=all-asdf832323

common=common-njj3n128

util=util-21827391jhd

Loading this file from your application, you can easily retrieve the new resource name (containing checksum encoded).

Let me know if you need more help.

Cheers,

Alex

Tim Bessie

unread,

Feb 17, 2012, 3:05:12 PM2/17/12

to wro4j

Thanks Alex! I shall attempt to put this to work and let you know if
I encounter any difficulties!

- Tim

On Feb 17, 11:03 am, Alex Objelean <alex.objel...@gmail.com> wrote:
> You can achieve this with groupNameMappingFile property:

> Extracted from mavenPlugin wiki page<http://code.google.com/p/wro4j/wiki/MavenPlugin>
> :
>
> *groupNameMappingFile* - (available since 1.4.0) is an optional parameter.

> When provided, it should be the path to the properties file where the

> mapping between original group names and renamed(by NamingStrategy<http://code.google.com/p/wro4j/wiki/OutputNamingStrategy>)

Tim Bessie

unread,

Feb 17, 2012, 3:41:30 PM2/17/12

to wro4j

By the way - since we are doing the naming at build time using WRO4J
itself, how are we to create this mapping file?
It sounds like we have to create it ourself, but it's not a separate
step from WRO4J... that is, I can't quite tell from the documentation
(perhaps I haven't read enough of it yet) if I can tell WRO4J to
generate these files AND generate the mapping file, as an integrated
part of the build (since we won't know ahead of time what these names
will be).

I'm still experimenting, but just wanted to check - do we need to
create this mapping file ourselves as a separate step, or does the
WRO4J maven plugin do that for us? My goal would be for wro.xml to
say "here's a group, name it according to this naming strategy, AND
add the mapping to this file".

- Tim

On Feb 17, 11:03 am, Alex Objelean <alex.objel...@gmail.com> wrote:

> You can achieve this with groupNameMappingFile property:

> Extracted from mavenPlugin wiki page<http://code.google.com/p/wro4j/wiki/MavenPlugin>
> :
>
> *groupNameMappingFile* - (available since 1.4.0) is an optional parameter.

> When provided, it should be the path to the properties file where the

> mapping between original group names and renamed(by NamingStrategy<http://code.google.com/p/wro4j/wiki/OutputNamingStrategy>)

Alex Objelean

unread,

Feb 17, 2012, 3:47:13 PM2/17/12

to wr...@googlegroups.com

Currently, maven plugin doesn't create a file. It just write the mappings to the specified location. If you think creating the file if this doesn't exist is useful, I can fix this for the next release.

The rest works exactly as you described it.

Alex

Tim Bessie

unread,

Feb 17, 2012, 4:00:34 PM2/17/12

to wro4j

Hi again...

A bit confused... is groupNameMappingFiled READ by the plugin, or
CREATED by the plugin?

The doc says:

groupNameMappingFile - (available since 1.4.0) is an optional

parameter. When provided, it should be the path to the properties file
where the mapping between original group names and renamed(by

NamingStrategy) are described. The generated file can be used to

identify the name of aggregated bundle name.

so I'm a little unsure of if "the generated file" means a file that
wro4j creates at the given location which contains the mappings it
itself is creating (and then can be used at runtime), or if it
contains a mapping file that I have to create, and which it uses for
reference.

Also, I just realized another several needs of ours:

1. All CS and JSS files need to be copied to {targetLocation}/
originalName-${checksum}.ext
2. The file in #1, under it's new name needs to be able to be included
in groups
3. Groups themselves need to be includeable in other groups
4. Groups included in served pages need to be referred to in the pages
by their $originalName-${checksum} name, not by their vanilla group
name, otherwise the cache-busting won't always work (since the browser
may think it already has the file and not request it from the server).

Can all of the above be done with the plugin? If the above sounds
unintelligible, I'll try to clarify; meanwhile, I'm trying to do the
above by playing some more with WRO4J so I can ask better-informed
questions. :-)

- Tim

Alex Objelean

unread,

Feb 17, 2012, 4:04:24 PM2/17/12

to wr...@googlegroups.com

The description is a bit wrong (sorry for confusion).

The plugin writes to the specified location. But it doesn't create the file if that doesn't exist.

I've created the task which will fix this problem and the file will be created if none exist: http://code.google.com/p/wro4j/issues/detail?id=380

Cheers,

Alex

Tim Bessie

unread,

Feb 17, 2012, 4:13:37 PM2/17/12

to wro4j

Thanks! Always like to make helpful suggestions! :-)

- Tim

Alex Objelean

unread,

Feb 17, 2012, 4:40:28 PM2/17/12

to wr...@googlegroups.com

Regarding your other questions described in 4 steps. Answers are inline:

1. All CS and JSS files need to be copied to {targetLocation}/
originalName-${checksum}.ext

As long as NamingStrategy is set, the bundled resources will be copied to correct location respecting the naming strategy. [OK]

2. The file in #1, under it's new name needs to be able to be included
in groups

Don't really understand this, but as long as you have a group named g1 which dependes on g2, you don't need to refer the absolute path to the generated resource (the one which has checksum encoded in its name).

3. Groups themselves need to be includeable in other groups

That works by default. Example (with groovy DSL):

groups {

g1 {

group-ref("g2")

}

g2 {}

}

4. Groups included in served pages need to be referred to in the pages
by their $originalName-${checksum} name, not by their vanilla group
name, otherwise the cache-busting won't always work (since the browser
may think it already has the file and not request it from the server).

The NamingStrategy decides the name of the bundled resource. If you use, for example HashEncoderNamingStrategy, then the cache busting will work as expected.

Alex

Tim Bessie

unread,

Feb 17, 2012, 4:44:41 PM2/17/12

to wro4j

Thanks again, Alex - very helpful! I'll let you know my results when
I've got it all integrated.

- Tim

On Feb 17, 1:40 pm, Alex Objelean <alex.objel...@gmail.com> wrote:
> Regarding your other questions described in 4 steps. Answers are inline:
>

> *1. All CS and JSS files need to be copied to {targetLocation}/
> originalName-${checksum}.ext*
> *As long as NamingStrategy is set, the bundled resources will be copied to
> correct location respecting the naming strategy. [OK]** *

> 2. The file in #1, under it's new name needs to be able to be included
> in groups

> *Don't really understand this, but as long as you have a group named g1

> which dependes on g2, you don't need to refer the absolute path to the
> generated resource (the one which has checksum encoded in its name).

> *3. Groups themselves need to be includeable in other groups
> *That works by default. Example (with groovy DSL):*
> *groups {*
> * g1 {*
> * group-ref("g2")*
> * }*
> * g2 {}*
> *}*

> 4. Groups included in served pages need to be referred to in the pages
> by their $originalName-${checksum} name, not by their vanilla group
> name, otherwise the cache-busting won't always work (since the browser

> may think it already has the file and not request it from the server). * *
> *The NamingStrategy decides the name of the bundled resource. If you use,

> for example HashEncoderNamingStrategy, then the cache busting will work as

> expected.*
> *
> *
> Alex
> *
> *

Tim Bessie

unread,

Feb 17, 2012, 8:32:31 PM2/17/12

to wro4j

Ah, hold on, I hadn't had stack traces turned on - I'd added it to the
wrong maven call.

Here 'tis:

[INFO] Exception occured while processing: Illegal group reference

[INFO]
------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.lifecycle.LifecycleExecutionException: Exception
occured while processing: Illegal group reference
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:
719)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:
556)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:
535)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:
387)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:
348)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:
180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:
328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:
138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at
org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:
60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at
org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.MojoExecutionException: Exception
occured while processing: Illegal group reference
at
ro.isdc.wro.maven.plugin.AbstractWro4jMojo.execute(AbstractWro4jMojo.java:
108)
at
org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:
490)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:
694)
... 17 more
Caused by: java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:713)
at
ro.isdc.wro.model.resource.processor.impl.css.AbstractCssUrlRewritingProcessor.parseCss(AbstractCssUrlRewritingProcessor.java:
106)
at
ro.isdc.wro.model.resource.processor.impl.css.AbstractCssUrlRewritingProcessor.process(AbstractCssUrlRewritingProcessor.java:
65)
at ro.isdc.wro.model.group.processor.PreProcessorExecutor
$3.process(PreProcessorExecutor.java:235)
at ro.isdc.wro.model.group.processor.PreProcessorExecutor
$2.process(PreProcessorExecutor.java:215)
at
ro.isdc.wro.model.group.processor.PreProcessorExecutor.applyPreProcessors(PreProcessorExecutor.java:
174)
at
ro.isdc.wro.model.group.processor.PreProcessorExecutor.processAndMerge(PreProcessorExecutor.java:
82)
at ro.isdc.wro.model.group.processor.GroupsProcessor
$1.processAndMerge(GroupsProcessor.java:78)
at
ro.isdc.wro.model.group.processor.GroupsProcessor.process(GroupsProcessor.java:
59)
at
ro.isdc.wro.manager.WroManager.getContentHashEntry(WroManager.java:
287)
at
ro.isdc.wro.manager.WroManager.serveProcessedBundle(WroManager.java:
175)
at ro.isdc.wro.manager.WroManager.process(WroManager.java:128)
at
ro.isdc.wro.maven.plugin.Wro4jMojo.processGroup(Wro4jMojo.java:213)
at ro.isdc.wro.maven.plugin.Wro4jMojo.doExecute(Wro4jMojo.java:
118)
at
ro.isdc.wro.maven.plugin.AbstractWro4jMojo.execute(AbstractWro4jMojo.java:
106)
... 19 more

Tim Bessie

unread,

Feb 17, 2012, 7:11:09 PM2/17/12

to wro4j

Oh, by the way, with regard to #1 - that's not exactly what I meant;
we sometimes refer to individual resources by name, and these names
have to have the checksum suffix as well (for the same cachebusting
reason).

Does this mean we'd need to create a separate group for each JS and
CSS file that exists, if we want to take advantage of renaming
individual resources like this, and be able to refer to them in JSPs
and in code? i.e. that mapping only happens with groups, not with
individual files, hmm?

- Tim