maven plugin 403 errors for certain external resources

43 views
Skip to first unread message

mich...@gmail.com

unread,
Mar 19, 2015, 1:20:37 PM3/19/15
to wr...@googlegroups.com
Hello, I'm using the maven plugin with the following group config:


<group name="core">
<js minimize="false">http://code.jquery.com/jquery-1.11.2.min.js</js>
<js minimize="false">http://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js</js>
<js minimize="false">http://cdn.datatables.net/1.10.5/js/jquery.dataTables.js</js>
</group>

wro4j is able to retrieve the first two resources with no problem, but the datatables.net returns a 403:

[ERROR] Cannot ignore missing resource:  ro.isdc.wro.model.resource.Resource@b326f69[JS,http://cdn.datatables.net/1.10.5/js/jquery.dataTables.js,false]
[ERROR] Exception while merging resources: Server returned HTTP response code: 403 for URL: http://cdn.datatables.net/1.10.5/js/jquery.dataTables.js
[ERROR] Exception occured while processing: ro.isdc.wro.WroRuntimeException: Exception while merging resources: Server returned HTTP response code: 403 for URL: http://cdn.datatables.net/1.10.5/js/jquery.dataTables.js, class: ro.isdc.wro.WroRuntimeException,caused by: java.io.IOException
ro.isdc.wro.WroRuntimeException: Exception while merging resources: Server returned HTTP response code: 403 for URL: http://cdn.datatables.net/1.10.5/js/jquery.dataTables.js
	at ro.isdc.wro.model.group.processor.GroupsProcessor.process(GroupsProcessor.java:86)
...

That URL works fine in a browser (or via wget, curl, etc).

I get the same error for highcharts files--e.g., 

<js>http://code.highcharts.com/highcharts.js</js>

Any ideas?

Thanks!

/mcr

Alex Objelean

unread,
Mar 19, 2015, 1:25:27 PM3/19/15
to wr...@googlegroups.com
Hi,

for some reason, when the request is performed on the server-side by wro4j locator, the server refuses to send the response. The 403 means Forbidden. I'm not sure why this happens, but a possible reason is that the CDN provider looks to the agent and refuses to serve anything else but browser calls.

A possible workaround could be save the url locally and update the path to use local resource or upload it to a different web location which doesn't have such limitations.

Thanks,
Alex

mich...@gmail.com

unread,
Mar 20, 2015, 8:35:32 AM3/20/15
to wr...@googlegroups.com
Strange, since requests via curl and wget from the command line don't get 403s. 
Maybe wro4j isn't sending some header that the CDNs want--e.g., HTTP_USER_AGENT?

mich...@gmail.com

unread,
Mar 20, 2015, 8:45:13 AM3/20/15
to wr...@googlegroups.com
Is there a way to turn on debugging from the maven plugin? (setting debug in the properties file didn't seem to work--must be a runtime option).

Alex Objelean

unread,
Mar 20, 2015, 10:09:08 AM3/20/15
to wr...@googlegroups.com
The debugging can be turned on using -X flag. 

There is a possibility that CDN expects some header. I'll take a look to see why exactly it doesn't work...

Alex

On 20 March 2015 at 14:45, <mich...@gmail.com> wrote:
Is there a way to turn on debugging from the maven plugin? (setting debug in the properties file didn't seem to work--must be a runtime option).

--
You received this message because you are subscribed to the Google Groups "wro4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wro4j+un...@googlegroups.com.
To post to this group, send email to wr...@googlegroups.com.
Visit this group at http://groups.google.com/group/wro4j.
For more options, visit https://groups.google.com/d/optout.

Alex Objelean

unread,
Mar 20, 2015, 11:13:07 AM3/20/15
to wr...@googlegroups.com
I can confirm that as long as User-Agent header is not set on request, the CDN returns 403. 
Open an issue and the url connection will add this header implicitly.

Cheers,
Alex

mich...@gmail.com

unread,
Mar 20, 2015, 11:19:38 AM3/20/15
to wr...@googlegroups.com
Thanks for investigating that, Alex.

Is there a way to configure the User-Agent header, or is that a code change?

Alex Objelean

unread,
Mar 20, 2015, 12:31:38 PM3/20/15
to wr...@googlegroups.com
There is a code change required in UrlUriLocator. 
You can patch it yourself until the fix is applied and released. All you have to do is to add the following line in the locate method of the UrlUriLocator class:

connection.setRequestProperty("User-Agent", "");

Cheers,
Alex

--
Reply all
Reply to author
Forward
0 new messages