GWT google crawler error undefined.cache.js

Zied Hamdi

unread,

Sep 30, 2016, 1:54:03 PM9/30/16

to GWT Users

I have a GWT application http://islamart.tn which fails on google crawling : The 'fetch as google' feature fails with a 'partial' status saying that it can't find undefined.cache.js

After some research, I found that this js is the app js when the user.agent is not recognized

I therefore changed my

UserAgentPropertyGenerator.UserAgent

enum value for gecko to recgnize google bot:

 gecko1_8("return (ua.indexOf('Googlebot') || ua.indexOf('gecko') != -1 || docMode >= 11);");

In my core GWT module I naturally added the line :

<property-provider name="user.agent" generator="fr.onevu.vume.client.common.UserAgentPropertyGenerator"/>

I installed 'Chrome UA spoofer" to emulate the Googlebot user.agent request, and what happened is that now I have a warning at startup saying that the runtime value safari is different than the compile time gecko_18

I still see some checking runtime code in the generated js

{$stackDepth_0=stackIndex_0-1;return 'gecko1_8'}$stackDepth_0=stackIndex_0-1;return 'unknown'}

but those files are very large, and I don't know from which classes they are generated.

Additionally, I had to fall back to the version 2.6.1 since compilation with 2.8.0RC2 and 2.7.0 don't finish the job (for some reasons I don't have the time to investigate now)

Any ideas please?...

I need an answer to one of these questions :

How to personalize the runtime user agent check generated code ?
How to make googlebot use the gecko_18 version of my app ?

p.s : I posted this message on http://stackoverflow.com/questions/39796166/gwt-application-crawler-error-undefined-cache-js

Thomas Broyer

unread,

Sep 30, 2016, 2:05:06 PM9/30/16

to GWT Users

Have you tried set-property-fallback?

Zied Hamdi

unread,

Sep 30, 2016, 2:36:16 PM9/30/16

to GWT Users

Hi Thomas,

Thanks for the fast reply :), it's been a while we didn't exchange !

I tried set-property-fallback, but this means I have to migrate to a newer gwt version (2.7 or 2.8). The issue is that the compiler for those two versions stops unexpectedly without any error message (it only writes "linking succeeded"). In addition, when it succeeds in compilation, I can't run the app in standard mode : the compiler of the super-dev-mode triggers even if I launch the app in standard mode (and I need to see what happens with the production js)

With all these issues and because the bug is critical to fix as fast as possible, I decided to downgrade back to 2.6.1 where I don't have these unexpected compiling failure without errors ... :-/

Actually I need to find a solution for the 2.6.1 version (where set-property-fallback doesn't apply), I will have more time to understand what happens with the new GWT compilers when this bug will be workarounded...

On Friday, September 30, 2016 at 7:05:06 PM UTC+1, Thomas Broyer wrote:

Have you tried set-property-fallback?

Vassilis Virvilis

unread,

Sep 30, 2016, 3:04:00 PM9/30/16

to google-we...@googlegroups.com

I was using this

APP_DIR=/var/lib/tomcat8/webapps/vizit/vizit; sudo ln -s `ls $APP_DIR/*.cache.js | head -n 1` $APP_DIR/undefined.cache.js; ls -lt $APP_DIR/

--
You received this message because you are subscribed to the Google Groups "GWT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit+unsub...@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at https://groups.google.com/group/google-web-toolkit.
For more options, visit https://groups.google.com/d/optout.

--

Vassilis Virvilis

Zied Hamdi

unread,

Oct 1, 2016, 8:27:51 PM10/1/16

to GWT Users

Hi Vassilis,

Thanks for your reply,

I found your post on stackTrace, the issue is that I can't do that since I don't have the control on my server : I'm under GAE where you can only delpoy your app. So I really need a solution in the source code. Is it possible to diable the assertion control? It seems the assertion code is generated in many js files, I don't know what classes are behind that code...

Zied Hamdi

unread,

Oct 3, 2016, 1:57:46 PM10/3/16

to GWT Users

Hi All,

I tried to run the app in standard dev mode under 2.8.0RC3 using the fallback property : <set-property-fallback name="user.agent" value="gecko1_8"/> and removing my own UserAgentPropertyGenerator

It shows up that the error still persits (when I emulate a googlebot user.agent), in addition of the compiler being run for my first page display (as if I was running in super dev mode) :

So I have two questions:

how do I configure the fallback user agent?
how do I force the compiler to run in production mode? without the super dev mode compilation at startup

The compile time user.agent value (gecko1_8) does not match the runtime user.agent value (safari).

Expect more errors.

com.google.gwt.useragent.client.UserAgentAsserter$UserAgentAssertionError: Possible problem with your *.gwt.xml module file.

The compile time user.agent value (gecko1_8) does not match the runtime user.agent value (safari).

Expect more errors.

at Unknown.Id_g$(Unknown Source)

at Unknown.Cgf_g$(Unknown Source)

at Unknown.Mgf_g$(Unknown Source)

at Unknown.Tgf_g$(Unknown Source)

at Unknown.rgf_g$(Unknown Source)

at Unknown.BM_g$(Unknown Source)

at Unknown.EM_g$(Unknown Source)

at Unknown.anonymous(Unknown Source)

On Friday, September 30, 2016 at 6:54:03 PM UTC+1, Zied Hamdi wrote:

Zied Hamdi

unread,

Oct 3, 2016, 2:17:29 PM10/3/16

to GWT Users

Hi Vassilis,

Since I'm under AppEngine, I could copy one of the compiled versions as /undefined.cache.js before uploading the app. But in that case I have two issues:

I need to find which is the js for gecko1_8 so that Googlebot can understand the js (the file names are random)
I need to separate the compile process from the upload process so that I can upload a modified version

These two points need some investigation, so I would prefer to find a real solution and let that workaroud as a last plan...

On Friday, September 30, 2016 at 8:04:00 PM UTC+1, Vassilis Virvilis wrote:

I was using this

APP_DIR=/var/lib/tomcat8/webapps/vizit/vizit; sudo ln -s `ls $APP_DIR/*.cache.js | head -n 1` $APP_DIR/undefined.cache.js; ls -lt $APP_DIR/

On Fri, Sep 30, 2016 at 9:36 PM, Zied Hamdi <zha...@gmail.com> wrote:

Hi Thomas,

Thanks for the fast reply :), it's been a while we didn't exchange !

I tried set-property-fallback, but this means I have to migrate to a newer gwt version (2.7 or 2.8). The issue is that the compiler for those two versions stops unexpectedly without any error message (it only writes "linking succeeded"). In addition, when it succeeds in compilation, I can't run the app in standard mode : the compiler of the super-dev-mode triggers even if I launch the app in standard mode (and I need to see what happens with the production js)

With all these issues and because the bug is critical to fix as fast as possible, I decided to downgrade back to 2.6.1 where I don't have these unexpected compiling failure without errors ... :-/

Actually I need to find a solution for the 2.6.1 version (where set-property-fallback doesn't apply), I will have more time to understand what happens with the new GWT compilers when this bug will be workarounded...

On Friday, September 30, 2016 at 7:05:06 PM UTC+1, Thomas Broyer wrote:
Have you tried set-property-fallback?

--
You received this message because you are subscribed to the Google Groups "GWT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit+unsub...@googlegroups.com.

To post to this group, send email to google-we...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-web-toolkit.
For more options, visit https://groups.google.com/d/optout.

--
Vassilis Virvilis

Reply all

Reply to author

Forward