Analytics and App Engine

[Eyal]

Hi all,

I'm trying to use the Analytics API with App Engine and running into a
"Timeout while fetching" IOException (the full stack trace below.)

The code doesn't do anything fancy, simply fetching some information
on a site. It works perfectly as a java application, but not as a web
app -- and I need it to be a web app. I tried GWT, and a simple JSP
page, both with the same timeout results.

Any idea how to solve this? I heard about increasing the timeout with
the URLFetch but it can only be up'ed to 10 seconds and it takes more
than that for my Analytics feed to come back.

I have also posted this to the App Engine group but so far with no
good hints.

Your help is greatly appreciated!

Eyal

Timeout while fetching:
https://www.google.com/analytics/feeds/data?max-results=50&start-date=2010-11-08&end-date=2010-11-08&metrics=ga%3Avisitors%2Cga%3Avisits%2Cga%3Apageviews%2Cga%3AnewVisits%2Cga%3Abounces%2Cga%3AtimeOnSite&ids=ga%3A9990197
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:
110)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:
39)
at
com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler
$Connection.fetchResponse(URLFetchServiceStreamHandler.java:404)
at
com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler
$Connection.getInputStream(URLFetchServiceStreamHandler.java:283)
at
com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler
$Connection.getResponseCode(URLFetchServiceStreamHandler.java:136)
at
com.google.gdata.client.http.HttpGDataRequest.checkResponse(HttpGDataRequest.java:
549)
at
com.google.gdata.client.http.HttpGDataRequest.execute(HttpGDataRequest.java:
530)
at
com.google.gdata.client.http.GoogleGDataRequest.execute(GoogleGDataRequest.java:
535)
at com.google.gdata.client.Service.getFeed(Service.java:1135)
at com.google.gdata.client.Service.getFeed(Service.java:998)
at com.google.gdata.client.GoogleService.getFeed(GoogleService.java:
631)
at com.google.gdata.client.Service.getFeed(Service.java:1017)
at
com.eyecarepro.manager.UpdateManager.getSiteBasicData(UpdateManager.java:
76)
at com.eyecarepro.manager.UpdateManager.<init>(UpdateManager.java:33)
at org.apache.jsp.Update_jsp._jspService(Update_jsp.java:67)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:
324)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:
292)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:236)
at com.google.appengine.tools.development.PrivilegedJspServlet.access
$101(PrivilegedJspServlet.java:23)
at com.google.appengine.tools.development.PrivilegedJspServlet
$2.run(PrivilegedJspServlet.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at
com.google.appengine.tools.development.PrivilegedJspServlet.service(PrivilegedJspServlet.java:
57)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
511)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1166)
at
com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:
58)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:
122)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
418)
at
com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:
70)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
152)
at com.google.appengine.tools.development.JettyContainerService
$ApiProxyHandler.handle(JettyContainerService.java:349)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
542)
at org.mortbay.jetty.HttpConnection
$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:
409)
at org.mortbay.thread.QueuedThreadPool
$PoolThread.run(QueuedThreadPool.java:582)

[Eyal]

I have also tried setting the connection timeout like this:
as.setConnectTimeout(10000);
but still no luck. Anyone else running into the same issue? Has anyone
had any luck integrating Analytics and App Engine? I'm stumped by
this.

Thanks,
Eyal

On Nov 8, 12:55 pm, Eyal <twoworldsf...@gmail.com> wrote:
> Hi all,
>
> I'm trying to use the Analytics API with App Engine and running into a
> "Timeout while fetching" IOException (the full stack trace below.)
>
> The code doesn't do anything fancy, simply fetching some information
> on a site. It works perfectly as a java application, but not as a web
> app -- and I need it to be a web app. I tried GWT, and a simple JSP
> page, both with the same timeout results.
>
> Any idea how to solve this? I heard about increasing the timeout with
> the URLFetch but it can only be up'ed to 10 seconds and it takes more
> than that for my Analytics feed to come back.
>
> I have also posted this to the App Engine group but so far with no
> good hints.
>
> Your help is greatly appreciated!
>
> Eyal
>

> Timeout while fetching:https://www.google.com/analytics/feeds/data?max-results=50&start-date...
>         at
> com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationExc eption(URLFetchServiceImpl.java:
> 110)
>         at
> com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchService Impl.java:

[Eyal]

Anyone? Is a person from Google monitoring this group? I can't seem to
get over the timeout issue when integrating Analytics and App Engine.
Anyone has a working example?

Thanks,
Eyal

[Nick]

Hi,

Yes, App Engine has some issues with the Google Analytics API.

Currently App Engine only allows a maximum of 10 seconds for each URL
Fetch request. Sometimes a query can take a long time. Some cases
include queries to the Account feed for a user with many accounts /
web props / profiles / goals or for Data Feed queries where we have to
reprocess the data on the fly.

There are a couple of ways to work around this.
1) Use the Management API with filers
2) Request smaller date ranges with the data feed
3) Re-write your app to make AJAX requests so that only one request
will be processed by App Engine at a time.

(In the current GA on App Engine demo, we used a more traditional
client-server architecture which requests both Account and Data Feeds
in each request. So time can time out for certain accounts)

-Nick

[Eyal]

Hi Nick,

Thank you very much for your response. I'm trying the Management API
example but getting unresolved compilation errors. In which jar are
these classes?

com.google.gdata.data.analytics.AnalyticsLink;
com.google.gdata.data.analytics.ManagementEntry;
com.google.gdata.data.analytics.ManagementFeed;

I have the following jars in my classpath as per the instructions on
http://code.google.com/apis/analytics/docs/mgmt/mgmtJava.html#setup

gdata-core-1.0.jar
gdata-client-meta-1.0.jar
gdata-client-1.0.jar
gdata-analytics-meta-2.1.jar
gdata-analytics-2.1.jar
google-collect-1.0-rc1.jar
jsr305.jar

Thanks,
Eyal

[Nick]

They should be in gdata-analytics-2.1.jar

Maybe you have and older version? I would try downloading the latest
here: http://code.google.com/p/gdata-java-client/downloads/list

-Nick

On Nov 10, 12:54 am, Eyal <twoworldsf...@gmail.com> wrote:
> Hi Nick,
>
> Thank you very much for your response. I'm trying the Management API
> example but getting unresolved compilation errors. In which jar are
> these classes?
>
> com.google.gdata.data.analytics.AnalyticsLink;
> com.google.gdata.data.analytics.ManagementEntry;
> com.google.gdata.data.analytics.ManagementFeed;
>

> I have the following jars in my classpath as per the instructions onhttp://code.google.com/apis/analytics/docs/mgmt/mgmtJava.html#setup

[Eyal]

Yes, that did it. Cool -- and thanks!

Eyal