Unable to establish connection with Google Cloud SQL from GAE

[Purushotham Rao]

Hello, 

we are users of GAE and use Google Cloud SQL to query our models. Experiencing some problems in establishing the connection with GQL from last 10 hrs. On initialization of application we will establish a connection with Cloud SQL, getting below exception and causing the instance to exit and creating a new instance.

Application Id : user-opinion-insights

As a result we are reaching maximum number of database connections and GAE instance is not up.

org.hibernate.tool.hbm2ddl.SchemaUpdate execute: HHH000299: Could not complete schema update
com.google.apphosting.api.ApiProxy$CancelledException: The API call rdbms.GetMetadata() was explicitly cancelled.
	at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:242)
	at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:69)
	at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:201)
	at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:198)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:198)
	at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:69)
	at com.google.appengine.tools.appstats.Recorder.makeSyncCall(Recorder.java:303)
	at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:120)
	at com.google.appengine.api.rdbms.RdbmsApiProxyClient$ApiProxyBlockingInterface.makeSyncCall(RdbmsApiProxyClient.java:93)
	at com.google.appengine.api.rdbms.RdbmsApiProxyClient$ApiProxyBlockingInterface.getMetadata(RdbmsApiProxyClient.java:63)
	at com.google.cloud.sql.jdbc.internal.SqlProtoClient.getMetadata(SqlProtoClient.java:141)
	at com.google.cloud.sql.jdbc.Connection.getMetadata(Connection.java:610)
	at com.google.cloud.sql.jdbc.DatabaseMetaData.getMetadata(DatabaseMetaData.java:1098)
	at com.google.cloud.sql.jdbc.DatabaseMetaData.getMetadata(DatabaseMetaData.java:1092)
	at com.google.cloud.sql.jdbc.DatabaseMetaData.getImportedKeys(DatabaseMetaData.java:281)
	at com.google.cloud.sql.jdbc.DatabaseMetaData.getImportedKeys(DatabaseMetaData.java:33)
	at org.hibernate.tool.hbm2ddl.TableMetadata.initForeignKeys(TableMetadata.java:162)
	at org.hibernate.tool.hbm2ddl.TableMetadata.<init>(TableMetadata.java:60)
	at org.hibernate.tool.hbm2ddl.DatabaseMetadata.getTableMetadata(DatabaseMetadata.java:140)
	at org.hibernate.cfg.Configuration.generateSchemaUpdateScriptList(Configuration.java:1171)
	at org.hibernate.tool.hbm2ddl.SchemaUpdate.execute(SchemaUpdate.java:212)
	at org.hibernate.tool.hbm2ddl.SchemaUpdate.execute(SchemaUpdate.java:178)
	at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:506)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1790)
	at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:96)
	at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:914)
	at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:899)
	at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:59)
	at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:63)
	at com.motorola.uoi.handlers.UserInsightsTableManager.<init>(UserInsightsTableManager.java:39)
	at com.motorola.uoi.handlers.UserInsightsTableManager.<clinit>(UserInsightsTableManager.java:42)
	at com.motorola.uoi.aggregation.AbstractAggregationServlet.<clinit>(AbstractAggregationServlet.java:81)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at java.lang.Class.newInstance(Class.java:370)
	at org.mortbay.jetty.servlet.Holder.newInstance(Holder.java:153)
	at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:428)
	at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:685)
	at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:199)
	at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:174)
	at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
	at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:527)
	at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
	at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
	at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:220)
	at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
	at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
	at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
	at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
	at java.lang.Thread.run(Thread.java:724)

Thanks for any help! 

[Vinny P]

On Mon, Feb 23, 2015 at 11:36 AM, Purushotham Rao <purusho...@motorola.com> wrote:

On initialization of application we will establish a connection with Cloud SQL, getting below exception and causing the instance to exit and creating a new instance.

Since you're running the SQL connection during init, it's difficult to figure out what exactly is the culprit: is the SQL connection taking too long to finish, then hitting the request deadline which cancels the getMetadata operation, or is there an error in the connection that cancels the function?

Can you try moving the connection code out of init and into a regular request handler, then hitting the handler from a cron request? Does the connection work then? If so, how long does the SQL connection take from start to finish? Have you configured anything unusual connection-wise in the SQL driver? Also, have you modified the max-concurrent-requests setting?

 

 

-----------------

-Vinny P

Technology & Media Consultant

Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com