how to avoid an out of quota of module.get_versions in appengine

[dg...@freightos.com]

I recently got an error in appengine:

com.google.apphosting.api.ApiProxy$OverQuotaException: The API call modules.GetVersions() required more quota than is available. The API call modules.GetVersions() required more quota than is available.
com.google.apphosting.api.ApiProxy$OverQuotaException: The API call modules.GetVersions() required more quota than is available. The API call modules.GetVersions() required more quota than is available.
com.google.apphosting.api.ApiProxy$OverQuotaException: The API call modules.GetVersions() required more quota than is available.
at com.google.apphosting.utils.runtime.ApiProxyUtils.convertApiError(ApiProxyUtils.java:54)
at com.google.apphosting.utils.runtime.ApiProxyUtils.getApiError(ApiProxyUtils.java:184)
at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:588)
at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:484)
at com.google.apphosting.runtime.http.HttpApiHostClient.receivedResponse(HttpApiHostClient.java:274)
at com.google.apphosting.runtime.http.JettyHttpApiHostClient$Listener.onComplete(JettyHttpApiHostClient.java:158)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:458)
at org.eclipse.jetty.client.HttpReceiver.responseSuccess(HttpReceiver.java:405)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.messageComplete(HttpReceiverOverHTTP.java:277)
at org.eclipse.jetty.http.HttpParser.handleContentMessage(HttpParser.java:601)
at org.eclipse.jetty.http.HttpParser.parseContent(HttpParser.java:1545)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1369)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:159)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:120)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:70)
at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90)
at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:115)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)

but this is really weird. why should i get an error of modules.get_version. Isn't this calculated once and cached?

Which quota do i need to raise to avoid this error?

Thanks,

[dg...@freightos.com]

I asked this in server fault also after reading that it may there be more appropriate:https://serverfault.com/questions/933927/why-do-i-get-an-out-of-quota-on-module-getversion-and-how-to-avoid-this

[Mohammad I (Cloud Platform Support)]

Hello dg...@freightos.com,

I have reproduced the issue that you have been experiencing and found that your code calling the modules.GetVersions() more then 86,400 times which is the daily safety limit quota for the Modules - Get Versions Calls. You can see quotas for your project by clicking here. The quotas for Modules - Get Versions Calls  86,400 per day seems reasonable as it is equivalent to call modules.GetVersions() each second for the entire day. However if you are a paid customer and believe daily quota for Modules - Get Versions Calls does not meet your requirement, please create a feature request issue in the issue tracker.

[dg...@freightos.com]

Hi,

I am a paid customer.

Google is making these calls to GetVersions() and not us. Please look at the stack trace.

Apparently Google is doing these API calls as part of some sort of internal logging.

Why is Google calling GetVersions() so much -- it seems to be consuming our quota.

[Mohammad I (Cloud Platform Support)]

Hello David,

Thank you for letting us know your concern. To investigate the issue further and inspect the StackTrace, we need the Project Id and preferably the Application Id (for Google App Engine). You can provide the information by creating a new private issue in the Public Issue Tracker. It will be appreciated if you provide further info how you came to the conclusion that Google Cloud is calling the API, modules.getVersions() for internal logging.

As you indicated you are a paid customer and in case you have paid support, you can go to the Google Cloud Platform support and select the Console Support Center to create a case instead of creating a private issue in the Public Issue Tracker.

[dg...@freightos.com]

Hi,

Thanks for your update:

I don't know why the Google library is doing this, for its own logging purposes or some other reason. I don't have access to source code. The one question is -- why is Google's AppEngine  using up our GetVersions quota?

Thanks,

David

[Harmit Rishi (Cloud Platform Support)]

Hello David,

After reviewing your stack trace as well as the recommendations provided by the previous Support Agent, it would be an appropriate action to create a private issue on Public Issue Tracker or  Google Cloud Platform support case to help resolve your issue that you are having with our Platform in a timely manner.

It would also be relevant to reiterate that Google by default provides at least 86,400 daily safety limit quota for your project. As your project seems to exceed this limit; identifying the underlying cause would definitely be beneficial to your project as well as be a cost-effective solution (rather than just raising your quota) as well.

Considering this is a public means of communication, I would like to mention that I will be providing you a suitable general answer to your question.

The stack trace that was provided does illustrate that this quota limit is being reached on our Platform (com.google.apphosting.api....). However, there are currently no known or reported issues that would be causing this on our side.

This issue seems more specific to your use of our platform and would require a deeper 1:1 investigation into the cause for this quota limit being reached.

I have also noticed that a Google Cloud Platform support case was created for this issue with us by someone in your organisation. It appears to be in progress currently by our support staff. Please see the appropriate individual who opened the case for further information on the matter.