ApiProxy$OverQuotaException without being over quota!!

[Sagar Mutha]

I have seen this error 2 days in a row now. My app is way below the quota but it keeps showing this exception.

`/api/seat

com.google.apphosting.api.ApiProxy$OverQuotaException: The API call remote_socket.SetSocketOptions() required more quota than is available.`

This is adversely affecting the user experience of my app. So please help me fix asap.

App Id - saseatavailability

[Patrice (Cloud Platform Support)]

Hi Sagar,

I believe that what's happening is that the "set socket options" has its own quota. Last time I saw a customer with a similar problem, it turned out he was calling a third-party library (if memory serves it was an Apache library) that was doing a lot of socket calls behind the scenes, causing that quota to grow faster than expected.

In general, If you're completely certain that your quota is below the limit, I would suggest looking into your daily budget. but I'd be surprised if that was your issue, as the call calls out the precise method that is triggering the error.

Can you show me the quotas you're seeing on your quota page?

Cheers!

[Sagar Mutha]

Hello Patrice,

Below is the screenshot of what I see on the quotas page. The app is working fine now after the quotas got reset at 12am but that was also the case yesterday. By around 6 in the evening it started showing the quota exceeded exception. I think the sockets created were about 56% when I checked then and frontend instance hours used were well below my daily budget.

[Sagar Mutha]

Hello Patrice,

I have started to see this error on my app again today and the sockets created is just at 36% From past experience this error will stay until the next quota reset i.e. 12am. Please help me out with this issue. A lot of my users have started to drop because of this. I have attached the screenshot of my quota and instance hours used below. This is what the logs say. 

com.google.apphosting.api.ApiProxy$OverQuotaException: The API call remote_socket.SetSocketOptions() required more quota than is available.

I found a thread with a similar issue in this group. Wonder if its a related problem? 

Hoping for an early resolution.

Regards,

Sagar

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/22cj_U1in5w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f2ae390c-0224-45de-8bdd-b1d4c032dda6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Sagar Mutha]

Forgot to mention the thread sorry - https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!searchin/google-appengine/ApiProxy$24OverQuotaException/google-appengine/w_GG2Y-yHRA/KDKZS3zhEKgJ

To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

Visit this group at http://groups.google.com/group/google-appengine.

[Patrice (Cloud Platform Support)]

Hi again Sagar,

Sorry for the delay.

I think that the quota that is mentioned by the error (SET_SOCKET_OPTIONS) might be different from the quota you're looking at, which is the "sockets created".

Is it possible that you reuse the same sockets a lot and you set your options one at a time? Could you show some code about how you use and create your sockets?

Cheers!

[Sagar Mutha]

Hello Patrice,

Thank you for your response! I am using the Apache library which does the setting of socket options I think. Here is what my code looks like.

@Path("/")

public class SparkResource implements Serializable {

 

    /**

     *

     */

    private static final long serialVersionUID = -5278229852877855876L;

    private final CloseableHttpClient httpClient = HttpClients.createDefault();

    private final Logger log = Logger.getLogger(RailwayResource.class.getName());

 

    @GET

    @Path("spark")

    @Produces({MediaType.APPLICATION_JSON})

    public String readSparkStatus(@QueryParam("param1") String param1 throws Exception {

         

        HttpPost httpPost = new HttpPost("http://www.spark.com");

         

        List <NameValuePair> nvps = new ArrayList <NameValuePair>();   

         

        nvps.add(new BasicNameValuePair("param1", param1));

         

        httpPost.setEntity(new UrlEncodedFormEntity(nvps));

         

        Document doc = null;

        CloseableHttpResponse response = null;

        try {

            response = httpclient.execute(httpPost);

            HttpEntity entity = response.getEntity();

            doc = Jsoup.parse(entity.getContent(), "UTF-8", "");

            EntityUtils.consume(entity);

        } catch(Exception ex) {

            log.log(Level.SEVERE, ex.getMessage(), ex);

        } finally {

            if(response != null) {

                response.close();

            }

            if(httpClient != null) {

                httpClient.close();

            }

        }

 

        ...

    }

}




Thank you,

Sagar

Cheers!

To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at http://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f2ae390c-0224-45de-8bdd-b1d4c032dda6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/22cj_U1in5w/unsubscribe.

To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Visit this group at http://groups.google.com/group/google-appengine.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/20ca7144-e2ae-4595-bed9-2d80881b4e53%40googlegroups.com.

[Patrice (Cloud Platform Support)]

Hi Sagar,

Yeah, this seems to be the same library I saw last time. I think there's something with how this library set its options that is not done in a "one shot" kinda way. They set options one at a time, which makes you run out of your options quota faster than your creation one.

Last time I saw this, the solution was to drop the library and use something else :S.

Cheers

On Tuesday, November 10, 2015 at 6:53:44 PM UTC-5, Sagar Mutha wrote:

Hello Patrice,

Thank you for your response! I am using the Apache library which does the setting of socket options I think. Here is what my code looks like.

@Path("/")

public class SparkResource implements Serializable {

 

    /**

     *

     */

    private static final long serialVersionUID = -5278229852877855876L;

    private final CloseableHttpClient httpClient = HttpClients.createDefault();

    private final Logger log = Logger.getLogger(RailwayResource.class.getName());

 

    @GET

    @Path("spark")

    @Produces({MediaType.APPLICATION_JSON})

    public String readSparkStatus(@QueryParam("param1") String param1 throws Exception {

         

        HttpPost httpPost = new HttpPost("http://www.spark.com");

         

        

...

[Sagar Mutha]

Not a solution I wanted to hear :( But, I guess I will have to do it if the library is the problem maker. Is there a library/approach that GAE recommends?

Thanks,

Sagar

[Patrice (Cloud Platform Support)]

Hi again Sagar,

There are two things that you can do (I got a bit more information because I realized you have a case opened with Support as well). What I gathered from the other case is twofold in what you can do, if you don't want to switch the library:

1- You could reuse your connections instead of opening a new one for each HTTP request

2- In your code you set the read timeout and the connect timeout manually, right? You might be better off setting these in your configuration file, as explained here[1]. That should save you some calls to set socket options.

Let me know if these help. I'm focusing on these since I know it can be painful to remove the library from your code.

Cheers

[1]https://cloud.google.com/appengine/docs/java/config/appconfig#url_fetch_timeout

[Sagar Mutha]

Hey Patrice,

The URLFetch code that I posted is not in production because it does not work for me. The request always times out after 10 seconds no matter what number I set in the timeout. And the funny thing is the URLFetch code works when I deploy it locally and test. But when I upload it to GAE, it starts timing out.

I tried setting the timeout in the xml but got an XML parse error. This is what I tried. Am I doing this wrong?

<?xml version="1.0" encoding="utf-8"?>

<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">

  <application>saseatavailability</application>

  <version>1</version>

  <!--

    Allows App Engine to send multiple requests to one instance in parallel:

  -->

  <threadsafe>true</threadsafe>

<appengine.api.urlfetch.defaultDeadline>10.0</appengine.api.urlfetch.defaultDeadline>

  <!-- Configure java.util.logging -->

<system-properties>

    <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>

  </system-properties>

  <!--

    HTTP Sessions are disabled by default. To enable HTTP sessions specify:

      <sessions-enabled>true</sessions-enabled>

    It's possible to reduce request latency by configuring your application to

    asynchronously write HTTP session data to the datastore:

      <async-session-persistence enabled="true" />

    With this feature enabled, there is a very small chance your app will see

    stale session data. For details, see

    http://code.google.com/appengine/docs/java/config/appconfig.html#Enabling_Sessions

  -->

</appengine-web-app>

Thanks,

Sagar

[Patrice (Cloud Platform Support)]

Hi again Sagar,

Just to keep you in the loop : I noticed the update that has been posted on your case has been answered and fully analysed by one of your specialists. I believe you should at this point have enough to move forward.

If you have follow up questions, I suggest you take them to the specialist, as he will be in a better position to answer, being more aware of what came up from his analysis than I am :).

Cheers!

[Sagar Mutha]

Sounds good! Thank you for all your help Patrice :)