SearchException - Failed to complete request

[Ananthakrishnan Venkatasubramanian]

Hi,

We have an application developed using Java running in google appengine.

When AppEngine's Search API is accessed for querying the records, sporadically we get the following exception (not always). But when we try again it is responding with results.

Stacktrace : 

Class : class com.google.appengine.api.search.SearchException
Message : Failed to complete request in 9498ms

at com.google.appengine.api.search.IndexImpl$4.convertException(IndexImpl.java:355)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:96)
at com.google.appengine.api.search.FutureHelper.getInternal(FutureHelper.java:73)
at com.google.appengine.api.search.FutureHelper.quietGet(FutureHelper.java:32)
at com.google.appengine.api.search.IndexImpl.search(IndexImpl.java:506)

Please let us know what is the issue behind this failure and what's the remedy for it.

Thanks in Advance.

Regards,

Ananthakrishnan.

[Nick (Cloud Platform Support)]

Hey Ananthakrishnan,

This appears to be some kind of internal error, and is not something you can necessarily prevent as a developer. It appears it has happened in the past and was related to an internal issue. I'll look into this and report back with any findings.

Sincerely,

Nick
Cloud Platform Community Support

[Ananthakrishnan Venkatasubramanian]

Hey Nick,

Thanks for your kind reply. Will keep an eye on this thread for further updates.

Regards,

Ananthakrishnan.

[Nick (Cloud Platform Support)]

Hey Ananthakrishnan,

While we are aware this can happen, as it has for other users in the past, and we commit ourselves daily to maintaining the uptime of our systems, I'm glad to hear that it is not consistent, and you have a retry logic. This is indeed a best practice for distributed systems which need to have a higher level of fault-tolerance and defensive programming (using incremental backoff, etc.) than traditional single-system applications.

Search API is a backend API service for App Engine, and it basically is a distributed system. Hence it can happen that one job is stuck, takes longer time to complete, and results in timeout. The exception only means that the particular request timed out. If you would like to experiment with different settings than a 10 second timeout, you can specify your desired timeout value using the setDeadline() method during building of the SearchService.

I hope this response has been helpful - let me know if you have any further questions and I'll be happy to assist.

Sincerely,

Nick
Cloud Platform Community Support

[Ananthakrishnan Venkatasubramanian]

Hey Nick,

Will try your suggestion.

Thanks for the quick response :)

[John Wieczorek]

I am have been experiencing symptoms similar to those described here and on the set of postings in the past as mentioned earlier in this thread. There are no reports of abnormal operations, but my index has essentially become non-functional, with timeouts when searching, or even deleting. I do use retry logic, but the queries are failing persistently. The problems began at 2016-07-18T14:47:10Z and have persisted since. 

Actually, I have just checked a production system that has been functional until now that uses a different index from the one I was having trouble with, and it too is timing out consistently for certain types of queries that did not fail in the past.

Any recommendations for how to find out what might be causing this, and how to recover from this situation?

[Nick (Cloud Platform Support)]

Hey John,

If you believe you've encountered a production issue, which this sounds like, you should file a Public Issue Tracker report. You can also get a support package for more on-demand response times, but we are very diligent with monitoring the Public Issue Trackers at any rate, and you should see a response quickly. Provide you project ID, the name of the indexes, and as much information as possible. Post the issue here once you've made it, so that I can take a look as well - I'm happy to help.

Regards,

Nick
Cloud Platform Community Support

[Nick (Cloud Platform Support)]

A link to the Public Issue Tracker for App Engine, left out of the last message.

On Monday, July 18, 2016 at 5:13:35 PM UTC-4, John Wieczorek wrote:

[John Wieczorek]

Hi Nick,

Here is a belated followup. I do not know why the production effects were seen, but they were ephemeral. The index I was loading also eventually became responsive, though many searches continued to fail. In the process I learned how to improve the search efficiency by changing the data type to Atom fields wherever reasonably possible. This improved search execution to the point where it was not even necessary to bump up the instance class nor to change the default url fetch deadline.

Thanks for your responsiveness when the issue arose.

[Nick (Cloud Platform Support)]

Hey John,

Glad to hear things worked out, and that the effects were ephemeral at any rate. Best of luck going forward!

Cheers,

Nick
Cloud Platform Community Support