Appengine search API doesn't work for deploys in flex environment

[Jianliang Zhao]

down votefavorite

I have been trying to deploy a service that uses the appengine golang search APIs. It works fine if I deploy it with vm: true option, but it's getting deprecated. When I deploy it with env: flex, I get this error: service bridge HTTP failed: Post http://appengine.googleapis.internal:10001/rpc_http: dial tcp: lookup appengine.googleapis.internal on 127.0.0.11:53: no such host How can I get around this issue? (Posted my question on stackoverflow too)

[George (Cloud Platform Support)]

You may try importing "cloud.google.com/go/datastore" in place of "google.golang.org/appengine/datastore". This solution is given as a reply to the "App Engine Flexible Golang can not access Datastore after successful deployment" question in stackoverflow. 

[Jianliang Zhao]

I saw the post regarding the use of datastore APIs. My question is for the search APIs in particular. There is no corresponding `cloud.google.com` library for use yet.

[George (Cloud Platform Support)]

The author of the reply to your stackoverflow question is right: the Search service is currently unavailable outside of the standard environment, as stated in the "Migrating Services from the Standard Environment to the Flexible Environment" online document. 

[Jianliang Zhao]

He also mentioned "It was available on AppEngine Flex using the compat runtime, but as you said, this is deprecated and will stop working on October 27th, 2017."

Is the date "October 27th" official?

I also have a followup comment that I wonder if you could help answer:

"Do you know if there is a plan to upgrade the go version for the appengine standard service ? Seems it's still on go 1.6 and the package vendoring doesn't work for me for the deploy time build. Also, do you know why it's not getting supported in the flex environment?"

[Robert _]

This thread highlights one of my main complains with Google's Cloud Platform: no roadmap! (at least not one they care to share with customers).  

The "everything is beta" environment might just be tolerable if there was a roadmap.  

As far as the Flex environment goes... you should think of it this way: it's a compute engine instance.  You can only do things in Flex that you can do in Compute Engine.  Google has written bridge libraries for many of the things in the Standard Environment (e.g. Datastore, Cloud SQL) but has left many things unsupported... Search API is one of those things.  I wouldn't hold your breath.  Deprecation of the Compat image is further proof that they've really given up on trying to bridge compute and appengine standard.  Perhaps the new focus is containers.  

[George (Cloud Platform Support)]

@Jianliang For official dates and the like, it is best to keep re-checking relevant pages from the official online documentation for the Search API. Sometimes these documents are updated more than once a week, to keep up with progress. 

@ Robert There are indeed plans for an update to version 1.8 of Go in the standard environment. The "Quick update about go1.8" groups posting provides more info. 

[Dheeraj Sarwaiya]

So what is the solution on flex environment? What libraries to use to create search indexes ?

[Julie (cloud platform support)]

The search service is unavailable outside the standard environment and as a workaround you can host any full-text search database such as ElasticSearch on Google Compute Engine and access it from both the standard and flexible environments. Details on how to create search indexes can be accessed here though it is only accessible for the standard environment. 

[Julie (cloud platform support)]

This issue tracker[1] can be used to track the request for Search API to be available for App Engine flex. The App Engine team is now aware of this feature and are currently looking into it. Please note that feature requests are not guaranteed and there is not ETA on them.

[1] https://issuetracker.google.com/120617455

[Dheeraj Sarwaiya]

Thanks for the response. 

The problem is that such inconsistencies and lack of information delays the whole product release process for small teams like ours. We now have to dedicate resources to learn this new thing specific to gcloud. I used Google app engine standard environment (go lang) earlier and it was simple enough for me to use  search api. Now when i am trying to make a bigger and better app I came across the absence of basic thing which was easily available earlier. On top of this, I am unable to find a document to produce equivalent outcome. At least you guys put a step by instruction on how to install elasticsearch in gcloud ecosystem and how to connect it from golang app. You guys completely forgot about your customer while making such decisions. It was more of convenient to you than to your customers.

I (and others) would really appreciate if you can share link(s) to help us solve such problem quickly. 

Meanwhile, thanks for raising the ticket.