Migrating to Compute Engine

[Thomas Baldauf]

If I manage to rewrite my App Engine Standard Edition Java app to not use the SDK but instead Google Cloud Client libraries and Redis Memcache service, should it be possible to run the app on Compute Engine? Can I still access my data in the Cloud Datastore (after moving it to the new project with the Datastore Admin backup tool)? Is it possible to run only one GCE VM with only one instance and have a decent uptime? As I read, security patches are applied with CentOS unattended updates and don't imply an image or instance restart, right? So what SLA or uptime can I expect there?

Many thanks!

[George (Cloud Platform Support)]

Hello Thomas,

By contrast to the standard environment, the Compute Engine environment would compare with a server you might simply run at home, in which case you have to provide what is automatically offered in the app engine standard environment. There is no scalability, so using one instance might prove not enough for your traffic. Your mentioned “decent uptime” depends on the envisaged traffic generated by your app, in that case. You are right about the CentOS; Google Compute Engine does not automatically update the operating system or the software on your instances. There is a 99.95% guaranteed uptime in the SLA.

An excellent alternative would be the flexible environment, that offers some services similar to the standard environment, especially important among these being scalability. You are right, you can simply adapt your app to the Flexible Environment using Google Cloud Client libraries.

The same Cloud Datastore data is available regardless of if you use the App Engine libraries, the Google Cloud client libraries, or call the API directly, so, yes: you can still access your data in the Cloud Datastore.

[Thomas Baldauf]

Thanks for the answers! I already migrated my app to Flex Env using the compat runtime and noticed a great performance gain with only 1 permanent instance in comparison to automatic scaling in Standard Edition (10x more throughput showed by jmeter). Now that the compat runtime is deprecated I'll sadly have to rewrite my code anyway. Flex Env also seems to be quite expensive that's why GCE could be the right thing for me. Of course deployments or updates are a problem there, but one powerful instance should be enough to handle the current traffic (100k requests/day, <10 requests/second) and also for the near future. As soon as a load balancer is needed, GCE seems to be in a similar price range as Flex Env, so it's really hard to decide where to go - Flex Env of course would be perfect if it was less expensive and would support a Memcache service (which will come as I read). If there was the compat runtime GA it would be just perfect and I would be willing to pay more not having to rewrite everything. I think there would be many small to mid-size apps being very happy about it.
However, and sorry for thinking loud too much, the biggest problem of migrating from GAE SE to GCE to Flex is the data. As we need a new project for each migration, is it still possible to access the datastore of the legacy project from each of them using the Cloud Client libraries? That would be great and give us even more flexibility to better adapt to business growth.
Thank you so much for your wonderful work and support!

[George (Cloud Platform Support)]

Grüß Gott, 

Good news: you are perfectly right, it is still possible to access the Datastore of the legacy project when using the Cloud Client libraries, if you identify it with the same project ID. In fact, there is no need to launch a new project with a new ID just for your flex migration; you can keep the same project. In this way, you maintain your app’s access to the same Datastore, which is associated to the app by way of your project ID. You might consider modifying your app within the project, exactly as you suggested initially, and have it run in the environment you prefer: recommended being the flexible environment. 

Tschüss

[Thomas Baldauf]

Very interesting, I'll try that as soon as possible!

Dankeschön :)

[Thomas Baldauf]

One more question before I do a test with the still available compat runtime on Flex Env: is it possible to have the legacy app running in the same project as the freshly ported Flex app just by using a different version name? Will it just add a GCE VM for that versioned instance and let the legacy app do its Standard Edition job in friendly coexistence?

[George (Cloud Platform Support)]

If you don’t absolutely need the same app name with a different version to run in the new environment, there is the option to rename your new app and dedicate a new service to it, for the new environment, all under the same project. As a result, your old app would continue running in the standard environment, whereas the new one, freshly re-baptised, runs in the flex environment, offering a new service. You would finally run 2 services accepting traffic, and that should take care of their “friendly coexistence”. 

Still, both apps must exist within the same project, to access the same Datastore, as the project ID secures common access. 

[Thomas Baldauf]

Thank you, but I'm not sure how to achieve this. I selected the project with my standard environment app in gcloud init and tried to deploy the flex version using appengine:deploy and the appengine-maven-plugin. This worked a few weeks ago with a new project, although it took about 15mins. This time, I waited for more than 30mins, then got the following failure:   

[INFO] GCLOUD: Beginning deployment of service [default]...

[INFO] GCLOUD: WARNING: Deployment of App Engine Flexible Environment apps is currently in Beta

[INFO] GCLOUD: Building and pushing image for service [default]

[INFO] GCLOUD: WARNING: No configuration information provided by runtime [java].ß

---- after about 30mins ---

INFO] GCLOUD: ERROR: (gcloud.app.deploy) ('The read operation timed out',)

[INFO] GCLOUD: This may be due to network connectivity issues. Please check your network settings, and the status of the service you are trying to reach.

----

I don't see any notifications in my project's activity log. What am I doing wrong? Could the (deprecated) <vm>true</vm> in my appengine-web.xml be the reason? Is there a possibility to set the log level of appengine-maven-plugin / gcloud under the hood to see more log output?

The other option you mentioned involved renaming the project. But how is this changing anything? The project-id cannot be changed and the project name is only an alias. So after changing the project name, deploying the flex app and revert the project name - how should that be different to my first approach?

Thank you!

[George (Cloud Platform Support)]

Hello Thomas,

Exactly as you supposed: the <vm>true</vm> entry now needs to read <env>flex</env> in your appengine-web.xml, or env: flex in the app.yaml file.

You can see your logs in detail at /var/log/app_engine/custom_logs, as described in the “Using Cloud logging in App Engine apps” document.

In the same project, you need to create different versions of the application for different environments: standard, flex, etc., specifying in the configuration file both the version and service names: service: service_name in the app.yaml file.

[Thomas Baldauf]

Thank you, unfortunately it's still not working. Now I found the logs of GCLOUD command and it says:

2016-11-30 07:07:41,608 DEBUG    root            Retrying request to url https://www.googleapis.com/resumable/upload/storage/v1/b/staging.typingscout-test3.appspot.com/o?uploadType=resumable&alt=json&upload_id=AEnB2UoYF-HOEjSoAK5IGRUi0IUL3bipk1LjfDNU6T3gZss3JkrnsAtSx3LKHHMRhmzXIrugy2bTD8wJZgDJbze--bg6qTVKaA&name=us.gcr.io%2Ftypingscout-test3%2Fappengine%2Fdefault.20161130t070559%3Alatest after exception [Errno 10053] Eine bestehende Verbindung wurde softwaregesteuert durch den Hostcomputer abgebrochen

What to do?

[Thomas Baldauf]

In the log I could also see that src.tgz (146MB) is being uploaded for 6 minutes and then the host computer shuts down the connection because of a read timeout. What can I do about it?

[George (Cloud Platform Support)]

Hallo Thomas,

What were you attempting to do exactly, when these logs got recorded? What was the exact command for deployment? If you tried to deploy to the flexible environment, was  the standard environment app modified appropriately prior to the deployment attempt? You need to adapt your app to the Flexible Environment using Google Cloud Client libraries. In doing so, you might have overseen some networking aspects: error 1053 is returned in Windows for “Software caused connection abort.”

To build your container images outside of Google Cloud Platform, you must first upload your images to a container image repository before you can deploy your images to App Engine with the gcloud app deploy command.

[Thomas Baldauf]

I just used the appengine-maven-plugin, which apparently is invoking gcloud under the hood. I was able to deploy the simple hello world example (https://github.com/GoogleCloudPlatform/getting-started-java/tree/master/helloworld-jsp) and modified it to access the datastore with the Google Cloud Datastore API. This is working, but performance is pretty poor (400-700ms per request). Last time I deployed my whole app with jetty9-compat performance was brilliant. I'm not sure what's going on there at App Engine flexible environment, but I get the feeling it's way too early to switch over there. In contrast to my earlier attempts, I could not find the machine type of the VM in the Cloud Console anymore. A few weeks ago it was shown in the Compute Engine view, but now that's all empty (even though the app is running as a new version).

One more thing I also noticed: whenever I set one or more of these config parameters of appengine-maven-plugin

<deploy.promote>false</deploy.promote>

<deploy.stopPreviousVersion>false</deploy.stopPreviousVersion> 

<deploy.version>flextest</deploy.version> 

I get the following error and deployment fails:

2016-11-30 22:22:39,253 ERROR    root            (gcloud.app.deploy) Error Response: [13] Deployment Manager operation failed, name: operation-1480540945143-5428b4d1c54d9-31101d50-e4631c66, error: [{"code":"RESOURCE_ERROR","location":"/deployments/aef-default-20161130t222126/resources/aef-default-20161130t222126-00","message":"Unexpected response from resource of type compute.alpha.regionInstanceGroupManager: 403 {\"code\":403,\"errors\":[{\"domain\":\"global\",\"message\":\"Required 'Alpha Access' permission for 'Compute API'\",\"reason\":\"forbidden\"}],\"message\":\"Required 'Alpha Access' permission for 'Compute API'\",\"statusMessage\":\"Forbidden\",\"requestPath\":\"https://www.googleapis.com/compute/alpha/projects/typingscout-test2/regions/us-central1/instanceGroupManagers\"}"}]

[George (Cloud Platform Support)]

Hello Thomas,

Reproducing your code with Datastore access, I can see access times in the order of 200 to 300 ms, but for only one write and one final read. By contrast, your code accesses the Datastore for 28 times at each request, and it is there that the difference in timing is generated. Your application writes data in the Datastore and immediately reads it to eventually return it to the user via the servlet’s get request. This is time-consuming. Data may be cached in Memcache and saved asynchronously by a separate background process.

If you are using the Datastore, Google recommends that you design your application so that it will not need to update an entity group more than once per second for a sustained period of time. There is no difference in performance between the current version of the flex environment and previous ones.

Trying various combinations for the config parameters of the appengine-maven-plugin, I do not encounter any error. You may consider re-checking your local networking parameters. Providing more detail on the matter should bring more clarity.

[Thomas Baldauf]

Thank you for investigating my issues. I'll try to follow your recommendations.